U.S. Department of Education

Providing Effective Teachers 

for All Students:

Examples From Five Districts

A PPSS Lessons From the Field Report

A PPSS Lessons From the Field Report 

Providing Effective Teachers for All Students:

Examples From Five Districts

Prepared by

Mariann Lemke

Kerri Thomsen 

Andrew Wayne 

Beatrice Birman

American Institutes for Research

Washington, DC 

Prepared for

U.S. Department of Education 

Office of Planning, Evaluation and Policy Development 

Policy and Program Studies Service

2012 
This report was prepared for the U.S. Department of Education under Contract Number 

ED-04-CO-0025/0021 with the American Institutes for Research. Elizabeth Eisner served as contracting 

officer’s representative for this report. The views expressed herein do not necessarily represent the 

positions or policies of the Department of Education. No official endorsement by the U.S. Department 

of Education is intended or should be inferred. In addition, any mention of trade names, commercial 

products, services, or organizations in this report does not imply endorsement by the U.S. Department 

of Education. 

U.S. Department of Education

Arne Duncan 

Secretary

Office of Planning, Evaluation and Policy Development

Carmel Martin 

Assistant Secretary

Policy and Program Studies Service

Stuart Kerachsky 

Director

June 2012 

This report is in the public domain. Authorization to reproduce it in whole or in part is granted. While 

permission to reprint this publication is not necessary, the suggested citation is U.S. Department of 

Education, Office of Planning, Evaluation and Policy Development, Policy and Program Studies Service, 

Providing Effective Teachers for All Students: Examples From Five Districts, Washington, DC, 2012. 

This publication is also available at the Department’s website at 

http://www2.ed.gov/about/offices/list/opepd/ppss/index.html. 

On request, this publication is available in alternative formats, such as Braille, large print or computer 

diskette. For more information, contact the Department’s Alternate Format Center at 202-260-0852 or 

202-260-0818. 

Content Contact

Elizabeth Eisner

202-260-1109

elizabeth.eisner@ed.gov
Contents 

Contents

List of Exhibits .......................................................................................................................... v 

Acknowledgments ................................................................................................................... vii

Executive Summary .................................................................................................................. ix

Identifying Effective Teachers......................................................................................................................... x

Using Information About Teachers’ Effectiveness in Human Resource Policies ..................................xi

Taking Targeted Action to Improve Equitable Access to Effective Teachers........................................xii

I. Introduction and Overview .................................................................................................. 1

The Focus and Organization of This Report ................................................................................................ 2

The Five Districts Profiled for This Report .................................................................................................. 3

II. Identifying Effective Teachers ............................................................................................ 5

Measuring Teacher Effectiveness Through Student Achievement Growth............................................. 6

Measuring Teacher Effectiveness Through Classroom Observation Methods ......................................12

Summary............................................................................................................................................................20

III.Using Information About Teachers’ Effectiveness in Human Resource Policies ........... 21

Preparing and Hiring Effective New Teachers...........................................................................................21

Improving the Effectiveness of Current Teachers .....................................................................................23

Retaining Effective Teachers and Removing Ineffective Teachers..........................................................26

Summary............................................................................................................................................................30

IV.Taking Targeted Action to Improve Equitable Access to Effective Teachers ................. 31

Analyzing Effectiveness Data to Determine Whether and Where Action Is Needed ..........................31

Attracting Effective Teachers to and Retaining Them in High-Need Schools ......................................33

Offering Principals in High-Need Schools Additional Assistance With Hiring .....................................35

Summary............................................................................................................................................................35

References................................................................................................................................ 37 
Contents iv

Appendix A .............................................................................................................................. 41 

Appendix A Contents............................................................................................................................... 43 

Appendix A–1: Eagle County Award Computation ............................................................................ 45 

Appendix A–2: Columbus TAP Award Computation ........................................................................49 

Appendix A–3: Columbus Performance Advancement System (PAS) Gains Calculation............ 53 

Appendix A–4: Columbus Value-Added Professional Development Materials .............................. 57 

Appendix A–5: Hillsborough Current Observation Rubric ............................................................... 65 

Appendix A–6: Hillsborough New Observation Rubric ....................................................................69

Appendix A–7: Eagle County Professional Practices Rubric.............................................................79

Appendix A–8: Houston Decision-Making Framework..................................................................... 97 

Appendix A–9: Hillsborough Teacher Summary Sheet ....................................................................103 

Appendix B: Site Descriptions .............................................................................................. 109

Appendix B–1: Columbus City School District Site Description....................................................111 

Appendix B–2: Eagle County School District Site Description.......................................................115 

Appendix B–3: Hamilton County School District Site Description ............................................... 119

Appendix B–4: Hillsborough County Public Schools Site Description ..........................................123 

Appendix B–5: Houston Independent School District Site Description .......................................127 

Appendix C: Study Methods................................................................................................... 131 

Appendix D: List of Resources.............................................................................................. 135 
Exhibits v

Exhibits

I. Introduction and Overview

Exhibit 1 Policies and Practices in Use in the Five Districts Profiled for This Report...........................4

II. Identifying Effective Teachers

Exhibit 2 Sample Teacher Effectiveness Scores...........................................................................................7

Exhibit 3 Hillsborough County’s Method for Computing Teacher Effectiveness ..................................8

Exhibit 4 Tests Used for Columbus Performance Advancement System ................................................9

Exhibit 5 Houston’s Web Portal ...................................................................................................................11

Exhibit 6 Description of Hamilton County’s Planned Measure of Teacher Practice............................12

Exhibit 7 Hamilton County’s Guidance on Conducting Observations ...................................................17

Exhibit 8 Results of Houston’s Survey on the Classroom Observation Framework ............................18

III. Using Information About Teachers’ Effectiveness in Human Resource Policies

Exhibit 9 Hamilton County’s Proposed Cycle of Feedback to Improve Teacher Effectiveness........24

Exhibit 10 Houston’s Staff Management Decisionmaking Framework Excerpt...................................29

IV. Taking Targeted Action to Improve Equitable Access to Effective Teachers

Exhibit 11 Houston’s Analysis of Distribution of Effective Teachers, 

by School Poverty Level.............................................................................................................32

Appendix B: Site Descriptions

Appendix B–1: Columbus City School District Site Description

Exhibit B-1.1 District Snapshot ................................................................................................................113

Exhibit B-1.2 District Proficiency Rates in Mathematics and Reading, 

Grades 3–10, for 2008–09................................................................................................. 113

Appendix B–2: Eagle County School District Site Description

Exhibit B-2.1 District Snapshot ................................................................................................................117

Exhibit B-2.2 District Proficiency Rates in Mathematics and Reading, 

Grades 3–10, for 2008–09................................................................................................. 117

Appendix B–3: Hamilton County School District Site Description

Exhibit B-3.1 District Snapshot ................................................................................................................121

Appendix B–4: Hillsborough County Public Schools Site Description

Exhibit B-4.1 District Snapshot ................................................................................................................125

Exhibit B-4.2 District Proficiency Rates in Mathematics and Reading, 

Grades 3–10, for 2008–09................................................................................................. 125
Exhibits vi

Appendix B–5: Houston Independent School District Site Description

Exhibit B-5.1 District Snapshot ................................................................................................................129

Exhibit B-5.2 District Proficiency Rates in Mathematics and Reading, 

Grades 3–10, for 2008–09................................................................................................. 129
Acknowledgments vii

Acknowledgments

We wish to thank the many individuals who contributed to the completion of this report. 

Particularly helpful were Liz Eisner and Andy Abrams of the Policy and Program Studies Service (PPSS) 

at the U.S. Department of Education (ED). In addition, we benefited from the input of other ED 

officials, including Judy Wurtzel, Ann Whalen and Brad Jupp. 

District site visits were conducted by teams of American Institutes for Research (AIR) staff, including 

Courtney Zmach, Delphinia Brown, Susan Caceres, Cheryl Pruce, Anna Mark, Meredith Ludwig and 

Jeannette Moses. Rita Kirshstein provided senior quality review of the report, and Jane Coggshall also 

provided helpful suggestions. Production assistance was provided by Marty Leff, editor, and Alexis 

Carpenter, desktop publisher. 

Perhaps most important, we would like to acknowledge district officials, principals and union officials at 

Columbus City Schools, Eagle County Schools, Houston Independent School District, Hillsborough 

County School District, and Hamilton County Public Schools, who welcomed us to their districts and 

provided thoughtful and thorough responses to our interview questions. 

Executive Summary ix

Executive Summary 

One of the most important things school systems can do to promote student achievement is to ensure 

that all students have effective teachers. In fact, effective teachers may be particularly important for our 

most disadvantaged students (e.g., Gordon, Kane and Staiger 2006). Over the years, however, research 

has shown that many of our traditional methods of ensuring teacher effectiveness, such as requiring 

particular credentials, experience, or education, are not strongly related to teachers’ effectiveness in 

promoting student achievement (for example, see Wayne and Youngs 2003; Kane, Rockoff and Staiger 

2008). In response, federal, state and district policymakers are increasingly investing in alternative 

measures of teacher effectiveness, including data on growth in student achievement, as well as new or 

revised measures of instructional practice. With improvements in the ways teacher effectiveness is 

measured, new opportunities are available for states and districts to use information about teachers’ 

effectiveness in districtwide human resource policies and in efforts focused on ensuring equity. 

The U.S. Department of Education commissioned the American Institutes for Research (AIR) to create 

this report as a resource for state and district policymakers who are working to ensure that all students, 

regardless of background, have effective teachers. This report is intended to illustrate the steps districts 

take and the challenges they face in assessing teacher effectiveness as a precursor to providing effective 

teachers for all students. In so doing, the report focuses on three areas of state and district policy that 

play a mutually reinforcing role in promoting equitable access to effective teachers: 

1. Identifying effective teachers. The first challenge to ensuring equitable access to effective 

teachers is the development of better measures to identify teacher effectiveness. States and 

districts are starting to develop measures of teacher quality that rely directly on student academic 

growth and on observations of teachers’ practice. These two types of measures—referred to here 

as measures of effectiveness—generate information that can be used in human resource policies and 

in efforts to ensure equity.

2. Using information about teacher effectiveness in human resource policies. Where 

information about teacher effectiveness is available, states and districts are beginning to 

incorporate this information into the design of human resource policies including recruitment, 

selection and placement, induction, mentoring, professional development, performance 

management (i.e., evaluation), compensation and instructional leadership (Odden and Kelly 

2008). The use of information about teacher effectiveness in human resource policies is a key 

component of efforts to ensure that all students have effective teachers (Weisberg and others 

2009).

3. Using information about teachers’ effectiveness in efforts to ensure equity. While 

districtwide human resource policies are necessary for ensuring that all students have effective 

teachers, such policies may not be sufficient. Studies have documented that, on average, teachers 

migrate toward working in affluent, high achieving schools (Hanushek, Kain and Rivkin 2004; 

Goldhaber 2008). Thus, another component of efforts to ensure that all students have effective 

teachers is a set of policies that specifically targets high-need schools. On the basis of 

information about teachers’ effectiveness, states and districts can monitor the effectiveness of 

teachers in high-need schools, in comparison with other schools (see, e.g., Tennessee 

Department of Education 2007; Carr and Oxnam 2009), and use information about teacher 

effectiveness to design targeted programs for high-need schools. For example, incentives to 

teach in high-need schools can be offered to effective teachers only, rather than to all teachers.
Executive Summary x 

This report draws on the experiences of five districts to provide examples that may be relevant to state 

and district policymakers considering action in these three areas. The five districts are: 

• Columbus City Schools (Ohio) 

• Eagle County Schools (Colorado) 

• Hamilton County Public Schools (Tennessee) 

• Hillsborough County Public Schools (Florida) 

• Houston Independent School District (Texas) 

The report addresses three questions: 

1. How did the districts identify effective teachers and what were some of the challenges associated 

with the measures that the districts used?

2. How did the districts use information about teachers’ effectiveness in human resource policies?

3. How did the districts use information about teachers’ effectiveness in their efforts to make the 

distribution of effective teachers more equitable? 

By summer 2010, all five districts had begun to measure teacher effectiveness through student 

achievement growth in at least some schools and most were implementing new classroom observations 

of teachers’ practice. By summer 2010, the five districts had used those measures in human resource 

policies in a variety of ways, and some had other policies in place for promoting equitable access to 

effective teachers. AIR staff visited each district in June and July of 2010 to learn about their specific 

approaches to teacher effectiveness, including the way the districts selected those approaches and 

resolved key dilemmas and challenges.

This report highlights examples of the way study districts identified effective teachers, and draws on that 

information for human resource policies and efforts to ensure equity. Because most of the practices 

discussed in this report are relatively new and will likely be refined over time, monitoring and evaluation 

will allow districts and states to determine whether their actions are successful and to adjust their 

practices accordingly. 

Identifying Effective Teachers 

As of July 2010, the five study districts had developed measures of teacher effectiveness that drew on 

data on student achievement growth and four had measures of classroom practice. While the reasons for 

developing these measures varied, compensation reform was a key driver in four of the five districts. 

Specifically, Columbus, Eagle County, Hillsborough, and Houston all developed or improved their 

measures of teacher effectiveness in order to offer performance incentives for high-performing teachers 

districtwide or in select schools. Funding for performance incentives or to support the development of 

measures came from different sources at different times. For example, Eagle County’s efforts began 

with local funds which were supplemented later with TIF funds. As of summer 2010, Hamilton County, 

Columbus, and Hillsborough planned to continue their work to develop and refine measures of teacher 

effectiveness as part of their states’ Race to the Top grant efforts. All the districts’ efforts suggest a 

number of key challenges that other districts and states may need to address as they consider developing 

measures of teacher effectiveness: 
Executive Summary xi

• Communication and engagement. Interviewees noted the importance of efforts to ensure 

that teachers and principals were appropriately informed about new measures of teacher 

effectiveness. Whether with respect to the design of a new measure or to its implementation, 

study districts responded to the need to allow for input and to provide clear and comprehensive 

information to teachers and principals in a variety of ways. Some provided opportunities for 

teachers or others to help develop or refine measures like classroom observation frameworks. 

Others focused on providing information about their measures. One provided a systematic, 

ongoing process by mandating regular stakeholder input for updates to the teacher evaluation 

process. 

• Limited scope of student achievement growth measures. A key challenge for study districts 

was the fact that data on student achievement growth were available only for teachers in grades 

and subjects in which state tests were implemented. Study districts used different approaches to 

supplement these measures, including creating or repurposing other district assessments, or 

applying schoolwide or other aggregate measures to teachers in untested grades or subjects. 

• Implementation of classroom observation frameworks. Interviewees noted challenges in 

implementing classroom observation systems that were both rigorous and feasible. For example, 

interviewees noted challenges in finding the right observers and in ensuring and maintaining 

reliability. Study districts took a variety of approaches in response to these challenges, including 

providing scheduling support, providing ongoing training, and working with teachers and 

principals to identify the right observers for each district’s context. 

Using Information About Teachers’ Effectiveness in Human Resource 

Policies

The five study districts had developed policies for using effectiveness information in their human 

resource policies. Across the districts, there were policies or programs to address the entire continuum of 

teacher employment and development, from hiring to dismissal, but study districts most often focused 

on the following: 

• Compensation initiatives. All study districts except Hamilton County had in place 

performance pay initiatives that made use of differing teacher effectiveness measures. As noted 

earlier, these programs often served as the impetus to design or redesign teacher effectiveness 

measures.

• Feedback through observation. Where study districts had new or revised observation 

frameworks in place, these frameworks appeared to serve as a linchpin in district efforts to 

provide useful feedback to teachers and aid them to improve their instructional practices. The 

format of the observations and feedback varied from district to district.

• Career paths. Several interviewees spoke about their efforts to use effectiveness information to 

identify teachers for leadership positions, as well as to release those they determined to be 

ineffective. Districts used different types of information for these purposes. For example, 

Houston described using value-added data as a potential source of evidence for dismissal and for 

granting leadership opportunities; Eagle County spoke about using observational data for 

dismissal purposes and Columbus noted how it could be used for identifying potential teacher 

leaders. 
Executive Summary xii

Taking Targeted Action to Improve Equitable Access to Effective 

Teachers

Several of the five study districts had developed policies for using effectiveness information in efforts to 

ensure teacher effectiveness in high-need schools. One district, Houston, analyzed effectiveness 

information to determine where inequities within the district might be. The focus of most study districts 

was monetary incentives to attract and retain effective teachers in high-need schools. In one case, 

Houston’s Effective Teacher Pipeline project, such incentives were paired with efforts to address 

working conditions in high-need schools by encouraging movement of many effective teachers to a small 

set of high-priority, high-need schools. Placing multiple effective teachers in high-need schools was 

intended to create a culture of supportive peers. 

The study districts and many other districts around the nation are at the forefront of learning how to 

ensure teacher effectiveness for all students, and their experiences may prove informative to others. In 

addition to the performance of ongoing monitoring and adjustments, districts can learn from one 

another about teacher evaluation strategies. This report is one step in that direction. 
Introduction and Overview 1 

I. Introduction and Overview

This report is intended to illustrate the steps districts take and the challenges they face in assessing 

teacher effectiveness as a precursor to providing effective teachers for all students. The report draws on 

descriptive information from case studies of five districts that have developed and implemented new 

measures of teacher effectiveness based on gains in student achievement, observations of instructional 

practice, or based on both types of data. The report discusses the ways these districts are using such 

measures in their human resource policies, including efforts to ensure the equitable distribution of 

effective teachers. Research has documented the importance of having effective teachers. For example, 

Gordon, Kane and Staiger (2006) found that students who had teachers in the top quartile of 

effectiveness gained 10 achievement percentile points relative to similar students who had teachers in the 

bottom quartile of effectiveness. 

Federal programs have increasingly focused on teacher quality as a tool for closing achievement gaps. 

The most recent reauthorization of the Elementary and Secondary Education Act (ESEA) in 2002 placed 

substantial policy emphasis on the key role of teachers by requiring that, by the end of the 2005–06 

school year, all core subjects be taught by highly qualified teachers (HQTs).1

1 The federal definition of highly qualified is a teacher who is fully certified and/or licensed by the state in which he or 

she teaches, who holds at least a bachelor’s degree from a four-year institution, and who demonstrates competence in 

ways identified in the statute in each core academic subject area taught (No Child Left Behind Act of 2001).

In addition, ESEA required 

that states provide assurances and develop plans to “ensure that poor and minority children are not 

taught at higher rates than other children by inexperienced, unqualified, or out of field teachers” (Section 

1111(b)(8)(C)). In 2009, the American Recovery and Reinvestment Act (ARRA) requirements reinforced the 

focus on equitable distribution of teachers by requiring states applying for education stimulus (State 

Fiscal Stabilization) funds to provide updated assurances and to publicize their plans to address potential 

inequities. ARRA also established competitive grants to help states build their pool of effective teachers 

and address inequities in the distribution of teachers, through, for example, the Race to the Top (RTT) 

program, for which one priority area is effective teachers and leaders. 

In addition to their increasing focus on the role of teacher quality in improving student academic growth 

and closing achievement gaps, federal programs have been promoting shifts in the way teacher quality is 

measured, away from a primary focus on teacher qualifications and credentials and toward effectiveness. 

In this report, the term measure of teacher effectiveness is used to refer to measures based on growth in 

student achievement, teacher practice, and a combination of these two approaches. Among the 

arguments supporting this shift, recent research has documented the limited extent to which commonly 

measured qualifications, such as possession of a master’s degree, are related to student outcomes (for 

example, see Wayne and Youngs 2003; Kane, Rockoff and Staiger 2008). In response to these findings 

and to several current federal and foundation initiatives, states and districts have begun to use data on 

growth in student achievement, as well as new or revised measures of instructional practice, to judge 

teacher effectiveness. Federal programs such as the Teacher Incentive Fund (TIF) and RTT have 

provided incentives for states and districts to move in this direction, including funds to support some of 

the technical aspects of developing measures of teacher quality. 

With improvements in the way teacher quality is measured, new opportunities are available for states and 

districts to examine whether students in high-need schools are as likely as other students to have 

effective teachers. For decades, data on teacher qualifications have indicated that students in 

high-poverty and high-minority schools are less likely than others to have teachers with certain 

qualifications. For example, studies have found that students in high-poverty or high-minority schools 

 
Introduction and Overview 2 

are more likely to be taught by less qualified and less experienced teachers (Coopersmith 2009; Clotfelter 

and others 2007; DeAngelis and others 2005). These students are more likely to be taught by “out-offield” teachers, teachers who have neither certification nor a major in the subject that they are teaching 

(Education Trust 2008; Lashway 2004), or by teachers trained at less selective colleges or universities 

(Imazeki 2007; Wayne 2002). 

Today, analyses using data on growth in student achievement are available in some states and districts, 

and provide a sharper understanding of the distribution of teacher quality (see, e.g., Tennessee 

Department of Education 2007; Sass and others 2010). 

The Focus and Organization of This Report

This report focuses on three areas of state and district policy that play a mutually reinforcing role in 

promoting equitable access to effective teachers. Policies in the first area, identifying effective teachers, provide 

the foundation for policies in the latter two areas: using information about teachers’ effectiveness in human resource 

policies and using information about teachers’ effectiveness in efforts to ensure equity.

1. Identifying effective teachers. The first challenge to ensuring equitable access to effective 

teachers is the development of better measures of teacher quality. States and districts are starting 

to develop measures of teacher quality that rely directly on student academic growth and on 

observations of teachers’ practice. These two types of measures—referred to here as “measures 

of effectiveness”—generate information that can be used in human resource policies and in 

efforts to ensure equity.

2. Using information about teachers’ effectiveness in human resource policies. Where 

information about teacher effectiveness is available, states and districts are beginning to 

incorporate this information in the design of human resource policies. Odden and Kelly (2008) 

identify eight key areas of human resource policies: recruitment, selection and placement, 

induction, mentoring, professional development, performance management (evaluation), 

compensation, and instructional leadership. The use of information about teacher effectiveness 

in human resource policies is a key component of efforts to ensure that all students have 

effective teachers (Weisberg and others 2009). 

3. Using information about teachers’ effectiveness in efforts to ensure equity. While 

districtwide human resource policies are necessary for ensuring that all students have effective 

teachers, such policies are not sufficient. Studies have documented that, on average, teachers 

migrate toward working in affluent, high-achieving schools (Hanushek, Kain and Rivkin 2004; 

Goldhaber 2008). Thus, another component of efforts to ensure that all students have effective 

teachers is a set of policies that specifically target high-need schools. On the basis of information 

about teachers’ effectiveness, states and districts can monitor the effectiveness of teachers in 

high-need schools, in comparison with teachers in other schools (see, for example, Tennessee 

Department of Education 2007; Carr and Oxnam 2009). States and districts can also use 

information about teacher effectiveness in the design of targeted monetary and nonmonetary 

incentives for high-need schools. For example, incentives to teach in high-need schools can be 

offered to effective teachers only, rather than to all teachers. 

This report includes one chapter on each of these three policy areas, drawing on a group of five study 

districts as a source of concrete examples of actions in these areas. The report addresses three main 

questions: 
Introduction and Overview 3 

1. How did the districts identify effective teachers and what were some of the challenges associated 

with the measures that the districts used?

2. How did the districts use information about teachers' effectiveness in human resource policies?

3. How did the districts use information about teachers’ effectiveness in their efforts to make the 

distribution of effective teachers more equitable? 

The appendixes provide additional concrete examples (Appendix A), profiles of each of the five districts 

(Appendix B), a description of study methods (Appendix C), and a list of resources for further support 

(Appendix D). 

The Five Districts Profiled for This Report

This report draws on the experiences of five districts to provide concrete examples that may be relevant 

to state and district policymakers considering similar approaches. The districts vary in size and region, 

from very large and urban to small and rural. The five districts are 

• Columbus City Schools (Ohio) 

• Eagle County Schools (Colorado) 

• Hamilton County Public Schools (Tennessee) 

• Hillsborough County Public Schools (Florida) 

• Houston Independent School District (Texas) 

AIR selected the five districts because, by summer 2010, all had begun to measure teacher effectiveness 

through student achievement growth in at least some schools and most were implementing 

newclassroom observation of teachers’ practice. They had also used such measures in a variety of human 

resource policies, and three of the districts had at least one policy for using such measures to promote 

equitable access to effective teachers. Several of the districts had received private, federal, or state grants 

which supported their efforts to measure effectiveness or use effectiveness data in human resource 

policies. For example, Eagle County and Houston were TIF grantees; Columbus received TIF funds 

through a grant to the state of Ohio. Hamilton County, Hillsborough County, and Columbus were all 

expecting to receive Race to the Top funds through their states. The site descriptions at the end of this 

report provide more information about each district and funding supports received for their efforts. 

AIR visited each district in June and July 2010 to interview district-level staff, teachers’ association or 

union representatives and principals about current and planned approaches to gathering and using 

information on teacher effectiveness, including the ways in which the districts selected those approaches 

and resolved key dilemmas and challenges. AIR also collected and analyzed documents both before and 

during site visits. AIR prepared a summary report for each district and shared each report with the 

district for verification. In some cases, districts noted during AIR visits that they were in the process of 

making changes to their policies and practices related to teacher effectiveness. These changes were noted 

when they were described to interviewers. This report focuses primarily on strategies or policies in place 

as of AIR’s visits in summer 2010. 

The chapters that follow describe the policies and practices in use in each district as of summer 2010, as 

summarized in Exhibit 1 and the bulleted items below: 
Introduction and Overview 4 

• All five districts used student achievement growth as one measure of teacher effectiveness for 

some or all teachers. Several districts were updating these student achievement growth measures. 

In addition, four districts used new or revised observation-based assessments or were in the 

process of developing them (see chapter II). 

• All five districts used their measures of teacher effectiveness in some human resource policies. 

For example, four used effectiveness information in performance pay initiatives, and five 

planned to use data from their observation frameworks or student achievement scores to guide 

professional development decisions. Three districts promoted the use of effectiveness 

information to remove ineffective teachers. Many of these practices were in an early stage of 

development, with the districts making adjustments to traditional practices on the basis of a 

review of effectiveness information (see chapter III). 

• Three of the five districts had policies that targeted high-need schools, drawing on effectiveness 

information. All three offered financial incentives to teachers to move to or stay in high-need 

schools, and one had hiring and transfer policies designed to provide principals in high-need 

schools additional opportunities to hire effective teachers (see chapter IV). 

Exhibit 1

Policies and Practices in Use in the Five Districts Profiled for This Report

Policies

Districts

Columbus Eagle County

Hamilton 

County

Hillsborough 

County Houston

Identifying Effective Teachers

Value-added or student growth scores in use • • • • •

Classroom observation methods in use • • • •

Using Information About Teachers’ 

Effectiveness in Human Resource Policies 

Initial preparation • •

Initial hiring • •

Professional development • • • • •

Compensation • • • •

Leadership • •

Tenure review or dismissal • • •

Transfer eligibility or hiring • •

Taking Targeted Action to Improve 

Equitable Access to Effective Teachers 

Analyzing data •

Programs to attract or retain effective teachers in 

high-need schools • • •

Additional hiring support for high-need schools •

Note: Policies and practices used districtwide or in select schools as of summer 2010 are shown in this table.
Identifying Effective Teachers 5 

II. Identifying Effective Teachers 

In a recent study of teacher performance measurement practices in 10 districts, researchers found that 

more than 99 percent of teachers were rated as satisfactory. Even where districts offered three or more 

rating categories, more than half the teachers were given the highest rating (e.g., “Outstanding,” 

“Superior”) (Weisburg and others 2009). These ratings became common in part because of 

underinvestment in the systems for measuring teacher performance. These systems traditionally relied on 

classroom observations by building administrators, conducted no more than three times in one year 

(Danielson and McGreal 2000; Porter, Youngs and Odden 2001). Weaknesses in those systems included 

observation instruments based on outdated research on what instruction should look like, lack of 

observer training on the observation instrument, lack of observer expertise in specific assignment areas 

and lack of time for observers to fulfill their responsibilities (Consortium for Policy Research in 

Education 1998; Danielson and McGreal 2000; Weiss and Weiss 1998). 

Districts are now reconsidering the way they measure teacher performance, in part because studies of 

teacher-linked student achievement data indicate sizable differences in teacher performance. For 

example, Gordon, Kane, and Staiger (2006) found that the achievement of students who had teachers in 

the top quartile of effectiveness, as measured by student achievement growth, gained 10 percentile points 

in achievement relative to similar students who had teachers in the bottom quartile. These findings 

underscore the need to better measure differences in teacher performance, as discussed in this chapter, 

and to build such information into district human resource and equity policies, as discussed in chapters 

III and IV. 

Using student achievement 

growth, rather than only 

observations, to measure 

differences in teacher 

performance has become 

increasingly feasible with the 

availability of teacher-linked 

student achievement data. 

Because many teachers teach 

grades or subjects that are 

currently untested, and 

because methodological 

issues concerning student 

growth measures remain to 

be resolved (e.g., Koedel and 

Betts 2009), many states and 

districts are developing both 

types of teacher 

effectiveness measures: 

those that use student 

achievement growth and 

those that use new or 

revised classroom 

observations. Some 

programs, such as RTT and 

TIF, require the use of multiple measures to determine teacher effectiveness.

The Teacher Advancement Program (TAP)

The goal of the Teacher Advancement Program is to attract and 

retain talented teachers through a reform model aimed at increasing 

the rewards of being a teacher. TAP provides professional 

development, opportunities for collaboration, classroom evaluations, 

and opportunities for career advancement. TAP is based on four 

elements: 

• Multiple career paths that allow teachers to take on greater 

responsibilities without leaving the classroom 

• Ongoing applied professional growth based on collaborative 

time and focused on identified needs 

• Instructionally focused accountability based on the TAP 

teaching skills, knowledge and responsibility standards 

• Performance-based compensation based on teachers’ 

performance in the classroom and their students’ academic 

performance 

TAP is operated by the National Institute for Excellence in Teaching. 

The institute works with districts or groups of schools to implement 

TAP. Source: http://www.talentedteachers.org (May 2010). 
Identifying Effective Teachers 6 

In the five study districts, data from student achievement growth or observation measures were used. 

Hillsborough County and Hamilton County were in the process of creating effectiveness ratings by 

combining data from multiple measures (see text box below). Meanwhile, Eagle County and Columbus’ 

Teacher Advancement Program (TAP) schools (see text box above) used data from multiple measures to 

award bonuses without actually creating an overall effectiveness score based on combined data; that is, 

teachers could earn separate salary increases for observational scores and for student achievement 

growth (Appendix A-1 and Appendix A-2 describe these calculations in more detail). 

This chapter’s two main sections discuss (1) teacher effectiveness measures that use student achievement 

growth and (2) teacher effectiveness measures that use classroom observations. A final section draws 

from the experiences of the study districts to provide potential lessons for other districts to consider as 

they pursue similar policies. 

Creating an Effectiveness Rating Using Multiple Measures

in Hillsborough County and Hamilton County

When AIR visited Hillsborough in summer 2010, the district planned to use its Bill & Melinda Gates Foundation 

Empowering Effective Teachers (EET) grant to create a new system combining a student growth measure and an 

observation measure, weighted at 40 percent and 60 percent, respectively. Both principals and teachers trained as 

observers would conduct classroom observations. The combined scores from these measures would then be divided 

into five categories (one through five), with categories three through five being considered “Effective.” These scores 

would determine where a teacher stood on a career ladder, which in turn would determine his or her compensation. 

Scores would also be used for tenure and in contract renewal decisions, and for determining the professional 

development needs of teachers.

As a result of Tennessee’s RTT grant, Hamilton County planned, as of summer 2010, to implement a new system to 

combine data and determine an overall effectiveness score for teachers, beginning in 2011–12. Educators would 

receive a rating of one through five for student achievement growth and another rating of one through five for teacher 

practice, on the basis of a qualitative appraisal instrument. These ratings would be combined into a single rating, 

through a system not yet determined during the site visit. 

Measuring Teacher Effectiveness Through Student Achievement

Growth 

One approach to measuring teacher effectiveness is to analyze teacher-linked student achievement data. 

Each of the five study districts implemented such measures to some degree. In Columbus, for instance, 

individual teachers in science, social studies, reading and mathematics in grades 3 through 8 were eligible 

to receive reports on their students’ growth through the Teachers Connecting Achievement and Progress 

(TCAP) program. Reports for individual teachers compared the actual amount of student growth over 

the course of one school year to the amount predicted by a statistical value-added model (see Exhibit 2). 
Identifying Effective Teachers 7 

Exhibit 2 

Sample Teacher Effectiveness Scores

Value-added scores are sometimes expressed as standardized scores, such as from –2 to 2, or –1 to 1, with 0 as the 

average. A teacher’s score can either be greater than 0, less than 0, or not distinguishably different from 0. The 

sample report below from the Columbus TCAP program shows a “Teacher Effect” of –0.2, meaning that a point 

estimate of the growth for the students of this teacher was less than expected, according to an analysis of average 

growth for students with similar academic and demographic characteristics. On the far right, the report indicates that 

this difference was NDD, or not detectably different, meaning that the difference was not statistically different from 

0 and, therefore, that the teacher’s performance was not measurably different from that of an average teacher. 

Note: Data in this exhibit are fictional.

Source: Columbus City Public Schools presentation, “TCAP—Teachers Connecting Achievement and Progress: 

Teachers In-Service,” April–May 2007.

The experiences of the study districts in selecting and implementing measures of teacher effectiveness 

based on student achievement data revealed challenges and practices that might inform other districts on 

the following issues: 

• Selecting an analytic method for linking teacher performance to student achievement 

• Using student growth measures for teachers of nontested grades and subjects 

• Linking students to the right teachers 

• Building principal and teacher understanding of measures of teacher effectiveness 

Selecting an Analytic Method

There are several methods available for calculating teacher effectiveness on the basis of student 

achievement growth. Choosing a specific calculation method involves a number of decisions, including 

the type of analytic method to use, which control variables to include, and the number of years of 

achievement data to include. Data collected from study districts indicate that some districts took the 

following actions when developing and rolling out measures of teacher effectiveness based on growth in 

student achievement data:

• Developed different measures of teacher effectiveness for different purposes. For 

low-stakes informational and planning purposes, teachers participating in the Teachers 

Connecting Achievement and Progress program (TCAP) in Columbus received reports that 
Identifying Effective Teachers 8 

estimated their relative effectiveness on a continuum (as shown earlier in Exhibit 2). A teacher 

receiving such a report might seek professional development in subject areas in which he or she 

was less effective. For purposes of determining performance incentive payouts, schools 

participating in a different program—Columbus’ Teacher Advancement Program (TAP)—used 

value-added scores produced using a different methodology (an extension of the state’s model). 

• Monitored initial results to inform changes to measures. Hillsborough County officials 

first started using their student growth measure in 2007, to award performance bonuses. They 

made adjustments based on initial results (see Exhibit 3) and as of summer 2010 were planning a 

new calculation method. 

Exhibit 3

Hillsborough County’s Method for Computing Teacher Effectiveness

When visited in 2010, the Hillsborough County School District was in the process of transitioning to a 

value-added teacher effectiveness measure. The prior system used a “value table,” whereby each student’s 

pretest and posttest scores were converted into levels between 1 and 5. If a student made greater gains than 

the average gain of other students at the same pretest level, the teacher earned points, and the number of 

points awarded depended on how difficult the gain was to obtain, as determined by the number of students who 

were able to accomplish that gain. For example, as shown in the table below, a student with a pretest level of 

1 and a posttest level of 1 earned 0 points for a teacher, whereas a student with a pretest level of 1 and a 

posttest level of 4 earned a teacher 145 points. The teacher’s effectiveness score was the sum of all of the 

points earned by his or her students, divided by the total number of students.

American History

Pretest Score

Posttest Score

Low 1 High 1 2 3 4 5

Low 1 0 65 90 120 145 170

High 1 –25 60 90 115 140 160

2 –50 –25 55 105 155 205

3 –75 –50 –25 80 150 200

4 –100 –75 –50 –25 75 150

5 –100 –75 –50 –25 40 120

Source: Merit Award Program presentation, Hillsborough County (n.d.).

Using Student Growth Measures for Teachers of Nontested Subjects

There are many teachers and other school staff members for whom student growth data are currently 

unavailable, such as teachers of noncore subjects like art or physical education or some teachers of 

students with disabilities. In addition to using nontest data, such as observations, study districts 

employed a number of different strategies to provide effectiveness scores for these teachers:2

 

2 One approach to measuring student achievement growth in currently untested grades or subjects that none of the five 

case study districts was implementing, but that has been described elsewhere is teacher-determined “student learning 

objectives.” For example, the Austin Independent School District uses student learning objectives in which teachers set 

goals for themselves and their students and then measure whether or not those goals are achieved, using assessments 

agreed on with the principal. These could include standardized assessments or teacher-made assessments. The state of 

Rhode Island is considering a similar model. According to Goe (2011), this method allows teachers to have a major role 

in determining how growth is measured and how they will be evaluated. However the method also relies on teachers’ 

judgments; thus, it may be difficult to establish comparability across teachers.
Identifying Effective Teachers 9 

• Assigned aggregate scores to individual teachers. One approach several study districts used 

for teachers without assessment data suitable for computing student achievement growth was to 

assign grade-level, department-level or building-level scores to those teachers. In Columbus’ 

TAP schools, where value-added results affected compensation, teachers of nontested subjects 

or grade levels were given a value-added score based on the buildingwide average. Houston used 

a similar approach. One drawback, as one principal in Columbus commented, was that some 

teachers believed it was unfair for some teachers to benefit (or not) from the work of others. 

• Created new assessments. Hillsborough used a different approach for assessing teachers of 

noncore subjects: creating assessments in these subjects and determining student growth, using a 

combination of data from these tests and state assessments in other subjects. For example, a 

score for a social studies teacher might be based on the results of a district-created social studies 

exam, along with a reading score from the state test. However, one principal noted that 

introducing new assessments could raise new issues related to the perceived rigor and fairness of 

the assessments, and might be costly. 

• Repurposed existing assessments. Columbus’ Performance Advancement System (PAS) 

evaluated teachers without classroom value-added reports on the basis of their students’ pretest 

and posttest gains, using other existing district-administered assessments. These assessments 

included end-of-course assessments, benchmark assessments, alternative assessments, and

assessments of English-language proficiency (see Exhibit 4 for a description of what tests were 

used in the PAS program, and Appendix A–3 for the calculation in detail). 

Exhibit 4 

Tests Used for Columbus Performance Advancement System 

The PAS program allowed teachers in Columbus to engage in classroom action research using a variety of 

assessments to measure student progress, including state achievement tests, district end-of-course (EOC) tests or 

benchmark tests, alternative tests and English-language-proficiency tests. On the basis of the results of these 

assessments, teachers could receive awards of as much as $2,500. 

Possible Pretests for 2009–10 School Year Possible Posttests for 2009–10 School Year

Grades 9–12 Grades 9–12

Ohio Achievement Test (OAT) 2009 in reading, 

mathematics, science or social studies for 9th grade

OAT 2010 in reading, mathematics, science or social 

studies for 9th grade

EOC 2009 in reading, mathematics, science or social 

studies for 10th or 12th grade

EOC 2010 in reading, mathematics, science or social 

studies for 10th or 12th grade

Ohio Graduation Test (OGT) 2009 in reading, 

mathematics, science or social studies for 11th grade

OGT 2010 in reading, mathematics, science or social 

studies for 11th grade

Alternate Assessment 2009 for special education Alternate Assessment 2010 for special education

Ohio Test of English Language Acquisition (OTELA) 

2009 for all grades limited English proficient (LEP)

Ohio Test of English Language Acquisition (OTELA) 

2010 for all grades limited English proficient (LEP)

Pre-K to Grade 8 Pre-K to Grade 8

Get It, Got It, Go—Fall Get It, Got It, Go—Spring

Pre-K Benchmark—Fall Pre-K Benchmark—Spring

Kindergarten Reading or Mathematics Benchmark—Fall Kindergarten Reading or Mathematics Benchmark—

Spring

OAT 2009 in reading and mathematics for 4th–8th 

grades

OAT 2010 in reading and mathematics for 4th–8th

grades

Alternate Assessment 2009 Alternate Assessment 2010

OTELA 2009 for all grades LEP OTELA 2010 for all grades LEP

Source: Performance Advancement System (PAS) 2009–10 Guidelines, http://www.columbus.k12.oh.us/pas.
Identifying Effective Teachers 10

Linking Students to the Right Teachers 

Some of the interviewees in case study districts noted that linking students to the right teachers could be 

especially difficult for students with more than one teacher for the same subject (as in the case of 

coteachers) or for those who worked with specialists or resource teachers. 

Columbus and Hillsborough addressed this issue by requiring teachers to verify (1) the list of students 

whom they taught in the prescribed time periods and (2) the proportion of time that they spent teaching 

them. (An example of the instructions given to teachers in Columbus appears in the following text box.) 

These districts then assigned weights to the scores for these teachers on the basis of the percentage of 

time that the teachers actually spent teaching students. That is, teachers were only considered responsible 

for some proportion of students’ scores. 

Linking Students to the Right Teachers in Columbus

Prior to calculating value-added scores, the Columbus district asked teachers to verify their student rosters in order to 

properly account for shared teaching responsibilities. Teachers were provided with the following instructions 

regarding linking students to teachers and had to confirm the students for whom they provided instruction.

1. Make sure the electronically provided class roster accurately reflects the students you taught.

2. Make sure students who moved in or out of your classroom are accounted for.

3. Make sure that, if the instruction of some students was shared with another teacher, those students are 

accounted for.

Source: Columbus City Public Schools presentation, “TCAP—Teachers Connecting Achievement and Progress: 

Teachers In-Service,” April–May 2007.

Building Principal and Teacher Understanding of Measures of Teacher 

Effectiveness 

Staff in two of the five districts underscored the need to develop a comprehensive long-term training 

and communication plan about teacher effectiveness measures, specifying the type of information 

provided, when, how, and to whom. Study districts indicated that they had taken the following actions—

or were in the process of taking action—in order to build understanding of the teacher effectiveness 

measures: 

• Provided as much explanatory information as possible about how scores were generated. 

All study districts provided written descriptions of the way their measures of teacher 

effectiveness were generated. In response to questions from teachers in Hillsborough, the district 

showed specifically how many points on the value table teachers received for each of their 

students, so that they could better understand how the overall score was generated (as shown 

earlier, in Exhibit 3). 

• Provided information directly to teachers. Several principals in the districts indicated that 

they did not fully understand how the measures yielded effectiveness scores, and that they could 

not explain the system to teachers. 

At the request of principals, Houston provided information directly to teachers through an 

information portal, to ensure that there were consistent and accurate explanations available (see 

Exhibit 5 for a page from the portal, which contained links to value-added reports, information 

about value-added scores and other information). The portal was designed not only to assist 

teachers directly but to provide more information to principals about the measures. 
Identifying Effective Teachers 11

• Provided information in multiple formats. Houston’s portal included videos, online courses, 

brochures, and direct access to data. Columbus provided direct professional development for 

principals and lead teachers (see sample materials used in professional development in 

Appendix A–4). 

Exhibit 5 

Houston’s Web Portal

Houston’s Accelerating Student Progress—Increasing Results & Expectations (ASPIRE) project portal is designed to 

improve communication about effectiveness measures. It includes links to value-added score reports, information on 

what value-added is and how it is computed, links to professional development courses that teachers can sign up for

and other relevant information 

Source: http://portal.battelleforkids.org/Aspire/Value-Added/VAResults.html?sflang=en, cited 2010.
Identifying Effective Teachers 12

Measuring Teacher Effectiveness Through Classroom Observation 

Methods

In addition to student growth, the study districts measured teacher effectiveness through observation 

methods. In some of the case study districts, the results of the observations provided a basis for 

immediate, specific feedback to teachers (see Exhibit 6 for a description of the observation framework 

planned for use in Hamilton County as of summer 2010, with explanations of the performance levels 

associated with each measured dimension of teaching). 

Exhibit 6 

Description of Hamilton County’s Planned Measure of Teacher Practice

Hamilton County’s new classroom observation system was based on a rubric developed by an education consultant. 

The rubric contained the following six domains:

— Planning and preparation for learning

— Classroom management

— Delivery of instruction

— Monitoring, assessment and followup 

— Family and community outreach

— Professional responsibilities

Within each domain, there are 10 indicators. Each indicator has descriptors for four levels of performance. A sample 

indicator (“Knowledge”) and descriptors are shown for the “Planning and preparation for learning” domain.

Planning and Preparation

Highly Effective Effective

Improvement 

Necessary

Does Not Meet 

Standards

Knowledge Is expert in the 

content area and 

has cutting-edge 

grasp of how 

students learn

Knows the content 

well and has a good 

grasp of how 

students learn

Is somewhat familiar 

with the content and 

has a few ideas of 

ways students 

develop and learn

Has little familiarity 

with the content and 

few ideas on how to 

teach it and how 

students learn

Teachers would earn between 1 and 4 points per indicator (1 for “Does Not Meet Standards,” 2 for “Improvement 

Necessary,” and so on). Data were to be collected through 10 “mini observations” (5–10 minutes) by principals and 

assistant principals, as well as document reviews. For the summative evaluation, principals would total the points 

earned for each indicator. For example, a teacher who earned an “Effective” rating for 60 indicators would earn a total 

of 180 points (60 indicators × 3 points per indicator), giving him or her a final rating for the year of “Effective.” Along 

with the score, comments and specific examples were to be provided to the teacher.

continued next page 
Identifying Effective Teachers 13

Exhibit 6 (continued)

Description of Hamilton County’s Planned Measure of Teacher Practice 

Note: PIP refers to Performance Improvement Plan; TPP/IAP refers to Teacher Performance Plan/Intensive 

Assistance Plan.

Source: Hamilton County Teacher Evaluation Pilot, Draft presentation, 2010–11. 

Four of the five study districts used or were planning to use revised teacher observation systems, 

replacing traditional systems. The fifth district, Columbus, supplemented its existing classroom 

observation measure only in its six TAP schools. The experiences of the five study districts in selecting 

and implementing measures of teacher practice can be organized into the following areas: 

• Selecting an observation framework 

• Using effective teachers as observers 

• Ensuring consistency in scoring 

• Scheduling observations 

• Updating the observation framework 

Selecting an Observation Framework 

In choosing an observation framework, districts have to consider several questions (summarized in the 

following text box). At a more general level, districts have to choose between adopting an existing 

framework wholesale, adapting an existing framework or creating an entirely new one. Some existing 

frameworks may have evidence of validity and reliability. By adopting these frameworks, districts or 

states may also benefit from existing training or other support resources. However, districts or states may 

want to allow their staff or teachers to have input into the design of observation rubrics. Columbus 

adopted an existing framework developed for use in all TAP schools. The TAP rubric consists of a 

number of different dimensions related to instruction, planning instruction and the learning 

environment. Teachers receive an overall score of 1 to 5: 1 and 2, not proficient; 3, proficient; and 

4 and 5, advanced. 
Identifying Effective Teachers 14

Eagle County started using the TAP observation framework as originally designed but, over time, 

adapted the framework. For example, the county created some indicators that were observed over time 

and some that were scored in the course of a single observation. Hamilton County also took an existing 

framework and made extensive revisions. These districts had not yet determined the predictive validity of 

the resulting frameworks (see Appendixes A–5 to A–7 for the frameworks from study districts). 

Questions to Consider in Selecting an Observation Framework

Researchers from the University of Virginia suggest the following questions for consideration in selecting an 

observation framework:

Tier 1: High-Priority Questions

— What questions about classrooms does my organization want answered? Is the scope of this tool aligned with the 

questions about classrooms and teachers’ practices that we want to address? 

— Are the observation and scoring protocols standardized and clear? 

— Has this tool been shown to be reliable across observers and over time? 

— Are the outputs (scores) from this observation protocol proven to be related to outcomes of interest in our 

population (i.e., growth in students’ academic skills, students’ prosocial behaviors, teacher retention, students’ 

reports of feelings of belongingness, etc.)? 

Tier 2: Additional Considerations

— Does the system include complementary sources of information (such as student surveys, etc.) that could be used 

to obtain a more complete portrait of the classroom? 

— Does the observation include guidelines and support for using findings for professional development purposes? 

— Is the time required for observation feasible for your organization? 

Source: “A Practitioner’s Guide to Conducting Classroom Observations: What the Research Tells Us About Choosing 

and Using Observational Systems,” by Megan W. Stuhlman, Bridget K. Hamre, Jason T. Downer, and Robert 

C. Pianta, University of Virginia, 2010.

Using Effective Teachers as Observers 

Some districts used effective teachers as observers in their evaluation systems. There are both benefits 

and drawbacks to this approach. Effective teachers may have credibility among teachers and 

administrators. However, a drawback is taking effective teachers out of the classroom. Also, conducting 

observations may involve skills (e.g., providing constructive feedback to adults, communicating with 

adults) that are not the same as those that make a teacher effective in the classroom. Finally, teachers 

who are effective in one subject or at one grade level may not have the content background to effectively 

observe teachers of other subjects or grades. Especially at the high school level, teachers being observed 

may be concerned that observers do not have sufficient understanding of the academic material to be 

able to accurately interpret lessons and provide valid and useful feedback. To respond to these concerns, 

study districts took the following actions: 

• Created time-limited positions, so that effective teachers were not permanently out of the 

classroom. Hillsborough County debated this issue and concluded that using effective teachers 

as observers or evaluators benefits many teachers and students. However, the district limited 

observers’ positions to two-year appointments, so that teachers could return to the classroom 

after serving as observers. 
Identifying Effective Teachers 15

• Developed selection criteria for observers that went beyond effectiveness. Several Eagle 

County principals and district officials spoke about how the process of hiring master and mentor 

teachers who serve as observers had become more selective over time as they focused not just 

on effectiveness but also on a teacher’s ability to work with other teachers.

• Used school-based observers. In Eagle County and in Columbus TAP schools, master and 

mentor teachers were school based, and one principal spoke about the benefits and challenges of 

having a former teacher in the school become a master or mentor teacher. On the one hand, 

such a teacher could have a strong reputation as an effective teacher and good personal 

relationships with teachers; on the other hand, these same relationships could result in concerns 

about objectivity. Having school-based observers could also reduce the possibility of having 

observers with specialized content knowledge, because schools could only select a few observers 

and might have many different types of courses taught by different teachers. 

• Assigned observers by subject area. As of summer 2010, Hillsborough planned to try to 

assign observers by subject area, matching as many as possible. Teacher observers would be 

assigned to a number of schools and observe teachers in a particular grade level or content area 

across schools. 

Ensuring Consistency in Scoring 

A major issue in the implementation of classroom observation frameworks is ensuring consistency and 

reliability in scoring, particularly for rubrics with many indicators. To address this challenge, the 

five districts took the following actions (or were considering these actions): 

• Provided training materials, such as handbooks or other reference materials, to 

observers. Eagle County’s professional development department created a handbook to go 

along with its observation rubric. This handbook included detailed information about the rubric 

indicators and examples. The district also provided online training modules for schools.

• Required ongoing training and certification. Eagle County reported that it required anyone 

who would conduct observations to be recertified each year. Those certified in the past attended 

a two-day refresher training and took another exam. New observers had a two-day training 

before the school year started, and then made live observations with a trained evaluator and 

returned six to eight weeks after school started for two final days of training and a final exam. 

Anyone who did not pass the exam would go through additional training.

• Monitored scores within and across schools. Two districts analyzed scores across schools to 

look for potential inconsistencies or patterns. One Eagle County principal mentioned that, in 

past years, the district analyzed building rubric scores compared with district averages, to look 

for outliers. The same principal had principals spend time in one another’s buildings to create 

shared understanding of scores. In Columbus, one principal scheduled initial observations with 

multiple observers as often as possible, so that they could calibrate their scoring within the 

school. 
Identifying Effective Teachers 16

Scheduling Observations 

Once an observation framework has been selected and observers selected and trained, these observers 

must find time to complete all the required observations. Particularly in large schools, simply scheduling 

and completing observations can be difficult. Study districts reported a number of strategies to address 

these issues: 

• Developed observation systems that involved a feasible number of observations. In 

Hamilton County’s new observation system, which was being piloted at the time of data 

collection in summer 2010, the plan was to have administrators observe each teacher, using 10 

mini-observations of approximately 5 to 10 minutes each. The observations were not to include 

preconferencing. Feedback was to be provided to the teacher in a short conversation, which was 

to last approximately 5 to 10 minutes. While this system required many observations for each 

teacher, each observation was to be very short in duration. Other study districts required fewer 

observations, with each observation lasting longer. For instance, Eagle County required three 

formal class-length observations, and more were encouraged. Researchers at the University of 

Virginia, who support the implementation of the Classroom Assessment Scoring System 

(CLASS) observation instrument, have determined that four observations are needed to ensure 

reliability in their system (Pianta, La Paro and Hamre 2007). 

• Provided specific supports for observers, to help them make time for the observation 

work. Hamilton County provided guidance to help administrators plan their observation 

responsibilities (see Exhibit 7). Because the district required many short observations of each 

teacher, administrators needed to complete approximately two teacher observations per day, 

throughout the year. To provide more time for principals to conduct observations and work 

with teachers, Houston was working on having the district central office take on some 

administrative tasks formerly provided at the school level (e.g., processing payments for 

professional development). 

• Established guidelines for the timing of observations. Eagle County’s system, which 

included both teacher and principal observers, purposely scheduled principal observations for 

the second half of the school year. This schedule allowed teachers to get feedback from peers 

and work on their practice before having a principal, whose scoring counted more heavily in a 

teacher’s summative evaluation, conduct his or her observation. In addition, observations did 

not begin until several weeks into the school year. 
Identifying Effective Teachers 17

Exhibit 7 

Hamilton County’s Guidance on Conducting Observations

Hamilton County’s communication to principals included general guidance on ways to schedule observations, as well 

as specific scenarios for different numbers of teachers and administrators, so that they could be completed as 

required. 

Source: Hamilton County Teacher Evaluation Pilot, Draft presentation, 2010–11.

Updating the Observation Framework 

As classroom observation frameworks are implemented widely, teachers and principals may discover 

problems or areas that should be modified. Each of the case study districts reported at least one of the 

following strategies for identifying ways that the observation framework might need to be updated or 

revised: 

• Collected feedback regularly from all stakeholders. Houston district officials recently 

conducted a survey of all principals and teachers to gather feedback on the observation tools in 

use in the district. Thirty-eight percent of high school teachers and 48 percent of elementary 

school teachers believed that the framework allowed appraisers to assess their instructional 

performance accurately (see Exhibit 8). About two-thirds of teachers reported that their 

confidence in the performance appraisal system would increase if it addressed factors such as 

clear timelines for the process, clear standards for observation and appraisal, ongoing surveys to 

seek input on the system, professional development based on the system, and training and 

evaluation of appraisers. At the time of data collection, Houston was considering adopting a new 

observation framework.

• Instituted regular updates. One study district, Eagle County, built a requirement into its 

policies that a committee review the district’s framework and its implementation every two years 

(see the text box, pg. 20, for the language of this requirement in Eagle County policy). A 

committee of principals, teachers, and union and district staff was to conduct the reviews. 
Identifying Effective Teachers 18

Exhibit 8 

Results of Houston’s Survey on the Classroom Observation Framework

A 2010 study by the New Teacher Project, in partnership with the Houston Independent School District, found little confidence in the statewide rubric used in 

Houston—the Professional Development and Appraisal System (PDAS) or a Modified PDAS (MPDAS), used with senior teachers.

Source: Presentation to the Houston Independent School District Board of Education, “Core Initiative Update: Effective Teacher in Every Classroom,” June 3, 

2010.

continued next page 
Identifying Effective Teachers 19

Exhibit 8 (continued)

Results of Houston’s Survey on the Classroom Observation Framework

Source: Presentation to the Houston Independent School District Board of Education, “Core Initiative Update: Effective Teacher in Every Classroom,” June 3, 

2010.
Identifying Effective Teachers 20

Eagle County’s Policy on Reviewing the Observation Framework and Its Use

Eagle County had a written policy that it would review its evaluation system every two years and make changes if 

necessary.

The Board of Education shall appoint an advisory school Personnel Performance Evaluation Council, which shall 

have the purpose of assisting in the district’s compliance with state law regarding licensed staff evaluations. This 

council will not evaluate the performance of employees, but rather will review the evaluation system a minimum of 

once every two years to determine if, in its judgment, the system meets the intent of state legislation and to provide 

appropriate recommendations to the Board and Superintendent as to any needed improvements. The licensed staff 

members will be provided the opportunity to submit recommendations regarding the refinement or improvement of the 

evaluation system.

Source: Eagle County Schools administrative policies, 2010.

Summary 

The five study districts developed measures of teacher effectiveness that drew on data on student 

achievement growth and classroom practice. While their reasons for developing these measures varied, in 

four of the five districts compensation reform was a key driver. Specifically, Columbus, Eagle County, 

Hillsborough and Houston all developed or improved their measures of teacher effectiveness in order to 

offer performance incentives for high-performing teachers districtwide or in select schools. In summer 

2010, Hamilton County, Columbus, and Hillsborough were planning to continue their work to develop 

and refine measures of teacher effectiveness as part of their states’ RTT grant efforts. All the districts’ 

efforts to date suggest some inherent challenges that other districts and states may need to address as 

they consider developing measures of teacher effectiveness: 

• Communication and engagement. Interviewees noted that efforts were needed to ensure that 

teachers and principals were appropriately informed about new measures of teacher 

effectiveness. Whether with respect to the design of a new measure or to its implementation, 

study districts responded to the need to allow for input and to provide clear and comprehensive 

information to teachers, principals and other stakeholders in a variety of ways. Some provided 

opportunities for teachers or others to help develop or refine measures like classroom 

observation frameworks. Others focused on providing information about their measures. 

One provided a systematic, ongoing process by mandating regular stakeholder input for updates 

to the teacher evaluation process. 

• Limited scope of student achievement growth measures. A key challenge for study districts 

was that data on student achievement growth data were available only for teachers in grades and 

subjects in which state tests were implemented. Study districts used different approaches to 

supplement these measures, including creating or repurposing other district assessments, or 

applying schoolwide or other aggregate measures to teachers in untested grades or subjects. 

• Implementation of classroom observation frameworks. Interviewees noted challenges in 

implementing classroom observation systems that were both rigorous and feasible. For example, 

interviewees noted challenges in finding the right observers, and in ensuring and maintaining 

reliability. Study districts took a variety of approaches in response to these challenges, including 

providing scheduling support, providing ongoing training, and working with teachers and 

principals to identify the right observers for each district’s context. 
Using Information About Teachers’ 21

Effectiveness in Human Resource Policies

III. Using Information About Teachers’ Effectiveness 

in Human Resource Policies 

Human resource policies, such as teacher hiring, professional development, retention and dismissal, are 

important levers in efforts to ensure that all students have effective teachers (Odden and Kelly 2008). 

The absence of meaningful information on teacher effectiveness can hamper the development of human 

resource policies. For example, many teacher professional development activities are typically designed 

to be “one size fits all,” rather than tailored to teachers’ individual strengths and weaknesses (Weisberg 

and others 2009). 

With information about teachers’ effectiveness available, districts have opportunities to use that 

information to design human resource policies. This chapter illustrates several approaches to using such 

information, including those that treat teachers differentially on the basis of effectiveness information, as 

well as those that apply to districtwide practices, such as hiring. Chapter IV addresses approaches to 

using effectiveness information to more equitably distribute effective teachers among schools. The five 

study districts used specific strategies in three areas to improve teacher effectiveness: 

• Preparing and hiring effective new teachers 

• Improving the effectiveness of current teachers (e.g., professional development) 

• Retaining effective teachers and removing ineffective teachers 

These strategies are discussed in the sections that follow. A summary of the types of policies in use in 

study districts is provided at the end of this chapter. 

Preparing and Hiring Effective New Teachers

One way to improve teacher effectiveness is to ensure that new teachers are well prepared and that 

hiring practices include appropriate selection criteria. Some of the study districts used information on 

teacher effectiveness to inform efforts to hire new teachers, specifically by developing the supply of 

applicants and by refining the criteria for selecting applicants. This section addresses these two practices 

in turn: 

• Providing input on the design of teacher preparation programs 

• Using effectiveness information to refine new-hire selection criteria 

Providing Input on the Design of Teacher Preparation Programs 

One way to improve the quality of new hires is to provide input into the design of teacher preparation 

programs, on the basis of the performance of recent hires. As part of a federal Teacher Quality 

Partnership grant, Columbus City Schools and a higher education partner, the Ohio State University 

(OSU), established a strong working relationship in which the district was represented in conversations 

regarding changes to OSU’s preparation programs. Representatives from the district’s mentoring 

program for first-year teachers were members of the committee that discussed changes to OSU’s 

program. The district used input from the mentoring program representatives on areas in which current 

first-year teachers struggled to guide changes to preparation programs. 
Using Information About Teachers’ 22

Effectiveness in Human Resource Policies

Another feature of the partnership between Columbus and OSU was a model teacher residency 

program. The teacher residency program provided each preservice student teacher with a current 

Columbus district teacher to serve as a mentor, and with improved field experiences so that student 

teachers would be better prepared for urban teaching. 

Using Effectiveness Information to Refine the Selection Criteria for Hiring 

One factor that may affect the quality of new hires is a district’s selection criteria. To date, there is little 

research to support the use of specific tools or strategies to identify applicants who may be effective 

teachers. Research that does exist suggests modest relationships between selection instruments, such as 

attitudinal surveys, and teacher effectiveness (e.g., Rockoff and others 2008; Novotny 2009; Metzger and 

Wu 2008). 

In the absence of more research, one way districts can refine their selection criteria is by analyzing the 

relationship between first-year teachers’ effectiveness and their prior scores on hiring selection 

instruments. Houston developed a plan to measure the percentage of screened applicants who were 

found to be effective, as shown in the following text box. An annual report will provide “detailed 

analysis and proposed action related to the key metrics for recruitment and selection.” 

Houston’s selection process had a four-stage selection pipeline that screened applicants on the basis of 

paper qualifications, a 5-minute sample of teaching, a 30-minute structured interview and then a final 

review by committee. The 30-minute interview used the Haberman Educational Foundation’s (HEF’s) 

Star Teacher Pre-Screener, which assesses applicant values and attitudes believed to be associated with 

successful teaching of high-need students. 
Using Information About Teachers’ 23

Effectiveness in Human Resource Policies

Sample Houston Metrics to Monitor Hiring Efforts

Houston developed a strategic plan to develop an effective teaching force, which included goals and metrics to 

monitor progress. This excerpt from that plan shows the overall goal, some key metrics and reporting requirements 

related to hiring. Note that EVAAS stands for “Education Value-Added Assessment System,” a commercially 

available system for measuring teacher effectiveness, using student growth, and that the “screener” is the Haberman 

Education Foundation’s Star Teacher Pre-Screener.

Attract and hire top talent through proactive search strategies and rigorous selection criteria for every job position.

KEY METRICS

Recruitment and Selection

— Number of new teacher hires

— % of teacher applicants rated in the acceptable range on screener

— % of principal applicants rated in the acceptable range on screener

— % of Human Resources–screened teachers rated in top 10% of EVAAS value-added data

— % of Human Resources–screened teachers rated in top two quartiles of EVAAS value-added data

— Teacher yield %: # of offers made to teachers versus # of teachers that accepted offer

Report: The administration will report to the Board of Education on the outcomes of the recruitment season, including 

comprehensive analysis of the efficacy of recruitment efforts. The report will include detailed analysis and proposed 

action related to the key metrics for recruitment and selection.

Source: Houston Independent School District Board of Education Board Monitoring System 2009–10. 

http://www.houstonisd.org/HISDConnectEnglish/Home/Board%20of%20Education/Board%20Images/Ne

w%20BMS%20Document-Final%20Rev2.pdf.

Improving the Effectiveness of Current Teachers

After teachers have been hired, districts have a variety of opportunities to support teachers’ growth and 

improve their effectiveness, and most districts offer a range of teacher professional development 

opportunities. 

This section describes three approaches used in the study districts to increase the effectiveness of current 

teachers: 

• Using observations to provide teachers with frequent feedback 

• Using effectiveness information to create professional development plans for individual teachers 

• Using effectiveness information to plan district professional development

Using Observations to Provide Teachers With Frequent Feedback 

The study districts with new or revised observation systems used feedback (or planned to use feedback) 

from the observations as a key part of professional development for teachers. For example, in Hamilton 

County, teachers were to receive brief feedback within 48 hours after each of the 10 mini-observations 

occurring during the course of the year (see Exhibit 9). 

Such feedback occurred multiple times, ranging from 3 to 10 times, during the course of the school year 

in the study districts, providing teachers with “job embedded” opportunities to improve their practice. In 

research on the link between professional development and teacher practice, this feature correlates with 
Using Information About Teachers’ 24

Effectiveness in Human Resource Policies

changes in practice (e.g., Garet and others 2001). In fact, in Eagle County, where a system of frequent 

feedback had been in place since 2001, district officials reported survey results showing that 80 percent 

of teachers said that they were better teachers because of the rubric and the associated feedback. 

Frequent feedback can be used as part of a general improvement strategy for all teachers but can also 

focus on teachers who may have been identified as in need of assistance, using other effectiveness 

information. Columbus’ intensive Peer Assistance and Review (PAR) program supported teachers who 

were struggling by providing a dedicated mentor, called a consulting teacher, to observe lessons and 

provide immediate feedback. Teachers could potentially be observed up to 40 times, according to the 

district. 

Exhibit 9 

Hamilton County’s Proposed Cycle of Feedback to Improve Teacher Effectiveness

Hamilton County’s evaluation measure would allow for a year-long process of observation, feedback and professional 

development guidance and goal setting.

Source: Hamilton County Teacher Evaluation Pilot, draft presentation, 2010–11.

Using Effectiveness Information to Create Professional Development Plans 

for Individual Teachers 

Effectiveness information can also be used to develop long-term professional development plans for 

individual teachers. The system in Hamilton County (described in Exhibit 9) would take into account 

both student growth and observation data to guide plans for individual teachers. Houston’s staffing 

review process (see the following text box) was intended to help principals use data on teacher 

effectiveness to develop individualized plans for teacher support and professional development. The 

process involved district-level professional development staff in conversations with principals, to assist 

them in creating these plans, using available data on teacher effectiveness. 
Using Information About Teachers’ 25

Effectiveness in Human Resource Policies

Description of Houston’s Staffing Review Process

In spring 2010, principals in Houston began to use teacher performance information to develop individual plans for 

teacher recognition and professional development. Through the staffing review process, principals placed teachers in 

one of four performance categories and then met with district human resources and professional development 

representatives to create plans for each teacher. For example, for veteran teachers in the lower two categories, 

principals developed remediation plans, including strategies such as team teaching, mentoring and professional 

development (see Appendix A–8 for the decision-making framework used by principals). An excerpt from a district 

memo describing the staffing review process is shown here: 

The purpose of this process is to ensure that teachers receive recognition for their excellent work, targeted 

professional development to meet their needs and important feedback regarding their performance. Principals should 

implement this staff review process for all teachers on their campus.

Step One: Categorize/group your teachers.

Principals should group all of his/her teachers in one of the following four categories. Decisions should be based on 

effectiveness by using all performance data, including but not limited to, principal walkthroughs, classroom 

observations, review of student work products, formative student assessment data and value-added scores, 

when available.

a. Highly Effective—teachers who consistently achieve student academic growth and/or student academic 

outcomes that are better than expected, based on value-added or other student performance data

b. Proficient—teachers who achieve expected levels of student academic growth and/or expected student 

academic outcomes, based on value-added or other student performance data

c. Developing—teachers who show potential for improvement but who achieve student academic growth and/or 

student academic outcomes that are less than expected and who may need additional supports to improve their 

instructional practices, based on value-added data or other student performance data

d. Low-Performing—teachers who consistently achieve low levels of student academic growth and/or student 

academic outcomes that are significantly less than expected, based on value-added or other student performance 

data

Step Two: Attend a staff review meeting to be scheduled by Academic Services.

Academic Services will be scheduling sessions with principals to conduct the staff review process. In preparation for 

these meetings, principals should be prepared with information on your staff and have completed Step 1 of this 

process. In addition, you should be considering career development strategies for teachers in each group.

Source: Houston Independent School District Memo from superintendent on staffing review process, March 5, 2010.

Using Effectiveness Information to Plan District Professional Development 

In addition to planning professional development for individuals, some study districts also used 

effectiveness information to inform professional development practices more generally. In Houston, a 

group of teachers who were rated “highly effective” on the basis of student achievement growth 

developed a set of recommendations for principals and teachers. As of summer 2010, Eagle County 

planned to collect instructional materials from teachers identified as “excellent” through Eagle County’s 

“Excellence in Teaching” process (modeled after National Board Certification process) as models. 

Aggregate analysis of effectiveness information pooled across the district may also inform professional 

development practices. In discussions about teacher professional development in Hamilton County, the 

district decided that data from each of the brief observations that principals conducted would be 

electronically saved in the district’s data system, and information about the needs of teachers would be 

shared with content specialists in the district, who could then plan for professional development. 
Using Information About Teachers’ 26

Effectiveness in Human Resource Policies

Retaining Effective Teachers and Removing Ineffective Teachers

Once a system has been designed to identify effective teachers, districts need strategies to retain them. 

These strategies can include performance-based compensation or bonuses and leadership opportunities. 

Conversely, once teachers are identified as ineffective, and if strategies to help them improve are 

unsuccessful, districts may wish to consider dismissal. As part of its strategic plan, Houston developed 

key metrics to monitor its efforts to retain and remove teachers on the basis of their effectiveness, as 

shown in the text box. 

Houston’s Sample Metrics to Monitor Progress in Using Effectiveness Information for 

Retention and Termination Decisions

Houston developed a number of key metrics that draw on teacher effectiveness information to track the district’s

progress in retention and termination.

Critical Outcome: Human Capital Assessment and Retention

Provide every employee ongoing and annual feedback that creates opportunities for recognizing excellence and 

developing skills and leadership, and retains high-performing staff in every job position.

KEY METRICS

Human Capital Assessment and Retention

— % of probationary teachers who receive a term contract

— % teachers in the top 10% of EVAAS value-added data who are retained

— % teachers in the top two quartiles of EVAAS value-added data who are retained

— % principals in the top 10% of EVAAS value-added data who are retained

— % principals in the top two quartiles of EVAAS value-added data who are retained

— % of teachers in the top 10% of EVAAS value-added data who are terminated or who retire

— % of teachers in the bottom 10% of EVAAS value-added data who are terminated or who retire

— % of teachers in the bottom two quartiles of EVAAS value-added data who are terminated or who retire 

— % of principals in the top 10% of EVAAS value-added data who are terminated or who retire

— % of principals in the bottom 10% of EVAAS value-added data who are terminated or who retire

— % of principals in the bottom two quartiles of EVAAS value-added data who are terminated or who retire

— % of employees on a performance improvement plan by school or department

— % of regressive value-added performers on performance improvement plan

— % of regressive value-added performers on improvement plans who attain positive value-added scores following 

remediation outlined in the plan

Report: The administration will report to the Board of Education on the outcomes of personnel assessment metrics.

Notes: EVAAS is the district’s value-added model, the Educator Value-Added Assessment System.

Regressive value-added refers to teachers with scores of less than 0, using the district’s measure.

Source: Houston’s Board Monitoring System, updated February 2010.
Using Information About Teachers’ 27

Effectiveness in Human Resource Policies

The availability of data on teacher effectiveness creates new opportunities for districts to design policies 

to reward and retain effective teachers and to revisit policies related to teacher tenure and dismissal. This 

section describes four approaches used in the study districts: 

• Using effectiveness information in teacher compensation 

• Using effectiveness information to create new career paths

• Using effectiveness information in tenure review and dismissal 

• Using effectiveness information in transfer eligibility and transfer hiring decisions 

Using Effectiveness Information in Teacher Compensation 

With the exception of Hamilton County, all the study districts were engaged in a performance pay 

initiative of some kind. Using effectiveness information in compensation systems raises a variety of 

questions about the design and implementation of such systems, including whether or how much of an 

award should be based on individual performance, compared with collective performance; whether 

performance-based awards should replace or supplement existing salary or bonus structures; the 

amounts of awards needed to successfully retain teachers; and how to ensure that the system is 

transparent for stakeholders. 

For example, Eagle County’s performance-based compensation system drew on both individual and 

collective measures of effectiveness, and based payments on effectiveness information, as well as 

qualifications (see the following text box). This system was designed in response to concerns raised 

about the effects of individual incentives on teacher collaboration, as well as stakeholder beliefs about 

the utility of qualifications. Several principals mentioned that, especially in certain subjects, they looked 

for teachers with master’s degrees, perceiving this as a sign that the person is likely to be effective. 

Indeed, research suggests that, in certain subjects, qualifications related to content partially predict 

effectiveness (Wayne and Youngs 2003). 
Using Information About Teachers’ 28

Effectiveness in Human Resource Policies

Description of Eagle County’s Performance-Based Compensation System

Eagle County’s performance-based compensation system included both yearly incentives and permanent salary 

adjustments based on performance, and replaced a traditional salary schedule. Salary increments for teachers and 

other employees were based on effectiveness, as measured through observations each year. Thus, base salary 

increased each year by a percentage associated with the level of performance (in the example below, a 3 percent 

increase for a “high-performing” evaluation). Bonuses were awarded each year as a percentage of base salary using

buildingwide and districtwide student achievement growth on the Colorado State Assessment Program (CSAP), 

American College Testing (ACT) and the Northwest Evaluation Association (NWEA) MAP assessment (in the 

example below, a 2.5 percent increase based on these scores). As described in the section on measures of teacher 

effectiveness in chapter II, study districts employed a variety of methods to try to improve communication about 

measures and their uses for performance pay (see Appendix A–1 for a sample of the letter sent to teachers in Eagle 

County describing the way the performance-based compensation is calculated). The example shown here is an 

abbreviated version presented on the Eagle County website to describe the system to prospective teachers.

Putting It Together: An Example

Teacher Joe makes $45,000 as his contract salary. He also adds $3,000 for an advanced degree and $2,500 for 

working in a “high-poverty” school as “à la carte” salary additions. These additions are NOT included as his contract 

salary. They will increase as district needs and market conditions require them to; so we use the $45,000 for 

determining bonus and the next year’s raise. Joe will earn

— $1,125 bonus from a 2.5% increase based on his assessment index for the district and his school (CSAP, NWEA 

and ACT)

— $900 salary increase for the Negotiated/Inflationary Component, set for the next year at 2%

— $1,350 salary increase for Joe’s “High-Performing” evaluation which was 3%

New annual salary: $47,250

Source: http://www.eagleschools.net.

Several principals in study districts mentioned that they did not think their district’s performance 

bonuses were large enough to have a big impact on teacher retention or motivation. Odden and Wallace 

(2007) recommend a range of 4 percent to 8 percent of salary, consistent with a review from private 

sector studies that found a range of 3.5 percent to 11 percent. Bonuses in study districts were in the 

$1,500 to $3,000 range (which with a median teacher salary of about $51,000 across districts would 

represent a range of about 2 to 6 percent), although Eagle County had a temporary “Excellence in 

Teaching” award program, which offered a one-time $10,000 bonus to teachers who applied for the 

program. Research studies conducted to date show mixed results for performance pay on teacher 

retention. A few rigorous studies (e.g., Springer and others 2008; Wiley, Spindler and Subert 2010) find 

increased retention of effective teachers with the use of performance incentives, while others do not 

(e.g., Glazerman and McKie 2010). 

Using Effectiveness Information to Create New Career Paths 

A study by the Center for Teaching Quality found that teachers who held leadership positions were 

significantly more likely to plan to stay in the profession over the next three years (Berry, Daughtrey and

Wieder 2010). Most study districts offered teachers opportunities to be coaches or mentors. For 

example, master and mentor teachers in Eagle County and Columbus conducted observations and 

provided individualized coaching to teachers. These teachers were selected in part on the basis of their 

effectiveness. Columbus’ TAP program specified that applicants must submit a video of their classroom 

teaching, which would be scored using the TAP observational rubric, “evidence of student 

achievement,” and references designed to gather information about the candidate’s leadership skills, 

among other things. 
Using Information About Teachers’ 29

Effectiveness in Human Resource Policies

Using Effectiveness Information in Tenure Review and Dismissal

In addition to better identifying ineffective teachers, some study districts used information about 

effectiveness in their tenure reviews and dismissals. For example, Houston updated its policy on annual 

renewal of the contracts of untenured teachers. A consistent lack of student progress, as measured by the 

value-added formula, was added to the list of reasons for termination of an untenured teacher. The 

revised policy stated, “In accordance with its data-driven culture, the District is including as a reason for 

nonrenewal insufficient student academic growth as reflected by value-added data.” Both Houston and 

Hamilton County district officials discussed changes they had recently made to the period of service 

preceding the district decision to grant tenure, to allow increased attention to teacher effectiveness in 

tenure decisions. 

Two study districts also discussed how they supported principals in teacher dismissal processes. In its 

staffing review process, Houston provided a decision-making document to principals, to assist them in 

considering actions to take for each teacher, including proposing termination (see Exhibit 10 for an 

excerpt from the policy focused on tenured or “term” teachers, and Appendix A–8 for the full 

document). In addition, Houston offered three-day training for principals on how and when to move 

toward termination. According to the district, the staffing review process resulted in an increased 

number of teachers’ not being granted tenure after three years (from 7 in 2009 to 154 in 2010), an 

increased number of teacher contract terminations (from 45 in 2009 to 100 in 2010) and an increased 

number of resignations in lieu of termination (from 24 in 2009 to 61 in 2010). 

Exhibit 10 

Houston’s Staff Management Decision-making Framework Excerpt

The following excerpt shows how the Houston school district is providing guidance to its principals on how to use 

effectiveness information for termination decisions about teachers (in this case, for “term” or tenured teachers).

Contract Type Management Decision Framework Options

Term 1. Contract should continue if:

— Teacher performance meets or is above expected levels

• But if a teacher has a Running Cumulative Average that is regressive (i.e., less than 0) 

in any one subject, a growth plan is required (i.e., team-teaching, mentoring, 

professional development)

— There is no other compelling reason to terminate the contract

2. Termination should be proposed if:

— Unsatisfactory performance on one or more of the 34 criteria as part of the district’s 

evaluation system for teachers.

Source: Houston Independent School District Memo from superintendent on staffing review process, March 5, 2010.

Eagle County encouraged its principals to make additional efforts to gather and review data on the 

effectiveness of untenured teachers, who could be dismissed through nonrenewal. On average, over the 

last few years, the annual rate of nonrenewal for probationary teachers was 20 percent. By comparison, 

the rate of nonrenewal in six case study districts in a recent study was less than 1 percent in five districts 

and 3 percent in the sixth district (Denver Public Schools) (Weisberg and others 2009). 
Using Information About Teachers’ 30

Effectiveness in Human Resource Policies

Using Effectiveness Information in Transfer Eligibility and Transfer Hiring 

Decisions 

Some of the study districts used effectiveness information in the rules governing transfers from one 

school to another. Such rules vary by district and usually take into account teacher preferences. These 

rules may be important places to insert effectiveness criteria, if teachers rated as unsatisfactory by their 

principals transfer to a school where the principal may be more lenient, or principals who are unwilling 

or unable to dismiss ineffective teachers may encourage them to transfer. 

Houston tightened its transfer eligibility criteria so that teachers with unsatisfactory performance were no 

longer eligible to transfer (see following text box). Hillsborough County adopted a different approach, 

focusing on the principals in schools that are considering which prospective transfer teachers to receive. 

Hillsborough provided those principals access to a “teacher summary sheet” about prospective transfer 

teachers. The sheet provided information on the teacher’s value-added and teaching history (see 

Appendix A–9). 

Description of Houston’s Policy Establishing Eligibility Criteria for Teachers To Transfer

The Houston School Board placed restrictions on transfers to make sure that ineffective teachers didn’t skip from 

school to school, never addressing their deficiencies.

In order to be eligible for transfer, teachers must meet the following criteria:

— Hold a valid Texas teacher certificate in the requested subject-area vacancy.

— Meet “Highly Qualified” standards of NCLB for the position.

— Must not be entering third- or fourth-year probationary status.

— Must not be on a growth plan.

Source: http://www.houstonisd.org/HISDConnectDS/v/index.jsp?vgnextoid=e2c2d5b853

548210VgnVCM10000028147fa6RCRD&vgnextchannel=9339e02e91b23110VgnVCM10000028147fa6RCRD.

Summary 

Although, across the districts, there were policies or programs to address the entire continuum of teacher 

employment and development, from hiring to dismissal, study districts often focused on: 

• Compensation initiatives. All study districts except Hamilton County had in place performance 

pay initiatives that made use of differing teacher effectiveness measures. As noted earlier, these 

programs often served as the impetus to design or redesign teacher effectiveness measures.

• Feedback through observation. Where study districts had new or revised observation 

frameworks in place, these frameworks appeared to serve as linchpins in district efforts to 

provide useful feedback to teachers and aid them in improving their instructional practices. The 

format of the observations and feedback varied from district to district. 

• Career paths. Several interviewees spoke about their efforts to use effectiveness information to 

identify teachers for leadership positions, as well as to release those determined to be ineffective. 

Districts used different types of information for these purposes. For example, Houston 

described value-added data as a potential source of evidence for dismissal and for granting 

leadership opportunities; Eagle County and Columbus interviewees spoke about the use of 

observation data for both purposes. 
Taking Targeted Action to Improve 31

Equitable Access to Effective Teachers

IV. Taking Targeted Action to Improve

Equitable Access to Effective Teachers 

Human resources policies that use effectiveness information may help ensure that all students have 

effective teachers, as discussed in chapter III. However, studies have documented that, on average, 

teachers migrate toward working in affluent, high-achieving schools (Hanushek, Kain and Rivkin 2004; 

Goldhaber 2008). Thus, another component of efforts to ensure that all students have effective teachers 

is a set of policies that specifically target high-need schools. 

Information about teachers’ effectiveness enables states and school districts to better address inequities 

in access to effective teachers in several ways. First, states and districts can monitor the distribution of 

teacher effectiveness, rather than qualifications, allowing better identification and targeting of actions to 

high-need schools that lack effective teachers. Second, in those targeted actions, states and districts can 

offer incentives to effective teachers, rather than to all teachers, and can focus attention on high-need 

schools. 

In the five study districts, targeted actions to improve equitable access to effective teachers fell into three 

categories: 

• Analyzing effectiveness data to determine whether and where action is needed 

• Attracting effective teachers to and retaining them in high-need schools 

• Offering principals in high-need schools additional opportunities to hire effective teachers 

Strategies in these areas are discussed in the following sections. A summary of study district efforts in 

these areas is provided at the end of this chapter. 

Analyzing Effectiveness Data to Determine Whether and Where Action 

Is Needed

There can be great variation across schools within a district in the quality and effectiveness of teachers 

(e.g., Clotfelter and others 2007; Goldhaber, Cramer and Choi 2007; Education Trust 2008; Sass and 

others 2010). To ensure that all students in a district have access to effective teachers, districts must 

analyze and monitor effectiveness information across schools. For example, Houston examined the 

proportion of effective teachers in high-and low-poverty schools, using their value-added measure (see 

Exhibit 11).3 Imazeki and Goe (2009) suggest that districts also take into account information on open 

positions or teacher turnover and teacher experience, using a combination of metrics to determine which 

schools may need targeted assistance. 

 

3 The state of Tennessee also analyzed teacher effectiveness across schools and found variation by subject across 

schools, indicating a need for programs targeting specific schools, and perhaps specific subject areas in those schools. 

For example, 28 percent of reading or language arts teachers were classified as highly effective in low-poverty schools, 

compared to 17 percent in high-poverty schools. Similar disparities did not exist in mathematics (Tennessee Department 

of Education 2007). Districts can also analyze effectiveness by minority enrollment or achievement status. In their Race 

to the Top applications, states described a number of ways in which they plan to analyze their data. For example, 

Tennessee reported plans to require its districts to examine retention data by effectiveness level, thus tracking the 

attrition of less effective teachers and leaders, for instance. Tennessee will also require its districts to chart the growth of 

teachers and leaders as they become more effective.
Taking Targeted Action to Improve 32

Equitable Access to Effective Teachers

Exhibit 11 

Houston’s Analysis of Distribution of Effective Teachers, by School Poverty Level

A study in Houston found, on the basis of value-added data, that there were fewer high-performing teachers at schools with higher percentages of students in 

poverty. In addition, high-performing teachers were less likely to transfer from a low-poverty school to a high-poverty school. 

Note: EVAAS is the Education Value-Added Assessment System, which Houston uses to calculate teachers’ contributions to student learning gains.

Source: Presentation to the Houston Independent School District Board of Education, “Core Initiative Update: Effective Teacher in Every Classroom,” June 3, 

2010.
Taking Targeted Action to Improve 33

Equitable Access to Effective Teachers

Attracting Effective Teachers to and Retaining Them in High-Need

Schools

Although districts may have the authority to reassign teachers as necessary, less coercive means may be 

preferable for bringing effective teachers to high-need schools. Strategies in the study districts and 

elsewhere included both financial incentives and other incentives, such as improvements in working 

conditions. 

Offering Financial Incentives to Attract and Retain Effective Teachers in 

High-Need Schools 

With the exception of Hamilton County, all study districts offered financial incentives to teachers to 

move to or stay in high-need schools.4 However, in only three of the districts (Columbus, Hillsborough, 

and Houston) were the incentives contingent on some measure of effectiveness. For example, National 

Board Certified Teachers (NBCTs) in Hillsborough County received an additional $4,500 for teaching at 

a Renaissance School (i.e., a school with 90 percent or more high-poverty students). They also received a 

$4,000 annual incentive provided to all NBCTs in the district. An additional bonus was given to teachers 

at those Renaissance Schools that reached achievement goals set for the school. That bonus was not 

awarded until the following year, to motivate teachers to remain at the school. Columbus was piloting a 

differentiated compensation program in which teachers with five years of experience and two years of 

demonstrated student achievement gains could seek transfer to a high-needs school and receive a 

$4,000 bonus each year for three years. Houston’s Effective Teacher Pipeline (described in the following 

text box), which was in its first year of implementation during the site visit, identified effective teachers 

by using value-added data and offered incentives to create clusters of effective teachers in targeted 

schools. 

4 One additional possible approach related to offering financial incentives is to allow retired teachers known to be 

effective to return to the classroom as part-time salaried teachers while continuing to draw retirement benefits. This type 

of strategy may need to be pursued at the state level or the district level, depending on local pension-funding 

arrangements.

Addressing Other Factors Related to Attracting and Retaining Teachers in 

High-Need Schools 

A key question is whether financial incentives alone are enough to attract or retain teachers, especially in 

the case of schools that may be perceived as difficult or failing. Teachers may be concerned with the 

working conditions in schools, including the support of leadership or peers, opportunities for 

professional development, or other factors. For example, Hillsborough conducted a study to determine 

whether additional compensation had any effect on the retention of high-quality teachers in high-poverty 

schools. In that study, 47 percent of teachers said they would not stay in their schools without the salary 

differential, and 43 percent said they would stay regardless of the extra pay. Among teachers who did 

leave, the main reason they cited was the leadership in the school (McLeod and Watson 2009). Hamilton 

County teachers reported that bonuses provided through an initial phase of the Benwood Initiative, a 

public–private partnership, were not the primary motivation for them to seek or retain employment at a 

struggling school. Instead, they reported being drawn by their personal commitment to traditionally 

disadvantaged students and by the opportunity to work with a visionary principal and a collegial 

professional community that fosters learning (Silva 2008). 

 
Taking Targeted Action to Improve 34

Equitable Access to Effective Teachers

States or districts may seek to address working conditions in tandem with financial incentives. For 

example, Houston’s Effective Teacher Pipeline project sought to address concerns about school working 

conditions by encouraging movement of many effective teachers to a small set of high-priority, 

high-need schools. Placing multiple effective teachers in high-need schools was intended to create a 

culture of supportive peers. In addition, the Effective Teacher Pipeline project provided extra 

professional development and leadership opportunities for teachers moving to those schools.5 

5 Charlotte-Mecklenburg’s Strategic Staffing Initiative goes further, moving principals and highly effective teachers to 

high-need schools as a team (Travers and Christiansen 2010). That is, effective principals are identified and then allowed 

to select effective members of their current school staff to take with them to a high-need school. Another example of 

efforts to address working conditions along with financial incentives is a statewide initiative in Delaware, slated to begin 

in fall 2012. The Delaware Fellows Program would offer $5,000 transfer bonuses to experienced, highly effective 

teachers and principals moving to high-poverty and high-minority schools. The program also would provide special 

summer training and monthly professional development, and in order to be eligible to receive staff, schools must show 

that they are implementing or continuing to implement schoolwide strategies to improve teaching and learning 

environments. It also may be that the geographic characteristics of a particular district or school make it extremely 

difficult to attract or retain effective teachers, even with financial or other incentives. In these instances, states or 

districts may need to consider alternatives that will allow effective teachers to serve high-need schools without actually 

moving to them. Examples of these types of programs could include distance-learning or coaching models, in which 

teachers identified as effective provide direct instruction to students by using technology. For example, Hawaii planned 

to implement a distance-learning program to provide geographically isolated, low-performing schools with access to 

effective teachers. Similarly, states or districts may be able to provide effective teachers to high-need schools as 

instructional coaches, working part-time with teachers and students.

Description of Houston’s Effective Teacher Pipeline Project

The goal of the Effective Teacher Pipeline project was to identify effective teachers and relocate them as clusters 

onto school campuses, where, in addition to teaching in their core subjects, they would serve as instructional leaders 

for other teachers. Effective teachers who agreed to transfer to the targeted low-performing schools received 

bonuses of $10,000 (funded by an outside foundation) each year for two years.

A teacher is eligible for the [Effective Teacher Pipeline] program for a particular subject (reading, language arts, 

mathematics, science and/or social studies) as follows:

Middle schools:

1. Teacher was in top 10% of all teachers in that subject within the district, ranked by value-added (Cumulative 

Gain Index [CGI]) for two of the last three years (06–07, 07–08, 08–09)

2. Teacher had positive value added (CGI greater than zero) for the year in which they were not ranked in the 

top 10%

3. Teacher taught in the subject during 08–09

4. Teacher has positive value-added scores (CGI greater than zero) in all other subjects taught, and

5. Teacher was placed in group 1, Highly Effective, or group 2, Proficient, during the 09–10 staff review process.

High schools:

6. Campus was in top quartile of value added (CGI) for that subject during both 07–08 and 08–09

7. Teacher was ascribed to that subject for the ASPIRE Award (teaching it for the majority of the day), and 

8. Teacher was placed in group 1, Highly Effective, during the 09–10 staff review process.

Items to Note:

— Middle school teachers can be designated in more than one subject. High school teachers cannot.

— Middle school teacher designation is based on three years of data. High school teacher designation is based on 

two years of data.

Value-added performance in the most recent school year (09–10) is not used, as this information will not become 

available until fall 2010.

Source: Houston Effective Teacher Pipeline Program description, June 2010.

 
Taking Targeted Action to Improve 35

Equitable Access to Effective Teachers

Offering Principals in High-Need Schools Additional Assistance With 

Hiring

In addition to attracting effective teachers to particular schools, another strategy for improving equitable 

access to effective teachers is to offer principals in high-need schools more opportunities to hire 

effective teachers directly. These opportunities could include early hiring options, or exemptions from 

policies that potentially reduce a principal’s ability to hire effective teachers. In many cases, these 

strategies can be undertaken within the context of existing collective bargaining agreements. 

One study district, Hillsborough, mentioned policies intended to offer such opportunities to principals. 

For example, during Hillsborough’s three-day job fair, the first day was limited to the high-poverty 

Renaissance Schools, to give those principals the ability to hire teachers first.6

6 In its Race to the Top application, the District of Columbia described a similar job-fair strategy called “Smart 

Targeting,” in which principals of high-need schools would meet prescreened applicants before principals of other 

schools would have a chance. The District of Columbia also planned to provide training to principals to help them 

conduct better interviews and find the best candidates for their particular positions.

In addition, according to district officials in Hillsborough County, principals in its Renaissance Schools 

were exempt from the requirement of selecting transferring teachers on the basis of seniority unless 

there were more teachers transferring than there were other positions available in the district. Instead, 

the principals could choose among those who wished to transfer or could hire teachers new to the 

district, in order to select the best candidates. Finally, when the Renaissance Schools in Hillsborough had 

open positions to fill, they were not subject to district hiring freezes. Instead, the principals could select 

new hires over transfers. 

Summary 

Three of the five study districts developed policies to use effectiveness information in efforts to ensure 

teacher effectiveness in high-need schools. One district, Houston, analyzed effectiveness information to 

determine where inequities within the district might be. The focus of most of these districts was 

monetary incentives to attract effective teachers to high-need schools and retain them. In one case, 

Houston’s Effective Teacher Pipeline project, such incentives were paired with efforts to address 

working conditions in high-need schools. 

 

References 37

References 

Berry, B., A. Daughtrey, and A. Wieder. 2010. Teacher leadership: Leading the way to effective teaching and 

learning. Hillsborough, NC: Center for Teaching Quality. 

http://effectiveteachers.org/images/uploads/research/CTQPolicyBriefOn_TEACHER_LEAD

ERSHIP__021810.pdf. 

Braun, H., N. Chudowsky and J. Koenig, eds., for National Research Council and National Academy of 

Education. 2010. Getting value out of value-added: Report of a workshop. Washington, DC: The 

National Academies Press. 

Carr, D. and Oxnam, G. 2009. “Addressing the Equitable Distribution of Teachers in Tennessee” 

(p. 100–109) in Goe, L., (ed.) America’s Opportunity: Teacher Effectiveness and Equity in K–12 

Classrooms. Washington, DC: National Comprehensive Center for Teacher Quality. 

Clotfelter, C. T., H. F. Ladd, J. L. Vigdor, and J. Wheeler. 2007. High-poverty schools and the 

distribution of teachers and principals. North Carolina Law Review 85 (5): 1345–79. 

Consortium for Policy Research in Education. 1998. Peer involvement in knowledge and skill assessment.

Handout from seminar Resigning Teacher Compensation, Madison, WI. 

Coopersmith, J. 2009. Characteristics of public, private, and Bureau of Indian Education elementary and secondary 

school teachers in the United States: Results from the 2007–08 Schools and Staffing Survey (NCES 

2009-324). Washington, DC: National Center for Education Statistics. 

Danielson, Charlotte, and Thomas L. McGreal. 2000. Teacher evaluation to enhance professional practice. 

Alexandria, VA: Association for Supervision and Curriculum Development. 

DeAngelis, K. J., Presley, J. B., and White, B. R. (2005). The distribution of teacher quality in Illinois.

Edwardsville, IL: Illinois Education Research Council. 

Education Trust. 2008. Core Problems: Out-of-field teaching persists in key academic courses, especially in America’s 

and high-poverty and high-minority schools. Washington, DC: Education Trust. 

Garet, M. S., A. C. Porter, L. Desimone, B. F. Birman, and K. S. Yoon. 2001. What makes professional 

development effective? Results from a national sample of teachers. American Educational Research 

Journal 38 (4): 915–45. 

Goe, L., and L. Holdheide, L. 2011. Measuring Teachers’ Contributions to Student Learning Growth for 

Nontested Grades and Subjects. TQ Research and Policy Brief. Washington, DC: National 

Comprehensive Center for Teacher Quality. Posted at: 

http://www.tqsource.org/publications/MeasuringTeachersContributions.pdf. 

Glazerman, S., and A. McKie. 2010. An evaluation of the Teacher Advancement Program (TAP) in Chicago: Year 

two impact report. Washington, DC: Mathematica Policy Research. 

Goldhaber, D. (2008). Addressing the teacher qualifications gap: Exploring the use and efficacy of incentives to reward 

teachers for tough assignments. Washington, DC: Center for American Progress. 

Goldhaber, D., L. Cramer, and H. Choi. 2007. A descriptive analysis of the distribution of NBPTS 

certified teachers in North Carolina. Economics of Education Review 26 (2): 160–72. 
References 38

Gordon, R., T. J. Kane, and D. O. Staiger. 2006. Identifying effective teachers using performance on the job

Hamilton Project Discussion Paper. Washington, DC: Brookings Institution. 

Hanushek, Eric A., John F. Kain, and Steven G. Rivkin. 2004. Why public schools lose teachers. Journal 

of Human Resources 39 (2): 326–54. 

Imazeki, J. 2007 (March). The distribution of teacher quality across schools. Paper prepared for the 

Teacher Supply and Demand Symposium for the National Center for Education Statistics, 

Washington, DC. 

Imazeki, J., and L. Goe. 2009. The distribution of highly qualified experienced teachers: Challenges and opportunities.

TQ Research and Policy Brief. Washington, DC: National Comprehensive Center for Teacher 

Quality. http://www.tqsource.org/publications/August2009Brief.pdf. 

Kane, T. J., J. Rockoff, and D. Staiger. 2008. What does certification tell us about teacher effectiveness? 

Evidence from New York City. Economics of Education Review 27: 615–31. 

Koedel, C., and J. R. Betts. 2009. Does student sorting invalidate value-added models of teacher effectiveness? An 

extended analysis of the Rothstein Critique. Washington, DC: National Bureau of Economic Research. 

Lashway, L. 2004. The mandate: To help low-performing schools. Teacher Librarian 31(5), 25–27. 

McLeod, J., and F. Watson. 2009. Evaluation of the Salary Differential Program. Tampa, FL: Hillsborough 

County Public Schools. 

Metzger, S. A., and M. J. Wu. 2008. Commercial teacher selection instruments: The validity of selecting 

teachers through beliefs, attitudes, and values. Review of Educational Research 78 (4): 921–40. 

Novotny, M. T. 2009. The relationship between TeacherInsightTM scores and professional development and appraisal 

system domain scores. Denton, TX: UNT Digital Library. 

http://digital.library.unt.edu/ark:/67531/metadc9835/. 

Odden, A., and J. Kelly. 2008. Strategic management of human capital in education. Madison, WI: University of 

Wisconsin, Consortium for Policy Research in Education. 

Odden, A. R., and M. Wallace. 2007. Rewarding teacher excellence: A teacher compensation handbook for state and 

local policymakers. Madison, WI: University of Wisconsin, Consortium for Policy Research in 

Education. 

Pianta, R., K. La Paro, and B. Hamre. 2007. Classroom assessment scoring system (observation protocol manual). 

Baltimore: Paul H. Brookes. 

Porter, A. C., P. Youngs, and A. Odden. 2001. Advances in teacher assessment and their uses. In 

Handbook of research on teaching, 4th ed., ed. V. Richardson, 259–97. Washington, DC: American 

Educational Research Association.

Rockoff, J. E., B. A. Jacob, T. J. Kane, and D. O. Staiger. 2008. Can you recognize an effective teacher when you 

recruit one? NBER Working Paper No. 14485. Cambridge, MA: National Bureau of Economic 

Research. http://www.nber.org/papers/w14485. 

Sass, T., J. Hannaway, Z. Xu, D. Figlio, and L. Feng. 2010, November. Value added of teachers in high-poverty 

schools and lower-poverty schools. Washington, DC: CALDER Working Paper No. 52. 
References 39

Silva, E. 2008. The Benwood Plan: A lesson in comprehensive teacher reform. Washington, DC: Education Sector. 

Springer, M., M. Podgursky, J. Lewis, M. Ehlert, T. Gronberg, S. Hamilton, D. Jansen, O. Lopez, 

B. Stecher, and L. Taylor. 2008. Texas Educator Excellence Grant (TEEG) program: Year 

Two evaluation report. Austin: Texas Education Agency. 

Stuhlman, M. W., B. K. Hamre, J. T. Downer, and R. C. Pianta. 2009. A practitioner’s guide to conducting 

classroom observations: What the research tells us about choosing and using observational systems. 

Charlottesville, VA: University of Virginia. 

Tennessee Department of Education. 2007. Tennessee’s Most Effective Teachers: Are they assigned to the schools 

that need them the most? Nashville, TN: Tennessee Department of Education. 

http://www.edvantia.org/publications/arccwebinar/docs/TNTeacherEffectiveness2007.pdf. 

Travers, J., and B. Christiansen. 2010. Strategic staffing for successful schools: Breaking the cycle of failure in 

Charlotte-Mecklenburg Schools. Watertown, MA: Education Resource Strategies. 

http://erstrategies.org/documents/pdf/CMS_case_study_APR16.pdf. 

Wayne, A. (June 13, 2002). Teacher inequality: New evidence on disparities in teachers’ academic skills. 

Education Policy Analysis Archives 10 (30). http://epaa.asu.edu/epaa/v10n30/(accessed on June 2, 

2011). 

Wayne, A. J., and P. Youngs. 2003. Teacher Characteristics and Student Achievement Gains: A Review. 

Review of Educational Research 73 (1): 89–122. 

Weisberg, D., S. Sexton, J. Mulhern, D. Keeling, J. Schunck, A. Palcisco, and K. Morgan. 2009. The widget 

effect. Washington, DC: The New Teacher Project. 

Weiss, E. M., and S. G. Weiss. 1998. New directions in teacher evaluation. ERIC Digest, ED 429052. Los 

Angeles: ERIC Clearinghouse for Community Colleges. 

Wiley, E. W., E. R. Spindler, and A. N. Subert. 2010. Denver ProComp: An outcomes evaluation of Denver’s 

alternative teacher compensation system: 2010 report. Boulder, CO: University of Colorado. 

http://static.dpsk12.org/gems/newprocomp/ProCompOutcomesEvaluationApril2010final.pdf. 

Appendix A 41

APPENDIX A 

 

Appendix A 43

Appendix A 

Appendix A Contents 

This appendix contains additional documents collected from districts illustrating their approaches to 

identifying effective teachers and using information about teacher effectiveness in human resource 

policies. The documents are organized according to their appearance in the report. An overview of each 

document is included here. 

Chapter 2: Identifying Effective Teachers

The appendixes relevant to Chapter 2 include documents related to teacher effectiveness calculations 

(including professional development materials describing these calculations), and observation 

frameworks. 

— Appendix A–1: Eagle County Award Computation—A letter to Eagle County teachers 

explaining how measures of effectiveness based on classroom observation and student 

achievement growth are weighted and combined to calculate salary increases.

— Appendix A–2: Columbus TAP Award Computation—A description of the eligibility 

requirements and weighting of measures of effectiveness for a Teacher Advancement Program 

(TAP) financial incentive.

— Appendix A–3: Columbus Performance Advancement System (PAS) Gains Calculation—

A detailed description of a process used in the district to identify teachers for financial awards 

based on student achievement growth. 

— Appendix A–4: Columbus Value-Added Professional Development Materials—Excerpted 

slides from a presentation explaining value-added computations in the district. 

— Appendix A–5: Hillsborough Current Observation Rubric—The classroom observation 

rubric used in Hillsborough through the 2009–2010 school year. 

— Appendix A–6: Hillsborough New Observation Rubric—The classroom observation rubric 

used in Hillsborough beginning in the 2010–2011 school year. 

— Appendix A–7: Eagle County Professional Practices Rubric—The classroom observation

rubric in use in Eagle County.

Chapter 3: Using Information about Teachers’ Effectiveness in Human 

Resource Policies 

The appendixes relevant to Chapter 3 include documents related to guidance for hiring, retention, 

termination, and remediation. 

— Appendix A–8: Houston Decision-Making Framework—Guidance provided to principals 

on how to use effectiveness information to consider staffing actions for teachers in their schools. 

— Appendix A–9: Hillsborough Teacher Summary Sheet—An example of the teacher 

summary sheet (containing effectiveness information) to aid principals in hiring decisions for 

transfer teachers.
Appendix A 44
Appendix A–1 45

APPENDIX A-1:

EAGLE COUNTY AWARD COMPUTATION

Appendix A–1 47
Appendix A–1 48
Appendix A–2 49

APPENDIX A-2:

COLUMBUS TAP AWARD COMPUTATION

Appendix A–2 51

Columbus City Schools TAP 

Performance Pay Award Calculations

BACKGROUND

Traditionally, the TAP payout structure allocates a minimum of $2,000 per teacher to establish 

the award fund (Note: many schools base their award fund on $2,500–$3,000 per teacher). The 

award fund is divided into award pools using a ratio of the career path level to the total number 

of teachers eligible for an award. The awards are divided into qualitative measures via the TAP 

Skills, Knowledge, and Responsibilities score and the quantitative student achievement scores 

calculated using value added analysis.1

1 Value-added analysis is a method for measuring the contribution of a teacher or school to gains in student achievement. The 

method uses individual student data linked from year to year, rather than cross-school or cohort averages. It applies statistical 

methods to (a) measure the academic gain or growth of each student over a period of time, and (b) attribute that gain or growth to 

the specific school and teacher(s) responsible for educating each student during that time.

Below are the minimum requirements on the Skills, Knowledge and Responsibilities to be 

eligible to earn the portion of the award pool set aside for that criterion: 

1. Master teachers must earn a SKR score of no less than “4.” 

2. Mentor teachers must earn a SKR score of no less than “3.5.” 

3. Career teachers must earn a SKR score of no less than “2.5.” 

Additionally, there are minimum requirements for both classroom and schoolwide achievement 

scores to be eligible to earn the portions of the award pool set aside for those criteria: 

1. All teacher types must earn a value-added score of no less than “3” on their individual, 

classroom achievement (a score of “3” means in the teacher’s students made one year’s 

expected growth on the state or comparable district assessment). 

2. The schoolwide achievement score must be a value-added score of no less than “3” (a 

score of “3” means the school, on average, made one year’s expected growth on the state 

or comparable district assessment). Schoolwide achievement scores of 3, 4, and 5 will 

earn 50 percent, 75 percent, and 100 percent of the schoolwide bonus, respectively. 

For example, if a career teacher received an SKR score of “3” and a classroom value-added score 

of “2,” they would only be eligible for the SKR portion of the award pool. 

Achievement Award Weights

For the career teacher with student achievement data, the award pool monies will be allocated as 

follows: 

• 50% Skills, Knowledge, and Responsibilities 

• 30% Classroom achievement gains 

• 20% School achievement gains 

For the career teacher without student achievement data, the award pool monies will be allocated 

as follows: 

• 50% Skills, Knowledge, and Responsibilities 

• 50% School achievement gains 

 
Appendix A–2 52

Columbus Public Schools Performance Awards for school year 2008–09 

Based on the results from last year, the school district of Columbus will award $251,550 in 

performance pay bonuses to the teachers in schools implementing the TAP System; $400,000 

was allocated. This differential is generated when schools do not achieve a value added score of 

‘5’ and allocated money is unused in a given fiscal year on the bonus awards. 

In the Columbus TAP project, only 26 of the 200 total teachers had individual value added 

scores (13%) for the 2008–09 school year. The middle schools in Columbus have chosen to use 

the 50-50 model mentioned above, even with teachers who teach tested grades or subjects.
Appendix A–3 53

APPENDIX A–3:

COLUMBUS PERFORMANCE ADVANCEMENT 

SYSTEM (PAS) GAINS CALCULATION

Appendix A–3 55

Data Analysis Procedures for PAS award in Columbus City Schools

The PAS award is given to those teachers whose average student achievement gain is equal to or greater 

than 3 NCE above the District gain from the same pre- and post-test assessments. Because dissimilar 

measures typically are used on the pretests and posttests, scores are converted to a standard scale 

called a z score. These z scores are then further converted to the Normal Curve Equivalent (NCE) for easy 

comparison. 

z Score 

The z score is the number of standard deviation units an individual student’s raw score is above or below 

the district mean. It has a one-to-one relationship with the standard deviation unit, one z-score unit = 

one standard deviation unit. On the z score scale the mean is set at zero. The z score is calculated by 

taking the raw score for a student, subtracting the district mean (average) of all student scores and 

dividing by the district standard deviation on the assessment. 

z score = (student raw score – district mean) /district standard deviation 

Normal Curve Equivalent 

The Normal Curve Equivalent is derived from the z score. On the NCE scale, the mean is set at 50% and 

each unit of standard deviation is represented by a 21.06% increase or decrease from the mean. The 

NCE has a range of 1 to 99%. The Normal Curve Equivalent can be thought of as the raw score 

percentage fitted to the normal distribution (bell curve). The NCE is calculated by multiplying the z score 

by 21.06 and adding 50. 

NCE = (z score x 21.06) + 50 

If you want to know the numbers used to calculate your class gain, contact the PAS Coordinator. She can 

print a spreadsheet detailing each of your students’ pre- and post-test scores, the z-scores, and the NCE 

equivalents used. She can also give you the district mean and standard deviation for the assessments 

used in your gains analysis. You can calculate your own gains by finding your class’ average pretest NCE 

score and subtracting it from the class’ average posttest NCE score. Remember, your students’ NCE 

scores already reflect the comparison to the district’s mean. If the difference is a positive 3.0 or greater, 

you are eligible for the award. 

A confusing phenomenon, negative gain, occurs if your students make gains, but not as much as the 

district. If subtracting the pretest from the posttest NCE average results in a negative number; that is 

known as a negative gain. 

Value-added 

For some teachers, the PAS award will be given if their mean student achievement gain exceeds one 

year of growth as determined by value-added calculation. The predicted mean approach will be used. 

For each student sample, a mean prediction of achievement on the next iteration of the Ohio 

Achievement Test will be calculated. Using test data for students with similar prior performance on 

common tests and the tests’ relationships to each other allows for the creation of statistically reliable 

predicted scores for each student in each subject. At the end of the year, a student sample mean of 

actual achievement will be calculated. The difference between the actual scores and the predicted 

scores is called the effect, or value-added score. 

It is not possible for a teacher to calculate their own value-added score as it is necessary to have 

multiple years of testing history for both the individual students and the comparison pool. There are 

about 40 Ohio school districts in the comparison pool. Districts are grouped into pools based on 

common testing histories at each grade-level cohort.
Appendix A–3 56
Appendix A–4 57

APPENDIX A–4:

COLUMBUS VALUE-ADDED PROFESSIONAL 

DEVELOPMENT MATERIALS

Appendix A–4 59
Appendix A–4 60
Appendix A–4 61
Appendix A–4 62
Appendix A–4 63
Appendix A–4 64
Appendix A–5 65

APPENDIX A–5:

HILLSBOROUGH CURRENT 

OBSERVATION RUBRIC

Appendix A–5 67
Appendix A–5 68
Appendix A–6 69

APPENDIX A–6:

HILLSBOROUGH NEW OBSERVATION RUBRIC

Appendix A–6 71
Appendix A–6 72
Appendix A–6 73
Appendix A–6 74
Appendix A–6 75
Appendix A–6 76
Appendix A–6 77
Appendix A–6 78
Appendix A–7 79

APPENDIX A–7:

EAGLE COUNTY PROFESSIONAL

PRACTICES RUBRIC

Appendix A–7 81

Domain of Planning Instruction and Assessment

Instructional Plans

This Indicator and Domain are evaluated over time (multiple observations/lessons being observed). 

Planning instruction encompasses decisions based on the district standards combined with knowledge of 

the disciplines taught, research-based professional practices, and the students who are to learn the curriculum. — Paula Rutherford 

Exemplary – 5

Descriptors are met thoroughly and

have significant impact on student learning.

Professional – 3

Descriptors are met effectively 

and impact student learning.

Unsatisfactory – 1

• The format of the standards-based instructional plans 

includes: review, introduction, presentation, activity, and 

closure.

• Instructional plans include all five of the following 

components:

• Content that

- Connects to previous learning and 

other powerful ideas

- Is broken down into the concepts and 

vocabulary to be learned 

• Activities and materials that

- Support the learning objective(s)

- Provide students with choices

- Are relevant to students’ lives

• Grouping size, compositions, and tasks that

- Maximize opportunities for student 

practice, student interaction, and for 

students to learn from each other

- Hold students accountable for group

and individual work

• Questioning that is

- Varied 

- Sequenced to the learning objective(s)

- Purposeful and coherent

• Differentiated instruction that

- Varies content, process, and product

• Instructional plans are observable and utilized.

• The format of the standards-based instructional plans 

includes: review, introduction, presentation, activity, 

and closure. 

• Instructional plans include four of the following 

components:

• Content that

- Connects to previous learning and 

other powerful ideas

- Is broken down into the concepts and 

vocabulary to be learned 

• Activities and materials that

- Support the learning objective(s)

- Provide students with choices

- Are relevant to students’ lives

• Grouping size, compositions, and tasks that

- Maximize opportunities for student 

practice, student interaction, and for 

students to learn from each other

- Hold students accountable for group

and individual work

• Questioning that is

- Varied 

- Sequenced to the learning 

objective(s)

- Purposeful and coherent

• Differentiated instruction that

- Varies content, process, and product

• The format of the standards-based instructional 

plans does not include: review, introduction, 

presentation, activity, or closure.

• Instructional plans do not include the following 

components:

• Content that

- Connects to previous learning and 

other powerful ideas

- Is broken down into the concepts 

and vocabulary to be learned 

• Activities and materials that

- Support the learning objective(s)

- Provide students with choices

- Are relevant to students’ lives

• Grouping size, compositions, and tasks that

- Maximize opportunities for student 

practice, student interaction, and 

for students to learn from each

other 

- Hold students accountable for 

group and individual work

• Questioning that is

- Varied 

- Sequenced to the learning 

objective(s)

- Purposeful and coherent

• Differentiated instruction that

- Varies content, process, and 

product

SY 09-10, Page 1 of 16 
Appendix A–7 82

Domain of Planning Instruction and Assessment

Assessment Plans

This Indicator and Domain are evaluated over time (multiple observations/lessons being observed). 

Effective assessment can motivate the unmotivated, restore the desire to learn, 

encourage students to keep learning and ultimately increase student achievement. — Richard Stiggins 

Exemplary – 5

Descriptors are met thoroughly and 

have significant impact on student learning.

Professional – 3

Descriptors are met effectively 

and impact student learning.

Unsatisfactory – 1

Assessment plans:

• Are aligned with state content standards and 

instructional plans.

• Have a clear desired student performance outcome.

• Include a variety of formative measures aligned to 

the learning objective(s).

• Include summative measures.

• Include goal setting and documentation of student 

progress toward state content standards.

• Are observable and utilized to inform instruction.

Assessment plans:

• Are aligned with state content standards and 

instructional plans.

• Have a clear desired student performance outcome. 

• Include a variety of formative measures aligned to 

the learning objective(s).

• Include summative measures.

• Include opportunities for goal setting and 

documentation of student progress toward state 

content standards.

Assessment plans:

• Are not aligned with state content standards.

• Do not have a clear student performance outcome.

• Do not include a variety of formative measures 

aligned to the learning objective(s).

• Do not include summative measures.

• Do not include goal setting.

• Are not observable or utilized to inform 

instruction. 

SY 09-10, Page 2 of 16 
Appendix A–7 83

Domain of Instruction

Standards and Objectives

This Indicator is evaluated based on a snapshot in time (lesson observed).

Establishing and communicating a specific learning objective is essential to direct student learning and to measure student progress.

Exemplary – 5

Descriptors are met thoroughly and

have significant impact on student learning.

Professional – 3

Descriptors are met effectively 

and impact student learning.

Unsatisfactory – 1

• The standards-based learning objective(s) is specific, 

measurable, demanding, and meaningful.

• The teacher communicates, displays, and references

the learning objective(s) throughout the lesson.

• The teacher thoroughly communicates the purpose of 

the learning objective(s).

• The teacher makes the desired student performance 

outcome thoroughly clear to all students.

• The lesson is focused on a limited set of skills and/or 

knowledge selected to help students reach the learning 

objective(s).

• There is evidence that most students demonstrate 

mastery of the learning objective(s).

• The standards-based learning objective(s) is specific, 

measurable, and meaningful.

• The teacher communicates and displays the learning 

objective(s).

• The teacher communicates the purpose of the learning 

objective(s).

• The teacher makes the desired student performance 

outcome clear to most students.

• The lesson is mostly focused on a limited set of skills 

and/or knowledge selected to help students reach the 

learning objective(s).

• There is evidence that most students demonstrate 

mastery of the learning objective(s).

• The standards-based learning objective(s) is not 

specific, measurable, demanding, or meaningful.

• The learning objective(s) is not communicated.

• The purpose of the learning objective(s) is not 

communicated.

• Expectations for student performance are unclear.

• The lesson is not focused on a limited set of skills or 

knowledge selected to help students reach the learning 

objective(s).

• There is evidence that few students demonstrate 

mastery of the learning objective(s).

SY 09-10, Page 3 of 16 
Appendix A–7 84

Domain of Instruction

Presenting Instructional Content

This Indicator is evaluated based on a snapshot in time (lesson observed).

The teaching and learning process is the interaction between the teacher and the students with 

the content. An effective teacher delivers instruction so that the learners are actively engaged with the content. 

Exemplary – 5

Descriptors are met thoroughly and

have significant impact on student learning.

Professional – 3

Descriptors are met effectively 

and impact student learning.

Unsatisfactory – 1

The teacher:

• Previews what will be learned and connects it to 

previous learning. 

• Provides visuals to establish the organization of the 

lesson.

• Breaks down the concept to be learned and teaches 

each part using appropriate, effective strategies and/or 

tools.

• Highlights key concepts and ideas and connects them 

to other powerful ideas. 

• Provides for frequent student interactivity with content.

• Communicates all essential information, is on topic, 

and is succinct.

• Frequently includes internal summaries during the 

lesson.

• Responds to students’ cues to adjust instruction. 

The teacher:

• Previews what will be learned and connects it to 

previous learning. 

• Provides visuals to establish the organization of the 

lesson.

• Breaks down the concept to be learned and teaches 

each part using appropriate, effective strategies and/or 

tools.

• Highlights key concepts and ideas.

• Provides for student interactivity with content.

• Communicates most essential information, is on topic, 

and is succinct.

• Includes internal summaries during the lesson. 

• Responds to students’ cues to adjust instruction. 

The teacher:

• Does not preview what will be learned or connect it to 

previous learning. 

• Provides ineffective or no visuals to establish the 

organization of the lesson. 

• Does not appropriately break down the concept to be 

learned.

• Does not highlight key concepts and ideas or connect 

them to other powerful ideas.

• Does not provide for student interactivity with content.

• Communication is off topic.

• Includes no internal summaries during the lesson.

• Does not respond to students’ cues to adjust 

instruction. 

SY 09-10, Page 4 of 16 
Appendix A–7 85

Domain of Instruction

Learning Activities & Materials

This Indicator is evaluated based on a snapshot in time (lesson observed).

Learning activities and materials must provide coherent, relevant learning experiences 

that will evoke and develop the desired understandings, promote interest, and lead to excellent performance.

Exemplary – 5

Descriptors are met thoroughly and

have significant impact on student learning.

Professional – 3

Descriptors are met effectively 

and impact student learning.

Unsatisfactory – 1

Activities and materials:

• Support the learning objective(s).

• Generate and sustain student engagement.

• Provide opportunities for student-to-student 

interaction.

• Provide students with choices.

• Are relevant to students’ lives.

AND include at least one of the following:

• Student interactivity with games or game-like 

materials.

• Product creation.

• Student use of multimedia.

• Student use of technology.

• Self-direction.

• Self-monitoring.

• Student use of resources beyond the school 

curriculum texts and materials.

Activities and materials:

• Support the learning objective(s).

• Generate and sustain student engagement.

• Provide opportunities for student-to-student 

interaction.

AND include at least one of the following:

• Student interactivity with games or game-like 

materials.

• Product creation.

• Student use of multimedia.

• Student use of technology. 

• Self-direction.

• Self-monitoring.

• Student use of resources beyond the school 

curriculum texts and materials.

Activities and materials:

• Do not support the learning objective(s).

• Does not generate or sustain student engagement.

• Do not provide opportunities for student-to-student 

interaction.

AND do not include:

• Student interactivity with games or game-like 

materials.

• Product creation.

• Student use of multimedia.

• Student use of technology. 

• Self-direction.

• Self-monitoring.

• Student use of resources beyond the school 

curriculum texts and materials.

SY 09-10, Page 5 of 16 
Appendix A–7 86

Domain of Instruction

Learning Groups

This Indicator is evaluated over time (multiple observations/lessons being observed). 

When students are intentionally placed into groups they experience multiple 

ways of thinking, receive more feedback, and engage in higher levels of discussion and interaction. 

Exemplary – 5

Descriptors are met thoroughly and

have significant impact on student learning.

Professional – 3

Descriptors are met effectively 

and impact student learning.

Unsatisfactory – 1

Over the course of multiple observations:

• Group size, group composition, and group tasks 

create opportunities for student practice, student 

interaction, and for students to learn from each other.

• Group work requires students to set goals, reflect on 

the group process, and evaluate their individual 

learning.

• All students perform their responsibilities and are held

accountable for both group and individual work. 

Over the course of multiple observations:

• Group size, group composition, and group tasks 

create opportunities for student practice, student 

interaction, or for students to learn from each other.

• Group work requires students to reflect on the group 

process and evaluate their individual learning. 

• Most students perform their responsibilities and are 

held accountable for either group or individual work. 

Over the course of multiple observations:

• Group size, group composition, and group tasks do 

not create opportunities for student practice, student 

interaction, or for students to learn from each other.

• Group work does not challenge students to set goals, 

reflect on the group process, or evaluate their 

individual learning. 

• Few students perform their responsibilities and are 

held accountable for group or individual work. 

SY 09-10, Page 6 of 16 
Appendix A–7 87

Domain of Instruction

Questioning

This Indicator is evaluated based on a snapshot in time (lesson observed).

When teachers use questions skillfully, they engage their students in an exploration of content. 

A teacher’s skill in questioning and in leading discussions makes a powerful contribution to student learning. – Charlotte Danielson

Exemplary – 5

Descriptors are met thoroughly and

have significant impact on student learning.

Professional – 3

Descriptors are met effectively 

and impact student learning.

Unsatisfactory – 1

• Questions are varied (remembering, 

understanding, applying, analyzing, evaluating, 

creating).

• Questions are sequenced with attention to the 

learning objective(s).

• Questions are purposeful and coherent.

• Questions frequently require active student responses. 

• Adequate wait time is consistently provided. 

• The teacher provides opportunities for students to 

generate and answer questions relevant to the 

learning objective(s).

• The teacher asks a high frequency of questions to 

engage students.

• The teacher calls on volunteers, non-volunteers, and 

a balance of students based on race, ability, and

gender.

• Questions are varied (remembering, 

understanding, applying, analyzing, evaluating, 

creating).

• Questions are mostly sequenced with attention to 

the learning objective(s).

• Questions are usually purposeful and coherent.

• Questions require active student responses.

• Adequate wait time is usually provided. 

• The teacher provides opportunities for students to 

generate questions relevant to the learning 

objective(s).

• The teacher asks questions to engage students.

• The teacher calls on volunteers, non-volunteers, and 

a balance of students based on race, ability, and

gender.

• Questions are not varied (remembering,

understanding, applying, analyzing, evaluating, 

creating).

• Questions are not sequenced with attention to the 

learning objective(s).

• Questions are not purposeful or coherent.

• Questions do not require active student responses.

• Adequate wait time is rarely provided. 

• The teacher does not provide opportunities for students 

to generate questions relevant to the learning 

objective(s).

• The teacher does not ask a high frequency of questions

to engage students.

• The teacher rarely calls on students. 

SY 09-10, Page 7 of 16 
Appendix A–7 88

Domain of Instruction

Academic Feedback

This Indicator is evaluated based on a snapshot in time (lesson observed).

Effective academic feedback tells students where they are on the continuum of mastery, what they are doing right, 

and next steps to take. It is just-in-time, just-for-me information delivered when and where it will do the most good. 

Exemplary – 5

Descriptors are met thoroughly and

have significant impact on student learning.

Professional – 3

Descriptors are met effectively 

and impact student learning.

Unsatisfactory – 1

The teacher:

• Provides a high frequency of oral and/or written 

feedback that is timely, academically-focused, 

corrective/directive, and specific to the learning 

objective(s).

• Circulates to prompt student thinking, assess each 

student’s progress, and to provide individual feedback.

• Intentionally engages students in giving academicallyfocused, corrective/directive, and specific to the 

learning objective feedback to one another.

The teacher:

• Provides oral and/or written feedback that is timely, 

academically-focused, corrective/directive, and 

specific to the learning objective(s).

• Circulates to prompt student thinking, assess each 

student’s progress, and sometimes to provide 

individual feedback. 

The teacher:

• Does not provide oral and/or written feedback that is 

timely, academically-focused, corrective/directive, or 

specific to the objective. 

• Does not circulate to prompt student thinking, assess 

each student’s progress, or provide individual 

feedback. 

SY 09-10, Page 8 of 16 
Appendix A–7 89

Domain of Instruction

Critical Thinking and Problem Solving

This Indicator is evaluated over time (multiple observations/lessons being observed). 

The teaching of critical thinking and problem solving enables students to develop mental techniques or abilities

that allow them to reason, judge, or formulate thoughts. Teaching thinking skills consists of teaching students how to engage in these 

behaviors. 

Exemplary – 5

Descriptors are met thoroughly and

have significant impact on student learning.

Professional – 3

Descriptors are met effectively 

and impact student learning.

Unsatisfactory – 1

Over the course of multiple observations, all four 

thinking/problem solving approaches are intentionally 

and explicitly taught and utilized.

• Analytical thinking where students compare/contrast, 

evaluate/explain, classify/categorize, or draw/justify 

conclusions.

• Practical thinking where students use, apply, and 

implement concepts and ideas they learned to work on 

real-life tasks.

• Creative thinking where students generate ideas, 

create, design, and evaluate a final product.

• Inquiry-based thinking where students hypothesize, 

observe, experiment, record, and report results.

Over the course of multiple observations, three of the 

thinking/problem solving approaches are intentionally 

and explicitly taught and utilized.

• Analytical thinking where students compare/contrast, 

evaluate/explain, classify/categorize, or draw/justify 

conclusions.

• Practical thinking where students use, apply, and 

implement concepts and ideas they learned to work on 

real-life tasks.

• Creative thinking where students generate ideas, 

create, design, and evaluate a final product.

• Inquiry-based thinking where students hypothesize, 

observe, experiment, record, and report results.

Over the course of multiple observations, the teacher 

implements few learning experiences that thoroughly 

teach or utilize any type of thinking/problem solving.

• Analytical thinking where students compare/contrast, 

evaluate/explain, classify/categorize, or draw/justify 

conclusions.

• Practical thinking where students use, apply, and 

implement concepts and ideas they learned to work on 

real-life tasks.

• Creative thinking where students generate ideas, 

create, design, and evaluate a final product.

• Inquiry-based thinking where students hypothesize, 

observe, experiment, record, and report results.

SY 09-10, Page 9 of 16 
Appendix A–7 90

Domain of Instruction

Differentiated Instruction

This Indicator is evaluated over time (multiple observations/lessons being observed). 

Not all students are alike. We must not differentiate who will learn what but rather how we will teach 

so that all students have access to, and support and guidance in, mastering the content. –Paula Rutherford 

Exemplary – 5

Descriptors are met thoroughly and

have significant impact on student learning.

Professional – 3

Descriptors are met effectively 

and impact student learning.

Unsatisfactory – 1

Over the course of multiple observations, the teacher:

• Provides differentiated CONTENT according to 

individual student readiness, interest, and

learning profile.

• Provides differentiated PROCESS according to 

individual student readiness, interest, and learning 

profile.

• As appropriate, provides opportunities for 

differentiated PRODUCT according to individual 

student readiness, interest, and learning profile.

Over the course of multiple observations, the teacher:

• Provides differentiated CONTENT according to 

groups of students’ readiness, interests, and learning 

profiles.

• Provides differentiated PROCESS according to groups

of students’ readiness, interests, and learning profiles.

• As appropriate, provides opportunities for 

differentiated PRODUCT according to groups of 

students’ readiness, interests, and learning profiles.

Over the course of multiple observations, the teacher:

• Does not differentiate CONTENT according to student 

readiness, interest, or learning profile.

• Does not differentiate PROCESS according to student 

readiness, interest, or learning profile.

• Does not differentiate PRODUCT according to student 

readiness, interest, or learning profile.

SY 09-10, Page 10 of 16 
Appendix A–7 91

Domain of Instruction

Lesson Structure

This Indicator is evaluated based on a snapshot in time (lesson observed).

Time, structure, and routines blend together to create a framework for the effective delivery of a lesson. 

Exemplary – 5

Descriptors are met thoroughly and

have significant impact on student learning.

Professional – 3

Descriptors are met effectively 

and impact student learning.

Unsatisfactory – 1

• Lesson starts and ends on time.

• Lesson structure is coherent and includes these aspects:

review, introduction, presentation, activity, and closure.

• The teacher provides time throughout the lesson for 

reflection on what was learned and why. 

• Procedures and routines are well established, efficient, 

and demonstrated by all students so that instructional 

time is maximized. 

• Lesson starts and ends somewhat on time.

• Lesson structure is coherent and includes these aspects:

review, introduction, presentation, activity, and closure.

• The teacher provides time for reflection on what was 

learned and why.

• Procedures and routines are well established, efficient, 

and demonstrated by most students so that instructional 

time is maximized. 

• Lesson does not start and/or end on time.

• Lesson structure is not coherent and includes few of 

these aspects: review, introduction, presentation, 

activity, closure.

• The teacher does not provide time for reflection on 

what was learned.

• Procedures and routines are not well established or 

efficient.

SY 09-10, Page 11 of 16 
Appendix A–7 92

Domain of Learning Environment

Academic Expectations

This Indicator and Domain are evaluated over time (multiple observations/lessons being observed). 

Having high academic expectations for all students influences the instructional decisions and behavior of the teacher. 

By believing that all students can achieve, the teacher makes it a practice to behave in ways that communicate those high expectations to every student. 

Exemplary – 5

Descriptors are met thoroughly and

have significant impact on student learning.

Professional – 3

Descriptors are met effectively 

and impact student learning.

Unsatisfactory – 1

• High and demanding academic expectations are set for 

every student.

• All students are consistently and meaningfully engaged.

• All students are encouraged to learn from mistakes.

• Learning opportunities are created where all students can 

experience success.

• Most students take initiative and follow through with 

their own work.

• Instructional time is optimized and expectations are set 

for better performance from every student.

• High and demanding academic expectations are set for 

every student.

• Most students are consistently and meaningfully 

engaged.

• Most students are encouraged to learn from mistakes.

• Learning opportunities are created where most students 

can experience success.

• Most students complete their work according to teacher 

expectations.

• Instructional time is optimized. 

• High and demanding academic expectations are not 

set for every student.

• Students are not consistently or meaningfully 

engaged.

• Students are not encouraged to learn from mistakes.

• Learning opportunities are not created where all 

students can experience success.

• Students do not take initiative or follow through with 

their own work. 

• Instructional time is not optimized. 

SY 09-10, Page 12 of 16 
Appendix A–7 93

Domain of Learning Environment

Managing Student Behavior

This Indicator and Domain are evaluated over time (multiple observations/lessons being observed). 

Effective classroom management is essentially invisible, because when students 

are well-behaved and engaged, the focus is on instruction and learning. — Rick Smith 

Exemplary – 5

Descriptors are met thoroughly and

have significant impact on student learning.

Professional – 3

Descriptors are met effectively 

and impact student learning.

Unsatisfactory – 1

Students are consistently well-behaved and on task 

because the teacher:

• Establishes clear standards of conduct.

• Attends to misbehavior quickly, respectfully, and 

appropriately.

• Addresses individual student behavior rather than the 

entire class and is sensitive to the students’ individual 

needs.

• Uses subtle responses to correct minor disruptions.

• Assigns natural and logical consequences.

• Overlooks inconsequential behavior when appropriate.

• Effectively uses research-based management 

strategies.

Students are usually well-behaved and on task because 

the teacher:

• Establishes clear standards of conduct.

• Attends to misbehavior quickly, respectfully, and 

appropriately.

• Addresses individual student behavior rather than the 

entire class and is sensitive to the students’ individual 

needs.

• Uses subtle responses to correct minor disruptions.

• Assigns natural and logical consequences.

• Overlooks inconsequential behavior when appropriate.

Students are not-well behaved or on task because the 

teacher does not:

• Establish clear standards of conduct.

• Attend to misbehavior quickly, respectfully, or 

appropriately.

• Address individual student behavior and is not 

sensitive to the students’ individual needs.

• Use subtle responses to correct minor disruptions. 

• Assign natural and logical consequences. 

• Overlook inconsequential behavior. 

SY 09-10, Page 13 of 16 
Appendix A–7 94

Domain of Learning Environment

Physical Environment

This Indicator and Domain are evaluated over time (multiple observations/lessons being observed). 

The physical environment is an important resource for learning and should provide flexibility in organizing students and activities.

Exemplary – 5

Descriptors are met thoroughly and

have significant impact on student learning.

Professional – 3

Descriptors are met effectively 

and impact student learning.

Unsatisfactory – 1

The classroom consistently:

• Is organized and understandable to all students.

• Has supplies, equipment, and resources easily and 

readily accessible.

• Displays relevant student work. 

• Is arranged to promote individual and group learning. 

• Is safe and clean.

• Displays the state standards.

• Displays content-specific references or resources.

The classroom usually:

• Is organized and understandable to most students.

• Has supplies, equipment, and resources accessible.

• Displays relevant student work. 

• Is arranged to promote individual and group learning. 

• Is safe and clean.

• Displays the state standards.

• Displays content-specific references or resources.

The classroom:

• Is not organized and understandable to all students.

• Does not has have supplies, equipment, or resources 

easily or readily accessible.

• Does not display relevant student work. 

• Is not arranged to promote individual or group 

learning.

• Is not safe or clean.

• Does not display the state standards.

• Does not display content-specific references or 

resources.

SY 09-10, Page 14 of 16 
Appendix A–7 95

Domain of Learning Environment

Respectful Culture

This Indicator and Domain are evaluated over time (multiple observations/lessons being observed). 

Creating a positive classroom climate begins with showing respect to one another and leads to a caring and supportive learning environment.

Exemplary – 5

Descriptors are met thoroughly and

have significant impact on student learning.

Professional – 3

Descriptors are met effectively 

and impact student learning.

Unsatisfactory – 1

Consistently,

• Interactions between teacher and student are respectful.

• Interactions between students are respectful. 

• Interactions among members of the class reflect 

warmth, caring, and sensitivity.

• Students take pride in their work. 

• The teacher seeks out the interests and opinions of all 

students.

• The teacher thoroughly reinforces and rewards effort.

Usually,

• Interactions between teacher and student are respectful.

• Interactions between students are respectful. 

• Interactions among members of the class reflect 

warmth, caring, and sensitivity.

• Students take pride in their work. 

• The teacher seeks out the interests and opinions of all 

students.

• The teacher reinforces and rewards effort.

• Interactions between teacher and student are not 

respectful.

• Interactions between students are not respectful.

• Interactions among members of the class do not 

reflect warmth, caring and sensitivity.

• Students do not take pride in their work. 

• The teacher does not seek out the interests or 

opinions of students. 

• The teacher does not reinforce or reward effort.

SY 09-10, Page 15 of 16 
Appendix A–7 96

Domain of Learning Environment

Classroom Procedures

This Indicator and Domain are evaluated over time (multiple observations/lessons being observed). 

Procedures are the railroad tracks...content is the train. — Rick Smith

Exemplary – 5

Descriptors are met thoroughly and

have significant impact on student learning.

Professional – 3

Descriptors are met effectively 

and impact student learning.

Unsatisfactory – 1

Consistently, classroom procedures and routines:

• Are explicitly taught, practiced, and understood by 

students.

• Are clear at the beginning, during the middle, and at 

the end of class, and for special situations. 

• Are efficient.

• Include non-instructional duties.

• Require all students to contribute.

Usually, classroom procedures and routines:

• Are explicitly taught, practiced, and understood by 

students.

• Are clear at the beginning, during the middle, and 

at the end of class, and for special situations. 

• Are efficient.

• Include non-instructional duties.

• Require all students to contribute.

Classroom procedures and routines:

• Are not explicitly taught, practiced, and or understood by 

students.

• Are not clear at the beginning, during middle, and at the end 

of class, or for special situations.

• Are not efficient.

• Do not include non-instructional duties.

• Do not require all students to contribute.

SY 09-10, Page 16 of 16 
Appendix A–8 97

APPENDIX A–8:

HOUSTON DECISION-MAKING FRAMEWORK

Appendix A–8 99

Contract Type Management Decision Framework Options 

Probationary Year 1 1. Probationary contract for year 2 should be issued if: 

• Teacher performance level meets or exceeds expectations, and there is no reason to 

non-renew. 

2. Termination should be proposed if: 

• Another teacher could be hired to better meet the needs of student s in the school 

• Summative Annual Appraisal is below Expectations or Unsatisfactory- for example: 

0 

0 

0 

Poor classroom management practices 

Poor relationships with st udents, parents, staff, etc. 

High rate of unexcused absences and/or instances of tardiness 

• Other legal reason 

Probationary Year 2 1. Probationary contract for year 3 should be issued if: 

• Teacher performance level meets or exceeds expectations, and there is no reason to 

non-renew 

0 However, if a teacher has a one-year Running Cumulative Average between 0 and 

-3.00 in any one subject, a growth plan will be required for the coming year (i.e. 

team-teaching, mentoring, professional development) 

2. Termination should be proposed if: 

• Another teacher could be hired to better meet the needs of students in the school 

• Summative Annual Appraisal is below Expectations or Unsatisfactory - for example: 

0 

0 

0 

0 

Poor classroom management practices 

Poor relationships with students, parents, staff, etc. 

High rate of unexcused absences and/or instances of tardiness 

If value-added data is available, the teacher has a one-year Running Cumulative 

Average of -3.00 or lower in any subject for the most recent school year-UNLESS 

a compelling reason is offered by the Principal and supported by the School 

Improvement Officer 

• Other legal reason 

Staff M anagement Decision Making Framework 
Appendix A–8 100

Probationary Year 3 1. Term contract should be issued if: 

• Teacher performance is above expected levels 

o If value-added data is available, the teacher's Cumulative Running Average is 0 (Zero) or 

higher for all subjects taught 

2. Termination should be proposed if: 

• Another teacher could be hired to better meet the needs of students in the school 

• Summative Annual Appraisal is below Expectations or Unsatisfactory on any one or more of 

the Performance Criteria -for example: 

o 

o 

o 

o 

Poor classroom management practices 

Poor relationships w ith students, parents, staff, etc. 

High rate of unexcused absences and/or instances of tardiness 

If available, for any subject the teacher has a -2.00 or lower Running Cumulative 

Average based on a 2-running average or a Running Cumulative Average of -3.00 or 

lower for the most recent school year if only one year of data is available-UNLESS a 

compelling reason is offered by the Principal and supported by the School 

Improvement Officer 

• Other legal reason 

3. Optional Probationary contract for year 4 may be issued if: 

o 

o 

The teacher has a Running Cumulative Average based on 2-years of va lue added data 

between 0 and -2 for any subject or based on 3-years of value added data between 0 

and -3.00 in any one subject AND the principal presents a compelling reason and it is 

supported by the School Improvement Officer 

A growth plan is required for all teachers receiving a Yea r 4 Probationary Contract (i.e. 

team-teaching, mentoring, professional development) 

Staff Management Decision Making Framework 
Appendix A–8 101

Probationary Year 4 1. Term contract should be issued if: 

• Teacher performance meets standards 

o If value-added data is available, the teacher has a Cumulative Running Average of 0 

(Zero) or higher for all subjects taught 

• Teacher performance meets standards 

o 

o 

BUT, if value-added data is available, for any one subject the teacher has a Running 

Cumulative Average of greater than -1.00 but less than a 0 (Zero) based on a 3-year 

running average or a Running Cumulative Average of greater than -2.00 or less than a 0 

(Zero) based on a 2-year running average or a Running Cumulative Average of greater 

than -3.00 but less than a 0 (Zero) for the most recent school year if only one year of 

data is available AND the principal presents a compelling reason and it is supported by 

the School Improvement Officer 

A growth plan is required for all teachers receiving a Term Contract that have value 

added scores below 0. (i.e. team-teaching, mentoring, professional development) 

2. Termination should be proposed if: 

• Another teacher could be hired to better meet the needs of students in the school 

• Summative Annual Appraisal Below Expectations or Unsatisfactory - for example: 

o 

o 

o 

o 

Poor classroom management practices 

Poor relationships with students, parents, staff, etc. 

High rate of unexcused absences and/or instances oftardiness 

If value-added data is available, the teacher has a -1.00 or lower Running Cumulative 

Average based on a 3-year running average or a -2.00 or lower Running Cumulative 

Average based on a 2-year running average or a Running Cumulative Average of -3.00 

or lower for the most recent school year if only one year of data is available-UNLESS a 

compelling reason is offered by the Principal and supported by the School 

Improvement Officer 

• Other legal reason 

Staff Management Decision Making Framework 
Appendix A–8 102

Staff M anagement Decision M aking Framework 

Term 1. Contract should continue if: 

• Teacher performance meets or is above expected levels 

0 But, if a teacher has a Running Cumulative Average that is regressive (i.e. Less than 0) in 

any one subject, a growth plan is required (i.e. team-teaching, mentoring, professional 

development) 

• There is no other compelling reason to terminate the contract 

2. Termination should be proposed if: 

• Unsatisfactory performance on one or more of the 34 criteria as part of the district's 

evaluation system for teachers. 

Continuing 1. Contract should continue if: 

• Teacher performance meets or is above expected levels 

0 But, if a teacher has a Running Cumulative Average that is regressive (i.e. Less than 0) in 

any one subject, a growth plan is required (i.e. team-teaching, mentoring, professional 

development) 

• There is no other compelling reason to terminate the contract . 

2. Termination should be proposed if: 

• Unsatisfactory performance on one or more of the 34 criteria as part of the district's 

evaluation system for teachers. 
Appendix A–9 103

APPENDIX A–9:

HILLSBOROUGH TEACHER SUMMARY SHEET

Appendix A–9 105
Appendix A–9 106
Appendix A–9 107
Appendix A–9 108
Appendix B 109

APPENDIX B:

SITE DESCRIPTIONS

Appendix B–1 111

APPENDIX B-1:

COLUMBUS CITY SCHOOL DISTRICT

SITE DESCRIPTION

Appendix B–1: 113

Columbus City School District 

Site Description

Columbus City School District Site Description 

District Background

Columbus City School District, located in Franklin 

County, in Columbus, Ohio, is approximately the 

76th largest school district in the United States and 

the largest school district in Ohio (see Exhibit B-1.1 

for more information on district and school 

characteristics). Columbus did not make adequate 

yearly progress (AYP) in 2008–09 (see Exhibit B-1.2 

for proficiency rates on state assessments in reading 

and mathematics). 

Exhibit B-1.1

District Snapshot

Schools 128

Students 55,000

Teachers 4,000

Student characteristics Percent of students

Free and reduced-price lunch 

(FRPL) 74%

English language learners 

(ELLs) 6%

Students with individualized

education programs (IEPs) 16%

Student race/ethnicity Percent of students

African American 62%

American Indian <1%

Asian 2%

Hispanic 6%

Multiracial 3%

White 27%

Exhibit B-1.2

District Proficiency Rates in 

Mathematics and Reading,

Grades 3–10, for 2008–09

Grade Reading Math

3rd grade 60% 66%

4th grade 67% 63%

5th grade 53% 44%

6th grade 59% 54%

7th grade 50% 48%

8th grade 49% 42%

9th grade NA NA

10th grade 77% 68%

Exhibit reads: In 2008–09, 60 percent of thirdgraders scored proficient or higher on the state 

test in reading and 66 percent scored proficient 

or higher on the state test in mathematics.

Source: http://ilrc.ode.state.oh.us/Districts/

Report.asp?Dssid=26FFE8B5455DB0101

D5B2FA632A473B1&desc=D.1.2

- Proficiency Test Results 

(District)&str=043802,

Columbus City,Franklin County (2008–09). 

History and Current Efforts to 

Measure Teacher and School 

Effectiveness

There were several different teacher effectiveness 

measures concurrently in use in the district. With the 

state of Ohio’s Race to the Top win, the district 

anticipated possibly making other changes to 

measures of teacher effectiveness. 

• As of the site visit in July 2010, the district’s 

official teacher evaluation process was based 

on as many as six formal observations by an 

administrator. Administrators rated teachers, 

using the negotiated evaluation checklist, with 

eight areas (e.g., pupil relations, teaching 

performance, and personal characteristics). 

An overall rating of satisfactory or 

unsatisfactory was issued. 

• Since 2007, the state has provided 

value-added data at the school and grade level 

in reading and mathematics to each Ohio 

school district as part of the state 

accountability system. These data are 

generated as a result of a 2003 state law 

requiring that value-added assessments be 

incorporated into school performance 

indexes. At the same time, the Ohio 

Department of Education worked with the 

nonprofit organization Battelle for Kids to 
Appendix B–1: 114

Columbus City School District 

Site Description

create a professional development program related to value-added data use. This program, called 

Project SOAR, was launched in 2002, and assisted 42 school districts across the state. Columbus 

City School District volunteered to be part of Project SOAR in 2002. 

• In 2006, building from its previous work, Battelle for Kids launched the Teachers Connecting 

Achievement and Progress (TCAP) program to assist participating school districts, including 

Columbus, with teacher-level value-added analysis. Participating schools in the district (there 

were approximately 40 as of the time of the site visit) received value-added data in science, social 

studies, reading and math at grades 3–8. Only teachers and principals received copies of 

teacher-specific reports, and the district received summary information at the school, grade and 

subject level. This information was used for professional development conversation and 

planning purposes. 

• Teachers could volunteer for the Performance Advancement System (PAS), in which their 

effectiveness in applying a particular instructional approach was evaluated. Teachers identified an 

academic area, selected students to study, applied a particular instructional approach with those 

students, and monitored their progress using student growth measures. At the end of the 

semester or year, teachers whose students had gains greater than the district average received a 

bonus of $2,500. Teachers with classroom-level value-added reports had their PAS gains 

calculated through the value-added method; otherwise gains were examined using other district 

or state assessment data. 

• There were additional teacher effectiveness measures used in selected schools in the district. In 

2004,7 with state support, four schools in Columbus began implementing the Teacher 

Advancement Program (TAP), which focused on teacher leadership, performance pay, teacher 

accountability, and professional development. In 2006, the Ohio Department of Education won 

a Teacher Incentive Fund (TIF) grant, and Columbus was able to expand TAP to additional 

schools. Due to school closings and reconstitution, in the 2010 through 2011 school year, 

six schools planned to participate in TAP. TAP schools used a value-added score and 

observation data to determine performance bonuses. Peers and administrators observed teachers 

a total of four times per year, using the TAP observation rubric, which is based on the Charlotte 

Danielson Framework for Teaching. The rubric consisted of a number of different dimensions 

related to instruction, planning, and the learning environment. Teachers were given a score from 

1 to 5: 1 and 2 not proficient, 3 proficient, and 4 and 5 advanced. 

7 http://edu.reportcard.state.tn.us/pls/apex/f?p=200:20:3308899578178648::NO

Strategies to Improve Equitable Access to Effective Teachers

Columbus officials did not report using teacher effectiveness measures to examine the distribution of 

effective teachers between high- and low-poverty or high- and low-minority schools. The district had 

several programs aimed at rewarding or supporting teacher quality in all schools, including support for 

National Board Certified teachers, a monetary reward for all teachers in schools that meet AYP, an 

intensive induction program for new teachers and an intervention program for struggling teachers (the 

Peer Assistance and Remediation Program), and a program for peers to consult on problem areas 

(instructional rounds). The district also highlighted several programs that targeted certain schools or 

teachers. One program (through the Teacher Quality Enhancement grant and Teacher Quality 

Partnership) provided internships to preservice teachers and attempted to place new teachers from the 

program in high-need schools. Another program provided a $4,000 stipend for three years to teachers 

with five years’ experience and demonstrated achievement gains (which could be based on a variety of 

measures, including value-added scores) who move to a struggling school. 

 
Appendix B–2: 115

Eagle County School District 

Site Description

APPENDIX B-2:

EAGLE COUNTY SCHOOL DISTRICT

SITE DESCRIPTION

Appendix B–2: 117

Eagle County School District 

Site Description

Eagle County School District Site Description 

District Background

Eagle County, Colorado, is located in the Rocky 

Mountains about 100 miles west of Denver. The 

county has a population of slightly more than 

53,000 people, but the district covers a relatively large 

geographic area. Exhibit B-2.1 shows overall 

information about student background characteristics 

in the district, but there are variations among schools, 

with several schools serving large majorities of ELL 

and FRPL-eligible students. 

The district did not make adequate yearly progress 

(AYP) in the 2008–09 school year, making that the 

third year of corrective action status (see Exhibit 

B-2.2 for proficiency rates on state assessments in 

reading and mathematics). 

Exhibit B-2.1

District Snapshot

Schools 19

Students 6,007

Teachers 495

Student characteristics Percent of students

Free and reduced-price lunch 

(FRPL) 35%

English language learners 

(ELLs) 35%

Students with individualized

education programs (IEPs) NA

Student race/ethnicity Percent of students

African American <2%

American Indian <2%

Asian <2%

Hispanic 52%

Multiracial NA

White 46%

Exhibit B-2.2

District Proficiency Rates in 

Mathematics and Reading,

Grades 3–10, for 2008–09

Grade Reading Math

3rd grade 71% 69%

4th grade 64% 69%

5th grade 68% 62%

6th grade 72% 67%

7th grade 70% 56%

8th grade 68% 57%

9th grade 67% 35%

10th grade 61% 23%

Exhibit reads: In 2008–09, 71 percent of thirdgraders scored proficient or higher on the state 

test in reading and 69 percent scored proficient 

or higher on the state test in mathematics.

Note: Results are reported here for the English 

version of the test only.

Source: http://www.cde.state.co.us/cdeassess/

documents/csap/csap_summary.html#2009.

History and Current Efforts to 

Measure Teacher and School 

Effectiveness

• In Eagle County, teacher effectiveness was 

measured through observations, as well as 

student achievement growth. 

• A detailed observation rubric was the basis of 

the teacher evaluation system in Eagle 

County. Teachers were observed formally at 

least three times a year (by master teachers, 

mentor teachers, and an administrator), and 

the observations were weighted to get an 

overall score. Eagle County had been using 

some form of this observation rubric since 

2001, when the district adopted the Teacher 

Advancement Program (TAP). In 2006, the 

district was awarded a Teacher Incentive 

Fund (TIF) grant to continue development 

and implementation of the TAP program. 

TAP’s performance pay component was 

controversial, and in the spring of 2007, 

many district leaders left or were asked to 

resign. An interim superintendent was hired for one year, and a permanent superintendent has 

been in place since 2008. Since then, the district moved away from the TAP version of the 

rubric, both adding and condensing elements. These changes were made by a committee 

including district staff, principals, teachers and union representatives. It was district policy that 
Appendix B–2: 118

Eagle County School District 

Site Description

this rubric be reviewed every two years. Scores from the observation rubric were used to 

determine salary increases for teachers. 

• At the individual teacher level, the district also obtained value-added information. This 

information was disseminated to schools for use in school improvement and professional 

development planning. 

• At the building level, the district also used value-added data and other measures of student 

achievement and growth, such as the Colorado Student Assessment Program (CSAP) and 

American College Testing (ACT) for performance pay purposes. 

Strategies to Improve Equitable Access to Effective Teachers

Eagle County did not report using its teacher effectiveness measures to examine the distribution of 

teachers between high- and low-poverty or high- and low-minority schools, although they were 

discussing within-school assignment of teachers as a result of a state grant focused on school 

improvement. The district described policies designed to improve effectiveness in all schools, including 

hiring strategies, professional development and induction, and the termination of unsatisfactory 

probationary teachers (on the basis of the district’s observation rubric). To attract teachers to high-need 

schools, the district set up a $2,500 one-time signing bonus for teachers who took a position at a 

high-need school. A two-year commitment was required. 
Appendix B–3: 119

Hamilton County School District 

Site Description

APPENDIX B-3:

HAMILTON COUNTY SCHOOL DISTRICT

SITE DESCRIPTION

Appendix B–3: 121

Hamilton County School District 

Site Description

Hamilton County School District Site Description 

District Background

The Hamilton County School District covers the 

county’s entire geographical area of 542 square miles 

and encompasses the city of Chattanooga and 

surrounding suburban and rural areas. Hamilton 

County is approximately the 123rd largest school 

district in the United States and the fifth largest in 

Tennessee (see Exhibit B-3.1 for more on the 

characteristics of the district and its students). 

In 2008–09 Hamilton County School District did not 

make adequate yearly progress (AYP). For all students 

in grades K through 8 in Hamilton County, 

89 percent were proficient in math and 90 percent 

were proficient in reading. For all students in grades 

9 through 12 in Hamilton County, 89 percent were 

proficient in math and 96 percent were proficient in 

reading.8

8 http://edu.reportcard.state.tn.us/pls/apex/f?p=200:20:3308899578178648::NO

Exhibit B-3.1

District Snapshot

Schools 75

Students 39,247

Teachers 2,836

Student characteristics Percent of students

Free and reduced-price lunch 

(FRPL) 59%

English language learners 

(ELLs) 3%

Students with individualized

education programs (IEPs) 16%

Student race/ethnicity Percent of students

African American 33%

American Indian <1%

Asian 2%

Hispanic 5%

Multiracial NA

White 59%

History and Current Efforts to Measure Teacher and School 

Effectiveness

During the site visit, Hamilton County was in the process of making changes to its systems to measure 

teacher effectiveness and use that data, in part as a result of Tennessee’s Race to the Top (RTT) grant. 

• A new measure of teacher practice, based on a new classroom observation framework, was being 

piloted. Until January 2010, Hamilton County’s teacher evaluation process involved three 

announced observations and walkthroughs by the principal. There was uniform dissatisfaction 

with this instrument: Almost everyone received a satisfactory evaluation; there was no inter-rater 

reliability and no consistency of use. In February of 2010, a consultant presented an idea for a 

new teacher evaluation rubric, and the superintendent approved the establishment of a steering 

committee consisting of principals, teachers, union representatives and district leaders to 

examine the rubric. This committee took the consultant’s rubric and made it the Hamilton 

County Teacher Performance Rubric. At the same time, the state was assembling a task force to 

review models of teacher evaluation as part of RTT. Hamilton County received approval from 

the state’s commissioner of education to pilot this new model. Data from the new rubric were 

intended to be used for immediate feedback and planning, as well as summative evaluation for 

individual teachers. The district also reported that they planned to use the data for professional 

development planning for the district as a whole. 

• In addition, through the Tennessee Value-Added Assessment Systems (TVAAS), Hamilton 

County received value-added data for individual teachers in all schools. While these data were 

available for use in planning, the district did not use them for a specific purpose. 

 
Appendix B–3: 122

Hamilton County School District 

Site Description

• As part of the state’s RTT plan, Hamilton County planned to construct a new measure of 

teacher effectiveness which would combine information based on student achievement growth 

and observations. Although the relative weights had been established by the state (50 percent to 

be based on measures of practice, 35 percent on value-added scores and 15 percent on “other”), 

other details and guidance were still being developed. 

Strategies to Improve Equitable Access to Effective Teachers

The district did not report tracking the distribution of effective teachers by school type, although the 

state has performed such an analysis across all schools in the state and found that there are disparities in 

the numbers of effective teachers serving low- and high-poverty schools. 

The district did not report any current activities specifically focused on increasing the numbers of 

effective teachers in high-need schools. However, in 2001, as part of the first phase of the Benwood 

Initiative, a partnership between the district and a local education foundation, the district restaffed a 

number of low-performing schools and provided incentives for effective teachers (as measured through 

value-added data) to apply for positions there. 

The district did describe a number of initiatives aimed at improving teacher effectiveness in all schools, 

for example, improving preparation, recruitment and hiring processes. As of summer 2010, the district 

was planning a “grow your own teachers initiative,” which would identify the top 10 percent of students 

in Hamilton County schools and recruit them into education programs, with the promise of a job as a 

teacher. In addition, both the University of Tennessee at Knoxville and the University of Tennessee at 

Chattanooga were working on alternative route programs and new residency programs for teacher 

preparation. Hamilton County was also trying to improve the screening and interviewing of candidates, 

and providing training to principals on writing good job descriptions and interviewing candidates. The 

county planned to base interviews and selection on the new teacher observation rubric and to work with 

deans of education schools to inform them about the expectations for teacher candidates. 
Appendix B–4: 123

Hillsborough County Public Schools 

Site Description

APPENDIX B-4:

HILLSBOROUGH COUNTY PUBLIC SCHOOLS

SITE DESCRIPTION

Appendix B–4: 125

Hillsborough County Public Schools 

Site Description

Hillsborough County Public Schools Site Description 

District Background

Hillsborough County Public Schools is the eighth 

largest school district in the United States. The county 

covers more than 1,000 square miles, encompassing 

the City of Tampa, as well as surrounding suburbs 

and rural areas (see Exhibit 1 for more detail on 

district characteristics). 

Among all students, 60 percent scored at or above 

grade level in reading and 67 percent did so in math. 

The district did not make adequate yearly progress 

(AYP) in 2008–09 (see Exhibit 2 for proficiency rates 

by grade for reading and mathematics). 

Exhibit B-4.1

District Snapshot

Schools 248

Students 191,975

Teachers 13,726

Student characteristics Percent of students

Free and reduced-price lunch 

(FRPL) 52%

English language learners 

(ELLs) 15%

Students with individualized

education programs (IEPs) 15%

Student race/ethnicity Percent of students

African American 22%

American Indian 1%

Asian 3%

Hispanic 28%

Multiracial 6%

White 41%

Exhibit B-4.2

District Proficiency Rates in 

Mathematics and Reading,

Grades 3–10, for 2008–09

Grade Reading Math

3rd grade 68% 76%

4th grade 72% 73%

5th grade 68% 62%

6th grade 64% 54%

7th grade 63% 59%

8th grade 52% 65%

9th grade 44% 67%

10th grade 38% 70%

Exhibit reads: In 2008–09, 68 percent of thirdgraders scored proficient or higher on the state 

test in reading and 76 percent scored proficient 

or higher on the state test in math.

Source: http://fcat.fldoe.org/results/default.asp.

History and Current Efforts to 

Measure Teacher and School 

Effectiveness

Hillsborough County Public Schools was in a time of 

transition during the site visit, in June 2010, in part 

due to the state’s Race to the Top (RTT) grant and 

Hillsborough’s own $100 million grant from the Bill 

& Melinda Gates Foundation for Empowering 

Effective Teachers (EET). The district had several 

measures of teacher effectiveness in use at the time, 

and was also in the process of developing new 

measures. 

• At the time of the visit, Hillsborough used a 

measure of teacher effectiveness based on 

student achievement growth as one criterion 

for awarding performance pay bonuses. The 

other criterion was a score from a checklist 

observation of teachers. Hillsborough County 

had a long history of performance pay 

initiatives, dating back to a system that used 

portfolios prior to the start of the state’s 

current Merit Award Program (MAP), which 

spurred the development of the student achievement measure. Beginning in 1999, a Florida state 

statute required that districts use their own funding to reward “outstanding performance,” based 

primarily on student achievement. Hillsborough developed its current system as a result of this 

law and the state programs that supported them. The district also used funds from a federal 

Teacher Incentive Fund (TIF) grant to provide additional monetary rewards in high-need 

schools. 
Appendix B–4: 126

Hillsborough County Public Schools 

Site Description

• The district was also in the process of developing a value-added measure and a new teacher 

observation rubric. The value-added student achievement measure and the observation measure 

would be combined to create an overall effectiveness rating, with the achievement measure 

weighted at 40 percent and the new teacher observation weighted at 60 percent. This 

comprehensive rating would be used for compensation, tenure, and nonrenewal contract 

considerations, as well as for determining the professional development needs of teachers. Both 

the value-added scores and the new observation rubric were rolling out in 2010–11. As of 

summer 2010, the new compensation system using these effectiveness measures was planned to 

begin in 2013, when the district would have three years of value-added data. 

Strategies to Improve Equitable Access to Effective Teachers

Hillsborough did not report using its teacher effectiveness measures to examine the distribution of 

teachers between high- and low-poverty or high- and low-minority schools. The district described several 

programs to reward and improve teacher effectiveness in all schools, such as the MAP awards and 

incentives for National Board Certified teachers. In addition, the district had several policies in place to 

try to get high-quality teachers to work in high-poverty schools, such as special hiring policies and 

additional monetary awards for teachers in the highest poverty schools, which it has designated 

Renaissance Schools. 
Appendix B–5: 127

Houston Independent School District 

Site Description

APPENDIX B-5:

HOUSTON INDEPENDENT SCHOOL DISTRICT

SITE DESCRIPTION

Appendix B–5: 129

Houston Independent School District 

Site Description

Houston Independent School District Site Description

District Background

The Houston Independent School District is the 

seventh largest district in the United States and the 

largest in Texas. The boundaries of the district include 

the city of Houston and extend to the greater Houston 

area (see Exhibit 1 for more details on district and 

student characteristics). 

Across all groups in grades 3 to 11, 84 percent of 

students were proficient on the Texas statewide 

reading test; 74 percent were proficient on the 

statewide mathematics test. The district did not make 

adequate yearly progress (AYP) in 2008–09 (see 

Exhibit 2 for proficiency rates, by grade, on state 

assessments in reading and mathematics). 

Exhibit B-5.1

District Snapshot

Schools 296

Students 200,225

Teachers 12,829

Student characteristics Percent of students

Free and reduced-price lunch 

(FRPL) 81%

English language learners 

(ELLs) 31%

Students with individualized

education programs (IEPs) 9%

Student race/ethnicity Percent of students

African American 28%

American Indian <1%

Asian 3%

Hispanic 61%

Multiracial NA

White 8%

Exhibit B-5.2

District Proficiency Rates in 

Mathematics and Reading, 

Grades 3–10, for 2008–09

Grade Reading Math

3rd grade 85% 82%

4th grade 82% 86%

5th grade 79% 84%

6th grade 86% 74%

7th grade 78% 74%

8th grade 89% 72%

9th grade 82% 57%

10th grade 83% 57%

Exhibit reads: In 2008–09, 85 percent of thirdgraders scored proficient or higher on the state 

test in reading and 82 percent scored proficient 

or higher on the state test in math.

Note: Results are reported here for the English 

version of the test only.

Source: http://www.houstonisd.org/Research

Accountability/Home/SP_TAKS/

Performance%20Reports

/2009/HISD%20District%20Summary.pdf.

History and Current Efforts to 

Measure Teacher and School 

Effectiveness

At the time of the site visit, Houston used a 

value-added measure of teacher effectiveness, largely 

for performance pay purposes, but was considering 

additional changes to its measures of teacher 

effectiveness, which could involve development of a 

new observation system or additional uses of 

value-added data. 

• Since 2007, the Houston Independent School 

District has used campus- and teacher-level 

value-added teacher effectiveness data to 

award performance bonuses to teachers, 

administrators, and other staff through the 

Accelerating Student Progress Increasing 

Results and Expectations (ASPIRE) Award 

program (with funding partly provided through 

a federal TIF grant). The district also reported 

that the data were used for professional 

development planning and identifying mentors 

and coaches. In addition, value-added data 

were made a possible criterion for termination 

decisions. 

• Houston used the state of Texas’ standard teacher evaluation process called the Professional 

Development and Appraisal System (PDAS), which included at least one classroom observation 
Appendix B–5: 130

Houston Independent School District 

Site Description

lasting a minimum of 45 minutes, a summative annual conference, and written documentation 

regarding job-related teacher performance.

• In 2009–10, the district tried a new approach to measuring teacher effectiveness called the 

staffing review process (often locally referred to as the “Bucket System”). Using this new 

method, principals were asked to evaluate various criteria of teacher performance, including the 

PDAS ratings and value-added data, to rate teachers as (1) Highly Effective, (2) Proficient, 

(3) Developing, or (4) Low-Performing. 

Strategies to Improve Equitable Access to Effective Teachers

The district has analyzed effectiveness data, by school characteristics, and found disparities in 

effectiveness. A 2010 study by the New Teacher Project for the Houston Independent School District 

found that schools with more than 75 percent of students eligible for FRPL had a smaller percentage of 

effective teachers, as measured by value-added data, compared with schools with 75 percent or less 

FRPL. 

In order to improve teacher effectiveness in all schools, the district had policies to increase the selectivity 

in hiring, use effectiveness data for termination decisions and prevent ineffective teachers from 

transferring. The district also had a mentoring and induction program for all new teachers. In addition, 

the district highlighted a specific new initiative targeting low-performing schools (the Effective Teacher 

Pipeline Project), which attempted to identify effective teachers and relocate them in clusters to 

low-performing schools. Teachers who agreed to transfer received a $10,000 stipend for two years. 
Appendix C: 131

Study Methods

APPENDIX C:

STUDY METHODS

Appendix C: 133

List of Resources

Study Methods

AIR staff worked with the five study districts to plan visits lasting approximately two days per district, to 

speak with district and school-level staff. Site visit teams, consisting of two AIR staff, reviewed district 

websites prior to the visits to learn about initiatives in the districts. From approximately May through 

August 2010, AIR staff visited the five districts to obtain detailed information from district-level staff 

and union representatives, as well as principals from four schools. We spoke to each respondent 

regarding 

• How the district’s approach to measuring teacher quality was being implemented from the 

perspective of the district, school leaders and teachers 

• How the district, school leaders, and teachers were using the information provided by the 

measures, with a special focus on schools serving disadvantaged students 

Interviews focused on the characteristics of the new measures of teacher quality, interviewees’ views of 

the measures, how they were being used (including successes and challenges), what they were learning 

from the new measures and what results they had observed. Interviews took about one hour each and 

were recorded. Documents were also collected both before and during site visits. Analysis of these 

interviews and extant data contributed to site-specific summaries, which were the basis for report 

writing. These summaries were reviewed by participating districts for accuracy. 

Appendix D: 135 

List of Resources

APPENDIX D:

LIST OF RESOURCES

Appendix D: 137 

List of Resources

List of Resources 

Following are short descriptions of suggested websites that provide additional information about 

identifying effective teachers, building systems to improve teacher effectiveness and taking targeted 

actions to improve equitable access to effective teachers. 

National Comprehensive Center for Teacher Quality

http://www.tqsource.org/

The National Comprehensive Center for Teacher Quality (TQ Center) is a federally funded technical 

assistance center focused on the quality of teaching—especially in high-poverty, low-performing and 

hard-to-staff schools. By using the center’s “filter by topic” or “keyword” functions, users can find 

resources related to teacher effectiveness, teacher evaluation, equitable distribution of teachers, 

professional development and other relevant topics. The TQ Center provides policy briefs and reviews, 

research syntheses, interactive tools and workbooks, and other resources. 

Center for Educator Compensation Reform

http://www.cecr.ed.gov/

The Center for Educator Compensation Reform (CECR) is a federally funded technical assistance center 

designed to raise national awareness of effective strategies for educator compensation reform. The 

website serves as the primary online repository for information, tools, and resources to support Teacher 

Incentive Fund (TIF) grantees in particular. The website also provides brief summaries of TIF grantee 

projects, as well as other compensation reform projects around the country, research syntheses, and 

annotated bibliographies, and policy briefs. 

Institute of Education Sciences Regional Education Laboratories

http://ies.ed.gov/ncee/edlabs

The Regional Educational Laboratory (REL) program consists of a network of ten laboratories that serve 

the educational needs of a designated region by providing access to high-quality scientifically valid 

education research through applied research and development projects, studies, and other related 

technical assistance activities. Using the REL Lookup feature, users can search for publications related to 

teacher effectiveness and equitable distribution of effective teachers. RELs produce information on 

policies in use in their regions, as well as large-scale rigorous research studies. 

The Consortium for Policy Research in Education

http://cpre.wceruw.org/

The Consortium for Policy Research in Education (CPRE) comprises seven universities in partnership to 

conduct research on education reform, policy, and finance. The University of Wisconsin CPRE website 

includes case studies on teacher compensation and links to resources on teacher evaluation. The main 

CPRE website, at http://www.cpre.org, provides additional research reports and policy briefs on teacher 

quality and compensation. 

National Center on Performance Incentives

http://www.performanceincentives.org

The National Center on Performance Incentives (NCPI) is a federally funded center designed to 

investigate the effects of financial incentives for teachers, administrators and schools on the quality of 

teaching and learning. NCPI’s work involves a series of rigorous research initiatives, including 

randomized field trials and evaluations of existing pay-for-performance programs. Resources available 

include reports of large-scale research and evaluation projects, as well as policy briefs related to 

performance incentives. 
 

The Department of Education’s Mission is to promote 

student achievement and preparation for global competitiveness 

by fostering educational excellence and ensuring equal access.

www.ed.gov