District of Columbia Public Schools: Important Steps Taken to Continue Reform Efforts, But Enhanced Planning Could Improve Implementation and Sustainability
Released: 2009-07-23T00:00:00

Letter

Background

The District’s Public School System

The District’s prekindergarten through grade 12 school system is composed of 128 public schools with enrollment for the 2008-2009 school year around 45,200. Historically, DCPS has had several problems that interfere with the education of its students. One primary problem was the dysfunction of the central office. For example, textbooks were not delivered on time or at all, parents complained about the lack of responsiveness of the central office, and teachers were not always paid on time. In addition, data systems were obsolete and inundated with errors, making it difficult to access basic information, such as the number of students enrolled at a school and student attendance rates. Such problems persisted in the D.C. public school system for several years despite numerous efforts to address them. In 1989, a report by the D.C. Committee on Public Education noted declining achievement levels as students moved through grades, the poor condition of the school system’s physical facilities, and the lack of accountability among D.C. agencies for the schools. Recent reports have continued to cite these problems. In 2004, the Council of the Great City Schools reviewed the D.C. school system and cited the continued failure to improve student achievement.

Efforts to improve the District’s schools often included new leadership to head the troubled school system. Over the last 20 years, DCPS has employed more than seven superintendents with an average tenure of 2.9 years. Such frequent changes in leadership may have further complicated efforts to improve student achievement, as each leader may have brought a different cadre of initiatives and goals which were not fully developed or implemented with the constant changes in leadership. In 2006, an analysis of the school system’s reform efforts by a consulting firm found no progress in student achievement and recommended a change in governance to improve student achievement and system-wide accountability.

The Reform Act

In response to the problems facing the District’s public school system, the D.C. Council (the legislative branch of the D.C. government) approved the 2007 Reform Act, which significantly altered the governance of the D.C. public schools. The Reform Act transferred the day-to-day management of the public schools from the Board of Education to the Mayor and placed DCPS under the Mayor’s office as a cabinet-level agency. Prior to the Reform Act, the head of D.C. public schools was selected by and reported to the Board of Education. The Reform Act also moved the state functions into a new state superintendent’s office, established a separate facilities office, and created the D.C. Department of Education headed by the Deputy Mayor for Education. The Deputy Mayor’s Office and the state superintendent’s office are also cabinet-level offices in the D.C. government structure.

Office of the State Superintendent of Education

Although the District of Columbia is not a state, its Office of the State Superintendent of Education serves as the District’s state education agency. Prior to the Reform Act, state functions and local functions were conducted in one office which led to problems with oversight and monitoring. Further, the District was and continues to be on the U.S. Department of Education’s (Education) high-risk list for its management of federal education grants. The Reform Act addressed such issues by clearly separating the two entities. Along with managing, distributing, and monitoring the use of federal funds across DCPS and the public charter schools, the office of the state superintendent has a significant policy role. For example, the state superintendent’s office works collaboratively with the State Board of Education to set standards of what students should learn in all the District’s public schools. In addition, in carrying out NCLBA, the state superintendent’s office is responsible for the state-wide assessment, or standardized test, that measures students’ progress in attaining proficiency and sets annual proficiency targets. The state superintendent’s office also delineates requirements for teacher licensure and, within the guidelines provided by NCLBA, determines the District’s definition of “highly qualified teachers.” In addition to these policy functions, the state superintendent’s office also provides support to D.C. public schools and the public charter schools. For example, the office can offer training and technical assistance on a variety of topics, such as the appropriate use and tracking of federal education funds.

NCLBA and the Recovery Act

In January 2002, Congress passed NCLBA which requires states to focus on increased expectations for academic performance and accountability. Under NCLBA, states are required to establish performance goals and hold schools that receive federal funds under Title I of NCLBA accountable for student performance by determining whether or not they have made adequate yearly progress (AYP). The failure to make AYP, or meet academic targets, for 2 or more consecutive years leads to specific actions that schools must take to improve student academic achievement. These actions, such as developing a school improvement plan or extending the school day, are more intensive the longer the school fails to meet academic targets. After 5 or more consecutive years of failing to meet academic targets, a school must make plans to restructure its governance and implement those plans the subsequent year. NCLBA specifies five options for restructuring schools: reopening as a charter school, replacing all or most of the school staff relevant to the failure to make AYP, contracting with another organization to run the school, turning the operation of the school over to the state, or undertaking another action that would result in restructuring the school’s governance.

NCLBA also establishes a federal requirement for teacher quality. It requires that teachers across the nation be “highly qualified” in every core subject they teach by the end of the 2005-2006 school year. In general, NCLBA requires that teachers have a bachelor’s degree, have state certification, and demonstrate subject area knowledge for every core subject they teach. States also have flexibility to set the requirements that teachers need to meet to demonstrate that they are highly qualified. In March 2008, the state superintendent’s office and the D.C. State Board of Education revised the District’s highly qualified teacher definition to better align it with NCLBA’s definition and allow more teachers to be considered highly qualified. Officials from the state superintendent’s office contend that the District’s previous highly qualified definition was more stringent than federal standards and disqualified good teachers from joining the D.C. public school system.

The Recovery Act was enacted in February 2009 to promote economic recovery, make investments, and minimize and avoid reductions in state and local government services. About $100 billion of the $787 billion funds included in the Recovery Act are targeted to support education at the state and local level. Some of the Recovery Act funds support existing programs, such as Title I of the Elementary and Secondary Education Act, as amended by NCLBA, and parts of the Individuals with Disabilities Education Act. In addition, the new State Fiscal Stabilization Fund provides funds to restore state support for elementary and secondary education, public higher education, and early childhood education programs and services. The District will receive an estimated $148 million of Recovery Act funds to support its education programs.

Pay for Performance Initiatives

The current teacher compensation system used by most school districts in the United States dates back to the 1920s and pays teachers based on their level of education and years of experience. However, many school districts have begun to experiment with alternative methods of compensation that reward teachers on certain elements of performance, such as improving student achievement, filling hard-to-staff positions, and taking on additional responsibilities. Some school districts offer bonuses for all staff or all teachers at schools who have met certain criteria (usually including an increase in student achievement). Other school districts offer differentiated pay to teachers based on characteristics other than education and years of experience. For example, the Denver Public School District has implemented a teacher compensation plan that allows multiple pathways to compensation bonuses. Bonuses can be based on professional evaluations using a standards-based system, progress toward objectives as agreed upon by teachers and their principal, and growth in student achievement on the Colorado Student Assessment Program. Teachers may receive additional incentives for filling hard-to-staff positions. The Denver plan is funded through a tax levy, federal grants, and private funding.

National teachers’ unions approve of some types of differentiated or incentive pay. Specifically, the American Federation of Teachers, which is the parent union of the Washington Teachers’ Union, has taken the position that teacher compensation plans could include financial incentives to teachers who acquire additional knowledge and skills or agree to teach in low-performing and hard-to-staff schools. In addition, the American Federation of Teachers supports incentive pay for school-wide improvement.

Early Initiatives Are Focused on Improving Student Achievement and DCPS Is Modifying Its Approach as It Moves Forward

During the first 2 years of its reform efforts, DCPS implemented several classroom-based initiatives to improve students’ basic skills in core subjects and implemented a new staffing model designed to give all students access to art, music, and physical education classes. In addition, as required by NCLBA, DCPS restructured 22 schools before the fall of 2008, after the schools failed to meet academic targets for 6 consecutive years. Restructuring will be ongoing as the vast majority of DCPS schools are in some form of school improvement status under NCLBA. In addition, DCPS and the state superintendent’s office are planning and developing new ways to use data to monitor student achievement and school performance. DCPS is refocusing or revising its approach to many of these initiatives as it continues to implement them.

DCPS Quickly Implemented Many Separate Initiatives to Improve Overall Student Performance and DCPS Is Refocusing Its Approach as It Moves Forward

During the first 2 years of reform, DCPS quickly implemented various initiatives intended to improve student achievement. For example, to improve students’ basic skills and standardized test scores in reading and math, DCPS introduced targeted interventions for students struggling in math and reading and provided additional instruction and practice to improve students’ responses to open-ended questions, including test questions. DCPS also introduced Saturday classes primarily targeted to students in grades 3 through 12 who were on the cusp of meeting academic targets on standardized tests. It also introduced initiatives designed to address student motivation and behavior. For example, DCPS piloted the Capital Gains program with the specific goals of improving student engagement, and ultimately student learning, by offering financial incentives to students for attendance, academic performance, and other positive behaviors. Table 1 provides a list of DCPS’s major initiatives to improve student outcomes, as well as descriptions and the status of these initiatives.

Recently, the Chancellor acknowledged that DCPS, in its effort to remedy the range of issues that plagued the District’s public schools, may have launched too many initiatives at once. The Chancellor noted that some schools may have lacked the capacity to implement so many programs effectively. In particular, some schools were undergoing significant organizational changes that may have affected their ability to implement these new academic initiatives. To support such schools, DCPS is considering offering a choice of programs for schools and allowing the principals to determine which programs best suit their schools’ needs and capacity.

DCPS does not yet know how successful these programs have been in improving student achievement. While DCPS students achieved gains on the 2008 state-wide test, increasing between 8 and 11 percentage points in math and reading for both elementary and secondary levels, it is unclear whether these gains can be attributed to the current reform efforts or to prior efforts. While DCPS officials told us that it is generally difficult to isolate and quantify the impact of any single program on student achievement, they were able to review an analysis of reading scores conducted by the vendor of one of its early reading programs. The vendor’s analysis showed that on some tests DCPS students who participated in the reading program generally scored higher than those who did not. Further, DCPS officials told us they plan to analyze, in late summer of 2009, student outcomes, including state-wide test scores, to assess the effectiveness of various interventions.

In addition, DCPS officials told us the success of the math and reading initiatives depended in part on how well teachers implemented them in the classroom. They also noted that there were varying levels of teacher quality and knowledge of effective teaching practices, and that it was difficult to ensure the extent to which teachers implemented the programs effectively. While DCPS had not defined “effective” teaching prior to the rollout of the above initiatives, officials told us that moving forward, they will focus on practicing effective teaching, as opposed to implementing various disparate programs. DCPS is developing a framework that is intended to help teachers understand the priorities moving forward, including understanding what students are expected to learn for each subject, how to prepare lessons, and effective teaching methods to be used. According to DCPS officials, this framework will be aligned to teacher evaluations. DCPS plans to implement this framework by the beginning of the 2009-2010 school year.

DCPS Encountered Challenges Allocating Teachers across Schools and Is Revising Its Approach for the 2009- 2010 School Year

In an effort to ensure that all students would have access to certain subjects and supports, DCPS changed the way it allocated teachers across its schools for the 2008-2009 school year. This new staffing model was intended to provide all schools with a core of teachers including art, music, and physical education, as well as social workers. It also was intended to provide all schools with reading coaches who work with teachers to improve reading instruction. Prior to this change, DCPS allocated funding to schools using a weighted student formula, which distributed funds to schools on a per pupil basis, so that the greater the enrollment of a school, the greater the amount allocated to that school. Principals then chose how to staff the school based on the amount of funding available, staffing requirements, and their perception of the school’s needs. Consequently, some schools—especially smaller schools— did not have the student enrollment to support programs, such as music and art, and other schools that had the funds to support those programs opted not to do so. While the new staffing model ensures a core staff at all schools regardless of enrollment, DCPS allowed principals to request changes based on their school’s needs. However, DCPS lacked a transparent process for making changes to the staffing allocation. In particular, DCPS did not establish or communicate clear guidance or criteria on how such requests would be treated. Further, DCPS granted or denied requests for changes to the original staffing allocation on a school- by-school basis, and it is unclear whether similar requests were treated in a consistent manner. A more transparent process, one that made public their rationale for decisions, would have helped assure stakeholders, including the D.C. Council, that changes to staffing allocations were made consistently and fairly. The D.C. Council and several community groups have criticized the process for its lack of transparency and questioned the fairness of the decisions made. For example, one independent analysis concluded that some schools received less per pupil funding than others with similar student populations. In addition, DCPS officials told us that in some cases, the changes to the original staffing model resulted in schools being granted allocations beyond their budgeted amounts.

DCPS revamped its approach for the staffing model for the 2009-2010 school year to address some of these challenges. For example, it established guidance about what changes it will allow principals to make to the staffing model and disseminated this guidance to school leaders at the beginning of the budgeting process. According to DCPS, the new guidance is expected to reduce the number of changes that principals request later in the process.

DCPS Closed 23 Schools Primarily Due to Low Enrollment, Restructured 22 Other Schools as Required by NCLBA, and Is Changing Its Process for Selecting Restructuring Options

During the summer of 2008, DCPS closed 23 schools primarily due to low student enrollment. Students from the closed schools, about 5,000 students according to DCPS, enrolled in 1 of 26 schools, referred to as receiving schools. DCPS updated facilities at these receiving schools to accommodate the influx of students from the newly closed schools. In addition, to assist these students and schools with the transition that this reorganization created, DCPS offered a more comprehensive version of its staffing model. In addition to the core staff of the standard staffing model, DCPS allocated additional staff, such as school psychologists and math coaches to the receiving schools. During the consolidation effort, DCPS also created several prekindergarten through grade 8 schools in some cases where elementary schools were underenrolled. In addition, according to DCPS, these prekindergarten through grade 8 schools were intended to create a smoother transition to middle school and reduce the number of elementary schools with different grade levels preparing students for the same middle or junior high school. By closing the 23 underenrolled schools, DCPS estimates it was able to redirect $15 million from administrative and facility costs to support these additional staff. The eight principals we interviewed at receiving schools provided mixed reports about the adequacy of their staffing allocations. On the one hand, three principals reported having adequate staff, and two others cited minor issues. The remaining three principals cited issues such as teacher skill levels, teacher vacancies, and inadequate training to accommodate an influx of special education students.

In addition, as required by NCLBA, DCPS restructured 22 of its lowest performing schools for the 2008-2009 school year after the schools failed to meet academic targets for 6 consecutive years. NCLBA specifies five options for restructuring schools, including replacing selected staff or contracting with another organization or company to run the school (table 2 lists the various NCLBA options and the options DCPS selected for the 2008-2009 school year). At 18 of the 22 schools in restructuring, DCPS replaced the school staff—principals, teachers, and/or administrative support staff—who were deemed relevant to the failure to meet academic targets. For the remaining schools in restructuring, DCPS elected to contract with other organizations or undertake other actions, such as adding more intensive school-level services to support students and families.

Restructuring underperforming schools will likely be an ongoing initiative for DCPS, as 89 of its 118 schools are in some form of school improvement status. (See fig. 2 for more details on DCPS’s school improvement status.)

DCPS revamped its process for determining the most appropriate restructuring option for the 13 schools that will be restructured in the 2009-2010 school year. Prior to implementing the first round of restructuring (i.e., for the 2008-2009 school year), DCPS officials told us there were insufficient school visits and inadequate training and guidance for teams assigned to evaluate which restructuring option was best suited for a given school. For example, the initial process called for review teams to visit each school once, which according to DCPS officials, did not allow the teams to obtain sufficient evidence to evaluate the schools’ condition. DCPS has addressed these issues by requiring two visits to each school, offering more training, and revising the form used to evaluate each school’s condition for the next round of restructuring.

In addition, DCPS officials told us they cannot continue to rely on replacing teachers and principals as the primary restructuring option because DCPS cannot terminate the teachers, and moving these teachers to other schools may undermine the District’s reform efforts. DCPS did not assess its capacity for replacing staff at schools restructured in the 2008- 2009 school year. According to DCPS, nearly half of the 160 teachers that were removed from these schools had to be placed at 38 other DCPS schools. For the 2009-2010 school year, DCPS has decided to replace select staff at 6 of the 13 schools that will be restructured. (For more details, see the section on teacher and principal quality later in this report.)

DCPS and the State Superintendent’s Office Are Working to Enhance and Create Data Systems to Monitor Student and School Performance

DCPS reported it has ongoing and planned initiatives to expand data access to principals and teachers, in part to monitor student and school performance. In particular, DCPS reported it made improvements to its primary student data system so central office users can better monitor school performance. For example, DCPS officials reported that they consolidated several student data systems by February 2009, including the system containing standardized test scores, into the primary student data system with the intent to improve data accuracy and consistency. They also told us they added software to the primary student data system that enabled central office employees to develop monthly reports of schools’ performance data, such as attendance and test scores. DCPS plans to eventually use these monthly reports to enable school leaders to better monitor student progress, and plans to develop an internal Web site that compiles various student and school information in one place for key stakeholders including central office staff and principals. However, DCPS officials told us they have delayed some of these efforts while they attempt to improve coordination among the various departments that were developing and disseminating information to school leaders. DCPS has not yet announced when the project will be completed. See table 3 for more details about key DCPS data initiatives and their status.

The state superintendent’s office also is developing a longitudinal database, called the Statewide Longitudinal Education Data Warehouse (SLED) that is intended to allow DCPS and other stakeholders to access a broad array of information, including standardized test scores of students and information on teachers. SLED is intended to allow the District to track student registration and movement among DCPS’s schools and the public charter schools more accurately, as well as expand the District’s ability to monitor student achievement and growth over time. According to officials in the state superintendent’s office, they revised the project schedule to allow more time to assist the charter schools with updating their data systems. In February 2009, the initial release of student data provided a student identification number and information on student eligibility for free or reduced-price lunches and other student demographics for all students attending DCPS’s schools and the public charter schools. The state superintendent’s office plans for SLED to enable DCPS to link student and teacher data by February 2010. (See table 4 for more details about the status of key SLED deliverables.) This link is to provide DCPS with data on the classes students enrolled in, the teachers that taught the classes, any academic interventions students received, students’ grades and test scores, and student demographics.

DCPS Replaced Teachers and Principals and Introduced Professional Development Initiatives, but Encountered Challenges in Implementation

DCPS is attempting to improve the quality of its teacher and principal workforce by hiring new teachers and principals and by providing professional development. After the 2007-2008 school year, about one-fifth of the teachers and one-third of the principals resigned, retired, or were terminated from DCPS. However, DCPS officials told us that the 2007-2008 and 2008-2009 teacher evaluation process did not allow them to assess whether the teacher workforce improved between these 2 school years. In addition, DCPS introduced professional development initiatives for teachers and principals, but late decisions about the program for teachers led to inconsistent implementation.

DCPS Focused on a Workforce Replacement Strategy to Strengthen the Quality of Teachers and Principals, but Is Unsure New Staff Are an Improvement

DCPS focused on a workforce replacement strategy to strengthen teacher and principal quality. DCPS maintains that the quality of teachers is the single greatest determinant to improving student achievement, and a growing body of research has shown that teacher quality is a significant factor in improving student academic performance. Yet it is often difficult to remove teachers for performance issues beyond their initial, or probationary, years in a given school system. For example, in the 2006- 2007 school year, only 1 teacher was removed from DCPS for poor performance out of more than 4,000 teachers. Representatives from the Washington Teachers’ Union agreed that there were several poor performing teachers in DCPS, but stated that the 2-year probationary period is the appropriate time to identify and dismiss poor teachers at will.

DCPS began implementing its teacher replacement strategy near the end of the 2007-2008 school year. Specifically, about one-fifth of the teachers and one-third of the principals resigned, retired, or were terminated from the school system at the end of the 2007-2008 school year. DCPS terminated about 350 teachers, approximately 100 of whom were released for underperformance at the end of their probationary period, when tenure decisions were made. The remaining 250 teachers were terminated because they did not meet specified time frames to become highly qualified under NCLBA. An additional 400 teachers accepted financial incentives offered by DCPS to resign or retire in the spring of 2008. A DCPS official told us there is anecdotal evidence suggesting DCPS lost some quality teachers through the contract buyouts, but officials noted that DCPS did not have measures in place to deter effective teachers from accepting the buyouts. In addition, DCPS did not renew the contracts of 42 principals, citing their failure to improve student achievement on standardized tests and to adequately implement school-wide programs.

To replace the teachers and principals who left the system, DCPS launched a nationwide recruitment effort for the 2008-2009 school year. DCPS hired 566 teachers and 46 principals for the 2008-2009 school year. Of the 566 teachers, 395 were hired from traditional backgrounds or other school systems and 171 came from nontraditional paths such as the D.C. Teaching Fellows program and Teach for America. (See fig. 3 for more details about the flow of teachers into and out of DCPS between the 2007- 2008 and 2008-2009 school years.)

However, DCPS did not have a new teacher contract in place due to ongoing negotiations with the Washington Teachers’ Union and officials told us this may have hindered their efforts to attract top-quality teachers. The Chancellor has stated that she wants to recruit and retain quality teachers by offering merit pay, which would reward teachers with higher salaries based, in part, on their students’ scores on standardized state tests. Under the plan, which has been in negotiation with the Washington Teachers’ Union since November 2007, teachers could voluntarily relinquish job protections in exchange for base salaries and bonuses totaling over $100,000 per school year. This plan relies on over $200 million in contributions from private foundations to fund the teacher contract, including salary increases and professional development. According to the Chancellor, private foundations continue to pledge their support, even with the current economic downturn. DCPS officials told us the higher annual salaries and bonuses would be sustainable with public funds if private funding is not available when the 5-year contract expires.

In addition, an official told us DCPS does not have an adequate means to assess whether its teacher workforce improved between the 2007-2008 and 2008-2009 school years because the current teacher evaluation system is not an effective way to assess teacher performance. Under this evaluation system, principals evaluate teachers’ subject matter knowledge, classroom management skills, and adherence to academic standards, among other elements. However, this system does not measure teachers’ impact on student achievement, which, according to DCPS, is a key factor in evaluating teacher effectiveness. In addition, according to DCPS, teacher evaluations conducted in prior years did not adequately distinguish excellent from poor performance—almost all teachers received satisfactory ratings. As a result, DCPS officials told us they cannot determine the quality of the 566 new teachers relative to the 817 teachers who left the system.

The current teacher evaluation system remains the primary mechanism for identifying teachers considered ineffective. During the 2008-2009 school year, principals used the evaluation system to place 147 tenured teachers deemed underperforming on 90-day improvement plans. At the end of 90 school days, principals decide whether to retain or terminate these teachers. In prior years, DCPS did not use the 90-day process to this extent.

DCPS plans to revise its teacher evaluation process to more directly link teacher performance to student achievement. The proposed system includes a value-added component that would measure teachers, in part, on their ability to improve students’ standardized test scores over the course of a school year. This value-added measure would only apply to about 20 percent of the teacher workforce, since not all grades and subjects are tested. DCPS plans to use a less formal student achievement measure for teachers in nontested grades and subjects in the short term, but is working to increase the number of teachers for whom student achievement growth data are available. In addition, DCPS’s proposed evaluation system would add classroom observations by third-party observers, called master teachers, who would be knowledgeable about teaching the relevant subject matter and grade level, to supplement school administrators’ observations of teachers. To solicit input on the proposed evaluation system, the Chancellor held a series of sessions in spring 2009 with teachers, teacher coaches, and other school staff, and engaged the Washington Teachers’ Union. DCPS officials told us that the feedback was generally positive and that teachers found the proposed evaluation system to be fair, transparent, and an improvement over the current evaluation.

However, some teachers were concerned about using students’ test scores as part of the evaluation.

For the 2007-2008 school year, DCPS revised the principal evaluation system, which holds principals accountable for improvements in students’ standardized test scores and achieving other standards. DCPS will be able to use this evaluation system to determine if principals performed better during the 2008-2009 school year than in 2007-2008.

DCPS Introduced Teacher Coaches and a Principals’ Academy to Provide Professional Development and Improve Skills, but Encountered Challenges in Implementation

In addition to the workforce replacement strategy, DCPS changed the way in which it develops its teacher workforce. DCPS began placing teacher coaches in schools to help teachers increase student achievement at their workplaces. Previously, DCPS’s teacher training was not systematic or aligned with the school district’s goals. For the 2008-2009 school year, DCPS hired about 150 teacher coaches to improve teachers’ skills in delivering reading and math instruction and boost student test scores. DCPS officials told us their decision to implement school-based teacher coaches was based on research demonstrating gains in student achievement as a result of teacher coaches collaborating with teachers to improve instruction. For the 2008-2009 school year, teacher coaches focused on helping new teachers and teachers with students in grades 3 through 10 in reading and math instruction. For example, teacher coaches, at the direction of principals, assisted teachers with interpreting student test scores, planning lessons, and using their classroom time constructively. DCPS is planning for teacher coaches to work with teachers in all grades and subjects for the 2009-2010 school year.

Late hiring of teacher coaches, however, affected the implementation of the professional development plan for the 2008-2009 school year. DCPS officials told us they made the decision to hire teacher coaches after their review of school restructuring plans in June 2008. DCPS officials told us that, as a result of this late decision, they were unable to adequately recruit a sufficient number of qualified staff to fill these positions. Specifically, qualified teacher coach applicants had accepted jobs elsewhere, since many school systems recruit staff from February through April.

DCPS intended to staff about 170 teacher coaching positions, however, as DCPS began the 2008-2009 school year, about 20 percent of the coaching positions remained open (19 reading coach vacancies and 16 math coach vacancies). As of late January 2009, there were 157 teacher coaches working on-site in the District’s public schools, with 14 total vacancies. Each vacancy represents a school without the full support (either a reading coach or both a reading coach and a math coach) that DCPS wanted to provide. As a result, the ratio of teachers to coaches was higher than it would have been had the positions been filled. In addition, according to DCPS officials and Washington Teachers’ Union officials we interviewed, teacher coaches were often unclear on their responsibilities and how to work with teachers, and received some conflicting guidance from principals. For example, these officials told us that some principals did not assign teacher coaches to their intended position. At the beginning of the school year, some principals assigned coaches to cover classes for absent teachers or to evaluate teachers—a practice not allowed under union rules—meaning the coaches were not able to work with teachers.

DCPS is also seeking to improve the quality of principals through the Principals Academy developed for the 2008-2009 school year. Consistent with DCPS’s belief that principals should be their schools’ instructional leaders, the academy’s goals include improving principals’ leadership skills, helping them interpret student test scores, and providing advice on how to use this information to improve their schools. The Principals Academy convenes monthly and also includes differentiated professional development workshops based on principals’ individual needs.

The State Superintendent’s Office and DCPS Have Developed and Begun Implementing Strategic Plans; However, DCPS Has Not Always Involved Relevant Stakeholders in Planning and Implementing Key Initiatives

The state superintendent plan is a “state-level” strategic plan that covers the District’s public schools (and public charter schools). This plan and DCPS’s strategic plan each contain elements GAO has identified as key to an effective plan, such as aligning short-term objectives to long-term goals in order to delineate how to attain those goals. While DCPS has recently increased efforts to involve stakeholders such as parents and the D.C. Council in key initiatives, past stakeholder involvement was inconsistent. DCPS has not yet developed a method for ensuring more consistent stakeholder involvement.

Both the State-Level and DCPS Strategic Plan Were Developed with Stakeholder Involvement and Contain Many Elements of Effective Plans

The state superintendent’s office and the State Board of Education collaboratively developed the District’s state-level, 5-year strategic plan, and released it in October 2008. This state-level plan spans early childhood and kindergarten through grade 12 education (including public charter schools). The plan was developed with stakeholder involvement throughout the process. Officials from the state superintendent’s office told us they involved District officials, and stakeholders representing early childhood education, business, and higher education communities, as well as other stakeholders while drafting the plan. In particular, they told us they involved DCPS and the D.C. Deputy Mayor of Education’s Office in discussions of the plan. In addition, in September 2008, the state superintendent’s office held one public forum to solicit stakeholder input on the draft of the document, and accepted comments on the draft on their Web site. The office released a revised version of the plan within a month of the public forum. Stakeholder involvement in formulating strategic plans allows relevant stakeholders to share their views and concerns. In addition, it affords stakeholders a way to understand the rationale for certain decisions. Ultimately, stakeholder involvement can result in increasing stakeholder support, or ownership, of the strategic plan.

The state-level plan details the state-level strategy for improving education in the District and delineates accountability measures for DCPS and the public charter schools. In addition, the state-level plan states the mission, vision, and goals of the agency. It includes three broad, long-term goals: to have all children ready for school, all schools ready to prepare students for success, and all District residents ready to be successful in the 21st century economy. Overall, the plan includes many key elements of an effective strategic plan such as the inclusion of objectives that delineate how the state superintendent’s office intends to attain each of its goals. The short-term objectives are supported by various strategies, objective measures, and performance targets. For example, one objective under the goal of having the District’s schools ready to prepare students for success is to ensure that all students receive rigorous instruction. This objective is broken down into objective measures, such as the percentage of elementary students scoring proficient or above on the state test. Further, the plan specifies annual performance targets for this objective for the years 2008 to 2013. See table 5 for more details on the elements of the state-level strategic plan.

DCPS released the draft of its 5-year strategic plan in late October 2008. In contrast to the state-level plan which includes the public charter schools, the DCPS plan is specific to prekindergarten through grade 12 education at its 128 schools. DCPS officials told us they based the draft on the Master Education Plan, which the prior DCPS administration developed with stakeholder involvement, and that they sought additional stakeholder input through a series of town hall meetings. After releasing the draft, DCPS held three public forums in the following 3 weeks where attendees provided DCPS officials with feedback on the draft strategic plan. In May 2009, DCPS released the revised draft, which incorporated stakeholder feedback.

The DCPS 5-year strategic plan outlines the organization’s vision and goals, and includes many elements of an effective strategic plan. For example the plan explains how DCPS’s six broad goals are interrelated and how they support the vision. (Table 5 lists the six DCPS goals).

In addition, the DCPS plan describes the condition of DCPS prior to the reform effort, the progress made to date, and the steps needed to achieve the long-term goals. However, the DCPS plan does not systematically delineate measurable outcomes with clear time frames and does not always identify key external factors that could increase the risk that an initiative may fail. For example, several objectives are aimed at improving teacher quality; however, the plan lacks specific targets for measuring the expected magnitude of such an improvement. Without such targets, it will be difficult for the public to evaluate DCPS’s progress toward improving its teacher workforce. In addition, while the strategic plan discusses increased performance-based pay for teachers, it does not specify the cost or explicitly mention the reliance on outside funding streams to achieve the increases. Yet, the reliance on outside funding for the initial 5 years is a risk that is not within DCPS’s control. Table 6 contains some key elements of the state-level and DCPS’s strategic plans.

Officials from the D.C. Deputy Mayor of Education’s office told us that as part of their office’s coordinating role, it ensured that DCPS and the state- level strategic plans were aligned. However, the office had no documentation showing its efforts to coordinate these plans, such as an alignment study. We found that the two plans were aligned in terms of long-term goals. For example, DCPS’s goals could support the state-level goal of having all schools ready. However, we could not evaluate whether more detailed, objective measures and performance targets were aligned because the DCPS strategic plan did not always include specific objective measures and performance targets.

DCPS Has Recently Increased Its Efforts to Involve Stakeholders in Various Initiatives, However It Has Not Systematically Included Stakeholders

DCPS officials have several planned and ongoing efforts to involve stakeholders in planning, implementing, and evaluating various initiatives. Stakeholder involvement can be instrumental in these areas because stakeholders can bring different knowledge, points of view, and experiences to planning and implementing reform efforts. DCPS officials told us they have a variety of approaches to involve stakeholders, including parents, students, and community groups, as well as institutional stakeholders such as the D.C. Council. For example, DCPS officials told us they reach out to parents, students, and the public by holding monthly community forums, meeting with a group of high school student leaders and a parent advisory group, responding to e-mail, and conducting annual parent and student surveys to gauge the school system’s performance. DCPS introduced monthly community forums in July 2008. These forums were generally informational sessions on topics chosen by DCPS officials, and were followed by questions from the audience. In some cases, such as the three forums focused on the strategic plan, DCPS officials facilitated discussions to elicit feedback. DCPS officials told us their efforts to involve students in reform efforts included a student leadership group that met regarding student concerns, and which was credited by DCPS officials for changes in the school lunch program as well as substantial changes to the discipline policy. DCPS also involved other stakeholders, such as parent organizations and the Washington Teachers’ Union in its process of changing the discipline policy. In addition, DCPS officials cited the Chancellor’s response to e-mail communications as a form of stakeholder involvement. While such communications may have provided stakeholders with a means of connecting to the Chancellor, e-mail communications are generally not public and do not lead to public debate or discourse.

In spring 2008, DCPS also conducted parent and student surveys to assess stakeholder satisfaction with DCPS schools. While DCPS officials told us they have completed the analysis of the parent survey, they have not yet released the results. Further, DCPS did not receive the student survey data until February 2009 due to complications with a vendor who was paid to collect these data. As a result of the delays, DCPS officials told us they have been unable to use student survey responses to inform decisions relevant to the 2008-2009 school year. However, officials said they will be able to use the information as a baseline for future surveys.

However, such activities do not ensure systematic stakeholder input in planning, implementing, and monitoring key initiatives. During our review, DCPS officials told us that stakeholder involvement was important to their reform efforts and that DCPS was taking steps to increase stakeholder involvement. However in some cases, according to two DCPS officials, DCPS did not have a planning process in place to ensure systematic stakeholder involvement, and we found that DCPS implemented some key initiatives with limited stakeholder involvement. For example, key stakeholders, including D.C. Council members and parent groups, told us they were not given the opportunity to provide input to inform DCPS’s initial proposals regarding school closures and consolidations, although DCPS did hold numerous meetings after the initial proposal, before finalizing decisions. Similarly, stakeholders told us DCPS did not include them in deliberations and decisions about the establishment of prekindergarten to grade 8 models at some schools. Representatives from one community organization told us that some parents had concerns about the structure and academic setting at the prekindergarten to grade 8 schools, but did not have a venue to express those concerns before decisions about grade configurations were made.

In addition, DCPS did not seek input from key stakeholders during the planning and early implementation of the new staffing model that placed art, music, and physical education teachers at schools and which fundamentally changed the way funding is allocated throughout DCPS. DCPS officials told us that they had not planned for the number of changes that were requested by principals. In particular, they told us that the vast majority of school principals requested changes to their initial staffing allocations. Stakeholders did not have a timely opportunity to raise concerns on the potential risks in implementing the staffing model, such as the uneven distribution of resources across schools and overspending at some schools. Stakeholders also said they were not given sufficient time to review the budget for the 2008-2009 school year or to understand the changes in the budget made after the school year began. DCPS officials told us the budget planning process for the 2010-2011 school year involved stakeholders extensively. In particular, DCPS invited the public to a preliminary budget meeting and also provided training on the budget process to some key stakeholders, such as school principals and community members.

Lack of stakeholder involvement in such key decisions led stakeholders, including the D.C. Council and parents groups, to voice concerns that DCPS was not operating in a transparent manner or obtaining input from stakeholders with experience relevant to the District’s education system. Further, these stakeholders have questioned whether the impact of reform efforts will be compromised because of restricted stakeholder involvement. Stakeholders from other urban school districts we visited told us a lack of stakeholder involvement leads to less transparency as key decisions are made without public knowledge or discourse. In addition, the lack of stakeholder involvement can result in an erosion of support for ongoing reform efforts and poor decisions. For example, officials in Chicago and Boston said public stakeholder involvement was critical to community support for various initiatives, such as decisions on which schools to close. Officials and stakeholders in New York cited lack of stakeholder involvement in decisions that were eventually reversed or revised. For example, changes made to school bus routes without consulting parents meant several route changes were later reversed because they proved to be unworkable.

DCPS and the State Superintendent’s Office Have Taken Steps to Improve Accountability and Performance, and DCPS Has Yet to Align Key Aspects of Its Performance Management System to Organizational Goals

DCPS and the state superintendent’s office have taken steps to improve accountability and performance of their offices. For example, both offices have started implementation of new individual employee performance management systems. While DCPS has taken steps to improve accountability and link its individual performance management system to organizational goals, it has not completed this process or used the results of surveys to improve central office operations.

Both DCPS and the State Superintendent’s Office Developed and Implemented a Performance Management System, and DCPS Has Not Yet Aligned Key Aspects of the System to Organizational Goals

To increase accountability of its central office, DCPS developed an accountability system and an individual performance management system for central office departments and employees. The central office, which is responsible for providing academic and nonacademic supports to DCPS, had operated without such accountability systems prior to the recent reform efforts. For example, previously, performance evaluations were not conducted for most DCPS staff. As a result, central office employees were not held accountable for the quality of services they provided to support schools.

To improve accountability for central office departments, DCPS developed departmental scorecards, as a part of its performance management system, to identify and assess performance expectations for each department. For example, the scorecard for the Office of Data and Accountability includes measures such as the number of users of the primary student data system. According to a DCPS official, these scorecards are discussed at weekly accountability meetings with the Chancellor to hold senior-level managers accountable for meeting performance expectations. For example, at the accountability meeting we attended, DCPS officials from the Office of Data and Accountability used scorecards to discuss their progress with collecting attendance data and setting up processes to strengthen the collection of these data. According to DCPS officials, some departmental leaders have established similar accountability meetings with their staff, although these are not required.

In January 2008, DCPS implemented a new performance management system for employees. Performance management systems for employees are generally used to set individual expectations, rate and reward individual performance, and plan work. DCPS developed its new performance management system in an effort to improve support services to the schools by improving the accountability and performance of central office employees. In particular, in past school years, teachers complained about not getting paid on time and beginning the school year with inadequate supplies. DCPS’s performance management system was put in place, in part, to improve these functions in the central office.

While DCPS developed and instituted a new performance management system, it did not fully align individual performance expectations and evaluations to organizational goals, which GAO has identified as a key practice of effective individual performance management systems. For example, while DCPS took important steps in developing and implementing its system, such as training department managers to set expectations and give feedback to employees, DCPS has not yet established a uniform policy for setting expectations. Further, DCPS has not yet instituted a system to track how and when such expectations are set. Instead, individual managers established processes specific to their office or department and, as a result, DCPS could not ensure that individual performance expectations were aligned to organizational goals as outlined in the DCPS 5-year strategic plan or in its annual performance plans. Without such alignment, employees may not be familiar with the overall organizational goals and their daily activities may not reflect these goals. An explicit alignment of daily activities with broader desired results helps individuals connect their daily activities and organizational goals and encourages individuals to focus on their roles and responsibilities to help achieve the broader goals. In addition, as we previously reported, DCPS developed individual performance evaluations in December 2007 as a part of its performance management system in order to assess central office employees’ performance. Such individual performance evaluations are used to rate central office employees on several core competencies twice a year. For example, employees are rated on how well they demonstrate a commitment to providing high-quality and timely customer service to both external and internal customers of District schools. Prior to our March 2008 testimony, DCPS officials told us that they intended to align the performance management system with organizational goals by January 2009, and DCPS has taken some steps to improve alignment. For example, DCPS officials told us they had better aligned their departmental scorecards to their 2009 annual performance plan. However, DCPS has not yet explicitly linked employee performance evaluations to the agency’s overall goals. DCPS officials told us they plan to link the individual performance evaluations with organizational goals in the summer of 2009 to ensure greater accountability in supporting schools.

The state superintendent’s office also implemented a new performance management system, effective October 2008, to hold its employees accountable and improve the office’s performance. The office is converting to a single electronic management system to track and evaluate employee performance. This new system, scheduled to be fully operational by December 2009, will replace the two separate systems that had operated on different cycles. According to an official from this office, the new system is uniform, user friendly, and allows for an easier transfer of performance information from manager to employee. In addition, this system links individual employee evaluations to overall performance goals and the office’s strategic plan. Under this new evaluation system, each employee is given a position description, which includes responsibilities and duties linked to the overall goals, mission, and vision of the state superintendent’s office. Individual and agency expectations are defined in an annual performance meeting with the employee. The office is currently training supervisory employees on how to use the system before its full implementation in December 2009.

DCPS Surveyed Staff at Schools to Measure Satisfaction with Central Office Services; However, DCPS Has Not Yet Used Survey Results to Improve Operations

In November 2007, DCPS conducted a survey of employees within District schools, including teachers and principals, to gauge satisfaction with District services, including central office services during the 2007-2008 school year. Personnel at the schools are key stakeholders in improving central office functions, and their feedback is important to help DCPS ensure resources are targeted to the highest priorities. The American Institutes for Research partnered with DCPS to administer the online survey of teachers, principals, aides, clerks, counselors, project directors/coordinators, related service providers, and other staff. They were asked to provide feedback on numerous topics, including the work environment, facilities and maintenance, professional development, and leadership, as well as central office services. With regard to central office services, the survey’s questions were focused on personnel services, budget and procurement services, district departments and support services, food and nutrition services, and technology and data.

Of those staff that completed the survey, more were satisfied with their schools, such as their work environment and fellow staff members, than with the support system provided by the central office. For example, they were least satisfied with the central office’s ability to provide goods and services in a timely manner, compute paychecks accurately, and allot budgeted funds when needed. In addition, staff who completed the survey were least satisfied with facilities office’s responsiveness to requests for school repairs, saying they were not completed in a timely manner. DCPS officials told us the results of the survey were shared internally with different central office departments in 2008, and focus groups were formed within a month of the release of the survey results to develop specific action plans to address identified issues. However, DCPS officials were unable to provide us with specific examples of improvements made in central office operations as a result of the survey. Three of the eight principals we met with regarding the school consolidation process stated that they could not always access budgeted funds when needed. In addition, four of the eight principals noted that school repairs were not made in a timely manner. One principal told us his payroll was often inaccurate, and some teachers were not always paid on time. DCPS officials told us another staff survey will be administered in spring 2009.

Conclusions

The challenge of reforming DCPS is daunting. NCLBA requires 100 percent proficiency by 2014 and the District’s students scored significantly lower than the District’s own proficiency targets for 2008 and below students in most other urban districts. In the past, support for reform efforts has waned as student achievement did not improve, as buildings deteriorated, and as new superintendents were ushered in every few years to address these problems.

The need for rapid reform and results is acute and the District’s Mayor and his education team have taken bold steps—such as implementing various classroom-based initiatives, reorganizing schools, and replacing teachers and principals—to improve the learning environment of the District’s students and ultimately increase student achievement. However, DCPS lacks certain planning processes, such as communicating information to stakeholders in a timely manner and incorporating stakeholder feedback at key junctures, which would allow for a more transparent process. In addition, DCPS did not gauge its internal capacity prior to implementing certain key initiatives, which, if addressed in the future, could help ensure the sustainability of initiatives. Without these planning processes, an organization risks having to revamp initiatives, leading to delays and compromising the implementation of timely, critical work. While having these planning processes in place will not eliminate all implementation issues, it will help to identify and mitigate risks associated with implementing bold initiatives and identify needed changes in the early stages of the initiative. Furthermore, a lack of these planning processes can result in decisions that are made on an ad hoc basis with resources unevenly distributed as was the case with the District’s new staffing model. Ultimately, the lack of such processes while planning and implementing initiatives has impeded the success of some of DCPS’s initiatives and could impede the District’s continued success and progress in reforming its school system.

Stakeholder consultation in planning and implementation efforts can help create a basic understanding of the competing demands that confront most agencies and the limited resources available to them. Stakeholders can then share their expertise and experience, and views on how these demands and resources can be balanced. Continuing to operate without a more formal mechanism—other than community forums or e-mails—for stakeholder involvement could diminish support for the reform efforts, undermine their sustainability, and ultimately compromise the potential gains in student achievement. As more initiatives are developed, the need to balance the expediency of the reform efforts with measures to increase sustainability, such as stakeholder involvement, is critical.

In addition, since the Reform Act, the District has taken several steps to improve central office operations, such as providing more accountability at the departmental level and implementing a new individual performance management system. However, DCPS has not taken steps to align its performance management system, including its individual performance evaluations, to its organizational goals, which could result in a disparity between employees’ daily activities and services needed to support schools. By ensuring that employees are familiar with the organizational goals and that their daily activities reflect these goals, DCPS could improve central office accountability and support to schools.

Recommendations to the Mayor of the District of Columbia

To help ensure the transparency, success, and sustainability of the District’s transformation of its public school system, we recommend that the Mayor direct DCPS to establish planning processes that include mechanisms to evaluate its internal capacity and communicate information to stakeholders and, when appropriate, incorporate their views.

To strengthen the new individual performance management system and ensure greater accountability of central office employees in their role supporting schools, we recommend that the Mayor direct DCPS to link individual performance evaluations to the agency’s overall goals.

Agency Comments and Our Evaluation

We provided a draft of this report to DCPS, the Deputy Mayor of Education, and to the Office of the State Superintendent of Education for review and comment. These offices provided written comments on a draft of this report, which are reproduced in appendix I. They also provided technical comments, which we incorporated when appropriate. All three entities concurred with our recommendations. However, they expressed concern with the way in which we evaluated their reform efforts and the overall tone of the draft report.

Specifically, District officials stated that we did not measure DCPS’s progress in terms of the condition of the school system prior to the reform efforts, but instead measured progress in terms of whether the ultimate goals of the reform efforts had been met. We disagree. We did not measure DCPS’s progress against “ultimate goals.” As is now reflected in the paragraph describing our approach to this study, we measured the progress of ongoing reform efforts by comparing DCPS’s progress to its own time frames for implementing various initiatives. In conducting our review, we spoke with numerous DCPS officials and repeatedly asked for documents and time frames in order to objectively gauge the District’s progress. In some cases, DCPS officials did not provide us with such documentation; however, we made a concerted effort to accurately identify current initiatives and related time frames. In addition, we measured completed initiatives against recognized standards. For example, we determined whether or not the DCPS and the state-level strategic plans contained elements that GAO has identified as key to an effective plan.

In addition, we described the conditions that existed prior to the reform efforts in order to provide context to the steps DCPS has taken. For example, we noted that prior to the reform efforts, DCPS’s teacher training was not systematic or aligned with the school district’s goals and that DCPS is now offering on-site professional development to improve teacher skills. We also cited the lack of individual performance evaluations for central office employees prior to the reform efforts that DCPS has made to improve in this area. Furthermore, we made every effort to provide balance and objectivity in our findings. For example, some stakeholders, such as parents groups, union representatives, and the D.C. Council, told us that DCPS made key decisions without their involvement. We revisited this issue with DCPS officials and described several of their efforts to improve stakeholder involvement in the initial draft of our report.

We visited four urban school districts with mayoral governance and conducted in-depth interviews to help us better understand the magnitude of the challenges that officials encountered while trying to reform their school systems. We also spoke with superintendents and officials from mayors’ offices in these districts about the key lessons they learned as they reformed their school systems, including the risks associated with not having systematic stakeholder involvement.

Finally, the District’s education offices stated in their response that we characterized the state superintendent’s efforts as positive and those of DCPS more negatively. While drafting this report, we intentionally avoided any comparison between DCPS and the state superintendent’s office, as their tasks and challenges are dissimilar. After reviewing our draft, DCPS provided us with more information and documentation regarding efforts to involve stakeholders in the development of the October 2008 draft of the DCPS strategic plan and steps taken to introduce alignment of accountability measures to organizational goals. We made changes to our report to reflect the updated information.

As agreed with your offices, unless you publicly announce its contents earlier, we plan no further distribution of this report until 30 days after its issue date. At that time, we will send copies of this report to the D.C. Mayor’s Office, relevant congressional committees, and other interested parties. Copies will also be made available upon request. In addition, the report will be available at no charge on GAO’s Web site http://www.gao.gov.

If you or your staff members have any questions about this report, please contact me at (202) 512-7215 or ashbyc@gao.gov. Contact points for our Offices of Congressional Relations and Public Affairs may be found on the last page of this report. Key contributors to this report are listed in appendix II.

Appendix I: Comments from the D.C. Mayor’s Office and District Education Offices

Appendix II: GAO Contact and Staff Acknowledgments

Acknowledgments

In addition to the contact named above, Elizabeth Morrison, Assistant Director; Nagla’a El-Hodiri, Analyst-in-Charge; Sheranda Campbell; Jeff Miller; and Vernette Shaw made significant contributions to this report in all aspects of the work. Susan Aschoff, Mark Bird, Timothy Case, Bryon Gordon, Jeffrey Heit, Janice Latimer, Jean McSween, Sandy Silzer, and Sarah Veale provided analytical assistance. Doreen Feldman and Sheila McCoy provided legal support and Lise Levie and Kimberly Siegal verified our findings.

Related GAO Products

District of Columbia Public Schools: While Early Reform Efforts Tackle Critical Management Issues, a District-wide Strategic Education Plan Would Help Guide Long-Term Efforts. GAO-08-549T. Washington, D.C.: March 14, 2008.

District of Columbia Opportunity Scholarship Program: Additional Policies and Procedures Would Improve Internal Controls and Program Operations. GAO-08-9. Washington, D.C.: November 1, 2007.

No Child Left Behind Act: Education Should Clarify Guidance and Address Potential Compliance Issues for Schools in Corrective Action and Restructuring Status. GAO-07-1035. Washington, D.C.: September 5, 2007 Charter Schools: Oversight Practices in the District of Columbia. GAO-05-490. Washington, D.C.: May 19, 2005.

Results-Oriented Cultures: Implementing Steps to Assist Mergers and Organizational Transformation. GAO-03-669. Washington, D.C.: July 2, 2003.

Results-Oriented Cultures: Creating a Clear Linkage between Individual Performance and Organizational Success. GAO-03-488. Washington, D.C.: March 14, 2003.

Agencies’ Strategic Plans Under GPRA: Key Questions to Facilitate Congressional Review (Version 1). GAO/GGD-10.1.16. Washington, D.C.: May 1997.

Executive Guide: Effectively Implementing the Government Performance and Results Act. GAO/GGD-96-118. Washington, D.C.: June 1996.