District of Columbia Public Schools: Implementation and Sustainability of Reform Efforts Could Benefit From Enhanced Planning
Released: 2009-07-23T00:00:00

Letter

DCPS Quickly Implemented Many Separate Initiatives to Improve Overall Student Performance and Is Refocusing Its Approach as It Moves Forward

During the first 2 years of its reform efforts, DCPS implemented several classroom-based initiatives to improve students’ basic skills in core subjects. For example, to improve students’ basic skills and standardized test scores in reading and math, DCPS introduced targeted interventions for students struggling in these subjects and provided additional instruction and practice to improve students’ responses to open-ended questions, including test questions. Table 1 provides a list of DCPS’s major initiatives to improve student outcomes, as well as descriptions and the status of these initiatives.

DCPS is modifying its approach to implementing many of these initiatives as it moves forward. For example, the Chancellor recently acknowledged that DCPS, in its effort to remedy the range of issues that plagued the District’s public schools, may have launched too many initiatives at once and some schools may not have had the capacity to implement so many programs effectively. In particular, some schools were undergoing significant organizational changes that may have affected their ability to implement these new academic initiatives. To support such schools, DCPS is considering offering a choice of programs for schools and allowing the principals to determine which programs best suit their schools’ needs and capacity.

DCPS does not yet know how successful these initiatives have been in improving student achievement. Our report notes that DCPS elementary and secondary students increased their reading and math scores between 8 and 11 percentage points on the 2008 state-wide test, but it is unclear whether these gains could be attributed to the current reform efforts or to prior efforts. Preliminary scores for the 2009 reading and math tests were announced on July 13, 2009. Elementary students made modest gains in reading (49 percent were proficient in reading, up from 46 percent in 2008) and more substantial gains in math (49 percent proficient in math, up from 40 percent in 2008). Preliminary scores for secondary students show that 41 percent are proficient in reading, up from 39 percent in 2008, and 40 percent are proficient in math, up from 36 percent in 2008. While DCPS officials told us that it is generally difficult to isolate and quantify the impact of any single program on student achievement, they plan in late summer 2009 to analyze student outcomes, including state-wide test scores, to assess the effectiveness of various initiatives.

DCPS officials also noted that there were varying levels of teacher quality and knowledge of effective teaching practices, and that it was difficult to ensure the extent to which teachers implemented the programs effectively. While DCPS had not previously defined “effective” teaching, DCPS officials told us they will focus on practicing effective teaching, as opposed to implementing various disparate programs. By the beginning of the 2009-2010 school year, DCPS plans to implement a framework that is intended to help teachers understand what students are expected to learn for each subject, how to prepare lessons, and what effective teaching methods are to be used.

DCPS also changed the way it allocated teachers across its schools for the 2008-2009 school year. This new staffing model was intended to provide all schools with a core of teachers including art, music, and physical education, as well as social workers. It was also intended to provide all schools with reading coaches who work with teachers to improve reading instruction. Prior to this change, DCPS allocated funding to schools using a weighted student formula, which distributed funds to schools on a per pupil basis, so that the greater the enrollment of a school, the greater the amount allocated to that school. The new staffing model was intended to ensure core staff at all schools regardless of enrollment. While DCPS allowed principals to request changes to the staffing model based on their school’s needs, it did not establish or communicate clear guidance or criteria on how such requests would be treated. Therefore, it is unclear whether similar requests were treated in a consistent manner. A more transparent process, one that publicly shared their rationale for such decisions, would have helped assure stakeholders, including the D.C. Council, that changes to staffing allocations were made consistently and fairly. The D.C. Council and several community groups have criticized the process for its lack of transparency and questioned the fairness of the decisions made. For example, one independent analysis concluded that under the staffing model some schools received less per pupil funding than others with similar student populations. DCPS revamped its approach for the staffing model for the 2009-2010 school year to address some of these challenges. For example, it established guidance about what changes it will allow principals to make to the staffing model and disseminated this guidance to school leaders at the beginning of the budgeting process. According to DCPS, the new guidance is expected to reduce the number of changes that principals request later in the process.

In addition, as required by NCLBA, DCPS restructured 22 schools before the fall of 2008, after the schools failed to meet academic targets for 6 consecutive years. NCLBA specifies five options for restructuring a school, including replacing selected staff or contracting with another organization or company to run the school. DCPS revamped its process for determining the most appropriate restructuring option for the 13 schools that will be restructured in the 2009-2010 school year. Prior to implementing the first round of restructuring (for the 2008-2009 school year), DCPS officials told us there were insufficient school visits and inadequate training and guidance for teams assigned to evaluate which restructuring option was best suited for a given school. DCPS has addressed these issues by requiring two visits to each school, offering more training, and revising the form used to evaluate each school’s condition for the next round of restructuring. Restructuring underperforming schools will likely be an ongoing initiative for DCPS, as 89 of its 118 schools were in some form of school improvement status as of June 2009.

Finally, DCPS and the state superintendent’s office are planning and developing new ways to use data to monitor student achievement and school performance. DCPS reported it has ongoing and planned initiatives to expand data access to principals and teachers, in part to monitor student and school performance. In particular, DCPS reported making improvements to its primary student data system so central office users can better monitor school performance. DCPS also plans to use monthly reports to enable school leaders to better monitor student progress, but DCPS officials told us they have delayed some of these efforts while they attempt to improve coordination among the various departments that were developing and disseminating information to school leaders. The state superintendent’s office also is developing a longitudinal database, called the Statewide Longitudinal Education Data Warehouse (SLED), intended to allow DCPS and other stakeholders to access a broad array of information, including standardized test scores of students and information on teachers. According to officials in the state superintendent’s office, they revised the project schedule to allow more time to assist the charter schools with updating their data systems. In February 2009, the initial release of student data provided a student identification number and information on student eligibility for free or reduced-price lunches and other student demographics for all students attending DCPS’s schools and the public charter schools. The state superintendent’s office plans for SLED to enable DCPS to link student and teacher data by February 2010.

DCPS Replaced Teachers and Principals and Introduced Professional Development Initiatives, but Encountered Challenges in Implementation

DCPS focused on a workforce replacement strategy to strengthen teacher and principal quality. After the 2007-2008 school year, about one-fifth of the teachers and one-third of the principals resigned, retired, or were terminated from DCPS. DCPS terminated about 350 teachers and an additional 400 teachers accepted financial incentives offered by DCPS to resign or retire in the spring of 2008. In addition, DCPS did not renew the contracts of 42 principals. To replace the teachers and principals who left the system, DCPS launched a nationwide recruitment effort for the 2008- 2009 school year and hired 566 teachers and 46 principals for the 2008- 2009 school year. DCPS did not have a new teacher contract in place due to ongoing negotiations with the Washington Teachers’ Union and DCPS officials told us a lack of contract may have hindered their efforts to attract top-quality teachers. Under the plan, which has been in negotiation with the Washington Teachers’ Union since November 2007, the Chancellor has stated that she wants to recruit and retain quality teachers by offering merit pay, which would reward teachers with higher salaries based, in part, on their students’ scores on standardized state tests.

In addition, DCPS officials told us that the 2007-2008 and 2008-2009 teacher evaluation process did not allow them to assess whether the teacher workforce improved between these 2 school years. According to DCPS officials, this system does not measure teachers’ impact on student achievement—a key factor cited by DCPS officials in evaluating teacher effectiveness. DCPS plans to revise its teacher evaluation process to more directly link teacher performance to student achievement. To supplement school administrators’ observations of teachers, DCPS is also seeking to add classroom observations by 36 third-party observers, called master teachers, who would be knowledgeable about teaching the relevant subject matter and grade level.

In addition, DCPS introduced professional development initiatives for teachers and principals, but late decisions about the program for teachers led to inconsistent implementation. For the 2008-2009 school year, DCPS hired about 150 teacher coaches to improve teachers’ skills in delivering reading and math instruction and boost student test scores. According to DCPS, teacher coaches assisted teachers with interpreting student test scores, planning lessons, and using their classroom time constructively. DCPS is planning for teacher coaches to work with teachers in all grades and subjects for the 2009-2010 school year. DCPS intended to staff about 170 teacher coaching positions; however, as DCPS began the 2008-2009 school year, about 20 percent of the coaching positions remained open (19 reading coach vacancies and 16 math coach vacancies) because of late hiring of teacher coaches. DCPS officials told us they made the decision to hire teacher coaches after their review of school restructuring plans in June 2008. The ratio of teachers to coaches was higher than it would have been had the positions been filled. In addition, according to DCPS officials and Washington Teachers’ Union officials we interviewed, teacher coaches were often uncertain about their responsibilities and how to work with teachers, and received some conflicting guidance from principals.

The State Superintendent’s Office and DCPS Have Developed and Begun Implementing Strategic Plans; However, DCPS Has Not Always Involved Relevant Stakeholders in Planning and Implementing Key Initiatives

The state superintendent’s office and DCPS each developed their 5-year strategic plans and involved stakeholders in the process. Stakeholder involvement in formulating strategic plans allows relevant stakeholders to share their views and concerns. The state superintendent’s office and the State Board of Education collaboratively developed the District’s state- level, 5-year strategic plan, and released it in October 2008. This state-level plan spans early childhood and kindergarten through grade 12 education (including public charter schools). Officials from the state superintendent’s office told us they involved District officials, and stakeholders representing early childhood education, business, and higher education communities, as well as other stakeholders while drafting the plan. In September 2008, the state superintendent’s office held a public forum to solicit stakeholder input and accepted comments on the draft on its Web site. The office released a revised version of the plan within a month of the public forum.

DCPS released the draft of its 5-year strategic plan in late October 2008. In contrast to the state-level plan which includes the public charter schools, the DCPS plan is specific to prekindergarten through grade 12 education in its 128 schools. DCPS officials told us they based the draft on the Master Education Plan, which the prior DCPS administration developed with stakeholder involvement, and that they sought additional stakeholder input through a series of town hall meetings. After releasing the draft, DCPS held three public forums in the following 3 weeks where attendees provided DCPS officials with feedback on the draft strategic plan. In May 2009, DCPS released the revised draft, which incorporated stakeholder feedback. Officials from the D.C. Deputy Mayor of Education’s office told us that as part of their office’s coordinating role, it ensured that DCPS and the state-level strategic plans were aligned. However, the office had no documentation showing its efforts to coordinate these plans, such as an alignment study. We found that the two plans were aligned in terms of long-term goals. For example, DCPS’s goals could support the state-level goal of having all schools ready. However, we could not evaluate whether more detailed, objective measures and performance targets were aligned because the DCPS strategic plan did not always include specific objective measures and performance targets.

DCPS recently increased its efforts to involve stakeholders in various initiatives; however, it has not always involved stakeholders in key decisions and initiatives. DCPS officials told us they have a variety of approaches to involve stakeholders, including parents, students, and community groups, as well as institutional stakeholders such as the D.C. Council. For example, DCPS officials told us they reach out to parents, students, and the public through monthly community forums, meeting with a group of high school student leaders and a parent advisory group, responding to e-mail, and conducting annual parent and student surveys to gauge the school system’s performance. DCPS also involved other stakeholders, such as parent organizations and the Washington Teachers’ Union in its process of changing the discipline policy. However, according to two DCPS officials, DCPS did not have a planning process in place to ensure systematic stakeholder involvement, and we found that DCPS implemented some key initiatives with limited stakeholder involvement.

For example, key stakeholders, including D.C. Council members and parent groups, told us they were not given the opportunity to provide input on DCPS’s initial proposals regarding school closures and consolidations, the establishment of schools that spanned prekindergarten to grade 8, or the planning and early implementation of the new staffing model that placed art, music, and physical education teachers at schools and which fundamentally changed the way funding is allocated across DCPS.

Lack of stakeholder involvement in such key decisions led stakeholders, including the D.C. Council and parents groups, to voice concerns that DCPS was not operating in a transparent manner or obtaining input from stakeholders with experience relevant to the District’s education system. Further, these stakeholders have questioned whether the impact of reform efforts will be compromised because of restricted stakeholder involvement. Stakeholders in the other urban school districts we visited told us a lack of stakeholder involvement leads to less transparency as key decisions are made without public knowledge or discourse. In addition, the lack of stakeholder involvement can result in an erosion of support for ongoing reform efforts and poor decisions. For example, officials in Chicago and Boston said public stakeholder involvement was critical to community support for various initiatives, such as decisions on which schools to close. Officials and stakeholders in New York cited a lack of stakeholder involvement in decisions that were eventually reversed or revised.

DCPS and the State Superintendent’s Office Have Taken Steps to Improve Accountability and Performance, and DCPS Has Yet to Align Key Aspects of Its Performance Management System to Organizational Goals

DCPS has taken steps to improve accountability and performance of its central office. To improve accountability for central office departments, DCPS developed departmental scorecards to identify and assess performance expectations for each department. According to a DCPS official, these scorecards are discussed at weekly accountability meetings with the Chancellor to hold senior-level managers accountable for meeting performance expectations. In addition, in January 2008, DCPS implemented a new performance management system for employees. Performance management systems for employees are generally used to set individual expectations, assess and reward individual performance, and plan work. In addition, as we previously reported in our March 2008 testimony, DCPS developed individual performance evaluations as a part of its performance management system in order to assess central office employees’ performance. Previously, performance evaluations were not conducted for most DCPS staff. Individual performance evaluations are now used to assess central office employees on several core competencies twice a year.

Prior to our March 2008 testimony, DCPS officials told us that they intended to align the performance management system with organizational goals by January 2009, and DCPS has taken some steps to improve alignment. For example, DCPS officials told us they had better aligned their departmental scorecards to their 2009 annual performance plan. However, DCPS has not yet explicitly linked employee performance evaluations to the agency’s overall goals. DCPS officials told us they plan to do so in the summer of 2009.

The state superintendent’s office also implemented a new performance management system, effective October 2008, to hold its employees accountable and improve the office’s performance. The office is converting to a single electronic management system to track and evaluate employee performance by December 2009. According to an official from the state superintendent’s office, this system links individual employee evaluations to overall performance goals and the office’s strategic plan. Under this new evaluation system, each employee is given a position description, which includes responsibilities and duties linked to the overall goals, mission, and vision of the state superintendent’s office.

Individual and agency expectations are defined in an annual performance meeting with the employee. The office is currently training supervisory employees on how to use the system before its full implementation in December 2009.

In addition to implementing a performance management system, the State Superintendent has begun to address long-term deficiencies identified by Education related to federal grant management. Education designated the District as a high-risk grantee because of its poor management of federal grants. If the District continues to be designated as a high-risk grantee, Education could respond by taking several actions, such as discontinuing one or more federal grants made to the District or having a third party take control over the administration of federal grants. As noted in a recent GAO report, the state superintendent’s office uses findings from an annual audit as part of its risk assessment and monitoring of subrecipients. The findings are used to design monitoring programs and determine risk levels for each school district, and the risk levels are used to develop monitoring strategies and work plans. The state superintendent’s office developed a corrective action plan, which it reports to Education and intends to use the plan to strengthen the monitoring of the school districts.

Implementation of Recommendations Could Improve Sustainability of Reform Efforts

The District’s Mayor and his education team have taken bold steps to improve the learning environment of the District’s students. As more initiatives are developed, the need to balance the expediency of the reform efforts with measures to increase sustainability, such as stakeholder involvement, is critical. DCPS currently lacks certain planning processes, such as communicating information to stakeholders in a timely manner and incorporating stakeholder feedback at key junctures, which would allow for a more transparent process. Stakeholder consultation in planning and implementation efforts can help create a basic understanding of the competing demands that confront most agencies and the limited resources available to them. Continuing to operate without a more formal mechanism for stakeholder involvement could diminish support for the reform efforts, undermine their sustainability, and ultimately compromise the potential gains in student achievement. In addition, since the Reform Act, the District has taken several steps to improve central office operations, such as providing more accountability at the departmental level and implementing a new individual performance management system. However, DCPS has not yet aligned its performance management system, including its individual performance evaluations, to its organizational goals, which could result in a disparity between employees’ daily activities and services needed to support schools. By ensuring that employees are familiar with the organizational goals and that their daily activities reflect these goals, DCPS could improve central office accountability and support to schools.

In our report that we publicly released today, we make two recommendations that could improve the implementation and sustainability of key initiatives in the District’s transformation of its public school system. We recommend that the Mayor direct DCPS to: Establish planning processes that include mechanisms to evaluate its internal capacity and communicate information to stakeholders and, when appropriate, incorporate their views.

Link individual performance evaluations to the agency’s overall goals.

In written comments on the report, all three District education offices— DCPS, the state superintendent’s office and the Deputy Mayor for Education—concurred with our recommendations. However, they expressed concern with the way in which we evaluated their reform efforts and the overall tone of the draft report. A summary of the District’s response to our findings and recommendations, as well as our evaluation of the response, are contained on pages 41 and 42 of the report.

Mr. Chairman and Members of the Subcommittee, this concludes my prepared statement. I would be happy to answer any questions that you may have at this time.

Acknowledgements

For further information regarding this testimony, please contact Cornelia Ashby at (202) 512-7215 or ashbyc@gao.gov. In addition, contact points for our Offices of Congressional Relations and Public Affairs may be found on the last page of this statement. Individuals who made key contributions to this testimony are Elizabeth Morrison, Assistant Director, Sheranda Campbell, and Nagla’a El-Hodiri.

This is a work of the U.S. government and is not subject to copyright protection in the United States. The published product may be reproduced and distributed in its entirety without further permission from GAO. However, because this work may contain copyrighted images or other material, permission from the copyright holder may be necessary if you wish to reproduce this material separately.