metadata: authors: 0: author: Yamauchi,Futoshi
count: Philippines
docna: 0: docna: An alternative estimate of school-based
            management impacts on students' achievements : evidence
            from the Philippines
docty: Policy Research Working Paper
subtopic: Primary Education,Educational Sciences,Educational Institutions & Facilities,Effective Schools and Teachers,Inequality
teratopic: Education
abstracts: cdata!: This paper aims to estimate the impact
            of school-based management on students' test scores in
            the Philippines. Estimation results using double
            differencing combined with propensity score matching show
            that school-based management increases the average national
            achievement test score by 4.2 points over three years. The
            increase in mathematics reached 5.7 points. This is larger
            than the estimate previously reported from the Philippines,
            probably because the sample schools had learned about
            implementation of school-based management from experiences
            accumulated in other provinces that introduced it earlier.
            The empirical results also show that schools with
            experienced principals and teachers are eager to introduce
            school-based management.
geo_reg_key: 517198,517191
repnme: repnme: An alternative estimate of school-based
            management impacts on students' achievements : evidence
            from the Philippines
display_title: An alternative estimate of
            school-based management impacts on students'
            achievements : evidence from the Philippines
listing_relative_url: /research/2014/01/18808747/alternative-estimate-school-based-management-impacts-students-achievements-evidence-philippines
topicv3: Teachers Management m1326119 1523,Primary Education m1326139 480,Poverty m1326687 695,Education m1327048 470,School-Based Management m1327730 2143
publishtoextweb_dt: 2014-01-01T00:00:00Z
historic_topic: Education
no_of_pages: 27
author_keys: 0: author_key: 1209647
model_extractions: WPS6747 Policy Research Working Paper 6747 An Alternative Estimate of School-based Management Impacts on Students’ Achievements Evidence from the Philippines _Futoshi Yamauchi_ The World Bank East Asia and the Pacific Region Education Sector Unit January 2014
Policy Research Working Paper 6747

# **Abstract**

This paper aims to estimate the impact of school-based management on students’ test scores in the Philippines. Estimation results using double differencing combined with propensity score matching show that school-based management increases the average national achievement test score by 4.2 points over three years. The increase in mathematics reached 5.7 points. This is larger than the estimate previously reported from the Philippines, probably because the sample schools had learned about implementation of school-based management from experiences accumulated in other provinces that introduced it earlier. The empirical results also show that schools with experienced principals and teachers are eager to introduce school-based management.

This paper is a product of the Education Sector Unit, East Asia and the Pacific Region. It is part of a larger effort by the World Bank to provide open access to its research and make a contribution to development policy discussions around the world. Policy Research Working Papers are also posted on the Web at http://econ.worldbank.org. The author may be contacted at fyamauchi@worldbank.org.

_The Policy Research Working Paper Series disseminates the findings of work in progress to encourage the exchange of ideas about development issues. An objective of the series is to get the findings out quickly, even if the presentations are less than fully polished. The papers carry the names of the authors and should be cited accordingly. The findings, interpretations, and conclusions expressed in this paper are entirely those of the authors. They do not necessarily represent the views of the International Bank for Reconstruction and Development/World Bank and its affiliated organizations, or those of the Executive Directors of the World Bank or the governments they represent._ Produced by the Research Support Team
**An Alternative Estimate of School-based Management Impacts on Students’ Achievements: Evidence from the Philippines**<sup>**1**</sup>

Futoshi Yamauchi<sup>2</sup> The World Bank

**Key words** : School based management, test score, Philippines

# **JEL Classifications: I21, O53**

> 1 I would like to thank Luis Benveniste, Yuki Ikeda, William Martin, Suhas Parandekar, and Ken Vine and the reviewer of the Journal for their useful comments. AusAID and the Government of Japan provided financial support. The opinions and conclusions expressed in this paper are those of its author and do not necessarily reflect positions of the World Bank and its member governments. Any remaining errors are mine.

> 2 Correspondence: Futoshi Yamauchi, The World Bank, 1818 H Street, NW, Washington D.C. 20433; Email: <u>fyamauchi@worldbank.org, Phone: 202 458 4262</u>
# **1. Introduction**

School-based management (SBM) aims to introduce autonomous school governance to reflect local needs with direct monitoring of school activities by their stakeholders, that is, parents and the community. Such a decentralization effort in the context of school education includes some of the following features – decentralized revenue generation, school administration, management, teacher hiring, and curriculum design.<sup>3</sup> These functions are devolved to local governments or schools. In this paper I assess impacts of SBM on students’ achievement, measured by their national achievement test scores, using school-level data available from the second round of SBM scale-up in the Philippines.

In the Philippines where our empirical study is drawn, there were two stages of SBM implementation. First, SBM was initially introduced in 23 poor provinces (divisions, interchangeably) as part of a large school intervention package, called the Third Elementary Education Project (TEEP), after 2001. Those provinces were identified as the most socially depressed in the Social Reform Agenda (see more details in Section 3). Second, once TEEP was completed in 2006, SBM was scaled up to other non-TEEP divisions with the support of neighboring TEEP divisions. In this period the introduction of SBM was the only major change experienced at non-TEEP schools, in contrast to the situation where TEEP schools, when implementing SBM, received other investments such as building construction and renovation, textbooks, teachers’ training, and other facility support.

I use data from the second stage of SBM implementation to avoid contamination effects from the other intervention components. More specifically, I compare schools that received SBM grants in 2006 and 2009 (thus, with a three-year interval) under the assumption that both the School Improvement Plan and the Annual Improvement Plan were proposed before receiving the grant (therefore, a three-year gap in the

> 3 When this policy includes financial decentralization, it may lead to poorer areas having much less funding for their schools.

2
improvement plans is assumed between the two groups).<sup>4</sup> The Matching method is used to control for pre-SBM differences, correlated with non-randomness of the SBM implementation between the two groups in the sample.

Our work is closely related to an earlier study by Khattr et al. (2012). They used the initial implementation stage of SBM during the TEEP period, when the government initially planned three batches by divisions to implement SBM (thus, enabling a pipeline method). Their empirical setting is characterized by two important features: (i) in reality, lags between batches were short for some reasons related to the actual project implementation, and (ii) it was at the very initial stage of SBM in the country. Their estimate of an increase in the total test scores from two-year exposure to SBM is 1.45 points, which is quite small probably due to the factors mentioned above.

In contrast to the above study, Yamauchi and Liu (2012b) evaluated the impact of the whole intervention package of TEEP (including SBM) on test scores. Their analysis does not attempt to purge out SBM impacts from the other components, so the impact estimate includes the SBM impacts. The estimation used the sample of schools from the macro-region of Visayas where treated and control divisions are geographically scattered. Based on double differences combined with propensity score matching, the impact is around an increase of score by 4.5 to 5 points over two years.<sup>5</sup> The paper is organized as follows. The next section reviews recent literatures on SBM. Section 3 describes our empirical setting, especially TEEP and non-TEEP divisions, and discusses why I restrict our sample to non-TEEP and non-city division schools. Section 4 describes our data. The data come from three sources: (i) national school census data, (ii) SBM fund data, and (iii) national achievement test score

> 4 Schools are required to set up school improvement plan (SIP) for the coming 5 years as well as more specific annual improvement plans (AIP) that describes their plan for a school year. Based on significance of their SIP/AIP, SBM grants were allocated to schools.

> 5 Interestingly, the simple double difference estimate (without matching) is similar to that of Kattr et al. (2012) for SBM.

3
data. Section 5 discusses the empirical method, namely double differences combined with propensity score matching.

The empirical results are summarized in Section 6. Double differencing with propensity score matching shows that SBM increases the national achievement test score by 4.2 points over three years. The mathematics score increased by 5.7 points. This is larger than the estimate previously reported from the Philippines, probably due to the fact that our sample schools had learned from SBM experiences accumulated in those schools that introduced SBM earlier. Our empirical results also show that schools endowed with more experienced principals and teachers are likely to be better prepared for introducing SBM. Conclusions are mentioned in the last section.

# **2. Literature Review**

Recently there has been a growing number of studies that evaluate the impact of SBM (see Bruns, Filmer and Patrinos, 2011, Table 3.4). Here I collect studies that used either randomization or a combination of panel data and matching methods. The first group used randomization approaches to identify SBM impacts (Duflo, Dupas and Kremer, 2012; Chaudhury and Parajuli, 2010; Das, 2008l; Pradhan et al., 2010; Gertler, Patrinos and Rodrigues-Oreggia, 2012). With an ideal introduction of randomized trials, the analysis is free from endogenous program placement that could bias the treatment effect. Duflo, Dupas and Kremer (2012) introduced randomized experiments to Kenyan schools to compare school resource effects and teachers’ incentive effects on learning outcomes. They find that improving school resources, measured by decreasing the pupil-teacher ratio, leads to reduced teacher effort in the absence of any other changes, but students who were assigned to locally-hired contract teachers (with the presence of parents’ involvement in school management) showed significant improvements in test scores.

Chaudhury and Parajuli (2010) and Gertler, Patrinos and Rodrigues-Oreggia (2012), using school-level randomized experiments in Nepal and Mexico, respectively, showed that devolution of decision making 4
to the school/community level improved grade progression (and reduced repetitions) but the impact on students’ learning outcomes, measured by test scores, was insignificant.

Pradhan et al. (2010) assessed randomized interventions to strengthen school autonomy in Indonesia. They found that, nearly two years after implementation, measures to reinforce existing school committee structures, the grant and training interventions, demonstrate limited or no effects, but measures that foster outside ties between the school committee and other parties, linkage and election, lead to greater engagement by education stakeholders and to an improvement in learning outcomes. The findings suggest that the way to combine different elements of school autonomy together with appropriate trainings is important to improving learning outcomes.

The second group of SBM impact evaluation studies combines panel data and matching methods to control endogenous program placements and potentially differentiated trends between treatment and comparison schools.<sup>6</sup> Skoufias and Shapiro (2006) evaluated the impact of increased school resources and decentralized management decisions combined in Mexico. They find that the reform decreased drop-out, failure and repetition rates, but the magnitude of the impacts is generally small. Related to the current study, Khattr, Ling and Jha (2012) recently used school-level test score data in the Philippines to assess the impact of SBM on learning outcomes. They showed that the introduction of SBM had a significant impact on students’ test scores but its magnitude is quite small, that is, an increase by about 1.5 points in percent score over two years.

# **3. Background**

> 6 King and Ozler (1998) and King, Ozler and Rawlings (1999) assessed the SBM impacts in Nicaragua by distinguishing between _de facto_ and _de jure_ autonomy. They found that the _de facto_ devolution of decision making to the school/community level matters but the _de jure_ devolution does not. Similar to many other studies, they only found significant impacts on progression/repetitions, not on learning outcomes. Parker (2005) basically confirmed the same conclusion in Nicaraguan schools – the SBM impact on learning outcome is ambiguous.

5
The Philippines introduced school-based management (SBM) in two steps. First, SBM was piloted and introduced to 23 school divisions (almost identical to provinces) as part of the Third Elementary Education Project, so called TEEP, during 2000-2006.<sup>7</sup> Second, the government decided to scale up SBM to non-TEEP divisions upon the completion of TEEP in 2006.

# **3.1 Third Elementary Education Project**

The Third Elementary Education Project (TEEP) was implemented from 2000 to 2006 by the Philippine Department of Education in all public primary and elementary schools<sup>8</sup> in the 23 provinces<sup>9</sup> identified as the most socially depressed in the Social Reform Agenda.<sup>10</sup> Note that TEEP was a pilot for SBM in the country as there are more than 80 provinces in the Philippines. The total project cost was US$221.16 million ($91.07 million from JBIC, $82.84 million from the World Bank, and $47.25 million from the Philippine government). The unique feature of TEEP is a combination of investments in school facility and education materials and school governance reform. Not only were school facilities and textbook supply improved, but the decision-making process was also decentralized to the school and community levels. TEEP introduced a package of investments to schools in the selected 23 provinces. Specifically, the package of investments included (1) school building construction and renovation, (2) textbooks, (3) teacher training, (4) school-based management, and (5) other facility and equipment support. Note that

> 7 See Yamauchi and Liu (2012a, 2012b) for the impacts on student test scores and subsequent schooling and labor market outcomes.

> 8 Primary schools cover grades 1 to 4 (or less), while elementary schools cover grades 1 to 6.

> 9 The program covered both primary (grades 1–4) and elementary (grades 1–6) schools. This paper analyzes the impacts on only elementary schools. However, converting primary schools to elementary schools by extending enrollment up to grade 6 was also an important part of the TEEP program. Students who complete primary schools are likely to attend elementary schools in grades 5 and 6, which changes the student body of those schools between grades 1–4 and grades 5 and 6.

> 10 The Ramos administration, along with their medium term development plan, called Philippines 2000, identified reforms as the key to bridging social gaps and alleviating poverty. The objective of enhancing development through social reforms led to the formulation of the blueprint for social development in the Philippines, the Social Reform Agenda (SRA), marked as the first instance of social reforms in the history of the Philippines (Ramos 1995). As a result of the initial success of the SRA, the Congress of the Philippines in 1998 passed Republic Act 8425, widely known as the Social Reform and Poverty Alleviation Act (Republic of the Philippines, Congress, 1998). The law institutionalized the poverty alleviation program and a host of grassroots development strategies.

6
except principal-led projects on school building, schools or communities did not influence initiation of the above interventions.

One important component of the program is school-based management, through which schools are given an incentive to manage proactively and more independently of the government. Schools were partnered with communities and parents to decide key issues such as improvement plans and school finance. Teachers were also trained systematically to improve teaching skills. Information management is being improved so that schools are responsible for systematically organizing information on enrollment, learning achievements, finance, and so forth and reporting it to the division office. Schools are required to set improvement plans every year and compare them with actual achievements. This dynamic process is monitored by the division-level education department. School finance is also being decentralized to some extent to relax the school budget constraints because Philippine public schools are not allowed to charge school fees. TEEP schools are free to raise their own funds from communities, parents, and others, though resources are admittedly limited in many poor communities. These reforms in public schools are expected to improve education quality, which would then in turn increase returns to schooling in labor markets (see Yamauchi 2005, on returns to schooling).

The selection of TEEP provinces was purposive because it intended to cover the most depressed provinces identified in the Social Reform Agenda. TEEP allocation is rather different in the Philippines’ three macro-regions. In the northern macro-region of Luzon, TEEP was concentrated in the Cordillera Administrative Region, a mountainous region in the center of northern Luzon. In the central macro-region of Visayas, TEEP divisions were relatively evenly distributed. In the southern Mindanao macro-region, TEEP divisions were clustered, though not as clustered as in northern Luzon.

# **3.2 Non-TEEP Schools**

7
In this paper, I use non-TEEP divisions where SBM was introduced after 2006. SBM was scaled up to non-TEEP divisions after the completion of TEEP. However, the implementation had lags due to uneven distributions of preparedness across divisions. TEEP divisions were partnered to their neighboring nonTEEP divisions to introduce SBM.<sup>11</sup> At that time, there was not a systematic large-scale school intervention, such as TEEP, so I can assume that the major change introduced in non-TEEP schools after 2006 was SBM.

As discussed below, I use two groups of schools: those schools that received SBM grants in 2006, and those received the grant in 2009. The distinction between them is simply the lag in receiving the SBM grant, which is correlated with school-level preparedness for SBM, reflected in the school improvement plan (5 years) and the annual improvement plan.

# **4. Data**

The analysis uses the Basic Education Information System (BEIS) database for elementary schools, national achievement test score data, and SBM grant information. They are panel data, so it is possible to estimate the effect of SBM introduction on changes in test scores, controlling for changes in school conditions.

To increase homogeneity, I restrict our sample to non-city divisions among non-TEEP divisions. First, schools from non-city divisions are used in the analysis since there are not enough numbers of treatment and control schools within a city division (relatively small area). Treatment and control schools are therefore compared within each division, excluding city divisions. By doing this, I can also avoid contagion effects of early SBM practitioners to latecomers in the same city.

> 11 However, there were provinces not located close to TEEP provinces. Examples include provinces in Central Luzon, Bocol and some parts of Mindanao.

8
As discussed, SBM was introduced in non-TEEP schools after 2006. In this process, TEEP divisions were partnered with their neighboring non-TEEP divisions to assist the implementation of SBM. Schools are required to set up a school improvement plan (SIP) for the coming five years as well as more specific annual improvement plans (AIP) that describe their plan for a school year. Based on significance of their SIP/AIP, SBM grants were allocated to schools. Thus, SBM grants reflect both school-level eagerness and division-level priority. The information on when SBM grants were received is available to us from the Department of Education.

The outcome measure used in this study is national achievement test (NAT) scores (percentage scores). Grade 6 students are required to take the exam each year. NAT data are available for all elementary schools in each school year.

Two years, 2006 and 2009, are used to define treatment and control groups. In 2006, non-TEEP schools started receiving SBM grants. To have an enough span to detect its impact on students’ achievements, measured by NAT scores, I decided to compare schools that received SBM grants in 2006 (treatment) and 2009 (control). Therefore, the treatment schools had received SBM grants three years earlier than the control schools.<sup>12</sup> As discussed in the next section, propensity score matching is used to control the endogeneity of SBM grant receipt in 2006 against 2009. BEIS data in SY2005 are used to control school types, resources: pupil-classroom ratio, pupil-teacher ratio, and total enrollment, and principal and teachers’ experience (i.e., capacity measured by salaries, correlated with their ranks). I suppose that schools that are well

> 12 The data show that no school received the grant in 2008. The numbers of recipient schools are comparable in 2006 and 2007. The analysis uses, as treatment, the schools that received the grant in 2006, rather than 2007, since this gives a 3-year period to assess the impact of SBM on test scores. However, the results reported below can be subject to potential selectivity that may be caused by years chosen for this study. It is also important to note that there are many schools that had not received the grant during the period of 2006 to 2010.

9
|(Y)<br>E(Y<br>=1)<br>DD=E(Y<br>|D=1)-E(Y<br>|D=0)=E(Y<br>|D=1)+B|
|---|
=E(Y |D=1)-E(Y |D=0)

E(Y |D=1)=E(Y |D = 0)

Kee ) L Dx] lv ) L DIP]
# **6. Empirical Results**

This section summarizes the empirical results. Before discussing the estimation results, I describe our empirical setting in more details. As explained, I compare schools that introduced SBM in 2006 and 2009. First, it is possible that the two groups of schools had different trends before 2006, which can be projected into the period in which the impact of SBM is assessed. More specifically, if the SBM/2006 schools had a larger trend in NAT than the SBM/2009 schools before 2006, the impact of SBM may be erroneously inferred from the pre-intervention differential trends in NAT.

Table 1 to be inserted Table 1 shows summary statistics of NAT scores and their changes for the treated and control schools. Interestingly, the average NAT score in 2006 was lower among SBM/2009 schools (control) than SBM/2006 schools. The trend during the pre-intervention period, 2004 to 2006, was higher among the SBM/2009 schools (control), but that was reversed in the post-intervention period. These observations support the possibility that SBM contributed to the increased trend observed among the treatment schools relative to the comparison schools.

Table 2 to be inserted Next, estimation results are discussed. First, a Logit equation is estimated to construct the propensity score. The results are reported in Table 2. The dependent variable takes the value one if the school received an SBM grant in 2006, and zero if 2009. The estimation includes division fixed effects (dummies).

12
First, the pseudo R-squared is 0.2190 with division fixed effects. Complete and mono-grade schools were likely to receive SBM grants.<sup>14</sup> Second, among school resource indicators, the pupil-teacher ratio significantly increases the likelihood of receiving the grant. Total enrollment and pupil-classroom ratio are insignificant. Third, total salaries of teaching staff – principal, head teachers, master teachers and teachers – all significantly increase the probability of receiving the SBM grant. Teachers’ human capital (measured by their experience) significantly increases the likelihood of introducing SBM. Therefore, the total amount of human capital endowed among teachers as well as the number of students per teacher are crucial determinants of early-stage SBM introduction. Since schools have to design good school improvement plans to have SBM grants, the above finding showing the importance of teachers’ human capital is sensible.

Figure 1 shows the computed propensities for the treated and control groups from the first specification. As expected, its distribution of the treated first-order dominates that of the control group. When constructing weights, the estimated propensity was trimmed by 0.15 and 0.85 (very small and large propensities are dropped).<sup>15</sup> The SBM impacts on test scores are summarized in Tables 3 and 4. Estimation in Table 3 uses the weights constructed from the propensities estimated in Logit (Table 2) I use total and subject test scores (mathematics, English, science, Filipino and hekasi: social science), both normalized as percentage scores. Propensity score matching with double differencing shows that SBM (introduced three years earlier) increases the national achievement test score by 5.69 (mathematics), 4.21 (English), 4.65 (science), 2.39

> 14 Some elementary schools are incomplete (not offering all grades) and/or multi-grade (combining more than a grade).

> 15 The minimum and maximum of propensity score in the control group are 0.0155647 and 0.9852484, respectively. Our results remain robust in an alternative method keeping schools only within this range.

13
(Filipino) and 4.09 (hekasi). The total score increased by 4.21, which is equivalent to about 33% of the standard deviation of NAT 2006 of non-SBM schools.<sup>16</sup> Table 4 controls school factors such as changes in total enrollment, pupil classroom ratio, pupil teacher ratio, and total amount of teachers’ salary by their position. If SBM changes school conditions through investments in school facilities etc., the inclusion of such variables as controls may bias the impact estimate. However, it is still important to check robustness of the main results (Table 3) by including changes in school conditions.<sup>17</sup> Table 4 confirms that the main findings remain robust, without any significant changes in the estimated impacts. The above results in Tables 3 and 4 show that (i) the SBM impacts on NAT are significant and relatively large, and (ii) whether or not controlling for school conditions does not affect the impact estimates.

It is, however, still possible that other factors contributed to the above findings. For example, NAT might have become easier in recent years for some reason. If so, low performing schools that also introduced SBM earlier were able to improve their test scores, regardless of SBM. Consistently it is also shown that the variation of NAT scores across schools has decreased in the same period (Table A1), while variations in other conventional school and teacher resource inputs have increased during the same period (Paradekar, Sipahimalani-Rao, and Yamauchi, 2012). This is consistent with the observations in Table 2 as well as the findings in Tables 3 and 4, since the average NAT score of the SBM/2006 schools caught up to that of the SBM/2009 schools, which can be attributed at least partially to the introduction of SBM.

> 16 Preliminary analyses show that, contrary to the initial conjecture, the change in test score is relatively large in 2006-2007, not in later periods, which implies that SBM had a rather immediate effect.

> 17 Double differencing without propensity score matching also gives qualitatively similar results. The impacts are estimated slightly smaller than those using propensity score matching, which implies that SBM grants were allocated to schools that performed relatively worse (despite that they developed school improvement plans earlier than others).

14
The SBM impact estimate of 4.2 in the total score can be translated into a change of pupil teacher or classroom ratio that is required to bring about NAT changes in the same magnitude. Table A2 shows the estimated effects of changes in the pupil teacher ratio, pupil classroom ratio and per-pupil teacher salary on NAT scores (using the 2005-2010 panel data). Though the estimates of the PCR and PTR effects are likely subject to upward bias (therefore, the impacts are underestimated), the results strikingly show that the impact of SBM, measured in a three-year period, is sufficiently large in terms of the impacts of school resource reallocations.<sup>18</sup>

# **7. Conclusions**

This paper showed a significant SBM impact on student test scores over three years. SBM was implemented earlier among schools that had experienced teachers but a large number of students per teacher.

Our estimate of the SBM impact over three years is larger than the previously available estimate from the Philippines (Khattri, Ling and Jha, 2012). There are several reasons. First, their estimate is based on a comparison between the first and second batch schools, scheduled during TEEP. In reality, however, interventions to the two groups occurred almost simultaneously. Second, TEEP not only introduced SBM but included other investments such as textbooks, school buildings, and teachers’ training. Though they attempted to control those non-SBM investments, their PSM method faced a serious challenge since those investments are also endogenous as part of the intervention package. In our case, SBM was the only major change experienced among schools in our sample.

> 18 Investments in classrooms and teachers during the period of 2005 to 2010 could be positively correlated with the initial shock to test score in 2005 (e.g., schools that performed well in NAT 2005 could receive more allocations from the government). In this case, the estimates in Table A2 and, therefore, the simulated changes of PCR and PTR are upwardly biased.

15
It is also possible that non-TEEP schools have learned from SBM experiences accumulated in TEEP schools, which improved the efficiency of SBM implementation. This can be an alternative explanation for our relatively large estimates. Social learning and/or knowledge transfer could have occurred from TEEP to non-TEEP schools, which significantly increased the impact of SBM after the initial trial and error stage.

16