metadata: authors: 0: author: Arcia,Gustavo J.
1: author: Macdonald,Kevin Alan David
2: author: Patrinos,Harry Anthony
count: Thailand
docna: 0: docna: School autonomy and accountability in
            Thailand: a systems approach for assessing policy intent and implementation
docty: Policy Research Working Paper
subtopic: Primary Education,Educational Sciences,Educational Policy and Planning,Educational Institutions & Facilities,Educational Populations,Education for Development (superceded),Educational Policy and Planning -
            Ministry of Education,Educational Policy and Planning -
            Institutional Development,Effective Schools and Teachers,Education For All
teratopic: Governance,Education,Public Sector Development,Macroeconomics and Economic Growth
abstracts: cdata!: There is a consensus on the need for
            Thailand to reform its education system to be able to
            compete with other high performing countries in the region.
            In terms of learning outcomes, the most recent evidence from
            the Programme for International Student Assessment shows
            little improvement over time. This paper uses the World
            Bank's Systems Approach for Better Education Results
            (SABER) approach in Thailand to contrast policy intent and
            policy implementation in school autonomy and accountability.
            The policy implementation data were obtained from a survey
            of school principals of the schools that participated in the
            Programme for International Student Assessment and merged
            the data sets. First, the study analyzes the gap between
            policy intent and policy implementation. Then it examines
            the effect of the gaps on various schooling outcomes while
            controlling for covariates. The analysis finds significant
            differences between the Systems Approach for Better
            Education Results indicators of policy intent and policy
            implementation in all areas assessed by the indicators.
            Schools in Thailand exercise more flexibility in their
            personnel management in practice than what is intended by
            policy; student assessments need to address issues of
            content, reliability, and validity and school accountability
            needs to improve the interpretation of student assessments
            to make schools more accountable. There is a positive
            association between the Programme for International Student
            Assessment scores and school autonomy and accountability.
geo_reg_key: 517198,517191
repnme: repnme: School autonomy and accountability in
            Thailand: a systems approach for assessing policy intent and implementation
display_title: School autonomy and accountability
            in Thailand: a systems approach for assessing policy intent
            and implementation
listing_relative_url: /research/2014/08/20141729/school-autonomy-accountability-thailand-systems-approach-assessing-policy-intent-implementation
topicv3: Teachers Management m1326119 1523,Primary Education m1326139 480,Education Policy and Planning m1326781 1513,Education m1327048 470,Learning Assessment m1327283 1555
publishtoextweb_dt: 2014-08-01T00:00:00Z
historic_topic: Governance,Education,Public Sector Development,Macroeconomics and Economic Growth
no_of_pages: 44
author_keys: 0: author_key: 428455
1: author_key: 962343
2: author_key: 95903
model_extractions: WPS7012 Policy Research Working Paper 7012 School Autonomy and Accountability in Thailand A Systems Approach for Assessing Policy Intent and Implementation _Gustavo Arcia Kevin MacDonald Harry Anthony Patrinos_ Education Global Practice Group August 2014
Policy Research Working Paper 7012

# **Abstract**

There is a consensus on the need for Thailand to reform its education system to be able to compete with other high performing countries in the region. In terms of learning outcomes, the most recent evidence from the Programme for International Student Assessment shows little improvement over time. This paper uses the World Bank’s Systems Approach for Better Education Results (SABER) approach in Thailand to contrast policy intent and policy implementation in school autonomy and accountability. The policy implementation data were obtained from a survey of school principals of the schools that participated in the Programme for International Student Assessment and merged the data sets. First, the study analyzes the gap between policy intent and policy implementation. Then it examines the effect of the gaps on various schooling outcomes while controlling for covariates. The analysis finds significant differences between the Systems Approach for Better Education Results indicators of policy intent and policy implementation in all areas assessed by the indicators. Schools in Thailand exercise more flexibility in their personnel management in practice than what is intended by policy; student assessments need to address issues of content, reliability, and validity and school accountability needs to improve the interpretation of student assessments to make schools more accountable. There is a positive association between the Programme for International Student Assessment scores and school autonomy and accountability.

This paper is a product of the Education Global Practice Group. It is part of a larger effort by the World Bank to provide open access to its research and make a contribution to development policy discussions around the world. Policy Research Working Papers are also posted on the Web at http://econ.worldbank.org. The authors may be contacted at hpatrinos@ worldbank.org .

_The Policy Research Working Paper Series disseminates the findings of work in progress to encourage the exchange of ideas about development issues. An objective of the series is to get the findings out quickly, even if the presentations are less than fully polished. The papers carry the names of the authors and should be cited accordingly. The findings, interpretations, and conclusions expressed in this paper are entirely those of the authors. They do not necessarily represent the views of the International Bank for Reconstruction and Development/World Bank and its affiliated organizations, or those of the Executive Directors of the World Bank or the governments they represent._ Produced by the Research Support Team
# **School Autonomy and Accountability in Thailand: A Systems Approach for Assessing Policy Intent and Implementation**

Gustavo Arcia Kevin MacDonald Harry Anthony Patrinos<sup>1</sup> JEL No: H4,I20,I25,J24,O15 Keywords: educational policy; educational autonomy; school accountability; Thailand 1 We thank Sunee Klainin, Institute for the Promotion of Teaching Science and Technology, Thailand, for her coordination of the school survey and data coding, and Angie Rivera Olvera, for her assistance with the school-level dataset. We are indebted to Derek Pornsima, Advisor to the Secretary-General of KHURUSAPHA (Teachers’ Council of Thailand) and Kraiyos Pratawart, Quality Learning Foundation, for their support during the different phases of this study. At the World Bank Office in Bangkok we thank Vachraras Pasuksuwan for her invaluable assistance with the management and logistics of the SABER survey, and Lars Sondergaard, for his assistance and advice during the policy dialogue with the Government. We also would like to thank Benjalug Namfa, former Deputy Secretary General, Office of the Basic Education Commission, and Wongduan Suwansiri, Academic Officer, Office of the Basic Education Commission, for their participation and support during the phase of policy dialogue within the education sector in Bangkok. Gustavo Arcia and Kevin MacDonald are World Bank Consultants and Harry Anthony Patrinos is Manager, Education, World Bank.
# **School Autonomy and Accountability in Thailand: A Systems Approach for Assessing Policy Intent and Implementation**

# <u>Summary</u>

- There are significant differences between the indicators of policy _intent_ and _implementation_ .

- Schools in Thailand exercise more flexibility in their budget planning and management, and in personnel management than is intended by policy.

- Student assessments need to address issues of content, reliability and validity.

- School accountability needs to improve the interpretation of student assessments to make schools more accountable.

- There is a positive association between reading scores and school autonomy in budgetary and personnel management.

|**Thailand: Benchmarking Policy****_Intent_ a**<br>**School Autonomy and Accou**|**nd****_Implementati_**<br>**ntability**|**_on_ in**|
|---|---|---|
|**Indicator**|**Policy**<br>**Intent**|**Policy**<br>**Implementation**|
|School Autonomy in Budget Planning and Approval|Emerging<br>|Established<br>|
|School Autonomy in Personnel Management|Latent<br>|Emerging<br>|
|Participation of the School Council in School Finance|Established<br>|Established<br>|
|Assessment of School and Student Performance|Advanced<br>|Established<br>|
|School Accountability|Established<br>|Emerging<br>|

Notes: Latent: reflects policy not in place or limited engagement; Emerging: reflects some good practice but still in progress; Established: reflects good practice with some limitations; and Advanced reflects international best practice 2
# **1. Introduction**

There is a general consensus on the need for Thailand to reform its education system to be able to compete with other high performing countries in the region (Patrinos _et al._ 2012). Education is crucial for economic growth (Jimenez, Nguyen and Patrinos 2012) and increasing the level of education of the labor force in rural areas contributed to 26 percent of the reduction in poverty. Nevertheless, Thai workers do not possess the skill set necessary to compete in the East Asian manufacturing markets (Bhaopichitr, Mala and Triratanasirikul 2012).

In terms of learning outcomes, the most recent evidence from PISA (OECD 2013) shows little improvement. In mathematics, the annualized rate of growth in Thai scores did not change since 2009, while reading scores increased by 1 percent and science scores by 4 percent. Despite these modest advances in reading and science, Thailand had a lower than average share of high scorers, and a higher than average share of low scorers, than the average for the 64 countries participating in PISA.

The Thai education system over-emphasizes inputs and has a large administrative structure that results in lower salaries for all, which reduces the probability of offering higher salaries for better teachers. This structural problem reduces equity and efficiency, and leads to a lack of higherlevel and critical thinking among students (Fry and Bi 2013).

Overall, there is a general consensus that Thailand will need to reconcile its intent in education system policies with the reality of the school level. One step in this direction is to assess the gap between policy intent and policy implementation, which is the main focus of this analysis.

The objective of this analysis is to compare policy intent with policy implementation in school autonomy and accountability in Thailand, with the purpose of assisting policy dialogue based on school-level evidence. Benchmarking policy implementation can be very useful for decision makers because the difference between policy intent and policy implementation can help fine tune the incentives and disincentives required at the school level to ensure that education policy has its desired effect.

**<mark>Box 1—SABER</mark>** <mark>.  The Systems Approach for Better Education Results (SABER) is an initiative of the World Bank with the purpose of producing comparative data and knowledge on education policies and institutions. Its aim is to help countries systematically strengthen their education systems. SABER evaluates the quality of education policies against evidence-based global standards, using new diagnostic tools and detailed policy data collected for the initiative. T</mark> he SABER approach encompasses: (i) an analytical framework to guide policy dialogue and foster policy reforms; (ii) relevant new data and information on education policies and institutional practices from other countries in the world, and (iii) concise analyses of how well the policies and institutions in each domain are aligned toward achieving Learning for All. SABER’s systems approach recognizes that improving education requires strengthening all factors that improve learning, and that a system's policies for governance, accountability, information systems, financing rules, and school management have to be aligned to ensure good performance. In the process, SABER identifies <mark>some common policy and institutional threads that run through most of the more successful experiences, allowing users to use the information as a tool for their own policy dialogue.</mark> Source: <mark>http://saber.worldbank.org/index.cfm</mark>

3
Data for this analysis comes from the assessment of indicators of School Autonomy and Accountability using a scale developed under the World Bank’s Systems Approach for Better Education Results, or SABER (see Box 1). The assessment of the indicators of policy intent in School Autonomy and Accountability was done in the first trimester of 2011, and validated during the East Asia Regional Conference on Systems Approach for Better Education Results (SABER), June 6-8, 2011 (World Bank 2012). The assessment of the indicators for policy implementation was done in 2012, using data collected in the 226 schools that were sampled by the 2009 PISA.

The report is organized as follows: Section 2 presents an overview of education in Thailand, setting the stage for the assessment of its education policies. Section 3 gives an overview of school autonomy and accountability through a conceptual framework that is used as the basis for the development of indicators of policy intent and policy implementation. Section 4 summarizes the results of the benchmarking on policy intent and discusses the content of the intent and implementation rubrics. This section presents the basic results of the two benchmarks. Section 5 presents the results of the implementation survey, and the links between the intentimplementation gaps with PISA results and selected covariates, and section 6 presents the conclusions and recommendations.

# **2. Education in Thailand: Searching for the path to high performance in education**

Education in Thailand is regulated by the Education Act of 2002, which establishes that all children should compulsorily study for nine years—six years of primary plus three years of lower secondary (OBEC 2003). For the past few years Thailand has made substantial gains in secondary education achievement, getting closer to the levels of the better performing countries in East Asia. UNESCO (2011) estimates that the school-life expectancy in Thailand is 12.3 years of education, which means that, on average, Thai youth are expected to finish secondary school.

Still, the Ministry of Education considers that Thailand has structural problems that are manifested in different forms:

- Education quality remains low due, mostly to the scarcity of good teachers. Based on learning assessments, there is a need to substantially improve learning in mathematics in primary school and in science in secondary school.

- Non-formal education still falls short of adequate coverage, particularly in rural areas, and problems continue in the area of education quality, which vary significantly by schools managed by different agencies.

- There is a need to improve primary and secondary education in order to produce students with higher levels of critical thinking and improved levels of problem-solving skills.

- Inequalities persist in education funding, especially between urban and rural areas.

Starting in 2002 school finances and school administration were decentralized at the Education Service Area level. The Education Service Area (ESA) is a division of the Ministry of Education responsible for the overseeing, monitoring, evaluation and dissolution of basic education institutions. The ESA also coordinates and promotes private institutions in the area and for the coordination with the Local Administrative Organizations (LAO, which are managed by the Ministry of the Interior). The ESA receives the transfers from the Ministry of Finance and 4
distributes the funds to the LOA or directly to schools. The LAO is the institution that will eventually administer schools autonomously. In the meantime, decentralization is in progress, transferring authorities from the ESA to the LAO as soon as it is feasible. Each LAO is administered by an Area Committee for Education, which includes community representatives, the Local Administrative Organization, the teachers’ association, the educational administrator associations, the parents’ association, and educational scholars. Geographically, the Education Service Area is above the local level but below the provincial level (UNESCO 2009). Under the decentralized framework schools are free to manage their non-salary expenditures. The Ministry of Education regulates policies and planning at the central level and each of the 76 provinces develops its own provincial plan, while local governments develop their own local policies.

Teachers are managed under the rules of the Teachers Civil Service Commission, which now allows for some managerial tasks to be performed by the Education Institution Committee at the local level. Teacher management has also been decentralized to the ESA level, but the Commission for Teachers and Administrative Personnel still has substantial influence, leaving the ESA only with the responsibility for the recruitment and deployment of teachers.

In terms of access, Thailand has made substantial progress in the last two decades. The gross enrollment rate for primary school is now above 100 percent; for lower secondary, it is 99 percent; and for upper secondary, 76 percent. In 2012 there were more than 5 million children enrolled in primary education, more than 4.7 million in secondary education and 2 million in tertiary education (UNESCO Institute of Statistics 2014). In addition, almost 1 million children attended vocational education (UNESCO 2011). Still, full coverage in basic education is not complete due to inaccessibility of some areas, and lower school access among the poor.

# **3. A Model for Assessing School Autonomy and Accountability**

Intuitively, good school management is a necessary but not a sufficient condition for learning. However, improper or inadequate management practices can be a substantial barrier to school and student performance. One can fix some managerial components and obtain no results or alter some other components and obtain good results. What combination of management practices are crucial for success are still under study, but the emerging body of practice point to a set of variables that foster managerial **_autonomy_** , the **_assessment_** of results, and the use of the assessment to promote **_accountability_** among all stakeholders (Patrinos, Velez and Wang 2013).

The lessons learned in the last 30 years indicate that by combining managerial autonomy, assessing student learning, and being accountable to parents and other stakeholders tends to produce good school performance and increased learning (Bruns, Filmer and Patrinos 2011). Together, these factors work like a system; that is, they work as a set of interconnected variables that can improve learning when they operate in close coordination. Moreover, these factors have well identified components that all schools can manage on their own, such as the management of the operating budget, the responsibility for selecting and managing teachers and staff, the management of the school’s infrastructure, and the monitoring and evaluation of teacher performance and of student learning outcomes.

_School autonomy_ is a form of school management in which schools are given decision-making authority over their operations (Patrinos 2011). Autonomy may take different forms and may include the hiring and firing of personnel, and the assessment of teachers and pedagogical 5
practices. School management under autonomy may give an important role to the School Council—representing the interests of parents—in budget planning and approval, as well as a voice/vote in personnel decisions.<sup>2</sup> By including the School Council in school management, school autonomy fosters accountability (Di Gropello 2004, 2006; Barrera, Fasih and Patrinos 2009).

_School accountability_ is defined as the acceptance of responsibility for one’s actions. In school management accountability may take other additional meanings, including: (i) the act of compliance with the rules and regulations of school governance; (ii) reporting to those with oversight authority over the school and (iii) linking rewards and sanctions to expected results (Heim 1996; Rechebei 2010).

Viewing the school as a closed system can lead to good insights on how to improve its performance. System performance is maximized when all variables are synchronized, allowing the system to close its internal loops. Under this framework, school management activities make sense when they contribute to system closure, otherwise they may have little impact on performance and become just a collection of isolated managerial activities. If school autonomy and school accountability regulate school management, managerial activities should foster an enabling environment for teachers and students, allowing for pedagogical variables, school inputs, and personal effort to work as intended, and helping achieve system closure.

In that vein it is clear that for managerial factors to become critical the system has to achieve _closure_ before their components reach a critical status. If a system closes, the sudden rupture in one of its components only leads to an imperfect solution. Therefore, defining a managerial system that can achieve closure is conceptually important. If the system is unable to close, are partial solutions effective? Yes, in a broad sense, in which schools can still function but their degree of effectiveness and efficiency would be lower than if the system closes. In this regard, the available evidence indicates that a school’s system can increase its chances of closure when it enforces enough autonomy to evaluate its results and use those results to hold someone accountable.

The main conclusion of the lessons learned on school-based management is that schools can improve learning for all students when they interconnect managerial autonomy, student assessments, and accountability, by aligning the functions of the school councils, the policies for improving teacher quality, and the functioning of the Education Management Information Systems (EMIS).

# **3.1 Autonomy and Accountability as System Components**

School autonomy and school accountability allow a school to close its internal system and maximize performance. Figure 1 shows how they interact with other system components and how they help achieve system closure and maximize performance. School Councils are crucial for implementing school autonomy because they become the representatives of the school clients: parents and students. As such, the School Council can be a resource to school management in the process of tailoring school services (curricula, teaching materials, school

> 2 For ease of presentation the term School Council Is used to represent the school-level organizations that represent parents and that are often referred to Parent-Teacher Association, Parent Council, among other names.

6
<!-- Start of picture text --> Fig. 1 School Autonomy and School Accountability in a closed system<br>Council<br>= Qualit<br><!-- End of picture text -->
for changes in pedagogical practices and to determine the training needs of their teachers. Both, pedagogical changes and teacher training are determinant factors of teacher quality (Vegas 2001). Finally, the role of EMIS on accountability has been well established and it is bound to increase as technology makes it easier to report on indicators of internal efficiency and on standardized test scores (Bruns, Filmer, and Patrinos 2011).

# **3.2 Linking the conceptual framework with SABER Indicators**

The last paragraph in the previous section helps define the types of indicators that would signal system closure and ensure better performance. Five indicators are used to benchmark school autonomy and accountability (SABER):

1. School autonomy in budget planning and approval – an increasing degree of local control over operating funds to better reflect the interests of parents and to improve operational efficiency

2. School autonomy in personnel management – giving the school more control over its teachers; depending on the laws, this control can range from teacher selection, to teacher contracting, and teacher salaries and bonuses 3. The participation of the school council (or the Parent Teacher Association as it is called in many countries) in school finance – dealing with their role in budget planning and management 4. The assessment of school and student performance – the regular measurement of teacher performance, student learning, or both; measuring school and student performance is a key precondition for ensuring accountability 5. School accountability to stakeholders – mechanisms in place to be held accountable to parents, local governments, and society at large Each of these indicators has a set of sub-indicators that give more details on the subject. Each indicator and sub-indicator is classified as _Latent_ , _Emerging_ , _Established_ or _Advanced_ . A Latent score indicates that the policy behind the indicator is not yet in place. An Emerging score indicates that the implementation of the program or policy is in progress. An Established score indicates that the program or policy is in operation and meets the minimum standards. An Advanced score indicates that the program or policy is in operation and reflects best practice.

# **3.3 Data Collection Process for the Policy Implementation Indicators**

All the 230 schools that participated in the PISA 2009 cycle were given a questionnaire for assessing school autonomy and accountability developed by the SABER initiative. The questionnaire was translated into Thai language with traces of modification as to fit to Thai education/schools system. Any modification was consulted and allowed by the SABER team.

The translated questionnaire was verified by Ministry of Education staff to ensure that the translation correctly interpreted the original English content, and that the translation was compatible with the Thai school context to ensure that school principals would understand the questions and reduce the number of ‘not _applicable’_ responses. Every school in the sample was contacted through the PISA 2009 schools coordinators. They were informed about the project 8
and its objectives. School coordinators were asked to contact schools principals to answer the questionnaires.

The questionnaire was reproduced and sent by registered post to the schools. One month was allowed for a response. A pre-paid envelope was provided to the schools along with a promise of a 500 Baht ($15) honorarium paid to the school for their collaboration. Most schools returned the questionnaire in time and those that did not were contacted by phone, through local education authorities and, for some schools, in person. In the latter case the survey staff had instructions to _sit and wait_ until the questionnaire was completed.

The answers to the questionnaires were keyed in the schools’ files and the schools’ PISA 2009 scores were matched to the questionnaire answers. The merged data has only the Schools ID since the school names are confidential.

School principals at all 230 schools filled out the implementation rubric by choosing the cell in the rubric that most closely described their own situation. The implementation rubric is shown in the annex. The responses were coded using a simple scale with the following values: Latent = 1 Emerging = 2 Established = 3 Advanced = 4 These values are used as identifiers of the indicators for the subsequent analysis. The calculation of an implementation score for an indicator is done with a simple formula: _Implementation Score_ = ( ( _Number of Latent scores_ * 1) + ( _Number of Emerging scores_ *2) + ( _Number of Established scores_ * 3) + (Number of Advanced scores * 4) ) / Number of school responses For example: For indicator 5.1 – defined as _Guidelines for using school and student assessments for accountability_ —in the SABER rubric, 127 schools responded as follows:

|Indicator 5.1|Latent|Emerging|Established|Advanced|Total|Implemen<br>tation<br>Score|
|---|---|---|---|---|---|---|
|Guidelines for<br>using school<br>and student<br>assessments<br>for<br>accountability|No<br>guidelines.<br>Assessments<br>are used only<br>by central<br>authorities<br>and school<br>staff|Municipal<br>governments<br>have guidelines<br>for using the<br>assessments,<br>but they cannot<br>take personnel<br>actions|Municipal<br>governments and<br>schools have<br>guidelines for<br>using the<br>assessments and<br>can take personnel<br>actions|Guidelines<br>are published<br>online and<br>can be used<br>by the public<br>to demand<br>accountability|||
||19|32|63|13|127|2.6|

With these responses the score for indicator 5.1 can be calculated as follows: Indicator 5.1 = ((19*1)+(32*2)+(63*3)+(13*4))/127 = (19+64+189+52)/127 = 324/127 = 2.6 For the overall score the formula remains the same, as it adds up all the scores in each category, multiply them by their respective factors, and divide the sum by the total number of schools in the sample.

9
# **4. Benchmarking Policy Intent and Implementation**

In 2011, the SABER School Autonomy and Accountability Scale (SAAS) – which measures policy _intent_ – was applied in Thailand to benchmark the existing framework of policies in this domain, and the results were discussed and validated with several institutions in the education sector. In 2012, a modified SAAS was applied in the 226 schools that participated in PISA 2009. The modified scale benchmarked the _implementation_ of school autonomy and accountability policies, with the objective of comparing policy _intent_ with policy _implementation_ in schoolbased management. The results of the assessment of **_policy intent_** are summarized in Table 1 and discussed in more detail in their respective subsections. Clearly, the key issue for policy dialogue is the gap that may exist between the policy intent outlined in Table 1 and the implementation of these policies at the school level.

|**Table 1: Thailand: Indicators of Policy****_Intent_ in School Autonomy and Accountab**<br>**1. School Autonomy in Budget Planning and Approval**|**ility 2011**|
|---|---|
|Schools have autonomy over non-salary expenditures, and their budget management plans<br>feed up to the Educational Service Area (ESA) level; education budgets are decentralized<br>at the ESA and to continue to the Local Administrative Organization level; administrative<br>systems are in place and well established at the ESA level.|Emerging<br>|
|**2. School Autonomy in Personnel Management**||
|Teachers are hired at the ESA level under the Teacher’s Civil Service Commission rules;<br>the selection process is bureaucratic and schools have little influence over their personnel<br>choices; there is limited scope for implementing a system for managing teacher incentives|Latent<br>|
|**3. Participation of the School Council in School Finance**||
|Parents participate in school board committees and play an advisory role at the local level,<br>and also contribute in fundraising activities; parents contribute up to 30% of total per<br>student expenditure in public schools, and the private sector another 10%; parent and<br>private donations are not registered in the school budget|Emerging<br>|
|**4. Assessment of School and Student Performance**||
|Office for National Education Standards and Quality Assessment (ONESQA) is an<br>independently run public organization that manages the periodic evaluations of school<br>performance; National Institute of Educational Testing Service (NIETS) is responsible for<br>standardized testing in basic and secondary education; national test scores and rankings<br>are publicized and used in education policy and private results are available to students<br>and parents<br>**5. School Accountability**<br>School boards can request the office of the Auditor General to perform financial audits;<br>school performance and learning outcomes are regularly made public; still, school<br>accountability is hampered by the lack of power of parents over school personnel and for<br>the weak linkages between student performance and teacher and school accountability|Advanced<br><br>Established<br>|

Source: World Bank 2013 10
# **4.1 School autonomy in budget planning and management**

The school autonomy in budget planning and management indicator has three sub-indicators:

- a. Level of autonomy in the management of the school’s operational budget

- b. Managerial autonomy of schools in managing non-teaching and teaching salaries

- c. Autonomy of principals to raise additional funds on top of what the school receives from the central and local governments

A _Latent_ indicator means that the school may not manage any money because the Ministry of Education or the local government sends goods directly to the school and pays for all services directly, without any intervention by the school. At the opposite end, an _Advanced_ indicator means that school administrators may receive a transfer that is deposited into the school’s bank account, disburse payments to teaching and non-teaching staff; pay suppliers of goods and services, and manage the school flow of funds.

In 2012, education funding in Thailand amounted to 441.5 million Baht ($13.6 million), or 18.7 percent of the government’s budget (Bureau of the Budget 2013). Of this amount, 341 million Baht were spent in basic education, and the rest in university education and other educationrelated expenditures. Funding is decentralized to the Educational Service Area (ESA), which may not correspond with geographical boundaries. Each ESA oversees the block transfers of operational funds to its schools. School principals have autonomy over the use of operational block transfers. Table 3 shows the results of benchmarking policy intent and policy implementation using the SABER rubric. A small panel of informants assessed policy intent, while the implementation score averages out the scores of 227 school principals.

**Table 2:  Thailand: Policy Intent and Policy Implementation Scores for Indicator of Budget Autonomy**

|**Sub indicator**|**Intent**<br>**Score**|**Implementation**<br>**Score**|**Intent**|**Implementation**|
|---|---|---|---|---|
|**1.1 Management of the**<br>**operational budget.**School<br>principals manage the operational<br>budget under ESAguidelines|**3.0**|**3.3**|Established<br>|Established<br>|
|**1.2 Management of non-**<br>**teaching staff & teacher**<br>**salaries.**ESA delegates managerial<br>authorityto schoolprincipals.|**1.0**|**2.8**|Latent<br>|Emerging<br>|
|**1.3 Raising additional funds for**<br>**the school.**School principals raise<br>additional funds from parents and the<br>private sector.|**3.0**|**3.0**|Established<br>|Established<br>|
|**Overall**|**2.3**|**3.0**|Emerging<br>|Established<br>|

Source: World Bank 2013 and school survey data. The differences between the intent and implementation scores are statistically significant at the 0.001 level except for indicator 1.3, where the difference is not statistically significant.

11
In terms of policy intent schools should not manage teacher and non-teacher salaries, as they are centrally regulated by civil service rules, resulting in a _Latent_ score of 1 (Table 2). In practice, however, school principals report that they do manage non-teaching and teaching staff salaries in coordination with their Education Service Area, resulting in a score of 2.7, which is the level of _Emerging_ . This difference shows flexibility on the part of ESA authorities for budget management at the school level, which bodes well for a possible increase in school autonomy. However, this is the only significant discrepancy between policy intent and policy implementation found in the indicator of budget autonomy.

# **4.2 School autonomy in personnel management**

The school autonomy in personnel management indicator refers to the level of autonomy that the school has in the everyday management of the school’s human resources. It has three subindicators:

- a. School autonomy in the hiring and firing of teachers

- b. Role of the School Council—as representative of parents—in decisions relating to teacher tenure or teacher transfers

- c. Level of autonomy in the hiring and firing of school principals

Teachers in Thailand are managed under the rules of the Teachers Civil Service Commission, which now allow for some managerial tasks to be performed by the Education Institution Committee at the local level. Teacher management has also been decentralized to the ESA level. However, the Commission for Teachers and Administrative Personnel retains administrative control over teachers. Moreover, at the ESA level teachers can only be recruited and deployed; they cannot be fired. As a result, teacher accountability is dissipated among the Civil Service Commission, the ESA, the Local Administrative Organizations, and the school, leading to situations where teacher accountability is low.

Personnel expenditures in education for 2012 amounted to 231 million Baht, or 52 percent of total government expenditures in education. Most of these expenditures go to pay teacher salaries (Bureau of the Budget 2013). Teacher management—the hiring and firing of teachers, and the administration of the payroll—has been recently decentralized to the level of Education Service Areas. Teacher hiring is regulated by the Teacher’s Civil Service Commission rules, which tend to protect job tenure. The teacher selection process is based on compliance of curricular requirements—such as the completion of a university degree—and without provision for more local control over teachers. Because teacher hiring and deployment is bureaucratic, schools have little room for maneuvering when assigned inadequate teachers. Moreover, since teacher salaries are regulated by Civil Service rules, there is limited scope for implementing a system of rewards and sanctions that could improve teacher incentives.

In terms of _policy intent_ , legal authority to manage school personnel is centralized, leading to an overall Latent score. Although an ESA can appoint a teacher, the law requires centralized approval, leading to a score of Emerging. In a similar vein, parents—acting through the School Council—have no legal authority over teacher tenure or over a request for a teacher transfer, rendering a Latent score to this sub-indicator. Finally, in terms of intent, the law says that the Ministry of Education appoints school principals, leading to a Latent score.

12
The assessment of _policy implementation_ shows less rigidity than the law currently allows, getting a benchmarking score of Emerging (Table 3). At the level of sub-indicator, municipallevel authorities seem to have more say over teacher hiring and firing than what the law allows, leading to a _score of 2.7 that could be called Emerging_ , which is more autonomous than the Emerging score given to policy intent. The same pattern is observed in the other two sub indicators, where principals report more involvement by municipal-level authorities than the law requires. In summary, personnel management is more flexible than what the law allows, and that flexibility is significant because parents and schools may use it for designing better teacher incentives.

|**Table 3: Thailand: Policy Intent and Implementation Scores for Indicator of Personnel**<br>**Management**|
|---|

|**Sub Indicator**|**Intent**<br>**Score**|**Implementation**<br>**Score**|**Intent**|**Implementation**|
|---|---|---|---|---|
|**2.1 School autonomy in teacher**<br>**appointment and deployment**<br>**decisions.**ESAappoint and deploy<br>teachers under union or civil service<br>agreements.|**2.0**|**2.8**|Emerging<br>|Established<br>|
|**2.2 School Council’s role in teacher**<br>**tenure or transfer.**Councils are<br>consulted and exert influence on transfer<br>requests.|**1.0**|**2.3**|Latent<br>|Emerging<br>|
|**2.3 Autonomy in the hiring and**<br>**firing of principals.**ESA has<br>autonomyinprincipal hiringand firing.|**1.0**|**2.9**|Latent<br>|Established<br>|
|**Overall**|**1.3**|**2.7**|Latent<br>|Emerging<br>|

Source: World Bank 2013 and school survey data. The differences between the intent and implementation scores are statistically significant at the 0.001 level.

# **4.3 Participation of the School Council in School Finance**

The participation of the council in school administration is very important because it is where parents can exercise their real power as clients of the education system. If the council has to cosign payments, then it has purchasing power automatically. The use of an operational manual is extremely important in this area, since it allows Council members to adequately monitor school management performance, help the principal with cash flow decisions, and become a catalyst for seeking additional funds from the community. The use of detailed operation manuals by the School Council is also a good vehicle for increased accountability and for the institutionalization of autonomy.

Policy intent for this indicator covers five sub-areas regarding the legal authority of the School Council:

- a. Participation in the preparation of the school budget

- b. Approval of the school budget

- c. Use of a manual outlining the participation of the School Council in school finances

13
- d. Role of the School Council in the implementation of the school budget

- e. Acceptance of the Council-approved budget by higher level authorities as a guide for detailed fiscal transfers to the school.  The more involved is the School Council in the planning and implementation of the school budget, the higher the probability for improving accountability.

Overall, in terms of _policy intent_ , Thailand gets an _Established_ score (Table 4). Parents participate in school board committees, which have a strong advisory role at the local level, and also contribute in fundraising activities. Since schools are responsible for drawing their own plans for managing non-salary resources, the role of parents in financial planning is very important. Currently, parents contribute between 20 and 30 percent of the total per student expenditure in public schools (UNESCO 2009) and another 10 percent is donated by the private sector following the lead of parents. Although parent participation is significant, current policies force schools to keep two sets of books in order to manage a combined budget of government and private funds, creating problems of operational efficiency and transparency. In terms of _policy implementation_ the participation of the School Council in school finance remains near the _Established_ stage. At the sub-indicator level participation is higher than the law allows in the areas of budget preparation and approval, with an _Emerging_ score for intent score and an _Established_ – for implementation.

**Table 4:  Thailand:  Policy Intent and Policy Implementation Scores for Indicator of School** **<u>Council Participation in School Finance</u>**

|**Sub-Indicator**|**Intent**<br>**Score**|**Implementation**<br>**Score**|**Intent**|**Implementation**|
|---|---|---|---|---|
|**3.1 Participation of the School**<br>**Council in budget preparation.**<br>SchoolCouncil has a voice in budget<br>planningandpreparation.|**2.0**|**2.8**|Emerging<br>|Established<br>|
|**3.2 School Council's authority to**<br>**approve the school budget.**School<br>Council consulted before approval.|**2.0**|**2.8**|Emerging<br>|Established<br>|
|**3.3 Manual for participation of**<br>**Councils in school finances.**There<br>are manuals, but participation by<br>Councils is less formal.|**3.0**|**2.8**|Established<br>|Established<br>|
|**3.4 Role of the School Council in**<br>**budget implementation.**ESA and<br>schools inform parents on off-budget<br>items on a regular basis.|**3.0**|**2.7**|Established<br>|Emerging<br>|
|**3.5 Use of budget prepared with**<br>**School Council's participation.**<br>Parent contributions used by ESA and<br>schools in budgetplanning.|**3.0**|**2.8**|Established<br>|Established<br>|
|**Overall**|**2.6**|**2.8**|Emerging<br>|Established<br>|

Source: World Bank 2013 and school survey data. The differences between the intent and implementation scores are statistically significant at the 0.001 level.

In the remaining three sub-indicators of School Council participation the intent and implementation scores are similar. The fact that schools must separate their real budget from the official budget means that School Councils have a say in budget planning and preparation, but it 14
is done with less certainty about the role of the council on budget management, and with lower chances for the real budget to be used as an input for education authorities above the school level. For the long term this gap reduces the transparency of school financing and reduces the potential for analyzing the efficiency of the school budget.

# **4.4 Assessment of School and Student Performance**

School assessments can have a significant effect on school performance because they force parents and teachers to agree on scoring rules and ways to keep track of them, allowing for accountability about student learning to emerge in a useful way. Measuring student assessment is another important way to determine if a school is effective in improving learning. A key aspect of school autonomy is the regular measurement of student learning, with the intent of using the results to inform parents and society, and to make adjustments to managerial and pedagogical practices. Without a regular assessment of learning outcomes school accountability is reduced and, with it, improving education quality becomes less certain.

There are five sub-indicators for assessment:

- a. Existence and frequency of school and student assessment

- b. Use of school assessments for making school adjustments

- c. Frequency of standardized student assessments

- d. Use of student assessments for pedagogical and personnel adjustments

- e. Publication of school and student assessments

Thailand has two government agencies in charge of system assessment: The Office for National Education Standards and Quality Assessment (ONESQA), which is responsible for school assessment, and the National Institute of Educational Testing Service (NIETS), which is responsible for student assessment. ONESQA is an autonomous agency responsible for conducting external evaluations of school quality in basic and post-secondary education, as mandated by the National Education Act of 1999 (Office of the National Education Commission 2003; Kingdom of Thailand 2000, 2003, 2004, 2009). Between 1999 and 2010, ONESQA conducted two complete rounds of school assessments in 40,000 schools, using an “amicable assessment model” to evaluate school performance. This approach compares a school’s performance with the standards set by the Ministry of Education using statistical data, indicators of quality, and a listing of the school’s strengths and weaknesses within an environment of cooperation in which consequences for low performance are framed in a positive way.

Assessments yield a set of recommendations, which should be implemented by failing schools during a grace period. Failing schools are revisited to verify compliance with the recommendations of the assessment. Every school in the country is assessed every five years by a team of evaluators employed and trained by ONESQA. Thus, the third round of evaluations began in 2011. For this round of evaluation ONESQA assessments began covering four areas: (1) learning outcomes; (2) school administration and management; (3) student-centered teaching and learning practices; and (4) internal procedures for quality assurance.

In 2010, an external evaluation committee reviewed ONESQA’s performance and gave it a _Good_ rating (85 percent), one level lower than the maximum rating of Very Good (90-100 percent). Most of the recommendations received by ONESQA were operational in nature and aimed at improving performance efficiency. However, a recent analysis of Thai education states that 15
ONESQA’s evaluations are too rooted on bureaucracy and do not reflect school quality as related to student learning (Tangkitvanich and Sasiwuttiwat 2012: 8-9). While the first round of ONESQA school assessment indicated that 65 percent of OBEC schools did not meet the minimum standards, the second round of assessments in 2010 indicated that only 20 percent of schools in basic education failed to meet the minimum standards (UNESCO 2011: 30). However, learning outcomes declined during the same period, suggesting that the assessments did not reflect school quality (Tangkitvanich and Sasiwuttiwat 2012). For the third round of assessments ONESQA has developed a different methodology, which is aimed at producing recommendations for improving education quality.

|**Table 5: Thailand: Policy Intent**|**and Imple**|**mentation Indicators o**|**f School and St**|**udent Assessment**|
|---|---|---|---|---|
|**Sub Indicator**|**Intent**<br>**Score**|**Implementation**<br>**Score**|**Intent**|**Implementation**|
|**4.1 Existence and frequency of**<br>**school and student**<br>**assessments.**Process well defined<br>but needs improvement in its<br>implementation.|**4.0**|**3.6**|Advanced<br>|Established<br>|
|**4.2 Use of school assessments**<br>**for making school**<br>**adjustments.**Links between test<br>results and adjustments not always<br>clear.|**4.0**|**3.2**|Advanced<br>|Established<br>|
|**4.3 Frequency of standardized**<br>**student assessments.**Process<br>well defined but needs improvement<br>in its implementation.|**4.0**|**3.7**|Advanced<br>|Established<br>|
|**4.4 Use of student assessments**<br>**for pedagogical and personnel**<br>**adjustments.**Links between test<br>results and adjustments not well<br>understood by principals and<br>teachers.|**4.0**|**3.4**|Advanced<br>|Established<br>|
|**4.5 Publication of school and**<br>**student assessments.**Process<br>well defined but needs improvement<br>in its implementation.|**4.0**|**3.6**|Advanced<br>|Established<br>|
|**Overall**|**4.0**|**3.5**|Advanced<br>|Established<br>|

Source: World Bank 2013 and school survey data. The differences between the intent and implementation scores are statistically significant at the 0.001 level.

The National Institute of Educational Testing Service (NIETS) was established in 2005, with the mission of implementing standardized student testing to measure learning outcomes in basic and vocational education. NIETS administers the Ordinary National Educational Test (O-NET), a standardized achievement test for grades 6, 9, and 12 in eight subject areas (Thai language, math, science, social science, religion and culture, health and physical education, art, career and technology, and foreign languages). Each year 950,000 students take O-NET in the 6<sup>th</sup> grade, 16
880,000 in the 9<sup>th</sup> grade, and 330,000 in the 12<sup>th</sup> grade. In addition, NIETS administers the National Test, a standardized test given to 3<sup>rd</sup> and 6<sup>th</sup> grade students. Local school systems administer their own tests to students in grades 2, 5 and 8. NIETS also administers two tests used to screen students for admission to the university: the General Aptitude Test (GAT) and the Professional and Academic Aptitude Test (PAT). These tests are given to students in Grade 12. The results of O-NET are published online, although the averages published without login-in are for the national level only.

In general, the principals’ assessment of policy implementation for this indicator is generally lower than the _Advanced_ score found in policy intent (Table 5). The lower score given by school principals may reflect the issues of reliability and validity, which lack meaningful analysis of test scores, or the low use of the results for making pedagogical and personnel adjustments because the school may exercise limited autonomy in these areas. It is important to point out that there is substantial room for improvement in the implementation of current school and student assessment policies. The qualifications to the existing system expressed by school principals reduce the rating of this sub-indicator to _Established_ , reflecting that the value of the indicator is near the Advanced level.

# **4.5 School Accountability to Stakeholders**

Accountability is at the heart of school-based management. The systemic connection between budgetary and personnel autonomy, parent participation in the financial and operational aspects of the school, and the measurement of learning outcomes are all aimed to reinforce accountability. Only by being accountable to parents can education quality be sustainable. The following indicators address the aspects of accountability that can be implemented within the framework of school-based management: the presence of written guidelines for the use of the results of school and student assessments; the strength of national institutions in charge of school and student assessments; the use of analytical comparisons to inform the public and education stakeholders about the performance of schools and students; the rights of school councils to request financial audits; and the presence of written guidelines for the school council with which to assess school financial performance.

The indicator of school accountability has five sub-indicators:

- a. School has guidelines for using school and student assessments for accountability

- b. There are national and/or regional systems for educational assessment

- c. There are reports with comparisons of school and student performance

- d. School Councils have legal authority to ask for financial audits

- e. There are manuals for the participation of the School Council on school audits

The results of the assessment of policy intent are mixed. Two sub-indicators received an _Emerging_ score: the presence of guidelines for interpreting school and student assessments, and the legal authority for school councils to request an audit also. Two other sub-indicators got an _Advanced_ score: the national system for assessing schools and students, and the presence of an audit manual for school councils. School boards do not have direct legal authority to ask for financial audits to enforce financial transparency and accountability, but can request the office of the Auditor General to perform one. Still, school accountability is hampered by the lack of 17
power of parents over staffing decisions, which can be a very important factor in improving learning.

Overall, the implementation of accountability needs improvement and it is rated as _Established -_ (Table 6). The implementation indicators for the functioning of a national system of school and student assessment and for the manuals for interpreting school audits are significantly lower than the indicators of policy intent. The gaps in these indicators suggest problems with key aspects of accountability to parents, especially in the areas of parent information about the meaning of test results, and on the use of results by schools to make modifications to their pedagogical practices.

|**Table 6: Thailand: Policy Intent a**|**nd Policy I**<br>|**mplementation Indic**<br>|**ators of Schoo**|**l Accountability**|
|---|---|---|---|---|
|**Sub Indicator**|**Intent**<br>**Score**|**Implementation**<br>**Score**|**Intent Score**|**Implementation**|
|**5.1 Guidelines for using school**<br>**and student assessments for**<br>**accountability.**ESAuses the<br>guidelines for teacher assessments<br>andpersonnel management.|**2.0**|**3.1**|Emerging<br>|Established<br>|
|**5.2 National or regional systems**<br>**of educational assessments.**ESA<br>autonomy in the interpretation of<br>guidelines maybe inconsistent.|**4.0**|**3.1**|Advanced<br>|Established<br>|
|**5.3 Comparisons of school and**<br>**student performance reports.**<br>Comparisons done at National,<br>regional and municipal levels, and<br>previous years. Schools often publish<br>summaryresults online.|**3.0**|**3.0**|Established<br>|Established<br>|
|**5.4 School Council's authority to**<br>**perform financial audits.**<br>Councils tend to rely on school-level<br>negotiations, even though they can<br>ask for formal audits.|**2.0**|**2.6**|Emerging<br>|Emerging<br>|
|**5.5 Manual for the participation**<br>**of the School Councils in school**<br>**audits.**Manuals exist but personal<br>communications are preferred by<br>Councils.|**4.0**|**2.5**|Advanced<br>|Emerging<br>|
|**Overall**|**3.0**|**2.8**|Established<br>|Emerging<br>|

Source: World Bank 2013 and school survey data. The differences between the intent and implementation scores are statistically significant at the 0.001 level, except for indicator 5.3.

# **4.6 Intent and Implementation Scores by Type of School Sponsorship**

The differences between the law’s intent and what happens on the ground may be due to many factors, but one that needs to be explored first is the type of school sponsorship. High schools in Thailand can be classified by type of institutional sponsor. Most schools are administered by the Office of the Basic Education Commission (OBEC), while others are administered by the Bangkok Metropolitan Administration (BMA), the Bureau of Local Educational Development (BLED), the Commission for Higher Education (CHE) also known as SATIT schools. Private 18
schools may or may not receive government funding, but those that do are under the supervision of the Office of the Private Education Commission (OPEC).

Resources available to each type of school vary widely. As reported by Patrinos _et al_ (2012), SATIT schools, which are governed by the Commission of Higher Education, have more resources and more teacher than other public schools. If school autonomy in the Education Service Areas vary in its interpretation by the schools—as measured by the SABER indicators— the impact on learning outcomes within the context of autonomy needs to be explored.

After analyzing PISA data for 2006 and 2009 using a Oaxaca-Blinder decomposition methodology, Patrinos _et al_ (2012) conclude that a decline in the reading scores among the poor was the net result of a decline in system quality, which was larger than the increase in student performance, especially among the poor. Relatedly, schools in Bangkok outperformed school in the rest of the country, and system performance was also a reflection of systems resources, which were correlated with student wealth. These findings have direct implications for the analysis of SABER indicators inasmuch they suggest a need to examine policy implementation by type of school and geographical area.

Table 7 shows the different implementation and scores by type of school sponsorship. There are no significant differences among the SABER sub-indicators by type of school sponsorship, but the lack of a significant difference is related to the high variation found within the indicators, which tended to reduce the statistical significance of the average differences between school types (see Table A6 in the Annex). In terms of average sub-indicator scores, however, the average differences between intent and implementation are statistically significant.

|**Table 7: SABER indicator scores fo**|**r policy im**|**plementa**|**tion by ty**|**pe of scho**|**ol sponso**|**r, 2013**<br>SABER|SABER|
|---|---|---|---|---|---|---|---|
|Indicator|OBEC|OPEC|BMA|BLED|SATIT|<br>Implem.|<br>Intent|
|1a. Legal authorityover management of budget|3.3|3.3|3.2|3.1|3.4|3.3|3.0|
|1b. Legal authorityover management of salaries|2.8|2.9|2.7|2.6|2.5|2.8|1.0|
|1c. Legal authorityto raise additional funds|3.0|3.1|3.2|3.0|2.8|3.0|3.0*|
|2a. Autonomyin teacher appointment decisions|2.8|2.9|2.7|2.7|2.9|2.8|2.0|
|2b. School council role in teacher tenure or transfer|2.3|2.2|2.2|2.5|2.2|2.3|1.0|
|2c. Autonomyin the hiringand firingofprincipals|2.9|2.9|2.5|2.8|2.8|2.9|1.0|
|3a. Participation of council in budgetpreparation|2.8|3.0|2.5|2.5|2.7|2.7|2.0|
|3b. School council authorityto approve budget|2.8|2.9|2.8|2.7|2.8|2.8|2.0|
|3c. Manual forparticipation of council in finances|2.8|2.7|3.1|2.5|2.6|2.7|3.0|
|3d. Role of council in budget implementation|2.7|2.7|3.2|2.6|2.5|2.7|3.0|
|3e. Use of budgetprepared with council|2.8|2.7|3.1|2.4|3.0|2.8|3.0|
|4a. Existence and frequencyof assessments|3.6|3.5|3.8|3.6|3.6|3.6|4.0|
|4b.Use school assessments for decisions|3.2|3.2|3.2|3.4|3.2|3.2|4.0|
|4c. Frequencyof standardized student assessments|3.7|3.7|3.8|3.7|3.7|3.7|4.0|
|4d.Use of assessments for adjustments|3.4|3.4|3.2|3.4|3.3|3.4|4.0|
|4e. Publication of school and student assessments|3.7|3.6|3.6|3.5|3.5|3.6|4.0|
|5a.Guidelines for assessments for accountability|3.1|3.0|3.1|3.2|3.2|3.1|2.0|
|5b. National or regional systems of assessments|3.2|3.1|3.2|3.2|3.0|3.1|4.0|
|5c.Comparisons ofperformance reports|2.9|2.8|2.7|2.9|3.3|3.0|3.0*|
|5d.Council authoritytoperform financial audits|2.6|2.8|2.2|2.8|2.4|2.6|2.0|
|5e. Manual forparticipation of council in audits|2.6|2.4|2.3|2.4|2.5|2.5|4.0|

Source: Estimated by authors. Difference between the implementation and intent score is statistically significant at the 0.00l level except for 1c and 5c Note: OBEC = Office of the Basic Education Commission; OPEC = Office of the Private Education Commission; BMA = Bangkok Metropolitan Administration; BLED = Bureau of Local Educational Development; SATIT = Commission for Higher Education 19
This statistical significance can be summarized as follows:

- a. In budgetary autonomy, schools tend to behave more autonomously than what is intended by education policy, and sponsorship does not seem to matter

- b. In terms of personnel management, autonomy implemented at the school level is higher policy intent, regardless of school sponsorship

- c. In the area of parent participation in school finance the score for policy intent in Bangkok Metropolitan Administration (BMA) schools is higher than the scores of other schools, and higher than the score for policy intent

- d. In the area of school and student assessment policy implementation lags behind intent; there are no clear differences among school sponsors

- e. School accountability is mixed, with some policies showing higher implementation scores and others showing higher intent scores, with no clear differences found among sponsors

# **5. Is policy** **_implementation_ of autonomy and accountability important for student learning?**

One of the key objectives of SABER is to assist governments in monitoring education policies conducive to improved education quality. Schools can be classified by type of school financing: public, private-subsidized, and fully private. In principle, classifying schools by the source of school financing should reflect the degree of autonomy that schools may have in selecting students and teachers, and in being more autonomous and, to some extent, more accountable.

Table 8 shows the implementation scores for schools classified by its source of financing. Public schools are completely funded by with government funds; private-subsidized schools receive government funds to cover their operational expenses but can have significant autonomy in selecting students and teachers; private-independent schools are reliant on private tuition and are completely autonomous in their selection of students and teachers.

The results of SABER implementation scores tend to be aligned with the source of school financing. Public schools tend to have lower implementation scores than private-subsidized schools, and both of these types of schools tend to show lower SABER scores than privateindependent schools.

20
**Table 8:  Thailand:  Implementation scores for SABER indicators of autonomy and accountability** **<u>(2012) and average PISA Scores (2009)</u>**

|School Type|Budget<br>autonomy|Personnel<br>management<br>autonomy|School<br>council's<br>participation|School &<br>Student<br>Assessment|Accountability|<sup>PISA</sup><br>Score|N|
|---|---|---|---|---|---|---|---|
|Public|3.0<br>(0.6)|2.6<br>(0.7)|2.7<br>(0.8)|3.6<br>(0.6)|2.9<br>(0.6)|425|201|
|Private<br>subsidized|3.3<br>(0.5)|3.6<br>(0.6)|3.1<br>(0.7)|3.4<br>(0.7)|3.0<br>(0.6)|403|19|
|Private<br>independent|3.7<br>(0.5)|3.7<br>(0.5)|3.3<br>(0.5)|3.5<br>(0.6)|3.3<br>(0.8)|432|6|
|Total|3.0<br>(0.6)|2.7<br>(0.8)|2.8<br>(0.8)|3.6<br>(0.6)|2.9<br>(0.6)|423|226|
|PISA 2009<br>Score ≥500|3.2<br>(0.6)|2.8<br>(0.7)|2.9<br>(0.9)|3.5<br>(0.6)|2.9<br>(0.8)|535|21|
|PISA 2009<br>Score ≤400|2.9<br>(0.6)|2.6<br>(0.8)|2.7<br>(0.7)|3.5<br>(0.7)|2.8<br>(0.6)|376|80|

Source: Estimated by authors Note: Numbers in parenthesis are the standard deviations PISA scores also show a slightly different mix than the SABER scores. The PISA scores in public schools tend to be higher than in private-subsidized schools, but lower than in privateindependent schools.

Schools with an average PISA score above 500 points show significantly higher indicators of budget autonomy than schools with PISA scores below 400 points. Higher performing schools have more budgetary autonomy. Private subsidized schools have higher indicators of budget autonomy than public schools, but have lower average PISA scores. Similar statements could be made for the differences between indicators of personnel management and accountability, but the differences between schools with high PISA scores and schools with low PISA scores is smaller than in the case of budget autonomy.

# **5.1 Exploratory Analysis of the Association between School Autonomy and Accountability and Academic Achievement**

The scores in Table 8 are just an approximation, since there is a need to control for several school characteristics in the sample population. Table 9 presents the results of regressing the PISA score for reading on the average total score for the SABER indicators of autonomy and accountability as independent variables, after controlling for selected school characteristics.

Overall, the results of the estimations show that PISA scores could be significantly associated with autonomy in personnel management. The overall impact of the SABER total score is significant, as shown in equation 12 in Table 9. A one-point increase in the total SABER indicator score accounts for an increase of 11 points in the PISA reading score. This could be very significant, as it suggests that schools with higher levels of autonomy and accountability have higher PISA scores.

At the sub-indicator level the results consistently show significant test score impacts associated with budgetary autonomy (Equations 1 and 2 in Table 9) and personnel management autonomy 21
(Equations 3 and 4 in Table 9). The equations for budgetary autonomy indicate that a one-point increase in the indicator is associated with more than 19 points in the PISA reading score for the school. After adjusting for covariates, the one-point increase in the indicator is associated with an increase of 9.4 points in the PISA reading score.

The equations for personnel autonomy also show similar associations. Schools that make personnel decisions at the school level tend to have higher PISA scores. Table 9 shows that a one-point increase in the implementation indicator of personnel autonomy is associated with an increase of 10.6 points in the school’s average PISA reading score. After adjusting for covariates, personnel autonomy at the school level is associated with an increase in the math test score of 9.5 points. Table 9 also shows that the indicators for school council participation, and for student assessment, are associated with higher reading scores. This effect, however, tends to be attenuated once the equation includes some key covariates.

The results of Table 9, particularly in the area of personnel autonomy, are consistent with the results of a recent factor analysis of SABER indicators for East Asia (Arcia, Lewis and Patrinos 2012) which shows that teacher management is one of two broad factors that explain most of the variation in SABER indicators of policy intent in 20 areas of influence in education.

22
**Table 9:  Exploratory Analysis of the Association between School Autonomy and Accountability and Test Scores (PISA 2009)**

|**Variables**|**1**|**2**|**3**|**4**|**5**|**6**|**7**|**8**|**9**|**10**|**11**|**12**|
|---|---|---|---|---|---|---|---|---|---|---|---|---|
|Overall SABER Score||||||||||||11.6*|
|Autonomy: Budget|19.2**|9.4*|||||||||15.0**||
|Autonomy: Personnel|||10.6**|9.5**|||||||3.9||
|School Council|||||9.0*|4.8|||||1.6||
|Student Assessment|||||||15.4**|5.3|||9.3||
|School Accountability|||||||||9.7|1.0|-0.8||
|Controls|NO|YES|NO|YES|NO|YES|NO|YES|NO|YES|NO|YES|
|Constant|363.4***|367.3***|392.8***|361.5***|396.3***|384.5***|367.1***|379.6***|393.3***|396.3***|331.1***|367.8***|
|R2|0.043|0.430|0.0190|0.440|0.015|0.410|0.020|0.430|0.012|0.430|0.055|0.391|
|F|10.15**|23.9***|4.37**|24.7***|3.41*|23.4***|4.68**|23.2***|2.62|23.0***|2.55**|28.3***|
|N|225|225|225|225|225|225|225|225|225|225|225|225|

Source: Estimated by the authors Notes: Significance: * = 0.1, ** = 0.05, *** = 0.01; PISA scores have a mean of 500 and a standard deviation of 100; controls: total school enrolment; student-teacher ratio; quality school resources; index academic selectivity; public school 23
# **6. Conclusions**

Benchmarking policy intent in education is useful for assessing a country’s position on certain education areas relative to the policy position of other countries. For purposes of a more focused policy dialogue, benchmarking policy implementation can be of added utility to decision makers, since the difference between the value of an indicator of policy intent and the value of the same indicator in policy implementation can help fine tune the incentives and disincentives required at the school level to ensure that the policy has its desired effect.

In terms of the differences between the indicators of policy intent in school autonomy and accountability, and the indicators of policy implementation, as reported by the school principals of 230 schools that participated in the 2009 PISA survey, the results show the following:

1. School principals are already managing budgets related to personnel in ways that go beyond what the law intends them to do.  The difference between policy intent and policy implementation in budget management suggests that authorities at the Education Service Area level have found that important personnel management functions could be delegated to school principals, thus deepening school autonomy.

2. Personnel autonomy—the hiring and firing of teachers—also seem to be delegated to local areas under significant influence from school principals. In general, in all areas of personnel autonomy, school authorities are making decisions that are later corroborated by the Education Service Area.

3. In terms of parent participation through the school council, the indicator results show that parents already have more say over the school budget than what the law currently allows. Again, this bodes well for increasing autonomy at the school level.

4. In terms of school and student assessment, the results show an opposite trend. In theory, the current system of school and student assessment is at an advanced level. However, principals give it a lower rating because of perceived problems testing at the school level.

5. School accountability shows a mixed picture. The implementation indicators for the functioning of a national system of school and student assessment and for the manuals for interpreting school audits are significantly lower than the indicators of policy intent. The gaps in these indicators suggest problems with key aspects of accountability to parents.

6. In terms of the relationship between SABER indicators of policy implementation in school autonomy and accountability, and student learning, the results show that school autonomy budget planning and management, and in in personnel management are associated with higher PISA scores. These are key conclusions for education policy, since they seem to confirm the trend found in high performing countries, where giving schools more autonomy leads to higher learning outcomes.

The exploratory regression results are important because their statistical significance reinforces the notion of evidence-based policy reform, which is at the core of the SABER approach. The results point towards a consistent relationship between budget and personnel autonomy on student learning. The evidence presented here is significant enough to suggest as a next step to include budget autonomy and personnel management autonomy as components of evidencebased policy dialogue.

24