metadata: authors: 0: author: Patrinos,Harry Anthony
1: author: Velez Maldonado,Eduardo Demy
2: author: Wang, Catherine Yan
count: World
docna: 0: docna: Framework for the reform of education
            systems and planning for quality
docty: Policy Research Working Paper
subtopic: Reproductive Health,Early Child and Children's Health,Nutrition,Early Childhood Development,Education Reform and Management,Educational Sciences,Educational Policy and Planning,Educational Institutions & Facilities,Educational Policy and Planning -
            Ministry of Education,Educational Policy and Planning -
            Institutional Development,Educational Policy and Planning -
            Politics of Education Reforms,Effective Schools and Teachers
teratopic: Education
abstracts: cdata!: In 2000, the goal that, by 2015, all
            children will have access to, and complete, free and
            compulsory primary education of good quality, was set.
            Despite the progress in terms of student enrollment and
            completion, the quality of learning produced in developing
            countries remains poor. Existing models of education
            production are inadequate for informing education reform for
            the purpose of improving school quality, as measured by
            student learning. Thus, a broader and more integrated
            approach of policy making is put forward. Building on theory
            and empirical evidence on what works, the paper puts forward
            a framework for improving the quality of education. The
            framework includes six factors: (1) assessment; (2)
            autonomy; (3) accountability; (4) attention to teachers; (5)
            attention to early childhood development; and (6) attention
            to culture. Going forward, there is a need to develop a
            system of international quality benchmarks drawing on a
            larger body of evidence. Most importantly, more empirical
            evidence from impact evaluations is needed.
repnme: repnme: Framework for the reform of education
            systems and planning for quality
display_title: Framework for the reform of
            education systems and planning for quality
listing_relative_url: /research/2013/11/18513211/framework-reform-education-systems-planning-quality
topicv3: Teachers Management m1326119 1523,Education, Skills Development and Labor
            Market m1326135 1484,Education m1327048 470,Education Reform m1327269 1518,Early Child Development m1327727 479
publishtoextweb_dt: 2013-11-01T00:00:00Z
historic_topic: Education
no_of_pages: 27
author_keys: 0: author_key: 95903
1: author_key: 2066053
model_extractions: WPS6701 Policy Research Working Paper 6701 Framework for the Reform of Education Systems and Planning for Quality _Harry Anthony Patrinos Eduardo Velez Catherine Yan Wang_ The World Bank Human Development Network Education Unit November 2013
Policy Research Working Paper 6701

# **Abstract**

In 2000, the goal that, by 2015, all children will have access to, and complete, free and compulsory primary education of good quality, was set. Despite the progress in terms of student enrollment and completion, the quality of learning produced in developing countries remains poor. Existing models of education production are inadequate for informing education reform for the purpose of improving school quality, as measured by student learning. Thus, a broader and more integrated approach of policy making is put forward. Building on theory and empirical evidence on what works, the paper puts forward a framework for improving the quality of education. The framework includes six factors: (1) assessment; (2) autonomy; (3) accountability; (4) attention to teachers; (5) attention to early childhood development; and (6) attention to culture. Going forward, there is a need to develop a system of international quality benchmarks drawing on a larger body of evidence. Most importantly, more empirical evidence from impact evaluations is needed.

This paper is a product of the Education Unit, Human Development Network. It is part of a larger effort by the World Bank to provide open access to its research and make a contribution to development policy discussions around the world. Policy Research Working Papers are also posted on the Web at http://econ.worldbank.org. The authors may be contacted at hpatrinos@worldbank.org. .

_The Policy Research Working Paper Series disseminates the findings of work in progress to encourage the exchange of ideas about development issues. An objective of the series is to get the findings out quickly, even if the presentations are less than fully polished. The papers carry the names of the authors and should be cited accordingly. The findings, interpretations, and conclusions expressed in this paper are entirely those of the authors. They do not necessarily represent the views of the International Bank for Reconstruction and Development/World Bank and its affiliated organizations, or those of the Executive Directors of the World Bank or the governments they represent._ Produced by the Research Support Team
# **Framework for the Reform of Education Systems and Planning for**

# **Quality**

Harry Anthony Patrinos, Eduardo Velez and Catherine Yan Wang<sup>1</sup> JEL Classifications: I21, O32, D23 Keywords: Benchmarking, Education systems, Education quality Sector Board: Education

> 1 Harry Anthony Patrinos, Manager, Education, World Bank <u>(hpatrinos@worldbank.org), Eduardo Velez, former Education</u> Sector Manager, East Asia Region, World Bank, and Catherine Yan Wang, Director, Department for International Exchange, National Institute of Education Sciences of China.  All views expressed here are those of the authors and should not be attributed to their respective organizations.
# **Introduction**

In 2000, the World Education Forum in Dakar, Senegal, set the goal that by 2015, all children will have access to, and complete, free and compulsory primary education of good quality. Despite the progress made in terms of enrollment and completion rates, the quality of learning outcomes in developing countries remains a formidable challenge. According to the most recent statistics, there were still 67 million primary school-age children out of school worldwide in 2009 (UNESCO UIS 2011). Moreover, schools in many developing countries are failing to teach foundational cognitive skills and significant numbers of students do not achieve the expected minimum levels of learning (World Bank 2010). Likewise, a large number of developed countries are yet to achieve the goal of quality measured by students’ learning outcomes to all the learners (OECD 2010).

Countries instituted structural, curricular and pedagogical reforms to improve students’ learning outcomes, yet problems persist. Quality is more and more becoming the central topic in education reform and planning in many countries. What are the key quality drivers underpinning the education reform and planning? What aims and measures are pivotal to the institutional development of education? To address an urgent gap in our understanding of the determinants of quality education, we put forward a framework that can help us understand policy making in the education sector. The framework serves as an overarching guide to policy making. Nevertheless, the relevance of the framework will only be proved when it is applied across a large number of countries. This paper strives to synthesize the major findings on factors contingent on educational quality and illustrate the patterns and critical pathways toward the goal of quality, shedding light on the rationale of education reform in both developing and developed countries.

# **Conventional Quality Framework**

How is student learning achieved? A common/traditional approach examines the quality of education focusing on particular components of the education system. At the micro-level, studies tend to focus on student characteristics with respect to their endowments and behaviors, as well as characteristics of their parents or families (Coleman et al. 1966), or focus on schools (Sammons, Hillman and Mortimore 1995; Creemers 1997) and/or behaviors of teachers (Bennett et. al 1976; Hanushek 1971; Wayne and Young 2003; Rivkin, Hanushek and Kain 2005). At the macro-level, quality studies often focus on specific policy actions, programs and reforms, as well as the education system and its organization and institutional characteristics (Bishop 1997; USAID/EQUIP2 2006; Woessmann 2003). Ideally, the economic, political and social contexts are taken into account. For example, some studies about student learning performance scrutinize economic and financial factors (Hanushek 1986, 1996; Greenwald, Hedges and Laine 1996; Levacic and Vignoles 2002). Other studies look into the social context relating to educational quality to dig out the social factors that hinge upon student performance (Hallinger and Murphy 2
1986; Alexander 2000). Figure 1 illustrates a summary of the traditional quality framework.

Yet, all these approaches are confined by analysis of the segmented education system. The findings and inferences from those studies are largely detached from the realistic complexities of education. When applied to diagnose educational problems and reforms, these amount to “treating the symptoms rather than the disease” prescriptions,<sup>2</sup> and often do not make a difference in student learning achievement.

**Figure 1: Traditional Quality Framework**

<!-- Start of picture text --> Economic, political,<br>and social context<br>Education<br>Policy<br>Policy actions Schools<br>Students<br>Specific Priori knowledge<br>Endowment and<br>programs and behaviors of<br>Behaviors of Systemic reforms Teachers<br>Students<br>Schools<br>Parents<br>Families<br>The Education<br>System<br>Its organization and<br>institutionality<br>Learning<br><!-- End of picture text --> Source: Vegas and Petrow 2007: 66 As an alternative, this paper suggests a systems approach to view the quality of education, echoing the motif of the World Bank Education Strategy 2020 (World Bank 2011). A key assumption is that analysis of the quality of education has to shift from a factor-dependent approach to a more systemic approach, including an increasing engagement of key quality drivers and prioritizing significant quality elements in the delivery of education services.

> 2 This is a Chinese saying used in Chinese herbal medicine, originally expressed as “ _zhi biao bu zhi ben_ ”, which means that the prescribed medicine treats the symptom of the disease while neglecting the cause of the disease.

3
# **Redefining the Quality Framework**

The approach employed here provides a broader and more inclusive perspective on education planning and reform than is frequently invoked in traditional educational studies. It builds on the theories and findings from related studies on educational quality. By analyzing the lessons and experiences across countries during the last two decades, the paper addresses gaps and hindrances in improving education quality and thus develops a new framework of quality education.

What are the major constraints against achieving the quality goal across countries? Studies suggest that three central institutional incentives affect learning outcomes: (i) choice and competition, (ii) school autonomy and (iii) school accountability (Bruns, Filmer and Patrinos 2011; World Bank 2003). These institutional incentives were reconceptualized as key policy levers of education systems; that is, autonomy, accountability and assessment – or, the ‘three A’s’ (World Bank 2010; Patrinos 2010a). Apart from the above three institutional factors, three structural quality elements are also pivotal to achieving the goal of quality, yet they are often neglected. These are Attention to teachers, Attention to early childhood development, and Attention to culture, each addressing a dimension contingent upon successful education reform. Together, these **_<u>6As</u>_** represent an input-output-outcome approach tackling the issue of quality. In this framework, quality is viewed as more than a goal; it also connotes aims, measures and destinations in concrete terms. Moreover, it integrates institutional, structural and contextual considerations. Policy makers may use these quality drivers to guide decisions about important education subsystems to achieve improved learning. Similarly, education projects and programs could be designed following the **_<u>6As</u>_** in line with the vision of the education authorities. The **_<u>6As</u>_** framework is illustrated in Figure 2.

4
**Figure 2: The** **_<u>6As</u>_** **<u>: An Integrated Approach to Quality Education</u>**

<!-- Start of picture text --> Autonomy<br>Achievement<br>Learning<br>Accountability Assessment<br>Attention to<br>teachers Attention to ECD Attention to Culture Other 1, 2, n<br><!-- End of picture text -->

# **Fallacy of the Quality Path/Cycle**

In the recent quality debate, more and more policy makers’ attention has shifted from inputs to outputs. More quality objectives in terms of assessment, autonomy and accountability appeared being embraced in education reform across countries.

Closer scrutiny identifies that the traditional definitions (benchmark) of autonomy, accountability and assessment are limited. In particular, autonomy is directly associated with school-based reform, school grants and/or accreditation. The accountability debate is dominated by information flow (World Bank 2003), including questions of teacher attendance, school committee functions, school report cards, budgets, and performance-based salaries. In the same vein, assessment is used interchangeably with sample-based tests, competency exams, and early grade reading assessment. But any concepts of autonomy, accountability or assessment mean much more.

Figure 3 shows the typical quality path/cycle that we are proposing to follow. It highlights the questions that should be answered to reflect the true impact of the **_<u>6As</u>_** : If autonomy, accountability, and assessment are implemented, do they lead to increased instructional time, textbook supply, more effective teacher training, and contemplation of subject matter? If yes, we may get closer to producing quality results. But even if the answers are on the positive side, another issue arises: Are inputs used in class for instruction? If they are, actual addition of instruction might be reinforced and semantic memory formation may increase. As a result, 5
students might score higher. In addition, dealing with teachers, ECD and culture are important to determine quality improvements. Even if the **_<u>6As</u>_** are provided in the right way, they might not lead to the desired learning outcomes _per se_ . Thus, the outcomes may be unsatisfactory. They only make a difference when they influence actual instruction in the classroom and contribute to semantic memory formation. For this end, quality elements such as early childhood development, good teachers and culture have to be designed with this implication in mind.

**Figure 3: Proposed Quality Diagnostic Path**

<!-- Start of picture text --> Autonomy Accountability Assessment<br>School Grants, Performance-based Sample-based tests,<br>SBE, accreditation, etc salaries, etc. competency exams, EGRA,<br>Irrelevant,<br>Increased instructional time, textbook supply, more effective teacher No more of the<br>training, and contemplation of subject matter or not? same<br>Yes Project outcome unsatisfactory<br>Were inputs used in Yes Effective instruction and Students<br>class for instruction?<br>semantic memory formation Learning<br>Achievement<br>Yes Yes<br>Were other “quality” inputs provided?<br>No<br>ECD, Good teachers, culture, etc.<br>Project design unsatisfactory<br><!-- End of picture text --> Source: Authors Obviously, the fallacy of the quality cycles lies in the absence of quality elements in the education system and/or inconsistencies and gaps between these elements. For example, focusing on improving schools as organizations or focusing on improving the performance of individual teachers may be effective, but focusing on improving schools and the performance of individual teachers at the same time will be more effective. The interaction of the **_<u>6As</u>_** is more important than the addition of the **_<u>6As</u>_** <u>. Many other examples can be given to illustrate the pivotal</u> difference that is made by pursuing a view of quality as an integrated process of factors that connect with one another. A broad approach of this kind permits simultaneous appreciation of the vital roles, in pursuit of quality education, of many different institutions, including educational institutions, governments and local authorities, market-related organizations and other stakeholders. The relationship between autonomy, accountability and assessment; and attention to teachers, early childhood education and culture is not additive, not even linear; rather, the model is iterative and interactive. In other words, it implies multipliers, not additivity. The following section attempts to elaborate the significance of and the relationship between the proposed quality elements.

6
# **Six Quality Elements**

In this section we elaborate on the six elements of quality mentioned above.

# <u>Quality Element 1: Assessment</u>

The failure of education reforms to achieve the goal of quality is partly attributable to absence of instruments and procedures that provide information to key consumers – namely teachers, administrators, policy makers but especially students and their families – on students’ learning. Take a typical example: studies indicate that students should read a minimum of one word per second, 60 words per minute on the average. If they do not read at this speed, they forget the content as they read, and by the end of the sentence they will not remember what they read at the beginning. So comprehension will not actually occur (Abadzi 2006, 2008). Presumably, the results would be rather different if a student’s reading skills were monitored against the corresponding standards and the pace, content and methods of instruction adjusted accordingly.

But the information needs to flow to parents. Student assessment systems are useful when the information is technically sound, but also when it is widely disseminated and used to make changes (Abdul-Hamid, Abu-Lebdeh and Patrinos 2011; Alvarez, Garcia Moreno and Patrinos 2007; Gertler, Patrinos and Rubio-Codina 2012). External scrutiny by parents and independent assessors is critical for using information to inform changes.

Such gaps persist on a larger scale. In developing/low-income countries, ineffective inputs prevail. Developmental delays, minimal instruction time, less practice, unsuitable pedagogy, together with linguistic barriers, such as spelling complexities and teaching in non-native languages, lead to low reading skills in grades one to three. It is common that the students cannot process the volume of text and thus cannot understand text used in the follow-up instruction. Typically, they find it difficult to solve math problems in grades three to six. These were eventually translated into low achievement scores in regional and national assessments or international assessments such as PISA and TIMSS in the short term, low basic skills upon graduation, and even low income and low quality of life over the long run. If both students’ and schools’ performance are assessed earlier and the assessment results are used to lever measures and policies to improve education systems, students will end up with different learning and labor market outcomes.

Hence, benchmarks and benchmark-based assessments are the cornerstone of the education planning and reform aiming at quality. Nonetheless, oftentimes assessment is discussed and used interchangeably with tests or examinations; the two _de facto_ are essentially different. While the former serves selection and certifying functions (Hill 2010), the latter collects information on operation of education systems including student achievement levels and contextual information in general. Scores stand at the center of examinations, often to the degree that they divert the 7
attention of students and schools to getting good examination results at the expense of learning outcomes, whereas standards and indicators are pivotal to the assessment that benchmarks students’ competencies and assists informed decision making on interventions to improve educational quality. For example, Finland, the best-performing country in PISA, has no national whole age cohort testing in preschool and basic education. The only national examination occurs at the end of upper secondary education (at the age of 18 or 19). Rather it uses a national assessment of learning outcomes (which only covers a sample of 10 percent of the age group, focusing on grades 3, 5, 7 and 9) and longitudinal assessments to pinpoint areas requiring further improvement in different subjects and within the entire school system (Halinen 2011).

Quality-oriented reform requires a shift from lack of assessment or examination-dominated assessment that tests individual students toward assessment to obtain information about the achievements of the education system as a whole for evidence-based planning and reform (Greaney and Kellaghan 2008). It needs both commitment and significant resource and efforts from government and other stakeholders. It is becoming a growing priority of the global development community to rebuild or build assessment systems and link them with policies, practices, and interventions to improve teaching and learning. Among others, they need to be aligned with autonomy and accountability reform, elaborated below.

# <u>Quality Element 2: Autonomy</u>

A growing body of evidence suggests that autonomy-driven reforms could improve students’ learning outcomes (Barrera, Fasih and Patrinos 2009; Bruns, Filmer and Patrinos 2011; Carnoy et al. 2008; Clark 2009; Patrinos 2010a). It was documented that most countries whose students perform well in international student achievement tests have instituted autonomy reform by devolving substantial decision-making to local authorities and schools (Patrinos 2010a). The autonomy reform is closely linked with empowering the schools by devolving part or all of decision-making power regarding school management. Nonetheless, the autonomy reforms are not limited to school-based reform. In the autonomy-based structure, the central administration retains responsibility for budgeting and funding, setting policy, and evaluations, decision-making in relation to management and operation of the schools are decentralized to local authority and schools. What counts most is that more stakeholders are engaged in the educational process in terms of ownership, resources and voice.

A well-designed autonomy reform features the following characteristics:

- To empower a school by giving it ownership

- To reinforce the school’s resource base by mobilizing social forces

- To build up the relevance of the school’s education

- To enhance the school’s competitiveness

8
The case of Netherlands illustrates how school autonomy yields efficiency gains and improves the quality of learning. Netherlands might be the country with the longest tradition of an autonomy-based education structure. The schooling system is characterized by freedom of education—freedom to establish schools, to organize teaching and to determine the principles that foster competition among the schools. While all schools are government-funded, most schools are administered by private school boards. At the same time, school choice is promoted as a means of increasing competition in the system. As a result, the majority of students are enrolled in private schools and the proportion is increasing. This competition eventually led to efficiency gains as both public and private schools try to improve their performance to compete for better students. Studies suggest that the general level of competition in the Dutch system has contributed to the overall high achievement level. Using econometric techniques to identify school choice and control for selection biases, it was found that private school attendance is associated with higher test scores, and the achievement effect of private schools in math, reading and science are 0.2 to 0.3 of a standard deviation over public school attendance (Patrinos 2010a). In addition, the Netherlands does exceptionally well in international academic achievement tests such as PISA and TIMSS. Even when controlling for level of national income and expenditure per student, the Netherlands achieves relatively high scores. This implies that the system is not only high achieving, but also cost-effective, achieving good results at relatively low cost.

However, the specific mechanics of autonomy reform vary from country to country, as does the scope and degree of devolving the decision-making power to local authorities. Autonomy reform could materialize in various school forms, such as government–dependent private schools, creation of new autonomous public schools, and expanded opportunities for homeschooling, along with new funding mechanisms to promote school choice (OECD 2010: 419). These schooling forms suit different contexts. For instance, while private school attendance increased students’ learning performance in Netherlands, in a top-performing country such as Finland, 97 percent of all students at the primary and lower secondary levels are enrolled in public schools (OECD 2011). Central to the autonomy reform is allowing schools to tailor instruction settings to particular groups of students with the involvement of parents and the community. Results from randomized trials of school autonomy and accountability reforms are presented in Table 1.

The evidence suggests that autonomy is instrumental to improving students’ learning. However, its potential for transforming education systems depends on whether the increased autonomy is accompanied by strengthened accountability mechanisms.

9
**<mark>Table 1: Evidence from Randomized Trials of School Autonomy Reforms</mark>**

|Country|Authors|Intervention|Findings|
|---|---|---|---|
|Nepal|Chaudhury 2011|Community<br>management of<br>school|Reduction in out of school children,<br>repetition; increased progression;<br>disadvantaged casteperform better|
|Kenya|Duflo, Dupas<br>and Kremer<br>2007|Training school<br>committee to<br>monitor teachers<br>performance and<br>hiring|Higher student test scores, lower teacher<br>absenteeism, small change in student<br>dropout|
|Indonesia|Pradhan et al.<br>2010|School-based<br>management|Positive effect on learning outcomes;<br>strongest for elections in combination<br>with linkage, increase<br>scores in language by 0.51 standard<br>deviations, math by0.46|
|Mexico|Gertler, Patrinos<br>and Rodriguez<br>2010|Parental<br>participation<br>(AGE)|Increased participation in first year;<br>reduced dropout, improved reading<br>scores 0.25 SD|
|Mexico|Gertler,<br>Patrinos, Rubio<br>and Garcia 2012|SBM grants in<br>Colima (PEC)|Improved learning outcomes 0.16 SD|

# <u>Quality Element 3: Accountability</u>

The autonomy reform basically alters the governance structure and demand accompanying accountability restructuring. While the decision-making power is redistributed among various stakeholders, local authorities, school principals, teachers, students and other stakeholders are given new responsibilities for resource deployment and school activities. This creates new relationships, such as relationship of accountability between school principals and parents, schools and educational authorities. In an autonomy-based structure, school principals are held accountable to authorities for (efficient) use of financial resources. Likewise, the school principals are held accountable to both parents and local authorities for improvement in learning environment and learning outcomes (Patrinos 2010b). These emerging relationships warrant new mechanisms to facilitate the operation of the restructured system.

Effective school performance should be based on a well-established accountability structure that clearly defines the roles of institutions, agencies, and individuals exercising control over the resources and activities of schools. In the United Kingdom, an education accountability system 10
was restructured through legislation of the 1988 Education Act. The Act clearly specified the roles and responsibilities of various stakeholders. The accountability relationships between the education authority, schools and parents were also redefined. As a result, the public schools were allowed to opt out of local authority control and become autonomous schools directly funded by the central government, named grant-maintained (GM) schools. Schools could acquire autonomy through proposing and wining majority vote among students’ parents. A new agency of the central government was accountable for funding the GM schools. Instead of the local school district, the GM schools have control over all staff contracts and ownership of school buildings and grounds given to GM schools. Each school was owned and managed by the school’s governing body, which was composed of the head teacher and teacher and parent representatives. GM schools were also given power over admissions, so that students could apply to the GM school directly. The Education Act was complemented by other policy reforms such as nationwide open enrollment and the publication of “league tables” of school performance, which ensured that all the schools compete in a highly competitive education market. A study suggests that the new accountability relationships were positively associated with students’ learning achievement. Almost one in three high schools voted on autonomy between 1988 and 1997. Large achievement gains were found at schools in which the vote barely won compared to schools in which it barely lost (Clark 2009). The success of this program led to major changes in the education system in recent decades, with the Academies in the 1990s and free schools in 2011.

Moreover, an accountability-based system is usually aligned with enhanced social and parental interest. It involves parents and the community in the operation and management of schools. Meanwhile it sets clear standards and goals for the restructured system. It usually entails a shift of locus of decision-making from government to community represented by school governing boards. This approach was piloted in China in 2004. A school accountability study implemented by the China National Institute for Educational Research was undertaken in Qingyang District, Chengdu City. The pilot reform introduced a school governance board into several public schools. The board consists of representatives of teachers, students, parents, communities and educational authorities. It had power of operation and management of the schools. One key function of the board is to elect school principals who could be appointed after the local authority’s endorsement. It was reported that the pilot school reform reduced government intervention in the school’s operations and enhanced school effectiveness (China National Institute of Educational Research 2010). Schools are held accountable for learning outcomes to students, teachers and parents. The role of government shifted from director and actor to facilitator and promoter of quality education. Although there are no quantitative findings of the effectiveness of the reform, the pilot has been well-received and is being extended into other areas.

Accountability reform also involves developing corresponding incentives and disincentives. The incentive schemes, when appropriately designed, ensure the fulfillment of the rearranged roles 11
and responsibilities. Central to such schemes are incentives that link pay and/or tenure directly with performance. For example, teachers’ compensation is linked with factors that influence student achievement, such as teacher attendance and working at a low-performing school, or teachers’ compensation is directly linked with students’ achievement gains. In recent years, both OECD and developing countries have introduced two particular teacher incentive policy reforms, contract tenure reforms and pay-for-performance reforms to strengthen teachers’ accountability for performance. Such incentive reforms, either offering positive rewards or strengthening the threat of sanctions, built up the existing policy framework (Bruns, Filmer and Patrinos 2011).

Typically, experiments in India and Kenya found that the contract teachers were strongly associated with decreased teacher absenteeism and increased student test scores. A study in West Africa where contract teachers are widely used shows that the presence of a contract teacher was positively correlated with the learning performance of low-ability students in the early grades (Bruns, Filmer and Patrinos 2011). In addition, eight rigorous evaluation programs in India, Israel, Brazil and Kenya observed significant improvement in students’ achievement in schools implementing pay for performance (Ferraz and Bruns Forthcoming; Glewwe, Ilias and Kremer 2010; Lavy 2009; Muralidharan and Sundararaman 2009).

The pay-for-performance reform has been introduced into many developed countries as well, although there have been very few rigorously evaluated programs or research results so far. In Australia, under the recent Teacher Quality National Partnership Agreement aiming at quality of teaching, teacher pay was restructured to reward quality teaching as well as those teachers and leaders who work in disadvantaged indigenous, rural/remote and hard-to-staff schools. Likewise, Finland instituted a new salary system in 2007, which linked teachers’ salaries with the tasks, requirements and the results of the work, the professionalism of the staff and work experience (OECD 2010: 285-286).

# <u>Quality Element 4: Attention to Teachers</u>

The quality of an education system cannot exceed the quality of its teachers. A survey of the top-performing school systems suggests that the main driver of the variation in student learning at schools is the quality of the teachers. It was articulated that the success of these school systems is largely attributable to: (a) selecting the right people into the profession; (b) teachers’ salary structure; and (c) developing teachers into effective instructors (McKinsey & Company 2007; Hanushek and Rivkin 2003).

First, these systems all develop effective mechanism for selecting teachers. According to the survey, the top-performing school systems recruit their teachers from the top third of each graduate cohort of their systems: the top 5 percent in South Korea, the top 10 percent in Finland, and the top 30 percent in Singapore and Hong Kong SAR, China. These countries usually 12
established rigorous selection procedures to admit the most suitable people into the teaching profession. There are essentially two models of selection. One is to select the people before the teacher training starts and limit the places in the training program; the other is to leave the training process until the prospective teachers have graduated and then select the best graduates to become teachers. Most top-performing systems follow the first selective-entry approach, as the second may lead to over-supply of potential teachers and make the teaching profession less attractive. Figure 4 shows the procedure used in Finland.

**Figure 4: Procedure of Teacher Selection in Finland**

<!-- Start of picture text --> Check for strong intrinsics: 300 question multiple choice<br>National Screening assessment testing numeracy, literacy and problem<br>solving<br>Assessment Tests Top 20% of the cohort from last round. Check overall<br>academic ability and literacy: tests evaluate ability to<br>(University)<br>process information, think critically, and synthesize data<br>Interview Check suitability for teaching: interview look for<br>motivation to teach, motivation to learn, communication<br>(University) skills and emotional intelligence<br>Check suitability for teaching group exercise and teaching<br>Group Work<br>demonstrations test communications and interpersonal<br>(University) skills.<br>Recruitment by On completion of their teacher training, candidates are<br>recruited by individual schools.<br>School<br><!-- End of picture text --> Source: McKinsey & Company 2007 Second, apart from the selection mechanism, these successful systems entail good starting compensation and salary structure. The top performing systems all pay higher than the OECD average starting salary, relative to GDP per capita. As Figure 5 shows, among the countries with available data that ranked highest in PISA 2008, the ratio of salary after 15 years of experience (minimum training) to GDP per capita is mostly higher than 1. Not only is compensation a critical factor for maintaining the quality of teaching, but also improvement of salary structure which can lead to higher student learning outcomes (OECD 2010: 392; Vegas 2005). Good salary structure can provide salary incentives and rewards to attract high-quality teachers and increase their job satisfaction and performance. Salary increases can be concentrated at different points in the salary structure, for example, early in the career or for more experienced employees, or a more linear structure with gradual salary increases throughout a teacher’s career (OECD 2010).

13
<!-- Start of picture text --> Figure 5: Ratio of salaries after 15 years of experience/minimum<br>training to GDP per capita (2009)<br>Korea<br>Portugal<br>Germany<br>Japan<br>Ireland<br>Spain<br>Denmark<br>Switzerland<br>England<br>Belgium<br>Australia<br>Netherlands<br>Greece<br>Finland<br>Italy<br>Austria<br>Norway<br>France<br>United States<br>Sweden<br>Iceland<br>Poland<br>Estonia<br>Hungary<br>0.0 0.5 1.0 1.5 2.0 2.5<br><!-- End of picture text --> Source: OECD 2011 Notes: 1. Figures refer to annual statutory teachers’ salaries in public institutions after 15years of experience at primary level, in equivalent $US converted using PPPs; 2. Belgium (Fl.) is used as proxy for Belgium Furthermore, designing and implementing effective teacher development for better teaching is instrumental for raising student learning outcomes. In fact, many measures instituted by governments seem not to work in many countries. Pre-service credentials do not predict teaching effectiveness well; conventional in-service professional development in many cases turned out not to be useful in altering teachers’ instructional methods or children’s experiences, just to name a few. Teacher development might well be a policy area requiring perhaps the greatest degree of innovation. Effective teacher professional development arms teachers with state-of-the-art subject content knowledge and adequate skills. Empirical studies have identified the following alternatives as components of effective professional development:

- Method-centered rather than theory-centered training that corresponds to prospective teachers’ future tasks;

- Updated curriculum for professional development, such as synthetic phonics including phonemic awareness, fluency, vocabulary, and comprehension;

- Focus on activities that are more relevant to classroom-based teaching and school-based activities;

- Active pedagogy that integrates real-life knowledge and skills with students’ learning; and

- Coherence between pedagogy for teacher training and taught pedagogy for teaching students (Vegas 2005).

14
Teachers’ professional development is essentially a process, not an event. There is no once-for-all therapy for any problem that a teacher might encounter in his/her career. It basically starts from teachers’ pre-service training and continues through various forms of non-formal and informal learning that build up and consolidate their knowledge base and skills continuously. Other aspects of education reform, even those not specifically designed to affect teachers, could also have significant effect on teachers, such as the afore-mentioned accountability. An interesting example comes from Shanghai, the top performer in 2009 PISA. Shanghai’s success may be partly due to upgrading teaching standards and raising teacher pay (Cheng 2010).

# <u>Quality Element 5: Attention to Early Childhood Development</u>

Early Childhood Development (ECD) constitutes another structural quality element in education planning and reform. Early childhood development is the foundation of a person’s learning and well-being (Heckman 2004). Recent scientific research has established that a tremendous burst of synapse formation occurs early in life, whereas the synapse number declines when a child grows up to primary school age, and apparently the decline extends into adolescence in some areas of the brain, as shown in Figure 6. This implies that the best time for nurturing sensing pathways, language abilities and higher cognitive function is the early years of childhood. That could explain why early childhood care and development during these early years strongly influence the extent to which a child’s cognitive and socio-emotional abilities may develop to their fullest potential. A plethora of evidence also suggests that experience-based brain development _in utero_ and during the early years of life can set brain and biological pathways that affect an individual’s health, behavior, and learning for a lifetime (Mustard 2007: 39).

**Figure 6: Human Brain Development: Synapse Formation**

<!-- Start of picture text --> Sensing Language<br>Pathways Higher<br>(vision, hearing) Cognitive Function<br>-6 -3 0 3 6 9 1 4 8 12 16<br>Months Years<br>AGE<br>Conception<br><!-- End of picture text --> Source: Shonkoff and Phillips 2010 15
Effective ECD programs can have a very high payoff by improving school readiness, and nurturing positive self-image and learning dispositions (World Bank 2010). The longitudinal study of the Perry Pre-school program indicates that those who had the preschool program had higher earnings, were more likely to hold a job, had committed fewer crimes, and were more likely to have graduated from high school than adults who did not attend preschool programs (Schweinhart 2003). Recent studies show that ECD is positively associated with enrollment in the first grade of primary education and potentially increases retention, completion and achievement. It was shown that in the United Kingdom children’s preschool participation contributed substantially to their intellectual development, independence, concentration and sociability in the initial three years of primary school (Sylva _et. al._ 2004, cited in UNESCO 2006). The gains were even higher when children participated longer in preschool education. Similarly, children in Bangladesh who received center-based preschool education outperformed their peers in the control group by 58 percent on a standardized test of school readiness (World Bank 2010). Effective investments in early child development can reduce inequalities perpetuated by poverty, poor nutrition and restricted learning opportunities. Increasing preschool enrolment to 25 or 50 percent in every low-income and middle-income country can have a benefit-to-cost ratio ranging from 6:4 to 17:6 (Engle et al. 2011). Early childhood is the most effective and cost-efficient time to ensure that all children develop their full potential. The returns on investment in early child development are substantial.

In reality, ECD remains lagging in many developing countries. The _2010 EFA Global Monitoring Report_ pinpoints that the gross enrollment rate of pre-primary education in developing countries is merely 36 percent in comparison to 80 percent in developed countries (2007 school year) (UNESCO 2010). What is worse, few countries have established national frameworks for the financing, coordination and supervision of ECCE programs for children aged 0-3 years. In terms of resources, OECD countries spent 9 percent of their educational budget on pre-primary education (OECD 2010a: 218), while most developing countries allocated less than 1 percent of the total education budget to early childhood programs (even including health expenditures), the figure is even lower in most African countries, less than 0.1 percent of the education budget (UNESCO 2010a).

The absence of appropriate ECD policies and programs largely undermines the effectiveness of basic education and education as a whole. By the time a child is old enough to attend school, there is already a wide disparity in cognitive skills and in emotional and behavioral development among children from households of different socioeconomic backgrounds. Low levels of cognitive development in early childhood strongly correlate with low socio-economic status as measured by wealth and parental education as well as malnutrition (Alderman 2011). Scarcity of investment in ECD could be a major barrier to the quality and cost-efficiency of overall educational delivery. An empirical study posits that lowering the parents’ price of schooling could stimulate increased enrollments and this proved effective in a wide variety of developing country settings. It was estimated that an investment of $7.5 billion per year would help 15 16
million children complete schooling that would drop out otherwise (Orazem, Glewwe and Patrinos 2009).

In short, in developing countries, there is a strong demand for ECD programs that provide the positive physical, social and psychological support for young children beginning at birth and even earlier in utero. More attention is urgently needed on the part of resources and institutions for effective ECD programs and services. The countries that invest earlier would gain an advantage in developing different sets of skills for their citizens and resulting economic progress (Young 2007).

# <u>Quality Element 6: Attention to Culture</u>

Last but not least, education reform is likely to fail without taking into account the cultural context. By culture here, we refer to existing social views of the value of education and educational leadership, often necessary to instill change. Education reform is more than curricular change; it also implies cultural change, as represented by values and mores in various institutions. The paradigm change underpinning education reform involves bringing in new values that may clash with existing values, thoughts and actions (Argyris 1992, cited in Dooley 1995). As for any other organizational change, education planning and reform has to recognize the extent to which the change process is vulnerable to powerful influences (Dooley 1995). Education planning and reform could hardly succeed without adapting to deeply-rooted institutions. This section builds upon several dimensions identified in Hofstede’s (2001) seminal work to illustrate the implication of culture for educational reform aiming at quality.

A significant dimension of national culture relating to education reform is _uncertainty avoidance,_ which education leaders often encounter when designing and implementing education reform. Reform always entails changes and transitions into a new stage full of uncertainties. Also, the political nature of education compounds the complexities engendered by the uncertainties. People’s anxiety and resistance against reform rising from uncertainty avoidance (Hofstede 2001) could become a major barrier in education reform. It is then the responsibility of the reformers to create a clear vision and build up conviction toward the success of the reform. In the case of the United States, Bruner (1996) illustrates the importance of leadership for school reform. What counts most is a school reform movement “with a better sense of where we are going, with deeper convictions about what kind of people we want to be” before the technical (assessment) instruments, he argues, only then can the nation “mount the kind of community effort that can truly address the future of our educational process” (Bruner 1996). Furthermore, different societies have developed different ways to adapt to uncertainty. These ways belong to the cultural heritages of societies and are reflected in collectively held values of the members of a particular society (Hofstede 2001: 146). It is important for the reformers to recognize and mobilize those values to engage the intended changes for the purpose of improving education quality. For example, China has capitalized on the experiences from opening up reform started 17
by experimentation with special economic zones and thus instituted curriculum reform by starting with small-scale experimentation, which accumulated valuable experiences for nationwide extension.

Education reform should attend to the influence of _individualism and collectivism_ . Ideal education caters to each individual student’s capabilities and interests (Aristotle, cited in Ornstein and Levine 1981). Nonetheless, individualism and collectivism featuring different societies (Hofstede 2001) largely affect the ways of organizing educational provisions, to the extent of impacting the effectiveness of instruction. Typically, in collectivist societies such as China and Japan, students are organized into administrative classes that follow the same schedule of lessons regardless of students’ interests, while in an individualist society such as the United States, students’ lessons are organized by subject and each student has his or her own schedule that fits into their own interests and capabilities. The former are prone to negligence of students’ individual characteristics. Without suitable counseling services in place, the students in those cultures often resort to off-campus tutoring to develop their own interests and subject-based knowledge. It is not surprising that tutoring services have developed rampantly in these societies and led to a heavy burden of homework for students (Bray 2007).

Finally, care should be given to handle the _short-term and long-term orientations_ of intended reform. While long-term oriented cultures are accustomed to working toward building up strong position, short-term oriented cultures tend to expect immediate results identified by studies (Hofstede 2001: 361). Given the nature of education, education policy reform is characterized by effect lag, that is, endeavors for changing the education system produce lagging effects long after the action is taken. Hence, in both cultures, education reform should take into account the immediate results as well as long-term impact. Specifically, education reformers should be cautious about the policy action intended for quick benefits, which might sacrifice the quality on the long-term basis. Education reform aiming at quality needs to be prepared for long-term efforts and commensurate resources and structural transformation. Expedient education reform measures can achieve great leaps in enrollments but perhaps at the cost of quality over the long run. For instance, China has expanded higher education rapidly since the end of the 1990s. As a result, tertiary enrollment rose 47 percent in one year. The gross enrollment rate in higher education increased from 10 percent in 1998 to 23 percent in 2009. The reform achieved immediate results of tertiary education expansion and stimulating economic growth, yet contributed to deteriorating quality over the long run. Among other things, public investment in education has not kept up with enrollment. While governmental allocation accounted for 60 percent of total higher education expenditure in 1998, it decreased to 53 percent in 2001 and further dropped to 43 percent in 2006. The universities rely on cost-sharing (charging tuition and fees) as well as income generated through commercial activities. This not only increased the financial burdens on students, but also diverted the university's attention from teaching and research to income-generating activities. The quality of higher education was reported deteriorating. An obvious measure is that a large number of graduates from expanded 18
# enrollment faced difficulties in finding a job (Yuan 2006).

There are many other culture-specific issues affecting education reform. Those who are alert and sensitive to cultural issues are more likely to succeed. One important area of culture that needs attention is language of instruction, especially as it relates to minorities and ethnic groups. Studies have found that bilingual education for disadvantaged students can produce significant human capital benefits. In the case of indigenous peoples in Guatemala, students of bilingual schools have higher attendance and promotion rates, and lower repetition and dropout rates. The program was successful not only because children were taught at first in their own native language, but also because the program took local culture into account. Drawing on the success of a program employing bilingual promoters in 1965 during the campaign to make Spanish the dominant language throughout Guatemala, a national bilingual education program was established (Patrinos and Velez 2009).

Education is associated with many benefits, economic and social. These benefits propel the individual and her family to demand more education. But there needs to be a market response. This is where policy plays a great role, either by eliminating the factors that inhibit a response, or creating the conditions for expansion of the system. In some cases public policy is needed to convince people that the benefits of education are for everyone and should be shared. This could be the case for girls’ education in some cases and perhaps for ethnic minorities in other cases. When information flows, then the benefits of education become widely known and demand will increase. But in some cases even information flow represents a significant change in society. At present, the returns to schooling overall are high, especially in developing countries. This information itself will propel change and demand.

# **Conclusions**

In conclusion, six factors, including three institutional factors and three structural quality elements, are pivotal for improving the quality of education. On the institutional level, quality-oriented reform requires a shift from lack of assessment or examination-dominated assessment toward assessment for evidence-based planning and reform; autonomy-driven reforms could also improve students’ learning outcomes by allowing schools to tailor instructional settings to particular groups of students with the involvement of parents and the community; and the assessment and autonomy reforms warrant accompanying accountability restructuring that redefines the roles and responsibilities of various stakeholders and developing corresponding incentives. On the structural level, a first driver is the quality of teachers which largely depends upon effective recruitment, competitive starting salary as well as effective professional development. Second, investment in early childhood development could largely improve the quality and cost-efficiency of overall educational delivery. Last but not least, culture counts; that is, successful education planning and reform should recognize and adapt to deeply-rooted cultural influences on institutions.

19
The **_<u>6As</u>_** essentially provide a reference framework for policy making aimed at improving education quality. The framework represents a holistic and inclusive approach to addressing education reform. Drawing on experiences and lessons of education reform during last two decades, it highlights the potential strategies that could help overcome the challenges to effective education planning and reform. It also offers a typology of examples that might inspire quality reform initiatives in developing countries. Presumably the quality elements could be used for the development of reliable and valid indicators (inputs, outputs and outcomes) that facilitate monitoring the performance levels of the related stakeholders.

We have to acknowledge that the six quality drivers and/or elements are not silver bullets or universally applicable recipes for education planning and reform across countries, especially when sophistication in the engineering of educating a child and variety of political, social and cultural contexts across countries is taken into account. Also, the evidence base needs to be expanded. Two priority areas for future research emerge from foregoing analysis.

First, there is a need to develop a system of international quality benchmarks drawing upon a larger body of evidence. This involves developing indicators and building a corresponding data base. Such a framework could be a useful reference for developing national education benchmarks.

Second, more empirical studies on impact evaluation are needed. This is a critical challenge as it is also necessary to convince policy makers of the importance of rigorous evaluation, especially on the part of activities that consume a large amount of resources, such as in-service teacher training. To complement the implementation of the **_<u>6As</u>_** or a more systematic quality framework, more profound empirical studies on evaluation of related policy interventions are needed. Both require concerted efforts of interest among countries and international organizations.

20