metadata: authors: 0: author: Serra, Danila
1: author: Barr, Abigail
2: author: Packard, Truman
count: Europe and Central Asia
docna: 0: docna: Education outcomes, school governance and
            parents' demand for accountability : evidence from Albania
docty: Policy Research Working Paper
subtopic: Public Sector Management and Reform,Educational Sciences,Educational Institutions & Facilities,Effective Schools and Teachers
teratopic: Education
abstracts: cdata!: The extent to which teachers and school
            directors are held to account may play a central role in
            determining education outcomes, particularly in developing
            and transition countries where institutional deficiencies
            can distort incentives. This paper investigates the
            relationship between an expanded set of school inputs,
            including proxies for the functionality of
            "top-down" and "bottom-up"
            accountability systems, and education outputs in Albanian
            primary schools. The authors use data generated by an
            original survey of 180 nationally representative schools.
            The analysis shows a strong negative correlation between
            measures of top-down accountability and students' rates
            of grade repetition and failure in final examinations, and a
            strong positive correlation between measures of top-down
            accountability and students' excellence in math.
            Bottom-up accountability measures are correlated to various
            education outputs, although they tend lose statistical
            significance once parent characteristics, school resources
            and top-down accountability indicators are considered. An
            in-depth analysis of participatory accountability within the
            schools focuses on parents' willingness to hold
            teachers to account. Here, the survey data are combined with
            data from lab-type experiments conducted with parents and
            teachers in the schools. In general, the survey data
            highlight problems of limited parental involvement and lack
            of information about participatory accountability
            structures. The experiments indicate that the lack of
            parental participation in the school accountability system
            is owing to information constraints and weak institutions
            that allow parent class representatives to be appointed by
            teachers rather than elected by parents.
repnme: repnme: Education outcomes, school governance and
            parents' demand for accountability : evidence from Albania
display_title: Education outcomes, school
            governance and parents' demand for accountability :
            evidence from Albania
listing_relative_url: /research/2011/04/14114257/education-outcomes-school-governance-parents-demand-accountability-evidence-albania
topicv3: School Directors m1326118 2144,Teachers Management m1326119 1523,Human Resources m1326304 1211,Governance Structures m1326526 1922,Education m1327048 470
publishtoextweb_dt: 2010-07-01T00:00:00Z
historic_topic: Education
no_of_pages: 61
model_extractions: WPS5643 Policy Research Working Paper 5643 Education Outcomes, School Governance and Parents’ Demand for Accountability Evidence from Albania _Danila Serra Abigail Barr Truman Packard_ The World Bank Europe and Central Asia Region Human Development Economics Unit April 2011
Policy Research Working Paper 0

# **Abstract**

The extent to which teachers and school directors are held to account may play a central role in determining education outcomes, particularly in developing and transition countries where institutional deficiencies can distort incentives. This paper investigates the relationship between an expanded set of school inputs, including proxies for the functionality of “top-down” and “bottomup” accountability systems, and education outputs in Albanian primary schools. The authors use data generated by an original survey of 180 nationally representative schools. The analysis shows a strong negative correlation between measures of _top-down_ accountability and students’ rates of grade repetition and failure in final examinations, and a strong positive correlation between measures of top-down accountability and students’ excellence in math. _Bottom-up_ accountability measures are correlated to various education outputs, although they tend lose statistical significance once parent characteristics, school resources and top-down accountability indicators are considered. An in-depth analysis of participatory accountability within the schools focuses on parents’ willingness to hold teachers to account. Here, the survey data are combined with data from lab-type experiments conducted with parents and teachers in the schools. In general, the survey data highlight problems of limited parental involvement and lack of information about participatory accountability structures. The experiments indicate that the lack of parental participation in the school accountability system is owing to information constraints and weak institutions that allow parent class representatives to be appointed by teachers rather than elected by parents.

This paper is a product of the Human Development Economics Unit, Europe and Central Asia Region. It is part of a larger effort by the World Bank to provide open access to its research and make a contribution to development policy discussions around the world. Policy Research Working Papers are also posted on the Web at http://econ.worldbank.org. The author may be contacted at tpackard@worldbank.org.

_The Policy Research Working Paper Series disseminates the findings of work in progress to encourage the exchange of ideas about development issues. An objective of the series is to get the findings out quickly, even if the presentations are less than fully polished. The papers carry the names of the authors and should be cited accordingly. The findings, interpretations, and conclusions expressed in this paper are entirely those of the authors. They do not necessarily represent the views of the International Bank for Reconstruction and Development/World Bank and its affiliated organizations, or those of the Executive Directors of the World Bank or the governments they represent._ Produced by the Research Support Team
Education Outcomes, School Governance and Parents‟ Demand for Accountability: Evidence from Albania

**Danila Serra**<sup>1</sup>

**Abigail Barr**<sup>2</sup>

**Truman Packard**<sup>3</sup>

**JEL Classification:** O1, I2, C8, C9

**Keywords** : Accountability, governance, education, lab experiment, Albania

> 1 Department of Economics, Florida State University, dserra@fsu.edu (corresponding author)

> 2 Nuffield College and the Department of International Development, University of Oxford, <u>abarr@economics.ox.ac.uk</u>

> 3 Human Development Economics, Europe and Central Asia Regional Department, World Bank, <u>tpackard@worldbank.org</u>
Page ii

# **Acknowledgements**

The authors would like to thank the Albanian **Ministry of Education and Science** (MoES) for their interest and assistance during the preparation of this work, in particular to **Mimoza Agolli** (MoES Economic Director), and **Floriand Nurce** (Budget Chief) who worked hard to gather and provide important statistical data. Thanks are also due to the **Institute for Development Research Alternatives** (IDRA), who collected the data, and especially to **Auron Pasha** , **Enkelejda Pashaj** and **Adela Gjergjani** for their technical excellence, professionalism, and willingness to explore and apply cutting-edge data collection techniques in difficult circumstances and against pressing deadlines.

This paper is a product of the World Bank‟s **Accountability for Better Governance Program** (P107759) of analytical advisory assistance to the Government of Albania, and was supported with financial assistance from the **Governance Partnership Facility (GPF)** . The team received support and input from **Lorena Kostallari** (Senior Operations Officer, Human Development, World Bank) and **Gentjana Sula** (Operations Officer, Human Development, World Bank) who also managed important discussions and exchanges of information with the Government from the World Bank‟s Office in Tirana. **Andrea Guedes** (Senior Operations Officer, Human Development, World Bank) provided guidance to ensure the operational relevance of this work. Administrative and logistical support was provided from Washington, by **Katarina Timina** (Program Assistant, ECSHD) and in Tirana by **Elda Hafizi** (Program Assistant, ECCAL).

The team worked under the direction of **Tamar Manuelyan Atinc** and **Mamta Murthi** (Prior and Current Directors, Human Development, Europe and Central Asia, World Bank); **Jane Armitage** (Country Director, Western Balkans, World Bank ); **Gordon Betcherman** and **Jesko Hentschel** (Prior and Current Sector Managers for Human Development Economics, Europe and Central Asia, World Bank); **Camille Lampart Nuamah** and **Kseniya Lvovsky** (Prior and current Country Managers, Albania, World Bank), and **Daniel Dulitzky** (Country Sector Coordinator for the Western Balkans, Human Development, World Bank).

The Peer Reviewers were, **Magnus Lindelow** (Senior Economist, Human Development, East Asia and Pacific Region, World Bank), **Mattias Lundberg** (Senior Economist, Human Development Network, World Bank), and **Waly Wane** (Senior Economist, Poverty Reduction and Economic Management, Africa Region, World Bank), who provided helpful guidance at the outset of the AAA program and held the team to a high technical standard of quality.

Special thanks are extended to **Clelia Rontoyanni** (Public Sector Specialist, Poverty Reduction and Economic Management, World Bank) for collaboration with the team during preparation of the Albania Country Governance Review, and for her suggestions that have strengthened this report, and to **Dena Ringold** (Senior Economist, Office of the Chief Economist, Human Development Network, World Bank) for ensuring consistency between this report and wider efforts in the World Bank‟s Human Development Network to improve governance and accountability in the social sectors.
<!-- Start of picture text --> route aN<br>pre of accoun;‘abilig,‘ 3<br>Short route<br><!-- End of picture text -->
Page 2 4. Doctors and teachers paid by line ministries are really accountable to national authorities only, even if on paper they should also be accountable to local government, communities and the end-users of services. While governments have a primary role in setting the rules and responsibilities of service provision through formal institutional arrangements, monitoring and penalty-enforcement cannot be supplied only by governments. Conflicts of interest and the risk of collusion between the monitors and the monitored call for a more participatory approach to accountability - an approach that directly involves service recipients.

5. This is the rationale for shortening the “route of accountability” by establishing robust participatory mechanisms that engage citizens in holding service providers accountable. Under the participatory approach, governments put institutions in place to allow citizens to play an active role in monitoring and sanctioning providers when their performance is unsatisfactory. Yet in most developing and transition countries, these mechanisms are still missing, or where they exist, they work poorly.

# **I.b. Improving Governance through Stronger Accountability in Albania’s Education Sector**

6. The Government of Albania is making rapid progress in strengthening governance across the public sector, particularly in education. Considerable advances have been made in the education sector in particular, with reforms to the service compact and how education services are supplied (represented in the right-hand side of Figure 1). These measures have reduced the room for discretion and collusive practices, increased fairness, and have brought to bear the disciplining forces of the market and greater transparency to improve the performance of the sector. The education sector reforms have been positively assessed, including in a recent World Bank country report (World Bank 2011) which presents data from a specially designed education governance module that was deployed with the 2008 wave of the Albania Living Standards Measurement Survey (LSMS).

7. The Ministry of Education and Science (MoES) is seeking to further improve governance by strengthening parent and community engagement in participatory accountability structures (the lefthand side of Figure 1). The Ministry has implemented reforms designed to strengthen the independence of school boards and parent committees, and is considering measures to increase school autonomy as well as make education financing more responsive to the priorities of stakeholders. Currently there is little autonomy or flexibility given to school-stakeholders to determine their needs and to make investments decisions. “Conditional transfers” from the national budget are channelled to Regional Education Departments (REDs to finance major inputs like teacher salaries, training, teaching materials and a textbook subsidy scheme. Local governments have discretion over the resources they receive as “unconditional transfers” from Ministry of Finance , which cover the costs of utilities and maintenance. Local governments also manage funds to finance investments which they receive on a competitive basis from MoES. These resources are used for improvements to schools such as rehabilitations, extentions, new buildings, and furniture for classrooms. Figure 2 presents a sylized representation of the _de jure_ service delivery and accountability relationships in Albania‟s pre-university education sector. However, until recently, parents and other community members did not have strong incentives nor effective structures to interact with school directors and teachers, in order to identify their particular issues and come up with solutions.

8. To create an enabling environment for greater parent and community engagement in school governance, the Ministry has given school boards greater responsibilities. The composition of these boards has been changed so that school principals should no longer be part of the boards, to give more representation to parents and community members (see Figure 2). The boards have been given greater responsibility for monitoring the actions of school administrators, and could eventually play a role in the selection of head teachers. There is some interest in providing block
Page 3 grants to schools so that they can decide on their own spending priorities. A pilot School Improvement Grant (SIG) scheme is being considered by MoES to give greater responsibility directly to school stakeholders for decisions over resources.

**Figure 2. Service Delivery and Accountability Relationships in Albania’s Pre University Education Sector**<sup>**4**</sup>

<!-- Start of picture text --> Government<br>MoES<br>RED<br>employs<br>holds accountable<br>School director<br>and teachers<br>provides<br>School services<br>board to<br>Parents<br><!-- End of picture text --> Source: Serra, Barr and Packard (2011) 9. A key objective of the _Accountability for Better Governance_ analytical program, initiated in the summer of 2008, is to measure household perceptions and experiences of governance in the education sector, and given this experience, their demand for accountability. Although planned improvements to education governance are ongoing, the analytical program provides important feedback to the government –in the form of responses to new household and facility user surveyson its efforts thus far. Additionally, since further measures to improve the performance of school governance structures as well as to increase school autonomy are still being considered, the program offers an opportunity to rigorously and scientifically explore the impact of reforms piloted as part of the _Albania Education Excellence and Equity Project (EEE-P)_ . Towards this end, the program provides a baseline of data on the performance and functioning of pre-university primary schools that was not previously available to the government. Strengthening accountability mechanisms in order to improve governance is one of the core development objectives of the _EEEP_ , supported with co-financing from the government, the World Bank, the European Investment

> 4 The figure simplifies the existing accountability relationships in two ways, although all the essential elements are represented.  First, the figure does not show local governments that are also subject to elections, but who have no power over REDs.  Second, according to MoES guidelines, a the community level parent members of the School Boards should be elected parent class representatives, The parent class representatives of each grade are together members of the school‟s Parent School Council.  Members of this Council elect from among themselves parent members of the School Board. Thus a necessary condition for the School Board to function as intended, is that its parent members are elected by elected parent class representatives, who themselves are chosen by parents participating in the accountability process.
Page 4 Bank (EIB), and the Council of Europe Bank. The reforms that remain on the agenda for the education sector involve giving greater means for stakeholders in local schools to hold the government and providers accountable for the quality of education.

10. With the support of the Governance Partnership Facility (GPF) a special survey was designed to examine how parents, community members and teachers interact in school governance structures. The _Albania School Stakeholder Survey and Lab-type Experiment_ was designed and deployed in 180 communities drawn randomly to represent the network of basic (primary) schools in Albania. Along with the survey, a set of controlled lab-type experiments (or “behavioral games”) was also conducted with parents and teachers in order to measure their demand for accountability.

11. This paper presents the findings of the survey and lab-type experiments, and is structured into three parts. The remainder of Part I summarizes the findings from analysis of the survey and lab-type experiments. Parts II and III provide greater technical details for interested readers who are statistically inclined. Appendicies explain the design and sampling strategy used to select schools, parents and other stakeholders for the survey and lab-type experiments, as well as more detailed data on school performance dissagregated by RED.

# **I.c. Summary of Findings from the School Stakeholder Survey and Lab-type Experiments**

12. The School Stakeholder Survey, collected in the autumn of 2009, was designed to provide policy makers with data on the functioning of schools and to inform the design of further reforms in the education sector. The survey was designed to: (i) quantify education resources and take stock of institutions for accountability, including both external (“top down”) and participatory (“bottom up”) structures; (ii) measure stakeholders‟ awareness of the schools‟ institutional framework in general, and participatory accountability institutions in particular; and (iii) gage actual parental involvement with the school. In what follows, we present descriptive statistics drawn from the survey to measure -or proxy- school inputs, school and teacher characteristics and effort, “top-down” accountability and transparency, participatory “bottom up” accountability and parental involvement, and parents‟ and community characteristics. The data reveal factors that are significantly correlated with school performance, as measured by student repetition rates, student drop-out rates, and students‟ performance on Albanian language and math exams.

13. Since the new school stakeholder data are drawn from a baseline survey, the analysis discussed in Part II is presented as a series of correlations rather than causal relationships. The results have to be interpreted cautiously as they are drawn from retrospective data collected with a cross-section survey, and are therefore vulnerable to biases due to omissions and endogeneity. Indeed the Albania School Stakeholder Survey was designed first and foremost to collect a baseline so that the impact of interventions the government is still considering can be measured. Nevertheless, the richness of the new data is unique, and allows researchers to investigate the relationship between school outcomes and: (1) school physical and human resources; (2) “topdown” accountability and transparency; (3) “bottom up” participatory accountability institutions and parental involvement; and (4) parents‟ and community characteristics. By employing statistical techniques at the level of RED and district, the problems of endogeneity can be attenuated. And while there is potential endogeneity of parental involvement, this is a greater risk where parents have a choice in determining in which school to enroll their children. In Albania parents appear to have this choice only in large urban areas.<sup>5</sup> In order to limit the problem of possible multicollinearity among variables, principal factor analysis is used on sets of variables that are

> 5 When asked why they chose that specific school to enroll their children, 88% of parents in rural areas point to the vicinity to their house, and 12% mention the presence of good teachers as a reason. These percentages change to 65% and 35% respectively in urban areas.
Page 5 meant to proxy for similar school, household or community characteristics and that appear highly correlated.

14. Part II presents indicators drawn from the school stakeholder survey in Albania that have been previously associated with school performance in other countries. The second part of the report starts by motivating the indicators selected from the survey with a review of the empirical literature that uses the “ _education production function_ ” as its analytical foundation. Disparities in key “inputs” and “outputs” across schools are then presented, and with the important caveats above about the limitations of what can be concluded from retrospective data, statistically significant associations are discussed. Students‟ performance on final year exams, repetition rates, and dropout rates are examined in relation to indicators of school resources (physical and human); teacher effort; parent and community characteristics that shape the context in which students learn; as well as “top down” and “bottom up” accountability structures.

15. Part III delves deeper into the role of local participatory governance structures and parents‟ demand for accountability in school performance **.** As discussed previously, better school performance can depend on parents holding teachers accountable through participatory structures. But even where these structures are in place, they are vulnerable to lack of parent capacity and capture by better informed teachers and school principals. Parent involvement depends on awareness of these institutions, and _their willingness_ to actively participate to try and keep teachers accountable. For this reason, parents‟ demand for accountability is critical. However, it is difficult to measure parent demand for accountability with only _subjective data_ on perceptions about problems in schools, _attitudinal data_ about how things ought to be changed, or _aspirational data_ about their willingness to do something about it. Techniques from experimental economics can overcome these limitations by drawing robust measures of demand for accountability that corroborate the subjective, attitudinal, and aspirational data from surveys. In Part III, the results from lab-type, controlled field experiments conducted with parents and teachers, are combined with survey data to investigate how parents‟ demand for accountability relate to their observable characteristics, such as education, income, occupation; existing school-level institutions for participatory accountability; parents‟ information about such institutions; and parents‟ actual involvement in the school.

# **_1.c.1 Significant variations in performance and resources across Albania’s school_** _s_

16. Starting with school performance, the survey results show that schools in urban locations appear to outperform schools in rural locations in students‟ achievement in their final exams. The data discussed in this section are presented in Tables 1-7 of Part II. Overall, the proportion of students who do extremely well in math was generally low and especially low in rural areas, where only 6 percent of the students who took the 2009 leaving examination earned a 9 or a 10. The dropout rate was significantly higher in rural locations. Conversely, schools in urban areas showed higher repetition rates (although the difference is not statistically significant).

17. Differences in student and school performance could reflect the wide disparities in physical and human resources found in the survey. Almost all schools in the sample reported problems of poor facilities and infrastructure. On average, as reported in the survey, 95 percent of the classrooms in the surveyed schools could not be used in poor weather conditions, and schools suffered power shortages for about four days in a typical month. Schools in rural locations were significantly smaller, more likely not to have a phone, to experience power shortages, and not to have a library. As for human resources, the student-teacher ratio was higher in urban locations, although more variation was found in rural schools, where there were up to 62 students per teacher. Teachers in urban schools had achieved higher levels of education and were more experienced; they
Page 6 are paid slightly more than rural teachers (which could simply reflect their higher average educational attainment), but they did not appear to be significantly more motivated.

18. Differences in performance could also stem from variation in teacher effort, the extent of quality control by education authorities, and perceptions of school governance. The survey provides key indicators of teacher effort, perceptions of school governance, and a glimpse at the use of formal “top down” accountability structures. On a positive note, the survey responses indicated that teacher absenteeism does not seem to be a major problem in Albanian primary schools: on average, the number of days of unjustified absence in a typical month was only 0.18, with a minimum of 0 and a maximum of 5 days. There were no significant differences in teacher absenteeism between rural and urban schools. The number of inspections from RED representatives in the semester prior to the survey varied greatly across schools, from a minimum of 0 to a maximum of 36. Rural schools were significantly less likely to receive an RED inspection. Overall, in 20 percent of schools, school directors were not subject to any form of appraisal and sanction if they underperformed. Moreover, about half of the surveyed teachers reported that they were not subject to any penalty if they underperformed. Perceptions of irregularities in the appointment of school directors and teachers were quite high, especially in rural areas. As many as 32 percent and 22 percent of the surveyed school stakeholders believed that school directors and teachers were appointed to posts for their personal connections rather than qualifications “often” or “very often”.

19. The existance and quality of “bottom up” participatory accountability structures – particularly those the Ministry has tried to strengthen – could be important factors for performance. The survey captured previously unavailable information about the bodies that have been established for participatory “bottom up” accountability. The number of school board meetings in the semester prior to the survey (January – May 2009) varied from a minimum of zero to a maximum of five. There were no significant differences in this indicator between rural and urban schools. However, contrary to the MoES directives, in about 30 percent of schools, the school director participated in school board meetings and expressed his or her vote. But again, there were no significant differences in this indicator between rural and urban schools. In about 81 percent of the schools, according to the answers of parent school board members, parent representatives were elected by parents rather than appointed by teachers; and in 43 percent of the schools teacher school board members were elected by teachers rather than appointed by the school directors. These proportions were significantly higher in urban schools as compared to rural schools.

20. The survey found a worrying lack of information among parents about participatory accountability structures in their children‟s schools. About 43 percent of the parents did not know school boards existed, and 28 percent did not know that there were class parent representatives. Only about 80 percent of parents who were informed about the existence of the school boards knew that there were parent members, and of these, 72 percent did not know their names. The lack of information among parents captured by the survey was more severe in rural schools than in urban schools. Of the 72 percent of parents who were aware of the existence of parent class representatives, only half knew their names; and of the parents who knew that parent representatives existed and were elected by parents, 77 percent did not vote in the most recent election.

21. Parent involvement in participatory accountability structures varied greatly across schools. The survey measured levels of parent engagement in accountability structures. The average number of meetings that parents said they had with teachers in the previous semester (January-May 2009) was around 4, yet this indicator varied considerably across schools, and was significantly higher in urban schools. The proportion of parents who usually participate in parent-teacher meetings also varied greatly across schools, from a minimum of 34 percent to a maximum of 100 percent, with an
Page 7 average of 72 percent. There was a significant difference between rural and urban schools in the proportion of parents that participated in parent-teacher meetings, albeit very small in magnitude. Lack of parental involvement was considered a problem by teachers in about 20 percent of the schools, but there were no significant differences between rural and urban locations in this indicator.

22. Household and community characteristics -significantly associated with school performance in other countries- also varied greatly across the surveyed stakeholders in Albania. Not surprisingly, the average years of schooling of both mothers and fathers of the students and the number of books owned by the household, were significantly higher in urban locations than in rural locations. Albanian parents have on average completed primary school and at least two years of secondary or vocational school. A household‟s economic status, as well as the proportion of employed parents -and the proportion of parents employed in occupations other than farming - were significantly higher in urban areas. About 90 percent of surveyed parents seemed to value the education of their children, but more so if the family lived in urban areas. Turning to community cohesion, urban areas were characterized by less community cohesion than rural areas. In urban areas there was less trust and cooperation than in rural areas -as shown by the significantly higher proportion of respondents who thought that “ _in their neighborhood there are people who want to take advantage_ ”, and the smaller proportion of those who thought that _“members of their community would be willing to help if needed_ ”, and would come together to jointly solve a common problem such as possible water shortage.

# **_1.c.2. Simple pair-wise correlations between education “inputs” and “outputs”_**

23. A school‟s physical and human resources were significantly correlated with the proportion of students who excelled in math and Albanian in the 2009 final examination. In particular, excellence in both math and language appeared more likely in urban schools, schools that have a phone (which could proxy the quality of local infrastructures), and schools that did not suffer as much from power shortages. The presence of a library in the school was also positively correlated with students‟ performance in the math test, but not in the Albanian test. Although no significant correlation was found between education outcomes and the characteristics of the school director, education of teachers and the student-teacher ratio, there was a significant, positive correlation between students‟ excellence in the language exam and teachers‟ years of experience, their motivation and wages. Teachers‟ motivation was also negatively correlated with students‟ drop-out rate.

24. Student and school performance were significantly correlated with perceptions of school governance, as well as with measures of “top-down” and participatory “bottom up” accountability structures. Perceptions of malfeasance in the appointment of teachers were positively correlated with the drop-out rate. The number of visits from RED inspectors was correlated positively with students‟ excellence in Albanian and negatively with the drop-out rate. Students‟ excellence in math and language was significantly more likely where parents were informed about participatory accountability structures; knew the names of the math and Albanian teachers; had more frequent meetings with teachers; and where lack of parental involvement was not considered to be a problem in the school. Parents‟ lack of information was also positively correlated with failure and repetition rates. Interestingly, the drop-out rate was found to be significantly higher in schools where there were fewer school board meetings, where the school director was participating and voting in these meetings, and where parents had fewer meetings with teachers.

25. Not surprisingly, family characteristics were also found to be important correlates of education outcomes. The average years of schooling of students‟ mothers and fathers; the number
Page 8 of books owned by the household; and an index of household socioeconomic status, were all strongly and positively correlated with student performance on final examinations.

# **_1.c.3. Findings from multivariate correlation analysis_**

26. The last section in Part II examines the robustness and relative importance of the correlations discussed above. For each measure of school performance – the drop-out rate, repetition rate, failure on exams, and student excellence in math and Albanian - four regressions were conducted, each including only one of four categories of independent variables: (1) school resources; (2) “top-down” accountability and transparency; (3) participatory “bottom up” accountability and parental involvement; and (4) family and community background. The correlation results are presented in Table 8.

27. School resources and top-down accountability structures appeared less significantly related to measures of school failure – drop-out rates and repetition - than parental involvement. There were very few correlates with drop-out, repetition and failure rates among school physical and human resources. The only significant results were a positive correlation between rural location and failure rate, and the negative relationship between teachers‟ motivation and drop-out rate. None of the top-down accountability and transparency measures appear to be correlated with drop-out, repetition and failure rates. In contrast, the number of school board meetings and whether the school director usually participates and votes in the meetings were, respectively, negatively and positively correlated with drop-out and repetition rates. Moreover, lack of parental involvement in the school, as assessed by all school stakeholders, was positively correlated with students‟ repetition rate. Among the proxies for the students‟ family background, the only significant result was a negative correlation between fathers‟ education and repetition rates.

28. School resources, teachers‟ characteristics and inspections, were more robust and significantly correlated with students‟ excellence in math and language. In particular, the percentage of students excelling in Albanian was significantly higher in urban schools which suffer fewer days with no electricity; where teachers are more motivated; and where they have more teaching experience. The percentage of students excelling in math was significantly higher in schools that had a telephone and in those with a library. More RED inspections and fewer days of teachers‟ absence were significantly associated with better performance of students in the Albanian exam. Teachers‟ absence had a significant negative association with student performance on the math exam, while the use of a register to record teacher absences was significantly and positively associated with student performance in math.

29. Parental involvement continued to show a robust, significant relationship with student achievement. A lack of parental involvement in the school was found to be negatively associated with students‟ excellence in both math and Albanian exams. Moreover, the proportion of students who excelled in math was higher the greater the number of meetings parents had with teachers and lower if parents had no information about participatory accountability structures.

30. However, some of the education “inputs” correlated differently with “outputs”. The indicators examined above were differently correlated with students‟ excellence in math than with student excellence in Albanian. For instance, the rural location of the school, and the experience and motivation of teachers were negatively correlated with students‟ performance in Albanian, but not in math. None of the proxies for top-down and participatory accountability, nor parental involvement or family background, were correlated with the proportion of students who got a 9 or a 10 in Albanian. The only variable that was significantly correlated with students‟ excellence in
Page 9 Albanian (with the expected negative sign) was lack of cohesion within members of the community. Students‟ excellence in math, however, was correlated with a larger set of factors. The proportion of students excelling in math was lower in schools with poor facilities. While teachers‟ characteristics did not appear significant, teacher appraisal and transparency with respect to teachers‟ absence from school were; indeed, students‟ excellence in math was more likely in schools where teachers believed they would be penalized if they underperformed, and where a teacher absence register existed and was regularly used. Lack of parental involvement in the school also correlated negatively, with students‟ performance in the final math exam. A father‟s education was strongly and positively associated with children‟s performance in math. And the extent of divide (lack of perceived cohesion) in the community appears to matter by reducing the proportion of students excelling in both math and language.

# **_1.c.4. Lab-type experiments and parents’ demand for accountability_**

31. The low levels of parent participation in school governance structures could reflect lack of information or parents‟ reluctance to hold teachers accountable. As discussed earlier **,** Albania has made concerted efforts to improve governance across the public sector. Its efforts in the education sector have been received positively by households. Seeking to complement governance improvements in education compact and the supply of education services, the Government has been investing in shortening the route to accountability, by strengthening participatory accountability structures (see again Figure 1). However, the data collected in the Albania School Stakeholder Survey indicate that parental participation in these structures is still very low. Is this lack of participation owing to poor institutions, a lack of knowledge and understanding, or a preference not to participate?

32. Parents appear seriously under-informed about accountability structures in their children‟s schools, and in many places these structures are not operating as they should. The findings summarized previously, highlight both a departure in _de facto_ accountability structures from how the MoES intends them to function _de jure_ , as well as a severe lack of information among parents about these structures. Contrary to MoES directives, in about 30 percent of schools, the school director participated and voted in school board meetings. In 81 percent of the schools, parent class representatives were elected by parents as intended, rather than appointed by teachers; and in only 43 percent of the schools were teacher school board members elected by teachers rather than appointed by the school directors. Only half of parents report participating in electing parent class representatives, and about 70 percent of those who did not participate were unaware either of the existence of parent class representatives or of the fact that they should be elected by parents. Moreover the survey identified a worrying proportion of parents who were unaware of the existence of the school board (43%), including a large proportion of elected parent class representatives (23%) and parents who had voted for parent class representatives (31%). This indicates that information problems are severe, even among parents who are participating in school governance in some way.

33. But if they had more information, would parents participate? The survey findings about the functioning of participatory accountability structures and parents‟ information, begs the question: _If parents were informed, would they participate in holding schools and teachers to account?_ Part III presents the conclusions of an analytical attempt to directly measure parents‟ preferences for participation through specially designed lab-type experiments (LTEs) that were conducted with a random sample of teachers and parents who responded to the school stakeholder survey.

34. Experimental techniques allow a controlled simulation of the parent-teacher accountability relationship. Lab-type experiments (LTEs) facilitate the measurement of individuals‟ values, beliefs and preferences that cannot be captured in survey data. They have been used in a number of
Page 10 countries and contexts, involving a wide variety of groups; from university students, and children, to adults without formal literacy or numeracy in foraging small-scale societies.<sup>6</sup> The specific details of each game will vary, depending on the specific preferences that they are designed to capture. LTEs are always designed with a clear research objective in mind, and employ real money in order to recreate as closely as possible the incentives that individuals face in the real-life situations that the games aim to simulate and elucidate.

35. The games were designed to yield insights into parents‟ preferences for participation, i.e. their willingness to forgo personal payoffs in order to hold teachers to account **.** In the games, participants‟ roles were matched to their real life roles in the school. The games were designed to mimic key aspects of the school context: teachers having resources that they might or might not pass on to their pupils, and parents having the choice to forgo personal resources to hold teachers to account. In the games, a player (teacher) is invited to divide a sum of money between herself and another player (a parent), and a third player (another parent) can pay to have money subtracted from the first player‟s allocation.<sup>7</sup> It is the third players‟ behavior that is of particular interest, as by choosing to “fine” they reveal both what they consider to be unfair behavior on the part of the first player, as well as their own willingness to pay to express this opinion. In other words, the third player is expressing their willingness to hold the first player to account for how the resources were divided (“fairly” or “unfairly”) between themselves and the passive second player.

36. However, the games were also designed to differ from the actual school context in several important ways, particularly with respect to the information parents have and how they can act on it. In the actual school context parents‟ decisions to hold school directors and teachers to account usually require that the parents expose themselves, whereas in the game parents were guaranteed at least some anonymity (teachers who were fined do not know which parents fined them). While in the school context the costs associated with participation may differ across parents, in the game the material cost of holding teachers to account was fixed and known by all. Finally, in the actual school context, parents may be uninformed about the existing participatory institutions and the actual performance of teachers, whereas in the game, parents were fully informed about how they can hold teachers to account and can condition their decisions on accurate measures of teacher performance. Therefore, the game can be viewed as a _counterfactual_ to the existing situation in Albania‟s primary schools. In this counterfactual information is not constrained, parents wishing to participate in holding teachers to account are afforded some protection (anonymity), and the consequences of their participation are clearly defined. Most importantly, in the game the manner in which parents could hold teachers to account was simple and clear, and the parents were fully informed. This being the case, the games afforded an opportunity to take a first glimpse at the parents‟ preferences for participation in holding teachers to account. The findings discussed are presented in Tables 10-12 in Part III.

37. Parents‟ decisions in the game were significantly related to their wealth and levels of education; however, their participation in actual school accountability structures was not significant. Not surprisingly, the decision to forgo personal resources to hold teachers to account (to fine) in the game was more likely among wealthier and better educated parents. Unexpectedly, parents who participate in actual school accountability structures (as parent class representatives of on school boards) were no more likely to engage in fining in the game. Further parents who were less informed about the schools their children attended were no less willing to participate in fining in the games.

> 6 See Barr et al. (2009) for an example of a cross-cultural experimental study based on the implementation of similar experimental designs to the ones we use here in 15 different small-scale societies.

> 7 The LTE discussed is a version of Fehr and Fischbacher‟s (2004) Third Party Punishment Game (TPG).
Page 11 38. How a school‟s participatory accountability structures were composed and functioned, as well as perceptions of school governance, were significantly associated with parent‟s decisions in the game. Critically, the extent to which participating parents held teachers accountable in the games depended significantly on whether the _de jure_ participatory structures – parent class representatives and school boards - have been _de facto_ compromised. Specifically, parent class representatives on the school boards that had been appointed to their position by teachers, rather than elected by parents, were significantly less demanding of teachers. Moreover, parents who perceived irregularities and malfeasance in the appointment of teachers were significantly more demanding of teachers in the game.

39. Parents‟ behavior in the game indicates that low participation in school accountability structures is likely to reveal constraints rather than parents‟ reluctance to hold teachers accountable. The lack of significant statistical correlation between the decision to participate in fining in the game, and parents‟ participation and information in the actual school context, could be an indication that the games were not externally valid, i.e. did not provide an accurate simulation of the actual parent-teacher accountability relationship. Alternatively, it could be an indication that the lack of participation and information in the actual school context are not the byproduct of distorted parents‟ preferences, but rather the result of the system‟s failure to generate _de facto_ participatory institutions, in which parents are represented by individuals who are accountable to them rather than captured by the interests of teachers, and are made aware of the available means to participate. Keeping in mind the possible limitations in external validity of the games, when the findings from the game (that appointed parent representatives are less demanding of teachers) is set in context - with the extensive information problems that emerged from the survey- a cautious conclusion can be draw. Namely, that lack of parental participation in Albania is more likely to be a problem of external constraints than one of preferences.

40. These findings are important for policy makers and sector specialists to keep in mind as they look for participatory structures that effectively shorten the route of accountability. Important messages can be drawn from these findings.

- First, the degree to which the parents of primary school students in Albania are willing to participate in the game bodes well for the government‟s objectives of increasing parent engagement in school accountability and governance structures.  This is good news for the government, and supportive of the steps it has taken and is still planning to strengthen school governance and autonomy.

- Second, the quality of accountability structures and how school governance is perceived matters to parents‟ decisions to become involved.  This should underscore to the authorities the importance of monitoring implementation of the school governance reforms to ensure de facto departures from their design and intent are minimized.

- Third, there is considerable scope for improving the degree to which parents are informed about the school governance reforms and how they can now assume a more active role in holding teachers and school administrators to account. This lack of information is found even among parents who are currently participating as elected class representatives.

41. The findings also show how future interventions can be designed and deployed to increase the likelihood of success. From a more technical and methodological angle, the results presented in this report shows that by making it possible to directly measure individuals‟ willingness to engage in holding teachers to account, lab-type experimental techniques could allow better calibration of participatory interventions. This could increase the likelihood of the interventions‟ success. Labtype experimental techniques implemented in the field can provide useful insights into the chances
Page 12 of success of alternative interventions, and/or allow for better tailoring of random controlled trials (RCTs) to different contexts. Moreover, if combined with surveys and RCTs, LTEs allow for useful and much needed investigations of the interaction between individuals‟ preferences and formal and informal institutions.
Page 13

# Part II. Accountability, Good Goveranance and School Performance

## **II.a. Are There Recipies for School Success? Lessons from Prior Research**

43. A large body of research is focused on identifying the determinants of student achievement in schools and school performance. The most commonly used approach for the study of educational outcomes is the so-called _production function approach_ (Hanushek, 1996), which relates students‟ and schools‟ performance to a number of measurable educational inputs, some of which (for example, physical and human resources available in schools) can be controlled by policy-makers while others (such as children‟s innate ability and socioeconomic background) cannot (although they can be indirectly affected by certain specific policies). The school (physical) resources usually investigated within the _production function framework_ are: the quality of building and infrastructures; the teacher-pupil ratio; and, when available, expenditure per pupil, and teachers‟ salaries. The most commonly used proxies for human resources are teachers‟ characteristics, such as years of schooling and teaching experience.

44. The empirical evidence on the effect of physical and human resources on students‟ achievement is mixed. Harbison and Hanushek (1992) provide a comprehensive review of 96 studies based on _retrospective data_ , i.e. based on ordinary (i.e. non-experimental data collected as responses to surveys) variation across schools and households in developing and transition countries, and illustrate that none of the physical and human resources listed above has been shown to be consistently and significantly related to educational outputs. In other words, “there seems to be no clear and systematic relationship between key inputs and educational outputs” (Hanushek, 1996). Further investigations in Ghana (Glewwe and Jacoby, 1994), Jamaica (Glewwe et al, 1995), India (Kingdon, 1996) and the Philippines (Tan et al. 1997) confirm that the quest for a causal link between teacher characteristics and school resources on the one hand, and educational attainment on the other, is still far from complete. Indeed, in the studies cited most variables proved statistically insignificant, and while each investigation found some teacher and school characteristics to be statistically significant, the specific resources tested or found significant varied greatly across the different studies.

45. Krueger (1999) highlights that “much of the uncertainty in the literature derives from the fact that the appropriate specification of the _education production function_ is uncertain”. Along the same line, Glewwe and Kremer (2005) stress that omitted variables at either the student, class, school, or local level may bias the estimates based on retrospective data either upward or downward. For example, unobservables such as children‟s innate ability and parents‟ motivation may be positively correlated with school overall performance, if parents enroll high-ability children in better performing schools. Similarly, teachers‟ unobservable characteristics, such as motivation, are likely to bias the estimates upward, if they are correlated with the observable characteristics included in the empirical specification.<sup>8</sup> 46. Researchers have tried to overcome these limitations by employing data generated by “natural experiments” or by conducting randomized control trials (RCTs). The resulting evidence on the impact of school resources on education outputs is still mixed. While studies exploiting natural experiments suggest, for instance, that reducing the pupil-teacher ratio increases test scores (Krueger, 1999; Angrist and Levy, 1999, among the others), RCTs conducted by Duflo et al. (2009)

> 8 The differences in the results obtained by different studies may therefore be the result of the different specifications used and, consequently, different omitted variables. Possible multicollinearity between the many school and teacher variables used with the attempt to reduce omitted variable bias, may also be responsible for the discrepancy in estimates across studies.  See Glewwe and Kremer (2005) for a comprehensive discussion of all possible limitations of estimates of education production functions based on retrospective data. Section 2 will also provide further discussion on this.
Page 14 in Kenya and by Banerjee et al (2010) in India provide null results. RCTs conducted in Kenya have also shown ineffectiveness of the use of flip-charts (Glewwe et al., 2004) and the provision of textbooks (Glewwe et al. 2009) to schools in improving education outcomes.<sup>9</sup> 47. Another explanation why school resources (physical and human) do not seem to generate the expected effects on education outputs is that resources intended for instruction may be diverted to other purposes or may not be efficiently used. In transition and developing countries, institutional deficiencies lead to accountability problems and incentives which result in poor performance. Teachers are frequently absent from the classroom.<sup>10</sup> Even when they are present, some may not be doing their job adequately.<sup>11</sup> School director and teachers‟ accountability is conditional on: (1) assignment of clear responsibilities; (2) being monitored while exercising these responsibilities; and (3) being subject to clear penalties when their performance falls short of predetermined targets (see Lewis and Petterson, 2009). The three conditions for accountability are usually all missing in developing and transition countries, resulting in teachers‟ absenteeism and/or low motivation, and, ultimately, poor education attainments, both in terms of quantity and quality.

48. Policy makers have tried to improve accountability using both “top-down” and “bottom-up” mechanisms. Top-down accountability typically takes the form of a monitoring system and a set of pre-established sanctions or rewards managed by higher levels of administrative authority in an institutional setting. In the case of the education sector, it often refers to school inspections from regional or local education offices, and resulting sanctions or rewards being dispensed to school director and teachers. Top-down accountability can also exist internally, within the school, in the form of the school director‟s efforts to regularly monitor the performance of teachers. However, empirical research shows top-down mechanisms are of limited effectiveness, as they are often subject to abuse and collusion between the “monitored” and the “monitor”.<sup>12</sup> 49. In response to the difficulties in implementing top-down accountability mechanisms effectively, bottom-up participatory programs have received increasing attention. The growing consensus is that while the government has a primary role in setting rules and responsibilities of service providers through formal institutional arrangements, monitoring and penalty-enforcement cannot be supplied by the government alone. In education, decentralization and local school management programs are advocated to capture the information advantage that parents and communities have with respect to the needs of their children; to harness their strong incentives to monitor the performance of teachers and school directors; and to exploit their comparative advantage in conducting the monitoring.

> 9 See Glewwe and Kremer (2005) and Kremer and Holla (2009) for comprehensive reviews of evaluations of the impact of school resources on education outcomes using RCTs.

> 10 Chaudhury et al. (2006) report results from surprise visits in primary schools in six developing countries; on average 19% of the teachers where found absent from school on the days of the visits – 27% in Uganda, 25% in India, 20% in Kenya, 16% in Bangladesh, 14% in Ecuador, and 11% in Peru.

> 11 Several researchers have found that even when teachers are in school, they may not necessarily teaching their students. In India, while 75 percent of teachers were present in the school, only about half of all teachers were actually teaching in the classroom when the enumerators arrived.

> 12 Several studies employing RCTs have. Kremer and Chen (2001) have implemented and evaluated the effectiveness of a top-down internal program that gave school directors the authority to give bonuses to teachers based on their presence at school. The program resulted highly ineffective in reducing teachers‟ absence, as school directors ended up giving bonuses to all teachers irrespective of attendance, which highlights one important limitation of top-down mechanisms: the possibility of collusion between the monitored and the monitor. Duflo et al. (2007) have implemented and evaluated a program implemented in NGO-run education centers in India, which provided bonuses to teachers based on an objective measure of their presence at school: photographs that teachers were required to take at the beginning and the end of the school day with a camera set with tamper-proof time and date. The program was highly successful in reducing teachers‟ absenteeism and improving test scores; however, it relied on an externally administered top-down system that would encounter strong resistance of teachers and teacher unions, had to be replicated in public schools.
Page 15 50. Empirical research focused on parental involvement in education governance through participatory “bottom up” accountability structures, is still in its infancy, but promising. While measures of parental background have for a long time been included in the empirical specifications employed in retrospective studies (again, with mixed results<sup>13</sup> ) attempts to measure and account for parental involvement in their children‟s education, and in participatory initiatives at the school level, are more recent. Retrospective studies conducted in the US (Houtenville and Conway, 2007) in Argentina (Eskeland and Filmer, 2007), in El Salvador (Jimenez and Sawada, 1999), in Mexico (Gertler et al., 2008; and Skoufias and Shapiro, 2006), among others, all show a significant impact of participatory programs and parental involvement on education outcomes.<sup>14</sup> Jimenez and Sawada (1999) found evidence that a local management system improved students' language skills and attendance in El Salvador. In Mexico Gertler, Patrinos, Rubio-Codina (2008) found that providing small grants ($500 to $700) to parent associations -to be used for infrastructure or materials that they considered important- improved student performance and reduced grade repetition by about 5%.<sup>15</sup> Skoufias and Shapiro (2006) investigated the effectiveness of the Mexican Quality Schools Program (PEC), within which parent associations developed and undertook school improvement plans funded by five-year grants of up to US$15,000. They found that participation in the program decreased drop-out, failure and repetition rates, and based on focus group interviews with parents and school directors, concluded that parents' involvement increased following the introduction of the program.<sup>1617</sup> 51. The evidence provided by studies based on RCTs is more mixed. In Kenya, alongside a program that allowed and funded public sector schools to hire extra teachers on short term contracts, some communities (both from the treatment and the control samples) were randomly selected for training in how to monitor teachers. The monitoring program had no impact on the attendance rates of the extra teachers and only a small, positive and insignificant effect on attendance rates of public sector teachers (Duflo et al. 2007). This was followed by Duflo, Dupas and Kremer (2009) implemented a controlled experiment in which 70 out of 140 Kenyan schools received an extra contract teacher, hired locally by a school committee made up of teachers and _elected_ parent representatives. At the end of the year, the school committees had to decide whether to hire the same contract teacher or a different one. Further, in half of the 70 schools, school committee members, including parent representatives, were trained on how to monitor and evaluate the performance of the contract teacher. The authors found that in the short term students assigned to the contract teachers performed better. However, this improvement in performance persisted only in schools in which committee members were trained and encouraged to meet and reassess the teacher. Also in Kenya, a program designed to strengthen ties between school committees and local educational authorities through training and joint meetings and under which school committees could award prizes to teachers who performed well, increased committee activity and the level of education of those elected to the committees but had no effect on teacher attendance (de

> 13 Mother‟s education seems to be the household characteristic most consistently related to students‟ performance.

> 14 For instance, Jimenez and Sawada (1999), Gertler, Patrinos, Rubio-Codina (2008) and Skoufias and Shapiro (2006) investigate the impact of specific participatory institutions in El Salvador and in Mexico respectively. In an ex post evaluation of the EDUCO program, Jimenez and Sawada (1999) found evidence that a local management system improved students' language skills and attendance.  The EDUCO program established community education associations, called ACEs, composed by _elected_ parents, which are given authority over the management of funds provided by the ministry, the hiring and firing of teachers, and the physical maintenance of the school. This program is therefore an example of the shortest possible route to teachers‟ accountability.

> 15 Parent associations also received training in the management of the funds and in participatory skills to increase their involvement in school activities. Based on qualitative data, such as interviews with parents, teachers and school directors, the authors argue that the program was successful in increasing parental participation in school matters. 16 Note that, compared to the EDUCO program, both AGE and PEC did not provide parents with direct means to hold teachers accountable, i.e. the route to accountability was much longer in this case.

> 17 Along the same lines, it may be the case that having parent associations in place but without financial resources is not as effective, that is it may be necessary to have a mandate and the resources.
Page 16 Laat et al. 2008). However, in Uttar Pradesh in India, neither informing communities about the quality of local schools, what they should expect from schools, local funding for schools, and the responsibilities of Village Education Committees (VECs), nor training communities and VECs in techniques for monitoring schools and assessing their children‟s progress, had any impact on school performance (Banerjee et al. 2009).

# **II.b. Accountability in the School “Production Function”**

52. Students‟ performance, and consequently school performance, can be measured on the basis of many different outcomes, e.g. students‟ average test scores, repetition rates, drop-out rates etc. Thus, the _education production function_ can take the form 53. where _qij_ is the performance of school _i_ in terms of output _j_ , and _xi_ represents the level of educational resources, _i.e._ school and teachers‟ characteristics, available in school _i_ . The set of accountability efforts made in school _i_ can be divided into two components: _a1i_ captures the **_topdown_** accountability efforts exercised by educational office inspectors and school director as well as transparency in school administration; and _a2i_ refers to the **_parental and local community participation_** in holding school director and teachers to account, as well as parents‟ general involvement in their children education; _zij_ comprises attributes of the students‟ parents and community. Finally, the term _ηij_ is a random error, which may includes school, community and household unobservable characteristics.

54. However, a note of caution is needed. This investigation into school performance in Albania also employs retrospective data and is therefore not exempt from possible biases due to omitted variables and endogeneity. Nevertheless, the richness of the new data from the Albania School Stakeholder Survey is unique, and allows researchers to investigate the relationship between school outcomes and: (1) school physical and human resources; (2) top-down accountability and transparency; (3) participatory accountability institutions and parental involvement; (4) parents‟ and community characteristics.

55. Since the data are retrospective in nature, establishing causal links is problematic due to omitted variables bias and possible endogeneity of school resources, parental characteristics and accountability measures. Fully addressing endogeneity problems is virtually impossible given the retrospective nature of the data. Indeed the Albania School Stakeholder Survey is intended first and foremost as a baseline to measure the impact of interventions the Government is still considering. That said, by employing fixed effects at the level of the Regional Educational District (RED) and clustering the error terms by district, the problems of endogeneity of school resources and top-down accountability measures can be attenuated (although not entirely eliminated).<sup>18</sup> As for the potential endogeneity of parental involvement, this is likely to be especially severe if parents have a choice in determining which school to enroll their children. In Albania this seems to be the case only in large urban areas.<sup>19</sup> Nevertheless, it is a reason for concern.<sup>20</sup>

> 18 The REDs are local education administration entities (thirteen in total) that are most directly accountable to The Ministry of Education and Science (MoES). While the MoES is directly responsible for the development of curricula, the selection of textbooks, the structure of the academic year, the workload of teachers, the allocation of resources among local education institutions, and teacher training (World Bank, 2005), the RED are responsible for the delivery of primary and secondary education in their respective districts –  37 in total.

> 19 When asked why they chose that specific school to enroll their children, 88% of parents in rural areas point to the vicinity to their house, and 12% mention the presence of good teachers as a reason. These percentages change to 65% and 35% respectively in urban areas.
Page 17 56. Conscious of the non-experimental nature of the data employed in this section of analysis, and unresolved endogeneity issues, in the sections that follow, care is taken to present results as correlations rather than causation. Moreover, in order to limit the problem of possible multicollinearity among the independent variables, principal factor analysis is used on sets of variables that are meant to proxy for similar school, household or community characteristics and that appear highly correlated.

57. The survey was designed to: (i) quantify educational resources and institutions for accountability, including both external (“top down”) and participatory (“bottom up”) mechanisms; (ii) measure the awareness of the different stakeholders of the school institutional framework in general, and participatory institutions in particular; and (iii) measure actual parental involvement with the school. In what follows, we present descriptive statistics drawn from the survey to measure or proxy school outputs, school and teacher characteristics, “top-down” accountability and transparency, participatory “bottom up” accountability and parental involvement, and parents‟ and community characteristics. In the first part of this section, we explain the measures that have been selected from the survey data. In the second part, we present descriptive statistics and the results of tests for significant differences between community groups of interest.

# _2.b.1. Measures of students’ performance at the school level_

58. Until the 2006-2007 academic year primary school leaving examinations were designed and administered by school teachers. This system generated opportunities for irregularities and prevented MoES from carrying out objective comparisons of students‟ performance across schools. Starting from June 2008, the locally administered final year examinations were replaced by standardized tests, centrally administered by the division of the MoES that is also in charge of _State Matura_ high school final year examinations. Therefore, since 2008 the performance of primary school students on leaving examinations has been comparable across schools.

59. We focus on five education outcomes at the school level. Three of them are generated by the leaving examinations corresponding to the academic year immediately preceding our data collection (the 2009 final year examinations). In each school, we look at: (1) the proportion of students who failed the final year examination; (2) the proportion of students who excelled (i.e. received a 9 or a 10) in the Albanian language test; and (3) the proportion of students who excelled (i.e. received a 9 or a 10) in the math test. In addition, we consider: (4) the drop-out rate (i.e. the proportion of students who were enrolled in the school in the 2008-2009 academic year and did not return to school the following year<sup>21</sup> ); and (5) the repetition rate (i.e. the proportion of students repeating a year at the time of the survey).

# _2.b.2. Measures of school physical and human resources_

60. The questionnaire directed at the schools‟ administrative representative provided extensive information about the status of the buildings and physical resources available in each school. Guided by the existing literature, we focus on the following school characteristics: the urban vs. rural location of the school; the distance (in kilometers) from the nearest town; whether the school can be reached by public transportation; the number of days with no electricity in a typical month; whether the school has a telephone; the proportion of classrooms in poor condition that cannot be

> 20 Gunnerson et al. (2009) use retrospective data to study the impact of parental participation and school outcomes on education outcomes in eight Latin American countries. They find that correcting for endogeneity substantially increase the estimated impact of parental participation on students‟ test scores.

> 21 Note that this may be also due to the family moving to a different location, or the student being transferred to a different school, where more than one are available.
Page 18 used in certain times of the year; the proportion of desks in poor condition; and whether the school has a library.

61. Both the questionnaire directed at school administrative representatives and that directed to teachers generated many important human resources variables, whose impact on students‟ performance has been investigated also by previous studies. These are: the student-teacher ratio; the average years of schooling of teachers, the average years of experience of teachers, the proportion of “local” teachers (i.e. those who grew up in the location where the school is), and the average and lowest wages received by teachers in the school. Moreover, we have information about the school directors‟ years of experience (as a school director), whether he or she received any school management training, and whether he or she is a “local” (i.e. grew up in the village/town where the school is located). Finally, we attempted to control for teachers‟ motivation – which is often considered a possible omitted variable in previous studies – by asking teachers whether “lack of teacher motivation” is a major or moderate problem in the school, as opposed to a minor or a nonexistent problem. In each school we compute the proportion of teachers who believe that teacher motivation is not a problem in the school.

# _2.b.3. Measures of “Top-down” accountability and transparency_

62. Regional Education Directorates (REDs) have no authority over the amount of funds allocated to schools – these are centrally established by MoES – and their administrative heads are centrally appointed; they are however responsible for the appointment and transfer of teachers, the distribution of administrative materials to schools, inspections of schools, as well as planning and supervising the construction of new facilities and major rehabilitation of existing schools.<sup>22</sup> Besides employing RED fixed effects in our regressions, we aimed to directly account for differences in topdown accountability across schools by asking school directors and teachers specific questions about monitoring and possible penalties. Our main variables of interest are: the number of visits of RED representatives in the previous academic year (January-May 2009); whether the school director is subject to penalties if he/she underperforms (self-reported by school directors); whether teachers are subject to penalties if they underperform (self-reported by teachers); and the days of unjustified absence of teachers in the month of May 2009, as reported in the teachers‟ absence registry, in the schools where such a registry is kept.

63. To measure transparency, we report whether the school keeps a teacher absence register and if it publicly displays teachers‟ absence and students‟ performance in the final year examination. We also generate a measure of corruption in the appointment of school directors and teachers based on perceptions of all the surveyed stakeholders. Each respondent was asked to evaluate how often school directors (teachers) are appointed to posts based on personal connections rather than their qualifications, using a 5-item Likert scale, ranging from “never” (1) to “always (5). In each school we consider the proportion of respondents that answered “often” or “always” to the above questions.

# _2.b.4. Parent Involvement in Participatory “Bottom up” Accountability Structures_

> 22 The REDs are also responsible for teachers‟ salaries, which are paid using the total budget allocated by the MoES, and for the collection of information at the district level and transmitting this information to the Ministry. The local municipalities/communes have very limited responsibility for the delivery of education services. Under the current arrangements, they receive conditional transfers from the MoES, through the REDs, to cover teachers‟ salaries and scholarships for pupils in secondary schools. They also receive unconditional grants from the Ministry of Finance to cover operational and maintenance expenditures, including minor repairs for the maintenance of buildings, and recurring costs (water, electricity, sewage, heating, bussing, etc.). The grants, however, are not earmarked for education, and municipalities and communes are also expected to finance other public services from it. The size of the block grant is decided based on population, area, and an urbanization coefficient (see Tibi, et. al., 2004).
Page 19 64. The institutional framework currently in place in Albania is centralized and top-down in nature, since it assigns the task of monitoring teachers and imposing penalties for underperformance to administrative units that are not accountable to service recipients. However, recently the MoES has been trying to improve teachers‟ performance and quality of education by promoting community engagement in local governance structures, such as parent committees and school boards. School boards have always existed within school administration units, but they have recently been given greater responsibilities for school governance. Moreover, in an attempt to reduce possible “elite capture” their composition has changed and the school principal is no longer allowed to be a member of the board.

65. Among the members of the school boards, there should be at least two parents, elected by elected representatives of parents of pupils in all classes, and at least one member of the community who should not be a parent in the school.<sup>23</sup> One teacher representative – elected by all teachers – and one or two student representatives are also members of the board. Moreover, a representative of the Regional Educational Department/ Educational Office, and a representatives of the local government unit (municipality or commune), appointed by the mayor, should also be part of the board.<sup>24</sup> 66. The main functions of the school board are to examine and approve (by voting) the school‟s mid-term plan and annual plan submitted by the school director, as well as the school‟s annual financial report. The board also has decisional power for the adoption of curricula and textbooks, as well as the school‟s budget (if any) generated by public funds, contributions of the community or other donors, or revenues from school‟s activities. School board members may also discuss problems related to the performance of teachers, or the school director, in response to complaints of parents, students, or teachers. MoES has been debating whether the responsibilities of the board should be extended in the near future to hiring and firing of teachers.

67. Regarding the composition and functioning of the school board, we register the number of meetings of the board in the previous semester (January-May 2009)<sup>25</sup> ; whether the school director usually participates in the meetings and votes; whether the parent representatives are elected by parents rather than appointed by teachers or the school director; and whether the teacher representatives are elected by teachers rather than appointed by the school director.

68. In order to account for the _de facto_ functioning of participatory institutions, we ask parents whether they are informed about the existence of school board and parent class representatives. Conditional on positive answers, we also collect information about parents‟ participation in the election of parent representatives. To further investigate the extent of parental involvement, we register whether parents know the names of their children‟s math and language teachers, and the number of parent-teacher meetings they attended in the previous semester (January-May 2009). We also gather information from teachers about the proportion of parents that usually participate in parent-teacher meetings, and we ask parents whether they ever filed a complaint concerning education matters. Finally, we elicit all the opinions of all surveyed stakeholders with respect to the extent of parental involvement in the school; we report the proportion of respondents who believe that lack of parental involvement is a moderate or serious problem in the school.

> 23 No details about the selection of the member of the community have been officially provided.

> 24 The Guideline for “Establishing and Functioning of the School Board” (MoES, 17 October 2007) established that “when the Regional Educational Department/ Educational Office or the local government (municipality, commune) covers a great number of schools, they are represented in board by their delegate, who as a rule should be a member of the scholastic community”.

> 25 We also asked about meetings in the current semester (September –December 2009), but the comparability of answers across schools is compromised by the fact that the survey was administered in different schools at different times of the first semester of the year 2009-2010.
Page 20

# _2.b.5. Parents’ and Community Characteristics_

69. Following the existing practice, we control for the average level of education of fathers and mothers of the students enrolled in the school. We also report whether at least one parent reports being unemployed, and among the possible occupations of the employed parents, we give special attention to farming. In order to further account for parents‟ preferences for education we register the number of books kept in the households (self-reported by parents), and we ask parents whether they agree or disagree (on a 5-point Likert scale) with the following statement: “There is not much point in sending my children to school, because education won‟t help them to find a better job when they grow up.” We report the proportion of parents in each school who disagree or strongly disagree with the statement. Finally, following Filmer and Pritchett (2001) we construct an index of household socio-economic status based on parents‟ answer to 13 questions aimed at registering whether the household owned a number of assets, such as a washing machine, a fridge, an oven, a car, a mobile phone etc. We conduct principal factor analysis on the answers to the 13 questions and report the resulting first factor, which represents our index of household socioeconomic status.<sup>26</sup> 70. While ethnic fragmentation is not a problem in Albania – only 2.5% of the surveyed stakeholders speak a language different than Albanian (Greek) in their households – divisions within the community based on wealth, and/or education and/or landholding, and/or political affiliation may prevent parents from collectively participate in the school, possibly leading to lower education outcomes.<sup>27</sup> In order to measure the extent of divide within the community we asked all surveyed stakeholders the following question: _“There are often differences between people living in the same village/neighborhood. For example, difference in wealth, income, social status, ethnic or ethnic-linguistic background. There can also be differences in religious or political beliefs, or there can be differences due to age or sex. To what extent do any such differences divide your village/neighborhood?”_ 71. We then registered the answers of the different school stakeholders on a scale from 1 (to a very small extent) to 5 (to great extent) and computed the average across respondents per school. In order to proxy the value given to education in the community, or lack thereof, we report the proportion of stakeholders that agreed or strongly agreed with the statement “Parents in my community still do not understand the benefits of education for their children”. We aimed to directly measure individuals‟ propensities to cooperate with other members of the community by asking respondents whether they agreed or disagreed (on a 5-point Likert scale) with the following two statements: (1) “In this village /neighborhood, there are people who want to take advantage”; (2) “most people in this village/neighborhood are willing to help if you need it”. We also asked how likely it is that if there were, for instance, a water supply problem people would cooperate to try to solve the problem. Finally, we registered whether in the past year, the respondent or any household member “participated in any community activities, in which people came together to do some work

> 26 Principal factor analysis is a technique used to summarize the information contained in a large number of variables to a smaller number by creating a set of mutually uncorrelated components of the data. The first factor is the linear index of all the observed variables that captures the most common variation among them. In the case of our index of socioeconomic status, the first factor captures 52% of the common variation among the 13 asset variables.

> 27 Retrospective studies of the relationship between education inputs and outcomes do not usually account for community characteristics among the possible determinants of school performance. However, Bjorkman and Svensson (2010), in their study – based on RCTs – of the impact of participatory programs on health outcomes in Uganda, have shown that division within the community, and particularly ethnic fractionalization, may prevent collective action for improved service provision, resulting in lower outcomes than in more homogeneous societies.  Galiani et al. (2008) in their study of decentralization efforts in Argentinean secondary schools (based on a “political natural experiment”), have shown that areas that are poorly administrated and characterized by populations that do not usually voice their preferences may be left behind and end up worse off rather than better off following decentralization.
Page 21 for the benefit of the community”. We aggregate the answers of all the surveyed school stakeholders in order to generate measure of community trust and cooperation.

# **II.c. Variation in Descriptive & Performance Indicators across Albania’s Primary Schools**

72. Table 1 presents our measures of school outputs, and compares schools in urban and rural locations (with less than 3,000 inhabitants). In the last column the results of tests for significant differences between rural and urban schools are reported, and where significant differences are found, are shown in bold. Table A.2.1 in the Appendix compares education outputs of schools located in the different REDs.

**Table 1. School Performance**

|||**Full s**|**ample**||**Sc**|**hools in **|**rural**<br> **ar**|**eas**|**Sch**|**ools inu**|**rban**<br> **ar**|**eas**|**Rural**<br>**vs.**<br>**urban**|
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
||**Mean**|**Min**|**Max**|**sd**|**Mean**|**Min**|**Max**|**sd**|**Mean**|**Min**|**Max**|**sd**|**p-value**|
|**School Outputs**||||||||||||||
|Drop-out rate (2009)|**0.04**|0.00|0.30|0.04|**0.04**|0.00|0.04|0.04|**0.03**|0.00|0.22|0.04|**0.089**|
|Repetition rate (2009)|**0.23**|0.00|0.79|0.18|**0.19**|0.00|0.70|0.16|**0.30**|0.00|0.79|0.20|0.250|
|Fail rate in the 2009 final exam|**0.01**|0.00|0.26|0.02|**0.01**|0.00|0.26|0.03|**0.01**|0.00|0.08|0.01|0.105|
|Students‟ excellence in math|**0.08**|0.00|0.45|0.11|**0.06**|0.00|0.45|0.09|**0.13**|0.00|0.40|0.11|**0.000**|
|Students‟ excellence in language|**0.23**|0.00|0.79|0.18|**0.19**|0.00|0.70|0.16|**0.30**|0.00|0.79|0.20|**0.000**|

Note: The p-values reported in the last column are generated by one-tailed tests of equality of means.

73. Schools in an urban location appear to outperform schools in rural locations in terms of students‟ achievement in the final exam. Overall, the proportion of students who do extremely well in math seems to be generally low and especially low in rural areas, where only 6% of the students who took the 2009 leaving examination got a 9 or a 10. While there does not seem to be a significant difference between rural and urban location with respect to the proportion of students who failed the final examination, the drop-out rate is significantly higher in rural locations. Conversely, schools in urban areas show higher repetition rates, although the difference is not statistically significant. A comparison between REDs (see Table A.2.1) shows Gjirokaster as the best performing RED, with 17% and 40% of the final year students excelling respectively in math and Albanian. The worst performing RED seems to be Kukes, where only 2% and 15% of the final year students got a 9 or a 10 respectively in math and in Albanian in the leaving examination.

74. Table 2 presents selected measures of the schools‟ physical and human resources. Overall, almost all schools in the sample seem to have problems relating to poor facilities and infrastructure. On average, 95% of the classrooms in the surveyed schools cannot be used in poor weather conditions, and schools suffer power shortages for about four days in a typical month. Schools in rural locations are significantly smaller, more likely not to have a phone, to experience power shortages, and not to have a library.

75. As for human resources, the student-teacher ratio is on average higher in urban locations, although there seems to be more variation in rural schools, where there can be up to 62 students per teacher. Teachers in urban schools have achieved higher levels of education and are more experienced; they are paid slightly more than rural teachers, but they do not seem to be more motivated. We do not observe significant differences across rural and urban schools with respect to school directors‟ characteristics. Table A.2.2 in the Appendix compares school resources across the different REDs.
Page 22

# **Table 2. School Resources**

|||**Full s**|**ample**||**Sc**|**hools in **|**rural**<br> **are**|**as**|**Sc**|**hools inu**|**rban**<br> **ar**|**eas**|**Rural**<br>**vs.**<br>**urban**|
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
||**Mean**|**Min**|**Max**|**Sd**|**Mean**|**Min**|**Max**|**sd**|**Mean**|**Min**|**Max**|**sd**|**p-value**|
|N. students|**354**|30|1559|289|**220**|30|1559|219|**590**|128|1218|243|**0.000**|
|**Infrastructures**||||||||||||||
|Can be reached by public transp.|**0.31**|1.00|0.00|0.54|**0.28**|1.00|0.00|0.55|**0.35**|1.00|0.00|0.52|0.29|
|No electricity (days in a month)|**3.76**|0.00|31.00|7.89|**4.72**|0.00|31.00|8.98|**2.06**|0.00|25.00|5.10|**0.015**|
|No phone|**0.66**|0.00|1.00|0.47|**0.90**|0.00|1.00|0.31|**0.25**|0.00|1.00|0.43|**0.000**|
|% desks in poor conditions|**0.14**|0.00|0.80|0.17|**0.15**|0.00|0.80|0.19|**0.12**|0.00|0.65|0.14|0.122|
|% classrooms that cannot be used in<br>poor weather conditions|**0.948**|0.00|1.00|0.145|**0.94**|0.00|1.00|0.168|**0.956**|0.00|1.00|0.883|0.294|
|Schools with a library|**0.783**|0.00|1.00|0.413|**0.713**|0.00|1.00|0.454|**0.908**|0.00|1.00|0.292|**0.002**|
|**Human resources**||||||||||||||
|School director years of experience|**3.784**|0.00|19.00|3.877|**3.557**|0.00|19.00|3.599|**4.187**|0.00|19.00|4.329|0.149|
|School director received school<br>management training|**0.59**|0.00|1.00|.49.23|**0.59**|0.00|1.00|0.49|**0.60**|0.00|1.00|0.49|0.545|
|School director is a “local”|**0.661**|0.00|1.00|0.474|**0.643**|0.00|1.00|0.481|**0.692**|0.00|1.00|0.465|0.254|
|Student-teacher ratio|**15.18**|3.20|62.30|6.73|**13.16**|3.20|62.30|6.58|**18.80**|5.82|38.92|5.39|**0.000**|
|% teachers with university degree<br>or more|**0.80**|0.00|1.00|0.22|**0.77**|0.00|1.00|0.25|**0.86**|0.40|1.00|0.15|**0.004**|
|Teachers‟ average years of<br>education|**14.94**|7.29|18.50|1.18|**14.78**|7.29|18.50|1.33|**15.22**|12.38|17.00|0.80|**0.008**|
|Teachers‟ average years of teaching<br>experience|**16.94**|4.60|34.20|5.69|**16.07**|5.40|34.20|5.66|**18.49**|4.60|33.60|5.44|**0.003**|
|% of “local” teachers|**0.569**|0.00|1.00|0.285|**0.576**|0.00|1.00|0.294|**0.557**|0.00|1.00|0.270|0.337|
|Motivation of teachers‟ in the<br>schools (self-reported)|**0.742**|0.00|1.00|0.246|**0.756**|0.00|1.00|0.251|**0.717**|0.00|1.00|0.237|0.150|
|Average salary paid to teachers|**35245**|27750|41000|2489|**34999**|28625|41000|2561|**35690**|27750|39900|2307|**0.038**|
|Lowest salary paid to teachers|**30909**|20000|41000|3453|**30394**|20000|41000|3744|**31842**|23000|36400|2633|**0.004**|

Note: The p-values reported in the last column are generated by either one-tailed tests of equality of means or Chi-square tests.

76. Table 3 reports statistics for our measures of top-down accountability and transparency in school practices. The number of inspections from RED representatives in the previous semester varies greatly across schools, from a minimum of 0 to a maximum of 36; the comparison between rural and urban schools highlights that rural schools are significantly less likely to receive inspections. Overall, in 20% of schools school director are not subject to any form of appraisal and sanction if they underperform. Moreover, about half of the surveyed teachers report that they are not subject to any penalty if they underperform. Perceptions of irregularities in the appointment of school directors and teachers are quite high, especially in rural areas, where 32% and 22% of the surveyed school stakeholders believe that often or very often school directors and teachers, respectively, are appointed to posts due to personal connections rather than qualifications. Teacher absenteeism does not seem to be a major problem in Albanian schools; on average, the number of days of unjustified absence in a typical month is only 0.18, with a minimum of 0 and a maximum of 5 days; it also does not seem to be different across rural and urban schools.
Page 23

**Table 3. Top-down Accountability**

(School averages)

|||**Full s**|**ample**||**Sch**|**ools in **|**rural**<br>**ar **|**eas**|**Sch**|**ools in**|**urban**<br> **ar**|**eas**|**Urban**<br>**vs.**<br>**rural**|
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
||**Mean**|**Min**|**Max**|**sd**|**Mean**|**Min**|**Max**|**sd**|**Mean**|**Min**|**Max**|**sd**|**p-value**|
|**Top-down accountability**||||||||||||||
|N. of inspections from RED|**5.04**|0.00|36.00|5.52|**3.90**|0.00|35.00|4.06|**7.02**|0.00|36.00|7.00|**0.000**|
|Underperforming school director is<br>not subject to penalties (self-<br>reported by school director)|**0.20**|0.00|1.00|0.40|**0.21**|0.00|1.00|0.41|**0.18**|0.00|1.00|0.39|0.698|
|Underperforming teachers are not<br>subject to penalties (self-reported<br>by teachers)|**0.49**|0.00|1.00|0.26|**0.48**|0.00|1.00|0.27|**0.51**|0.00|1.00|0.25|0.233|
|Teachers' days of absence in May<br>2009|**0.18**|0.00|5.17|0.70|**0.22**|0.00|5.17|0.83|**0.13**|0.00|2.25|0.37|0.203|
|**Transparency**||||||||||||||
|School keeps a teacher absence<br>register|**0.91**|0.00|1.00|0.29|**0.90**|0.00|1.00|0.31|**0.92**|0.00|1.00|0.27|0.546|
|Public display of teachers' absence|**0.12**|0.00|1.00|0.33|**0.15**|0.00|1.00|0.36|**0.08**|0.00|1.00|0.27|0.163|
|Public display of students' final<br>grades|**0.56**|0.00|1.00|41.50|**0.70**|0.00|1.00|0.46|**0.82**|0.00|1.00|0.39|**0.079**|
|% respondents who perceive<br>corruption in the appointment of<br>school directors|**0.28**|0.00|0.82|0.18|**0.26**|0.00|0.82|0.17|**0.32**|0.03|0.77|0.18|**0.008**|
|% respondents who perceive<br>corruption in the appointment of<br>teachers|**0.20**|0.00|0.64|0.15|**0.19**|0.00|0.64|0.15|**0.22**|0.00|0.64|0.15|**0.099**|

Note: The p-values reported in the last column are generated by either one-tailed tests of equality of means or Chi-square tests.

77. Simple pair-wise correlations between the different measures of accountability and transparency reveal that not keeping a teacher absence register is associated with underperforming school directors not being subject to penalties. Table A.2.3, in the Appendix, compares the schools in the different REDs with respect to accountability and transparency of school administrations. The data show that the number of RED inspections is on average significantly higher in the REDs of Fier and Shkoder, and significantly lower in Korce, where it is also most likely that a bad performing school director is not subject to penalties. Perceptions of corruption in the appointments of school director and teachers are most severe in Vlore, Diber and Elbasan.

78. To contrast with the indicators above, Table 4 provides information about the bodies that have been established for participatory “bottom up” accountability and parental involvement in the school. The top panel of the table reports statistics on the composition and meetings of the school board (SB). The number of SB meetings in the previous semester (January – May 2009) varies from a minimum of zero to a maximum of five, with no significant differences between rural and urban schools. Contrary to the MoES directives, in about 30% of schools, the school director participates in school board meetings and expresses his or her vote, again with no significant differences between rural and urban schools. In about 81 percent of the schools, according to the answers of parent SB members, parent class representatives are elected by parents rather than appointed by teachers; and in 43% of the schools teacher school board members are elected by teachers rather than appointed by the school directors. These proportions are significantly higher in urban schools as compared to rural schools.

79. The second panel of Table 4 reports the proportions of parents who are not informed about: the existence of the school board; the existence of parent class representatives; the name of their child‟s math teacher; and the name of their child‟s Albanian teacher. About 43% of the parents do
Page 24 not know about the existence of the school board<sup>28</sup> and 28% do not know that there exist class parent representatives. Lack of information is more severe in rural schools than in urban schools. A closer look at the data (not reported in the table) show that of the 72 percent of parents who are aware of the existence of parent class representatives, only half know their names; and of the parents who know that parent representatives exist and are elected by parents, 77 percent did not vote in the most recent election.<sup>29</sup>

**Table 4. Participatory “Bottom Up” Accountability and Parent Involvement**

(School averages)

|||**Full**|**sample**|||**Rural**<br>|**areas**|||**Urban**|**areas**||**Rural**<br>**vs**<br>**urban **|
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
||**Mean**|**Min**|**Max**|**sd**|**Mean**|**Min**|**Max**|**sd**|**Mean**|**Min**|**Max**|**sd**|**p-value**|
|**The School Board:**||||||||||||||
|N. of meetings in the past semester|**1.92**|0.00|5.00|0.76|**1.89**|0.00|4.00|0.71|**1.98**|0.00|5.00|0.83|0.215|
|School directors participates and votes|**0.29**|0.00|1.00|0.46|**0.30**|0.00|1.00|0.46|**0.28**|0.00|1.00|0.45|0.632|
|Parent representatives are elected by parents|**0.81**|0.00|1.00|0.39|**0.78**|0.00|1.00|0.42|**0.86**|0.00|1.00|0.35|0.113|
|Teacher representatives are elected by<br>teachers|**0.43**|0.00|1.00|0.50|**0.37**|0.00|1.00|0.48|**0.54**|0.00|1.00|0.50|**0.016**|
|**Lack of information about participatory**<br>**organs and teachers:**||||||||||||||
|Parents not knowing that there is a SB|**0.43**|0.00|1.00|0.25|**0.48**|0.00|1.00|0.24|**0.36**|0.00|1.00|0.26|**0.001**|
|Parents not knowing that there are parent<br>class representatives|**0.28**|0.00|1.00|0.24|**0.34**|0.00|1.00|0.24|**0.16**|0.00|0.70|0.18|**0.000**|
|Parents not knowing the name of the math<br>teacher|**0.25**|0.00|0.80|0.21|**0.26**|0.00|0.80|0.21|**0.22**|0.00|0.80|0.20|0.143|
|Parents not knowing the name of the<br>language teacher|**0.23**|0.00|0.80|0.20|**0.25**|0.00|0.70|0.21|**0.20**|0.00|0.80|0.19|**0.055**|
|**Parental involvement:**||||||||||||||
|N. meetings with math teacher, reported by<br>parents (past semester)|**4.22**|1.40|8.50|1.48|**3.99**|1.60|8.40|1.42|**4.62**|1.40|8.50|1.50|**0.003**|
|N. meetings with language teacher, reported<br>by parents (past semester)|**4.15**|1.50|8.10|1.44|**3.94**|1.70|8.10|1.38|**4.52**|1.50|8.10|1.49|**0.005**|
|% Parents who participate in parent-teacher<br>meetings, reported by teachers|**0.72**|0.34|1.00|0.11|**0.72**|0.34|1.00|0.12|**0.74**|0.43|0.90|0.10|**0.092**|
|Parents who filed a complaint in education<br>matters|**0.21**|0.00|0.80|0.18|**0.22**|0.00|0.80|0.19|**0.18**|0.00|0.50|0.15|**0.080**|
|% respondents who think that lack of<br>parental involvement is a major problem in<br>the school|**0.215**|0.00|0.93|0.17|**0.21**|0.00|0.73|0.19|**0.21**|0.00|0.93|0.17|0.49|

Note: The p-values reported in the last column are generated by either one-tailed tests of equality of means or Chi-square tests.

80. The bottom panel of Table 4 reports the number of meetings parents said they held with the math and the language teachers during the previous semester (January-May 2010), and the proportion of parents who usually participate in parent-teacher meetings, as reported by teachers. We also report the proportion of parents who filed a complaint concerning education matters at least once in their life. The average number of meetings that parents said they had with teachers in the previous semester (January-May 2009) is on average around 4, yet it varies considerably across schools, and is significantly higher in urban schools. The proportion of parents who usually participate in parent-teacher meetings also varies greatly across schools, from a minimum of 34% to a maximum of 100%, with an average of 72%, there seems to be a significant difference between rural and urban schools in the proportion of participating parents, albeit such difference is quite small in magnitude. Lack of parental involvement seems to be considered a problem in about 20%

> 28 Only about 80 percent of the parents informed about the existence of the SB know that there are parent SB members, and of these, 72 percent do not know their names.

> 29 The great majority of these parents (74 percent) did not vote because they were not informed about the date and location of the election.
Page 25 of the schools, with no significant differences between rural and urban locations. Table A.2.4 in the Appendix presents statistics of our measures of participatory accountability and parental involvement across schools located in the different REDs.

81. The indicators in Table 5 confirm that as expected the average years of schooling of both the mother and the father of the students, and the number of books owned by the household, are significantly higher in urban locations than in rural locations. Albanian parents have on average completed primary school and at least two years of secondary or vocational school. The household economic status, as well as the proportion of employed parents, and the proportion of parents employed in occupations other than farming are also significantly higher in urban areas. About 90% of the surveyed parents seem to value the education of their children, and more so if they are located in urban areas.

**Table 5. Parents’ Characteristics**

(School averages)

|||**Full S**|**ample**||**Sch**|**ools in **|**rural**<br> **are**|**as**|**Sch**|**ools inu**|**rban**<br> **ar**|**eas**|**Rural**<br>**vs.**<br>**urban**|
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
||**Mean**|**Min**|**Max**|**Sd**|**Mean**|**Min**|**Max**|**sd**|**Mean**|**Min**|**Max**|**sd**|**p-value**|
|Average years of schooling of<br>mothers|**10.14**|6.89|14.80|1.66|**9.31**|7.20|11.80|0.90|**11.62**|6.89|14.80|1.67|**0.000**|
|Average years of schooling of<br>fathers|**10.40**|6.67|15.78|1.54|**9.74**|7.60|12.80|1.04|**11.57**|6.67|15.78|1.60|**0.000**|
|N. of books in the household|**23.47**|0.70|122.9|21.70|**14.78**|0.70|81.50|12.42|**38.85**|1.10|122.9|25.83|**0.000**|
|Never helps children with<br>homework|**0.16**|0.00|0.70|0.16|**0.16**|0.00|0.60|0.17|**0.14**|0.00|0.70|0.14|0.215|
|Currently unemployed (self-<br>reported)|**0.23**|0.00|0.80|0.19|**0.25**|0.00|0.80|0.19|**0.19**|0.00|0.60|0.17|**0.028**|
|Most common occupation of<br>parents: farmer (self-reported)|**0.07**|0.00|0.60|0.11|**0.11**|0.00|0.60|0.12|**0.01**|0.00|0.30|0.04|**0.000**|
|Household economic status<sup>*</sup>|**0.017**|-1.291|1.693|0.596|**-0.269**|-1.291|0.831|0.388|**0.522**|-0.894|1.692|0.566|**0.000**|
|Disagree with:"There is not much<br>||||||||||||||
|point in sending my children to<br>school -education won‟t help<br>them to find a better job"|**0.90**|0.40|1.00|0.13|**0.88**|0.40|1.00|0.14|**0.93**|0.60|1.00|0.11|**0.009**|

Note:<sup>*</sup> the index of household economic status is the principal factor obtained from principal factor analysis over a set of 13 variables describing the assets owned by the household. The p-values reported in the last column are generated by either one-tailed tests of equality of means or Chi-square tests.

82. Table 6 reports some salient characteristics of the community to which the schools in the sample belong. The extent of “divide” (as defined in the earlier section) within the community seems to be moderate, as suggested by the 2.6 average of our index computed on a scale of 1 to 5, across all the surveyed school stakeholders. Urban areas seem to be characterized by greater divide than rural areas. Furthermore, in urban areas we observe less trust and cooperation than in rural areas, as shown by the significantly higher proportion of respondents who think that in their neighborhood there are people who want to take advantage, and the fewer proportion of those who think that members of their community would be willing to help if needed, and would come together to jointly solve a common problem such as possible water shortage.
Page 26

**Table 6. Community Characteristics**

(Averages of answers of all school stakeholders)

|||**Full Sa**|**mple**|||**Rural**<br>|**areas**|||**Urban**<br>|**areas**||**Rural vs.**<br>**urban**|
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
||**Mean**|**Min**|**Max**|**Sd**|**Mean**|**Min**|**Max**|**sd**|**Mean**|**Min**|**Max**|**sd**|**p-value**|
|**Divide within the village/neighborhood**||||||||||||||
|Extent of divide (1-low; 5- high)|**2.67**|1.55|3.89|0.48|**2.56**|1.55|3.77|0.43|**2.88**|1.83|3.89|0.50|**0.000**|
|Agree with: “Parents in my community<br>still do not understand the benefits of<br>education for their children”|<br>**0.489**|0.00|0.933|0.19|**0.481**|0.00|0.867|0.18|**0.504**|0.00|0.933|0.20|0.220|
|**Trust and cooperation**||||||||||||||
|Think that “In this village /neighborhood,<br>there are people who want to take<br>advantage”|**0.32**|0.03|0.77|0.17|**0.29**|0.03|0.72|0.16|**0.36**|0.05|0.77|0.18|**0.010**|
|Think that “most people in this<br>village/neighborhood are willing to help if<br>you need it”|**0.65**|0.13|1.00|0.19|**0.68**|0.13|1.00|0.19|**0.59**|0.18|0.90|0.18|**0.000**|
|Think that “village members are likely to<br>cooperate in the case of water shortage”|**0.69**|0.03|1.00|0.19|**0.71**|0.03|1.00|0.19|**0.66**|0.30|1.00|0.18|**0.048**|
|Participated in community activities in the<br>last year|**0.24**|0.00|0.73|0.17|**0.23**|0.00|0.73|0.17|**0.25**|0.00|0.67|0.17|0.249|

Note: The p-values reported in the last column are generated by either one-tailed tests of equality of means or Chi-square tests.

# **II.d. Correlates of School Performance**

83. In this section we aim to identify the factors that are significantly correlated with school performance (as measured by the five education “outputs” identified earlier). In the first half simple pair-wise correlations are presented. These are followed by estimates from regression analysis.

84. Table 7 presents only the variables that are significantly correlated with at least one of the five education outputs. In other words, if a variable for which descriptive statistics presented earlier is not reported in Table 7 that indicates that the variable is not correlated with any of the five measures of school performance. The top panel of Table 7 shows that most proxies of school resources are correlated with the proportion of students who excelled in math and Albanian in the 2009 final examination. In particular, excellence in both math and language appears more likely in urban schools, schools that have a phone (which might proxy the quality of local infrastructures), and schools who do not tend to suffer from power shortages. The presence of a library in the school also seems to be positively correlated with students‟ performance in the math test, but not in the Albanian test.

85. As for the availability and quality of human resources, there is no significant correlation between education outcomes and: characteristics of the school director; education of teachers; and student-teacher ratio (not reported in the table). We do however find years of teaching experience, teachers‟ motivation and wages to be significantly and positively correlated with students‟ excellence in the language exam. Teachers‟ motivation seems also to be negatively correlated with students‟ drop-out rate.

86. The second panel of Table 7 focuses on measures of top-down accountability and transparency. The only factors found to be correlated with at least one of the education outcomes are the number of visits from RED inspectors, and perceptions of corruption in the appointment of teachers. The former is correlated positively with students‟ excellence in Albanian and negatively with drop-out rate; the latter is positively correlated with drop-out rate.
Page 27

**Table 7. Pair-wise Correlations**

||**Drop-out**<br>**rate**|**Repetition**<br>**rate**|**Fail rate**|**Excellence in**<br>**Albanian**|**Excellence in**<br>**math**|
|---|---|---|---|---|---|
|**RESOURCES**||||||
|Rural location|0.104|0.051|0.096|-0.296***|-0.313***|
|Availability of public transportation|0.067|0.042|0.099|0.137*|-0.004|
|Days with no electricity|0.045|-0.057|0.004|-0.201***|-0.163**|
|No phone|0.134*|0.039|0.073|-0.280***|-0.384***|
|Library|0.046|-0.039|0.048|0.066|0.242***|
|Teachers‟ years of experience|0.063|0.018|0.083|0.137*|0.091|
|Teachers are motivated|-0.196*|-0.045|-0.078|0.228***|0.088|
|Average wage of teachers|0.004|-0.059|0.024|0.132*|0.113|
|**TOP-DOWN ACCOUNTABILITY AND TRANSPA**|**RENCY**|||||
|N. of RED inspections|-0.167**|-0.029|-0.063|0.209***|0.111|
|Perceptions of corruption in the appointment of teachers|0.191**|0.028|0.050|0.103|-0.017|
|**PARTICIPATORY ACCOUNTABILITY AND PA**<br>**INVOLVEMENT**|**RENTAL**|||||
|N. of SB meetings (Jan-May 09)|- 0.238***|-0.0805|- 0.045|0.043|0.045|
|School directors participates and votes|0.149*|-0.069|0.027|-0.039|-0.070|
|% parents not knowing that there is a SB|0.102|0.073|0.187**|-0.139*|-0.184**|
|% parents not knowing that there are parent class reps.|0.076|0.183**|0.136*|-0.123*|-0.189**|
|% parents not knowing the name of the math teacher|0.068|-0.028|0.159**|-0.236***|-0.175**|
|% parents not knowing the name of the language teacher|0.121|-0.052|0.161**|-0.190**|-0.159**|
|Number of meetings with the math teacher (Jan-May 09)|-0.248***|-0.008|-0.149**|0.204***|0.296***|
|Number of meetings with the math teacher (Jan-May 09)|-0.251***|-0.014|-0.119|0.198***|0.317***|
|“ lack of parental involvement in the school is a problem”|0.107|0.096|0.063|-0.132*|-0.166**|
|**PARENTS’ CHARACTERISTICS**||||||
|Average mother education|-0.227***|-0.184**|-0.178**|0.322***|0.358***|
|Average father education|-0.225***|-0.215***|-0.178**|0.263***|0.364***|
|Average n. of books in the household|-0.150**|-0.129*|-0.126*|0.180**|0.186**|
|% currently unemployed|0.111|0.048|0.024|-0.147*|-0.109|
|% farmers|0.118|0.186**|0.184**|-0.222***|-0.136*|
|Average index of economic status|-0.178**|-0.148**|-0.171**|0.282***|0.306***|

Note: ***significant at the 1% level, **significant at the 5% lever, * significant at the 10% level.

87. The third panel of Table 7 highlights that a large number of measures of participatory accountability and parental involvement are correlated with education outcomes. In particular, students‟ excellence in math and language appears significantly more likely where parents are informed about participatory organs, know the names of the math and Albanian teachers, have more frequent meetings with teachers, and where lack of parental involvement is not considered to be a problem in the school. Parents‟ lack of information is also positively correlated with failure and repetition rates. Interestingly, the drop-out rate seems to be higher in schools where there are fewer SB meetings, where the school director can participate and vote in such meetings, and where parents held fewer meetings with teachers.
Page 28 88. The bottom panel of Table 7 confirms that family characteristics are important correlates of education outcomes. The average years of schooling of students‟ mother and father, the number of books owned by the household and the index of socioeconomic status are all strongly correlated, with the expected sign, with all five school performance variables. None of the community characteristics presented in Table 6 appear to be significantly correlated with school-level education outputs.

89. Following the exposition and data analysis structure in previous sections, for each measure of school performance four OLS regressions have been conducted, each including only one category of independent variables: (1) school resources; (2) top-down accountability and transparency; (3) participatory accountability and parental involvement; and (4) family and community background. The OLS regressions have then been run including all four categories, and the risk of multicollinearity bias has been lowered by using principal factor analysis. In all regressions RED fixed effects have been used, and the standard errors have been clustered by district to account for possible unobservable inter-dependencies between schools belonging to the same district. Table 8 and Table 9 report estimates for drop-out, repetition and failure rates, and for excellence in math and Albanian, employing one category of explanatory variables at a time. Table 10 reports estimates for all school performance measures, and including all categories of explanatory variables.

90. The top panel of Table 8 shows very few correlates with drop-out, repetition and failure rates among school resources. The only significant results seem to be the positive correlation between rural location and failure rate and the negative relationship between teachers‟ motivation and drop-out rate. None of the top-down accountability and transparency measures appear to be correlated with drop-out, repetition and failure rates. In contrast, the number of SB meetings and whether the school director usually participates and votes in the meetings are, respectively, negatively and positively correlated with drop-out and repetition rates. Moreover, lack of parental involvement in the school, as assessed by all school stakeholders, is positively correlated with students‟ repetition rate.

91. Note that, in the participatory accountability regressions (Panel 3), in order to reduce the bias generated by multicollinearity between the different proxies of parental awareness and involvement, principal factor analysis was conducted on three groups of highly related variables: (i) the measures of parents‟ lack of information with respect to the existence of class parent representatives and school board; (ii) the measures of parents‟ lack of information of the names of head teacher, math teacher, language teacher and school director; (iii) the measures of parents‟ participation in parent-teacher meetings, with different teachers, between January and May 2009. Thus, in the regression, the resulting indexes have been included: parents‟ awareness of participatory organs; parents‟ awareness of teachers‟ names; and meetings with teachers. Similarly, when testing for family background we do not control for the unemployment status of parents, nor their possible occupation as farmers, since both variables are highly correlated with the household index of socioeconomic status. Among the proxies for the students‟ family background, the only significant result is the negative correlation between fathers‟ education and repetition rate.
Page 29

# **Table 8. Correlates with Drop-out, Repetition and Failure Rates**

||||||**Dependent**|**variable:**||||||
|---|---|---|---|---|---|---|---|---|---|---|---|
||**Drop-o**|**ut rate**|||**Repeti**|**tion rate**||**Fa**|**ilure rate**|**(final exa**|**m)**|
|(1)|(2)|(3)|(4)|(5)|(6)|(7)|(8)|(9)|(10)|(11)|(12)|
|**RESOURCES**||||||||||||
|**Rural school**<br>0.00<br>||||-0.00<br>||||**0.01***<br>||||
|(0.009)||||(0.002)||||**(0.008)**||||
|Public transportation<br>0.01<br>||||0.00<br>||||0.01<br>||||
|(0.007)||||(0.002)||||(0.009)||||
|No electricity (days)<br>0.00<br>(0.000)||||-0.00<br>(0.000)||||0.00<br>(0.001)||||
|No telephone<br>0.01<br>||||0.00<br>||||0.00<br>||||
|(0.011)<br>No library<br>-0.01||||(0.005)<br>0.00||||(0.007)<br>-0.01||||
|<br> <br>(0.008)||||<br>(0.003)||||<br>(0.012)||||
|**Motivated teachers**<br>**-0.04*****||||0.01||||-0.01||||
|<br> <br>**(0.012)**||||<br>(0.008)||||<br>(0.017)||||
|Teacher exerience<br>-000||||000||||000||||
|p<br>.<br>(0.001)||||.<br>(0.000)||||.<br>(0.001)||||
|Ln (teacher wage)<br>0.05||||-0.02||||-0.03||||
|<br>(0.046)||||(0.020)||||(0.045)||||
|**TOP-DOWN ACCOUNTABILIT**|**Y AND T**|**RANSPAR**|**ENCY**|||||||||
|RED inspections|-0.00||||-0.00||||-0.00|||
||(0.000)||||(0.000)||||(0.000)|||
|SD not subject to sanctions|<br>0.00<br>||||<br>-0.00<br>||||<br>0.01<br>|||
|Th t bt t ti|(0.009)<br>001||||(0.003)<br>000||||(0.012)<br>000|||
|eacers no sujec o sancons|-.<br>(0.014)||||-.<br>(0.004)||||.<br>(0.015)|||
|Th bM09|<br>000||||<br>000||||<br>001|||
|eacers asence-ay|.<br>||||-.<br>||||-.<br>|||
||(0.003)||||(0.001)||||(0.005)|||
|Teacher absence register|<br>-0.01<br>||||<br>0.00<br>||||<br>0.01<br>|||
||(0.020)||||(0.004)||||(0.017)|||
|Corrution in teacher aointment|<br>003||||<br>000||||<br>004|||
|p pp|.<br>(0.023)||||.<br>(0.007)||||.<br>(0.033)|||
|**PARTICIPATORY ACCOUNTA**<br>|**BILITY**|**AND PAR**<br>*******|**ENTS’ IN**|**VOLVE**|**MENT**|*****||||||
|**N. SB meetings**||**-0.01**||||**-0.00**||||-0.00||
|||**(0.004)**||||**(0.001)**||||(0.005)||
|**Director can vote**||<br>**0.01**||||<br>**-0.00****||||<br>-0.00||
|||**(0.009)**<br>||||**(0.002)**<br>||||(0.013)<br>||
|No info about participatory organs||0.01<br>(0014)||||0.01<br>(0009)||||-0.00<br>(0010)||
|No info about teachers‟ names||.<br>-0.00||||.<br>-0.01||||.<br>0.02||
|||(0010)||||(0006)||||(0011)||
|N. of meetings with teachers||.<br>-0.01||||.<br>0.00||||.<br>-0.01||
|||(0005)||||(0005)||||(0016)||
|**Lack of parental involvement in s**|**chool**|.<br>0.03||||.<br>**0.02***||||.<br>0.01||
|||(0.032)||||**(0.012)**||||(0.029)||
|**PARENTS’ CHARACTERISTIC**|**S**|||||||||||
|Mother yrs. of education|||0.00||||0.00||||-0.00|
||||(0.004)||||(0.002)||||(0.005)|
|**Father yrs. of education**|||-0.01<br>||||**-0.00***<br>||||-0.00<br>|
|N. books in hh|||(0.004)<br>0.00||||**(0.002)**<br>0.00||||(0.005)<br>0.00|
||||(0.000)||||(0.000)||||(0.000)|
|Economic status|||-0.00<br>0006||||-0.00<br>0006||||-0.02<br>0010|
|Constant<br>-0.42|0.05**|0.06***|(.)<br>0.10***|0.26|0.02|0.02*|(.)<br>0.04**|0.31|0.04*|0.06***|(.)<br> <br>0.08*|
|(0.483)|(0.024)|(0.015)|(0.030)|(0.216)|(0.015)|(0.013)|(0.019)|(0.479)|(0.022)|(0.020)|(0.040)|
|Observations<br>167|164|161|169|174|169|168|177|170|165|165|173|
|R-squared<br>0.218|0.176|0.231|0.199|0.093|0.068|0.130|0.102|0.116|0.117|0.131|0.124|

Note: Robust standard errors, in parentheses, have been adjusted to account for clustering within districts. *** p<0.01, ** p<0.05, * p<0.1.
Page 30 92. Table 9 reports results obtained by conducting the same empirical procedure, with the proportion of students who excelled in math (columns 1 to 4) and in language (columns 5 to 8) as dependent variables. The estimates in the top panel confirm that a number of school resources and teachers‟ characteristics are significantly correlated with students‟ excellence in math and language. In particular, the percentage of students excelling in Albanian is significantly higher in urban schools which suffer fewer days with no electricity, where teachers are motivated, and have more teaching experience. The percentage of students excelling in math is higher in schools that have a telephone and in those with a library.

93. Among the top-down accountability and transparency measures, more RED inspections and fewer days of teachers‟ absence are associated with better performance in the Albanian exam. Teachers‟ absence, and whether the school has a teacher absence register, seem to affect -negatively and positively, respectively- students‟ performance in the math exam.

94. The estimates presented in the third panel of Table 9 suggest that lack of parental involvement in the school is negatively associated with students‟ excellence in both math and Albanian. Moreover, the proportion of students who excelled in math is higher the higher the number of meetings parents had with teachers, and lower if parents have no information about participatory accountability bodies.

95. Among the family background proxies, fathers‟ education and household economic status seem to positively affect respectively students‟ excellence in math and excellence in Albanian language. Moreover, in communities where differences among community members create divisions to a larger extent, students are less likely to excel in both math and language.
Page 31

# **Table 9. Correlates with Students’ Excellence in Albanian and Math**

||||**D**<br>|**ependent**<br>|**variable:**<br>||||
|---|---|---|---|---|---|---|---|---|
||**% stud**|**ents who exc**<br>**(final exa**|**elled in Alb**<br>**m 2009)**|**anian**|**% st**|**udents who**<br>**(final ex**|**excelled in**<br>**am 2009)**|**math**|
||(1)|(2)|(3)|(4)|(5)|(6)|(7)|(8)|
|**RESOURCES**|||||||||
|**Rural school**|**-0.09****<br>||||-0.01<br>||||
||**(0.042)**||||(0.025)||||
|Public transportation|0.03<br>(0031)||||-0.01<br>(0017)||||
|**No electricity (days)**|.<br>**-0.00*****<br>**(0.001)**||||.<br>-0.00<br>(0.001)||||
|**No telephone**|<br>-0.02||||<br>**-0.06*****||||
||(0.043)||||**(0.020)**||||
|**No library**|<br>-0.00<br>(0.036)||||<br>**-0.04****<br>**(0.017)**||||
|**Motivated teachers**|<br>**0.23*****||||<br>0.06||||
||<br>**(0.076)**||||<br>(0.037)||||
|**Teacher experience**|<br>**0.00****||||<br>0.00||||
||<br>**(0.002)**||||<br>(0.002)||||
|Ln (teacher wage)|-0.03||||-0.04||||
||(0.226)||||(0.127)||||
|**TOP-DOWN ACCOUNTABILITY AN**|**D TRANSP**|**ARENCY**|||||||
|**RED insections**||**001****||||000|||
|**p**||**.**<br>**(0.004)**||||.<br>(0.001)|||
|SD not subject to sanctions||0.02||||0.03|||
|||(0.042)||||(0.023)|||
|Teachers not subject to sanctions||-0.01||||-0.03|||
|||(0065)||||(0031)|||
|**Teachers absence-May09**||.<br>**-0.03****||||.<br>**-0.01*****|||
|||**(0.012)**||||**(0.004)**|||
|**School has a teacher absence register**||<br>0.04||||<br>**0.05***|||
|||(0.056)||||**(0.023)**|||
|Corruption in teacher appointment||<br>0.06||||<br>0.01|||
|||(0.128)||||(0.066)|||
|**PARTICIPATORY ACCOUNTABILI**|**TY AND PA**|**RENTS’ INV**|**OLVEMENT**||||||
|N. SB meetings|||0.01||||-0.00||
||||(0017)||||(0009)||
|Director can vote|||.<br>-0.02||||.<br>-0.02||
||||(0048)||||(0016)||
|**No info about participatory organs**|||.<br>-0.02<br>(0033)||||.<br>**-0.03****<br>**(0.017)**||
|No info about teachers‟ names|||.<br>-0.04<br>(0.032)||||<br>-0.01<br>(0.018)||
|**N f ti ith th**|||<br>006||||<br>**005***||
|**. o meengs w eacers**|||.<br>(0.038)||||**.**<br>**(0.023)**||
|**Lack of parental involvement is a prob**|**lem**||<br>**-0.15***||||<br>**-0.09****||
||||<br>**(0.089)**||||<br>**(0.038)**||
|**PARENTS’ AND COMMUNITY CHA**|**RACTERIS**|**TICS**|||||||
|Mother yrs. of education||||0.03||||0.01|
|||||(0.021)||||(0.013)|
|**Father yrs. of education**||||0.00<br>(0019)||||**0.03*****<br>**(0009)**|
|N. books in hh||||.<br>-0.00||||**.**<br>-0.00|
|||||(0.001)||||(0.001)|
|**Economic status**||||<br>**0.08***<br>**(0.028)**||||<br>0.03<br>(0.023)|
|**Ett f diid i th it**||||<br>**007****||||<br>**003****|
|**xen o ve n e communy**||||**-.**<br>**(0.028)**||||**-.**<br>**(0.014)**|
|Constant|0.58|0.13|0.22***|<br>0.00|0.40|0.05|0.12***|<br>-0.18*|
||<br>(1.312)|<br>(0.078)|<br>(0.037)|<br>(0.175)|<br>(2.352)|<br>(0.037)|<br>(0.027)|<br>(0.088)|
|RED fixed effects|YES|YES|YES|YES|YES|YES|YES|YES|
|<br>R-squared|<br>0.250|<br>0.204|<br>0.200|<br>0.254|<br>0.325|<br>0.153|<br>0.218|<br>0.253|

Note: Robust standard errors, in parentheses, have been adjusted to account for clustering within districts. *** p<0.01, ** p<0.05, * p<0.1.
Page 32 96. Table 10 reports regression estimates for all five school performance measures, controlling for all categories of independent variables. Therefore, the estimates presented in Table 10 check for the robustness of the results highlighted in Tables 7, 8 and 9.

**Table 10. Correlates with School-level Education Outcomes**

|||**De**<br>|**pendent vari**<br>|**ables**<br>||
|---|---|---|---|---|---|
||**Drop-out**<br>**rate**|**Repetition**<br>**rate**|**Failure**<br>**rate**|**% students**<br>**excelling in**<br>**Albanian **|**% students**<br>**excelling in**<br>**math **|
||(1)|(2)|(3)|(4)|(5)|
|**Rural school**|-0.00|-0.01|0.01|**-0.07***|-0.02|
||(0.015)|(0.005)|(0.013)|**(0.039)**|(0.024)|
|**Poor facility and resources (first factor)**|-0.01<br>|0.00<br>|-0.00<br>|0.01<br>|**-0.02****<br>|
||(0.008)|(0.002)|(0.009)|(0.023)|**(0.009)**|
|**Teacher experience**|-0.00|0.00|0.00|**0.01****|-0.00|
||(0.001)|(0.000)|(0.001)|**(0.002)**|(0.002)|
|**Motivated teachers**|-0.02|**0.02***|-0.02|**0.19****|0.03|
||(0.017)|**(0.009)**|(0.023)|**(0.086)**|(0.038)|
|RED inspections|-0.00|0.00|0.00|0.01|0.00|
||(0.000)|(0.000)|(0.001)|(0.003)|(0.002)|
|SD not subject to sanctions|0.00|0.00|0.01|0.03|0.03|
||(0.011)|(0.003)|(0.012)|(0.039)|(0.018)|
|**Teachers not subject to sanctions**|-0.01|0.00|0.01|-0.01|**-0.05***|
||(0.015)|(0.004)|(0.014)|(0.066)|**(0.028)**|
|**Teachers absence-May09**|0.00|**-0.00****|**-0.01***|-0.01|-0.00|
||(0.003)|**(0.001)**|**(0.005)**|(0.013)|(0.008)|
|**School has a teacher absence register**|-0.00|0.00<br>|0.02<br>|0.03<br>|**0.05****|
||(0.020)|(0.005)|(0.016)|(0.056)|**(0.021)**|
|Corruption in teachers‟ appointments|0.03|0.01|**0.05***|-0.00|-0.04|
||(0.028)|(0.008)|**(0.030)**|(0.131)|(0.064)|
|**N. of SB meetings**|**-0.01****|**-0.00****|-0.00|-0.02|-0.01|
||**(0.004)**|**(0.001)**|(0.007)|(0.027)|(0.012)|
|No information about participatory organs (parents)|0.00|0.00|0.00|0.01|-0.02|
||(0.016)|(0.008)|(0.012)|(0.032)|(0.019)|
|No information about names of teachers (parents)|-0.01|-0.01|0.01|-0.03|-0.01|
||(0.009)|(0.007)|(0.014)|(0.033)|(0.020)|
|N. of meetings with teachers|-0.00|0.01|-0.01|0.04|0.03|
||(0.006)|(0.007)|(0.016)|(0.033)|(0.021)|
|**Lack of parental involvement in the school**|0.02|**0.03****|0.02|-0.09|**-0.09****|
||(0.034)|**(0.013)**|(0.029)|(0.097)|**(0.042)**|
|Mother years of education|0.00<br>|0.00<br>|-0.00<br>|0.02<br>|0.00<br>|
||(0.005)|(0.002)|(0.007)|(0.020)|(0.011)|
|**Father years of education**|<br>-0.01|<br>-0.00|<br>0.00|<br>-0.01|<br>**0.02***|
||(0.005)|(0.002)|(0.006)|(0.023)|**(0.010)**|
|N. of books in the household|-0.00|-0.00|0.00|-0.00|-0.00|
||(0.000)|(0.000)|(0.000)|(0.001)|(0.000)|
|Economic status|-0.00|-0.01|-0.01|0.05|0.00|
||(0.010)|(0.007)|(0.012)|(0.040)|(0.029)|
|**Extent of divide in the community**|0.00|-0.00|-0.01|**-0.07****|**-0.04****|
||(0.006)|(0.004)|(0.008)|**(0.031)**|**(0.019)**|
|Constant|0.13**|0.02|0.03|0.12|0.00|
||(0.053)|(0.020)|(0.072)|(0.244)|(0.127)|
|RED fixed effects|YES|YES|YES|YES|YES|
|R-squared|0.298|0.188|0.194|0.391|0.433|

Note: Robust standard errors, in parentheses, have been adjusted to account for clustering within districts. *** p<0.01, ** p<0.05, * p<0.1.

97. The estimates for drop-out, repetition and failure rates are not very informative as most of the variables are rendered statistically insignificant a large set of controls are included. Where variable appear significant, their substantive relationship to outcomes is difficult to interpret. For example, teachers‟ motivation appears positively correlated with repetition rates, possibly suggesting that more motivated teachers are more demanding of students. Along the same lines, repetition and failure rates are lower in schools where teachers‟ absence was more severe in May 2009. Drop-out and repetition rates appear respectively higher and lower the higher the number of school board meetings; repetition rates are also higher in the case of lack of parental involvement in
Page 33 the school. None of the proxies for family background seem to affect drop-out, repetition and failure rates.

98. Interestingly, different sets of variables seem to be correlated with students‟ excellence in math and in Albanian. For instance, the rural location of the school, and the experience and motivation of teachers are negatively correlated with students‟ performance in Albanian, but not in math. None of the proxies for top-down and participatory accountability, nor parental involvement or family background, are correlated with the proportion of students who got a 9 or a 10 in Albanian. The only variable that seems significantly correlated with students‟ excellence in Albanian, with the expected negative sign, is the extent of divide within members of the community.

99. The results obtained for students‟ excellence in math and Albanian are more enlightening. Students‟ excellence in math appears to be correlated with a larger set of factors. First of all, the proportion of students excelling in math is lower in schools characterized by poor school facilities.<sup>30</sup> While teachers‟ characteristics do not seem to play a role, teacher appraisal and transparency with respect to teachers‟ absence from school do; indeed, students excellence in math is more likely in schools where teachers believe that they would be penalized if they underperform, and where a teacher absence register exists and is regularly used. Lack of parental involvement in the school also appears to be correlated, negatively, with students‟ performance in the final math exam. A father‟s education also seems to have strong, positive, impact on children‟s performance in math. Finally, the extent of divide in the community appears to matter by reducing the proportion of students excelling in both math and language.

> 30 This was measured by an index obtained through principal factor analysis to reduce multicollinearity bias.  The index was constructed using: days with no electricity; absence of a telephone in the school; percentage of desks in poor conditions and absence of a library.
Page 34

# Part III. Parent Demand for Accountability and Better Governance

## **III.a. The Importance of Parents’ Preferences**

100. From a now widely accepted conceptual framework presented in earlier sections of this report, there are two routes of accountability: a long route and a short route. The long route connects the recipients of the services and policymakers in the ideal institutional setting, with wellfunctioning electoral systems, free from patronage and clientelism, and with perfect information on the standards that public service providers should meet. In such a setting, service recipients can hold policymakers accountable for the delivery of poor public services through the exercise of their votes. However, this ideal setting is rarely if ever the reality experienced by households, especially in developing and transition countries, where the route between service recipients and policymakers is often long, institutionally fragile, and strewn with obstacles. Decentralization may shorten the route to a certain extent, but electoral inefficiencies, elite capture and imperfect information can still hinder households from exercising accountability.<sup>31</sup> In such contexts, the shorter “local” route to accountability, which connects service recipients directly with service providers, is likely to be more viable and effective (World Bank, 2004).

101. In the education sector, as with other areas of public service, improving accountability and governance requires that school directors and teachers are assigned clear responsibilities; are monitored while exercising these responsibilities; and are subject to penalties when their performance falls short of predetermined targets. In education, the most direct -and therefore shortest- route of accountability would be for _all_ parents to be involved in both the formulation of the responsibilities and outcomes that enter the contracts signed by teachers, and the implementation of penalties that make these contracts binding.<sup>32</sup> But because the active participation of _all_ parents is logistically unfeasible, only a _sub-group_ of parents, representing the interests of all parents, are given decisional power in schools as active members of a school board or some other structure. There is general agreement that the design of the specific participatory institutions adopted by schools should have the clear objective of inducing (directly or indirectly) the participation of all parents while minimizing the chances of elite capture,<sup>33</sup> and that the existence and functioning of such institutions should be _known_ to all parents.

102. In Albania‟s basic schools, according to MoES guidelines, parent members of the School Boards should be elected parent class representatives. The parent class representatives of each grade are together members of the school‟s Parent School Council. Members of this Council elect from among themselves parent members of the School Board. Thus a necessary condition for the School Board to function as intended, is that its parent members are elected by parent class representatives, who themselves have been elected by parents participating fully in the accountability process. If the parent class representatives who are supposed to elect the parent members of the school board have conflict of interest because they have been appointed by teachers, the representation of parents' interests in the School Board is necessarily compromised.

> 31 See Oates (1972) and Azfar et al. (2001) for general discussion of the possible limitations of decentralization.

> 32 In practice, school-based management (SBM) programs may or may not give decisional power to the parents. Some programs give parents decisional power over the school budget only; others involve parents also in the choice of the curriculum and textbooks; others give parents responsibilities in the hiring and firing of teachers; others invite parents to participate in the development of a school improvement plan, and possibly in fund-raising. School-based management programs may also exclude parents from participating in the decision process.  There seems to be three different models of SBM (see Wohlstetter and Odden, 1992). In the first model, the primary objective is to give the community control over decision-making; in the second model, teacher- and not parent- empowerment is the primary objective of the reform; finally, in the third model it is the principal to be given the most control lover decision-making.

> 33 It is for instance reasonable to believe that parents are more likely to feel empowered and thus get involved if the sub-group of parents who are assigned decisional power are _elected_ by all parents rather than appointed to their position by the school director and/or the teachers, and/or the head of the community. Elected parent representatives are also less likely to be “captured” by the interests of school director and teachers.
Page 35 103. But do parents get involved in participatory accountability structures? What are the factors that determine whether parents get involved? And if they do participate, does this have an impact on student and school performance?

104. As discussed in earlier sections of this report, the empirical literature exploring the impact of parental involvement in education is relatively new. The existing studies can be categorized generally into two types: those using retrospective survey data of the sort presented earlier; and those that use an experimental or randomized controlled trial (RCT) approach, which involves the random allocation of participatory institutions of various types to different schools. Retrospective studies focus mainly on the existence of participatory institutions and actual parental participation, and their association with school outcomes. As discussed earlier, while these retrospective, ex post evaluations provide encouraging findings, they can be compromised by endogeneity and omitted variables<sup>34</sup> as schools with participatory institutions or more parental involvement in general may differ from other schools in a number of ways that may both drive the difference in parental participation and have an independent effect on school outcomes. This problem can, of course, be addressed by randomly allocating participatory programs to schools and measuring the effect by conducting ex ante and ex post surveys, but this approach can be costly.

105. Parents‟ willingness to participate in holding teachers to account depends on the actual and expected costs and benefits associated with participating. The actual costs include: transportation costs; the opportunity cost of time; and the internal psychological costs that parents may suffer when holding teachers to account, especially when they have personal relationships with teachers or feel somehow inferior or beholden to teachers. The expected benefits from participation include the better education that parents expect their children to receive if teachers are held to account and any intrinsic benefit associated with believing that holding teachers to account is “the right thing to do”. This section of the report discusses the use of experimental techniques to help measure these less tangible intrinsic costs and benefits to parents involvement in structures designed to help them hold service providers to account.

106. The experimental techniques that were deployed in parallel with the survey of school stakeholders in Albania, do not overcome the limitations of retrospective evaluations. Nor are they a substitute for the power of RCTs conducted with an ex-ante and ex-post evaluation strategy. However, they can provide valuable insights that help policymakers avoid what can be costly mistakes in the design and piloting of innovative interventions, particularly in country contexts where there is little or no prior information to guide their efforts.

# **III.b. Controlled Lab Type Experiments: Just Playing Games?**

107. Alongside the Albania School Stakeholder Survey presented in the part two of this report, parents and teachers were invited to participate in the specifically designed controlled, lab-type experiments (LTEs). The retrospective survey was designed to quantify the institutions for increasing accountability (including both external “top down” and participatory “bottom up” structures); to measure the awareness of stakeholders (especially parents), of the school institutional framework in general, and participatory institutions in particular; and to measure actual parental involvement with the school. To complement this valuable data, the LTE was designed to yield insights into parents‟ preferences for participation, i.e. their willingness to forgo personal payoffs in order to hold teachers to account.<sup>35</sup>

> 34 Gunnarsson et al. (2009), in their survey-based ex post evaluation of participatory institutions in eight Latin American countries, found that controlling for the possible endogeneity of school resources and parents‟ participation, substantially increases the estimated impact of parental participation on students‟ test scores.

> 35 It also provided a measure of teachers‟ levels of intrinsic motivation that will be analyzed elsewhere.
Page 36 108. Lab-type experiments (LTEs) –sometimes referred to simply as “behavioral games”facilitate the measurement of individuals‟ values, beliefs and preferences that cannot be captured in survey data. They have been used in a number of countries and contexts, involving a wide variety of groups; from university students, and children, to adults without formal literacy or numeracy in foraging small-scale societies.<sup>36</sup> The specific details of each game will vary, depending on the specific preferences that they are designed to capture. LTEs are always designed with a clear research objective in mind, and employ real money in order to recreate as closely as possible the incentives that individuals face in the real-life situations that the games aim to simulate and elucidate.

109. Experimentally generated measures of individual preferences have been shown to correlate with individuals‟ behavior in a variety of contexts of direct interest to policy makers. Barr and Packard (2000 and 200#) show that experimental measures of risk aversion and time preference are significantly associated with the choices of workers whether to take up formal pension plans in Chile, and even when choosing between plans structured either as individual savings (“defined contribution”) or risk sharing (“defined benefit”) in Peru. Carpenter and Seki (forthcoming) show that Japanese fishermen‟s preferences for cooperation, as measured in a lab-type public goods game, predict their productivity when pooling their catches with other fishermen in everyday life.<sup>37</sup> Karlan et al. (2005) find that individual trustworthiness, measured using a lab-type trust game conducted with members of a non-profit “village banking” organization in Peru, predicts loan repayment. Barr and Serneels (2009) show that individual tendencies to reciprocate, as measured using a lab-type trust game played by employees of manufacturing firms in Ghana, predict both individual- and firm-level productivity. Serra, Serneels and Barr (forthcoming) show that the prosocial motivations of final year medical and nursing students in Ethiopia, as measured using a labtype generalized trust game, predict their decision to work in non-profit health facilities rather than in the public or private for profit sectors. Finally, Barr and Zeitlin (2010) find that the decisions of Ugandan teachers towards their pupils‟ parents in games correlate strongly with the teachers‟ actual absenteeism rates.

110. With the exception of Barr and Zeitlin (2010), LTEs have not been incorporated into baseline surveys for RCTs aimed at improved accountability in the education sector. This is a pity, because specifically designed LTEs could help us address the main limitation of randomized evaluations, as highlighted by Bardhan (2005), i.e. their unsuitability for “deciphering the mechanisms through which certain outcomes are generated (the „why‟ and the „how‟) and the social dynamics that are involved” (p. 16). Moreover, they could provide early insights relating to important questions the full addressing of which would require costly and time-consuming RCTs. And, as such, they can lead to better RCT design and a corresponding increase in the information generated by time and money spent on RCTs.

111. In Albania, three LTEs were adapted and conducted with the school stakeholders who participated in the survey (see a fuller explanation in Appendix 3). In the LTE most directly related to parents‟ preferences for accountability, one player is invited to divide a sum of money between herself and another player, and a third player can pay to have money subtracted from the first player‟s allocation.<sup>38</sup> It is the third players‟ behavior that is of particular interest here as by choosing to “fine” they reveal both what they consider to be unfair behavior on the part of the first player, as well as their own willingness to pay to express this opinion. In other words, the third player is

> 36 See Barr et al. (2009) for an example of a cross-cultural experimental study based on the implementation of similar experimental designs to the ones we use here in 15 different small-scale societies.

> 37 Similarly, preferences measured in the context of a public goods game have been found to predict the likelihood of Brazilian fishermen to over-exploit their common fishing ground (see Fehr and Leibbrandt, 2008).

> 38 The LTE discussed is a version of Fehr and Fischbacher‟s (2004) Third Party Punishment Game (TPG).
Page 37 expressing their willingness to hold the first player to account for how the resources were divided (“fairly” or “unfairly”) between themselves and the passive second player.

112. The most important differences between the game as was conducted in Albania‟s primary schools, and how it has been used in other contexts are that: (a) the subjects are ordinary people, rather than university students in a lab; (b) the game roles were assigned to subjects with reference to and with the aim of matching the roles they (potentially) play in a real, ongoing accountability scenario. Specifically, teachers were invited to play in the role of “givers”, i.e. the individuals who have to decide how to divide an endowment between themselves and another subject. Parents were invited to play either in the roles of the “recipients” and “third party punishers”. The exact parameters and sequence of the games and important details about how they were conducted are explained in Appendix 3.

113. The objective when designing the game was to render observable the net effects of the parents‟ internal psychological costs and benefits associated with holding teachers to account. This is important as these intangible costs and benefits as correspond to parents‟ preferences for active participation in the teacher-parent accountability relationship. To this end, the game was designed to match the school context in terms of actual and possible relationships, while differing from it with respect to its simplicity and the availability of information to stakeholders. By matching the relational-space in the game to the actual or potential relational-space in the school context, the likelihood of parents‟ preferences that are salient when they are considering holding teachers to account also being salient in the game is increased.

114. There were, nevertheless, many important contextual differences between the relationships simulated in the game and the actual accountability relationship between parents and teachers outside of the game. Three of these are worthy of particular note. First, in the actual school context parents‟ decisions to hold school and teachers to account usually require that the parents expose themselves, whereas in the game parents were guaranteed at least some anonymity - teachers who were fined do not know which of the five parents fined them. Second, while in the school context the material costs associated with participation may differ across parents, in the game the material cost of holding teachers to account was fixed and known by all. Finally, in the actual school context, parents may be uninformed about the existing participatory institutions and the actual performance of teachers whereas in the game, parents were fully informed about how they can hold teachers to account and can condition their decisions on accurate measures of teacher performance. The only that was left ambiguous was how much teachers should allocate to parents, but prior studies employing this type of LTE on non-student subjects who are partially or fully integrated into monetized economies, provide a basis for expecting the 50:50 split to emerge as a reference point of “fairness” (the results presented below are consistent with this expectation). Therefore, assuming that the relational-space matching worked, the game can be viewed as a _counterfactual_ to the existing situation in Albania‟s primary schools, in which information is not constrained, parents wishing to participate in holding teachers to account are afforded some protection (anonymity), and the consequences of their participation are clearly defined.

115. Given this LTE design, it is only under very specific conditions that we expect parents‟ behavior in the game to correlate with their behavior and attributes in the actual school context and this being the case, if such correlations are observed, they can be taken as evidence that the specific conditions apply. A series of conjectures are formulated here to guide the empirical analysis and the interpretation of the findings from the game.

- If participation in fining in the game correlates with participation in the form of voting in the parent representative elections or being an elected representative, it indicates that the two types of participation are motivated by the same preferences and the LTE and school context
Page 38 are well matched. Conversely, lack of correlation may indicate either that the LTE lacks external validity, i.e., that the preferences that are salient in the school context are not invoked in the LTE or that, parent‟s participation in the school context is subject to external constraints that are not present in the LTE;

- If parents who participate in real life but in a compromised way -i.e. as parent representatives who have been appointed by teachers or school directors rather than being elected- are less willing to participate in fining in the LTE, it indicates precisely the sort of conflicts of interest that would arise in malfunctioning systems of accountability;

- If participation in the experiment correlates with being informed about the participatory bodies existing in the school, it indicates that being informed and participating in holding teachers to account in the LTE are motivated by the same preferences. Conversely, if they are not correlated, it indicates that being informed is not driven by a preference to participate and is, instead, owing to some sort of external constraint or system failure.

116. Moreover, it is reasonable to expect parents‟ individual characteristics relating to the expected costs and benefits associated with participation -such as parents‟ economic status, education and perceptions of malpractice at school- play a significant role in their decision to participate in the game and/or real life. In particular, parents‟ economic status could matter because, even if in the game setting the cost of fining is fixed, poorer parents are likely to have higher marginal utility for the cash made available in the game, and are therefore less likely to participate in fining. As for parents‟ education, it could affect the value parents‟ give to their children‟s education, and, consequently, the intrinsic benefit parents gain from holding teachers to account. Both greater economic status and education are also likely to diminish feelings of inferiority of parents toward teachers, reducing non-monetary costs associated with holding teachers to account. Finally, perceptions of malpractice in the appointment of teachers might encourage greater participation by reducing non-monetary costs and increasing non-monetary pleasures from fining teachers. However, while economic status, education and perceptions of corruption in the appointment of teachers are likely to increase participation in the game, they will affect participation in real life only if _de facto_ participation is possible in the school and parents are (or can take steps to become) informed about how to participate in accountability structures.

# **III.c. Results from Lab-type Experiments with Albania’s School Stakeholders**

117. Figure 1 summarizes the fining decisions made by the parents in the role of the third party observer with the power to punish teachers by fining if they consider the teachers‟ decisions selfish or unfair.<sup>39</sup> The data points marked by chevrons indicate the percentage of parents (vertical axis) who said they would fine a teacher who made a specified allocation (horizontal axis) to the passive parent. Only 16 percent of the parents decided never to fine teachers, regardless of their behavior in the game, even when teachers decided to keep the entire endowment (2000 Lek). As expected, the decision to fine depends negatively on the amount teachers allocate to (passive) parents.<sup>40</sup>

> 39 Figure 2 in appendix shows allocations of teachers to (passive) parents, i.e. the orange players, in the third-party punishment game. The mode at 50:50 is unusually strong, especially compared to results obtained from student samples. There are quite a lot of offers higher than 50%. This would be cause for concern in an experiment where those fulfilling the offering and receiving role were drawn from the same pool, but in this case they are not and know that they are not. If a teacher had a strong prefernece for fairness and considered himself materially better off than the average parent in the receiving role he would give more than 50%. Note that a teacher with no preference for fairness but who believes that the finers will fine unfair offers will rarely make a high offer – to do so he would have to have quite extreme beliefs about the likelihood of being fined.

> 40 The strategies of all parents are consistent, i.e. there are not irregular switches in the fining decisions (such as for instance, not fining a parent if he/she keeps all the endowment, but fining him/her for giving a positive amount). This might seem unusual to
Page 39

**Figure 3. Parents’ Fining Behavior**

<!-- Start of picture text --> 90<br>80<br>70<br>60<br>50<br>40<br>30<br>20<br>10<br>0<br>0 0.1 0.2 0.3 0.4 0.5 0.6 0.7<br>Amount the teachers gave to the (passive) parent<br>(% of endowment)<br>% third-party parents who fined<br><!-- End of picture text --> Source: Serra, Barr and Packard (2011) 118. Fining the teacher is costly to the third party parent, and the decision to fine depends on the intrinsic benefit from imposing the fine outweighing the fixed cost. Assuming that the intrinsic benefit to the parent of fining increases as the gap between what the parent believes the teacher ought to allocate and what they actually allocate widens (with the latter below the former), then only when the gap gets sufficiently wide will the fine be levied. For this reason, the minimum unfined offer chosen by each third party parent gives us a direct measure of that parents‟ intrinsic benefit from fining, i.e., their preference for participating in holding teachers to account.

119. Table 11 shows that for 77% of parents the minimum unfined offer is less than half of the endowment, 21% of the parents chose a minimum unfined offer of precisely one half of the endowment , and only 2% chose minimum unfined offers greater than one half. It is worth highlighting, particularly for readers not familiar with experimental methods and the results obtained from the same game in other countries (e.g., Barr et al. 2009), that parents in Albania are revealing a comparatively strong willingness to pay to fine teachers who share unfairly within the game.

researchers who are familiar with the implementation of strategy elicitation to non-students in lab-type experiments conducted in the field. However, the lack of inconsistencies can be explained by the fact that all parents were literate. The minimum years of schooling are 4 only for 1% of the sample. 44% of the parents completed 8 years of schooling and the remaining parents completed secondary school (27%) or even achieved an undergraduate or postgraduate degree (30%).
Page 40

**Table 11. Minimum Unfined Offer**

|**Minimum Un-fined Offer**|**Percent**|**Cum.**|
|---|---|---|
|0|16.44|16.44|
|0.1|14.22|30.67|
|0.2|10.67|41.33|
|0.3|13.00|54.33|
|0.4|22.33|76.67|
|0.5|21.44|98.11|
|0.6|0.78|98.89|
|0.7|0.78|99.67|
|0.8|0.33|100|

Source: Serra, Barr and Packard (2011) 120. To investigate the correlations between parents‟ behavior in game and their behavior and attributes in the school context two series of multivariate analyses are conducted. The first of these is a series of Probit analyses focusing on a dependent variable that equals one if a parent in the third-party role participates in fining and zero if the parent never fines (columns 1 to 3 in Table 12). The second is a series of linear regressions taking the minimum un-fined offer chosen by each parent in the third-party role as the dependent variable (columns 4 to 6 in Table 12). The sample upon which this analysis is based is restricted to the parents who fine at least once. Positive (negative) coefficients in the Probits identify variables that are associated with parents deciding to participate in (abstain from) fining. Positive (negative) coefficients in the linear regressions identify variables that are associated with a lower (higher) degree of tolerance of teachers‟ selfish behavior.

121. We start our empirical analysis by testing whether parent class representatives are more likely to participate in holding teachers to account in the game, while controlling for parents‟ individual characteristics, such as economic status, education, and perceptions of corruption in the appointment of teachers. We then look at the parent representatives who, contrary to the _de jure_ institutional provisions, have been appointed by teachers. We then expand the empirical specification to control for the decision to vote in the most recent elections of parent class representatives. Finally, we include a measure of parents‟ lack of information with respect to the existing participatory accountability structures.<sup>41</sup>

> 41 In order to reduce the bias generated by multicollinearity between the various proxies of parental awareness and involvement, we conducted factor analysis on the measures of parents‟ lack of information with respect to the existence of class parent representatives and School Board; we include the first factor in the empirical specifications appearing in columns 3 and 6 of Table 12.
Page 41

**Table 12. Parents’ Preferences for Accountability**

||**D**<br>**Dummy**<br> **part**<br>|**ep. Variable**<br>**equal to 1 if**<br>**icipates in fi**<br>|**:**<br>**parent**<br>**ning**<br>|**De**<br>**Minimu**<br>|**p. Variable:**<br>**m un-fined o**<br>|<br>**ffer**<br>|
|---|---|---|---|---|---|---|
||Probit|Probit|Probit|OLS|OLS|OLS|
||(1)|(2)|(3)|(4)|(5)|(6)|
|Parent representative|0.02<br>(0.025)|0.03<br>(0.025)|0.03<br>(0.024)|-0.02<br>(0.012)|-0.02<br>(0.013)|-0.02<br>(0.013)|
|Parent representative appointed by a teacher|0.05<br>(0.069)|0.05<br>(0.074)|0.06<br>(0.065)|**-0.09*****<br>**(0.035)**|**-0.09****<br>**(0.035)**|**-0.08****<br>**(0.035)**|
|Voted in the last parent rep elections||-0.03<br>(0.022)|-0.02<br>(0.023)||0.01<br>(0.011)|0.01<br>(0.012)|
|No information about participatory organs|||0.02<br>(0.017)|||0.01<br>(0.008)|
|Perceptions of corruption in the appointment of teachers|-0.01<br>(0.024)|-0.01<br>(0.025)|-0.02<br>(0.025)|**0.02***<br>**(0.012)**|**0.02***<br>**(0.012)**|**0.02***<br>**(0.012)**|
|Economic status|**0.03***<br>**(0.014)**|**0.03***<br>**(0.014)**|**0.03***<br>**(0.014)**|-0.00<br>(0.007)|-0.00<br>(0.007)|0.00<br>(0.007)|
|Years of schooling|**0.01***<br>**(0.003)**|**0.01***<br>**(0.003)**|**0.01****<br>**(0.003)**|0.00<br>(0.001)|0.00<br>(0.001)|0.00<br>(0.001)|
|Rural location|0.02|0.01|0.01|-0.01|-0.01|-0.02|
||(0.029)|(0.029)|(0.029)|(0.014)|(0.014)|(0.015)|
|Female|0.01|0.01|0.01|-0.01|-0.01|-0.01|
||(0.022)|(0.022)|(0.022)|(0.010)|(0.010)|(0.010)|
|% parents among friends and relatives in the session|0.04<br>(0.032)|0.05<br>(0.032)|0.05<br>(0.032)|-0.01<br>(0.011)|-0.01<br>(0.011)|-0.01<br>(0.011)|
|Constant||||0.36***<br>(0.034)|0.36***<br>(0.034)|0.35***<br>(0.035)|
|RED dummies|YES|YES|YES|YES|YES|YES|
|Observations|900|900|900|752|752|752|
|R-squared(orpseudo)|0.16|0.16|0.16|0.31|0.31|0.31|

Note: Robust standard errors, in parentheses, have been adjusted to account for clustering within sessions conducted in different schools. Columns 1 to 4 report marginal effects of continuous variables and the effect of a change from 0 to 1 for dichotomous variables. *** p<0.01, ** p<0.05, * p<0.1.

Source: Serra, Barr and Packard (2011) 122. The estimates presented in Table 12 suggests that the decision to actively participate in the accountability system and -conditional on participating- the extent to which parents hold teachers accountable, are motivated by different sets of factors. Parents‟ decision to participate in fining in the game appears to depend primarily, and positively, on parents‟ economic status and education. Parent representatives, no matter whether they were elected or appointed, and parents who participated in voting are no more willing to participate in holding teachers accountable in the game; conversely uninformed parents are no less willing to participate in the game.

123. The results obtained from the linear regressions taking the minimum un-fined offer as the dependent variable highlight how effective participatory accountability also depends on the extent to which the existing institutions are _de facto_ participatory. Indeed, the estimates in Columns 4 to 6 indicate that parent representatives that have been appointed to their position are relatively less demanding of teachers (i.e. their minimum un-fined offer is significantly lower). Although it is unclear if appointed parent representatives have become less demanding as a result of being appointed by teachers/school directors, or have been appointed because they are less demanding, the negative coefficient on the “Appointed” variable is evidence of a conflict of interest in parent representatives who are put in place in accordance with _de facto_ institutions that do not match the _de jure_ governmental provisions. This interpretation is supported by another important result: parents who believe that teachers are always or often appointed to positions owing to personal connections rather than their qualifications, are more demanding of teachers (i.e. their minimum unfined offer is significantly higher).

124. To summarize the results from the LTE, parents‟ willingness to take an active role in participatory accountability, as measured by the game, is positively associated with parents‟ economic status and education. Parents who are uninformed about the participatory accountability
Page 42 institutions available at the school level, and parents who are not currently involved with the school, are no less willing to forgo personal resources to hold teachers to account in the game. Once parents decide to hold teachers to account, the rigor with which they do so is lower if they have acquired a role in the _de facto_ accountability system in a compromised way, i.e. by being appointed parent class representatives by teachers or the school director, and higher if they believe that teachers have been appointed to their position in a compromised way.