Evaluation of the support to schools project in Czech Republic.
Date: 2021-10-20
Institutions: Joint Research Centre
Subjects: Czechia, European Regional Development Fund, European Social Fund, data collection, education statistics, educational system, nursery school, primary education, research report, school life, school results, social integration, socially disadvantaged class, teaching quality

Evaluation of the Support to Schools 
Project in Czech Republic

Francesca Agosti
Jan Dusatko
Elena Claudia Meroni
Jakub Vrobel 

2021  

EUR 30860 EN 

This publication is a Technical report by the Joint Research Centre (JRC), the European Commission's science and knowledge service. It 
aims to provide evidence-based scientific support to the European policymaking process. The scientific output expressed does not imply a 
policy position of the European Commission. Neither the European Commission nor any person acting on behalf of the Commission is 
responsible for the use that might be made of this publication. For information on the methodology and quality underlying the data used 
in this publication for which the source is neither Eurostat nor other Commission services, users should contact the referenced source. The 
designations employed and the presentation of material on the maps do not imply the expression of any opinion whatsoever on the part 
of the European Union concerning the legal status of any country, territory, city or area or of its authorities, or concerning the delimitation 
of its frontiers or boundaries.

Contact Information
Name: Competence Centre on Microeconomic Evaluation
Address: Joint Research Centre, Via Enrico Fermi 2749, 21027 Ispra (VA), Italy
Email: elena.meroni@ec.europa.eu
Tel.: +39 0332 785608

EU Science Hub
https://ec.europa.eu/jrc

JRC125324

EUR 30860 EN

PDF

ISBN 978-92-76-42311-9

ISSN 1831-9424

doi:10.2760/293062

Luxembourg: Publications Office of the European Union, 2021

© European Union, 2021

The reuse policy of the European Commission is implemented by the Commission Decision 2011/833/EU of 12 December 2011 on the 
reuse of Commission documents (OJ L 330, 14.12.2011, p. 39). Except otherwise noted, the reuse of this document is authorised under 
the Creative Commons Attribution 4.0 International (CC BY 4.0) licence (https://creativecommons.org/licenses/by/4.0/). This means that 
reuse is allowed provided appropriate credit is given and any changes are indicated. For any use or reproduction of photos or other 
material that is not owned by the EU, permission must be sought directly from the copyright holders.

All content © European Union, 2021

How to cite this report: Agosti, F., Dusatko, J., Meroni, E.C. and Vrobel, J., Evaluation of the Support to Schools Project in Czech Republic, EUR 
30860 EN, Publications Office of the European Union, Luxembourg, 2021, ISBN 978-92-76-42311-9, doi:10.2760/293062, JRC125324.

Contents

Foreword....................................................................................................................................................................................................................... 1

Acknowledgements................................................................................................................................................................................................. 2

Abstract ......................................................................................................................................................................................................................... 3

Executive summary ................................................................................................................................................................................................ 4

1 Introduction......................................................................................................................................................................................................... 6

2 Context and details of the intervention............................................................................................................................................... 6

2.1 The Czech educational system ....................................................................................................................................................... 6

2.2 The evaluated intervention ............................................................................................................................................................... 7

3 Literature review.............................................................................................................................................................................................. 8

4 Method................................................................................................................................................................................................................... 9

5 Data......................................................................................................................................................................................................................... 10

5.1 Description of the sample ................................................................................................................................................................. 10

5.2 Description of variables used as controls ................................................................................................................................ 11

5.3 Description of the outcomes considered................................................................................................................................... 13

5.3.1

``School climate outcomes'' .................................................................................................................................................... 13

5.3.2

``Cermat exams''............................................................................................................................................................................ 13

6 Binary treatment.............................................................................................................................................................................................. 16

6.1 Definition of the treatment variables ......................................................................................................................................... 16

6.2 Results .......................................................................................................................................................................................................... 18

6.3 Sample of schools starting in academic year 2016-2017............................................................................................. 23

7 Continuous treatment ................................................................................................................................................................................... 26

7.1 Definition of the treatment .............................................................................................................................................................. 26

7.2 Results .......................................................................................................................................................................................................... 28

8 Conclusions.......................................................................................................................................................................................................... 29

References ................................................................................................................................................................................................................... 30

Annexes ......................................................................................................................................................................................................................... 32

1 School need data ............................................................................................................................................................................................. 32

2 Details on the construction of the school outcome...................................................................................................................... 37

3 Generalized propensity score matching.............................................................................................................................................. 45

B Additional tables .............................................................................................................................................................................................. 46

i

Foreword

The purpose of this report is to evaluate an intervention (cid:16)Support to schools in form of simpli(cid:28)ed reporting
projects - Template for nursery schools and primary school(cid:17), by the European Social Fund (cid:28)nanced in the Czech
Republic, during the Programming period 2014-2020. This is a joint work of the JRC-CRIE (Center for Research
on Impact Evaluation) and one of the Czech managing authorities, Ministry of Education, Youth and Sports. The
motivation behind this collaboration is an Administrative Agreement, signed between JRC Unit I.1 and DG EMPL
unit G.5, whose aim is to provide scienti(cid:28)c expertise and methodological support to the Managing Authorities
and the European Commission for the evaluation of the impact of interventions funded by the European Social
Fund. This is part of the Quality Assurance Support activity. The activity started in Janaury 2020 and was
concluded in September 2021.

1

Acknowledgements

The authors acknowledge the help of Gianluca Mazzarella on an earlier version of the analysis.

Authors
Francesca Agosti, Jan Dusatko, Elena Claudia Meroni, and Jakub Vrobel.(Authors listed in alphabetical order)

Elena Claudia Meroni is part of the European Commission, JRC. When doing this work, Francesca Agosti was a
trainee at the European Commission, JRC
Jan Dusatko and Jakub Vrobel are part of the of the Czech ESF managing authority, Ministry of Education, Youth
and Sports of Czech Republic.

2

Abstract

The purpose of this report is to evaluate an intervention (cid:16)Support to schools in forms of simplifed reporting
projects - Template for Nursery schools and Primary schools") (cid:28)nanced by the European Social Fund in Czech
Republic, during the Programming period 2014-2020. This is a join work of the JRC-CRIE (Center for research
on Impact Evaluation) and one of the Czech managing authorities, the Ministry of Education, Youth and Sports.
In order to understand the short run effects of the call, the outcomes of participating schools are compared to
the outcomes of non-participating schools, using propensity score matching as a way to solve the selection bias
arising from the fact that schools self-select themselves, as participation to the call is voluntary. Analysis are
based on a combination of administrative data coming from schools' registries, and survey data collected on a
relevant sample of schools. The main outcomes of interested are performances in mathematics and language
test scores and some indicators capturing the level of school's climate (measuring features like inclusiveness,
discrimination, etc...). Preliminary results do not show any consistent pattern of effects on test scores. Some
mixed evidence is found for the school climate measures, where only some dimensions seem to be affected by
the program, but not in a clear and consistent pattern. Explanation for these results could be that very short time
passed between the intervention and the date when the outcomes were measured, which can be problematic if
one expects academic achievement and school climate to slowly change in response to such interventions. Also,
the academic achievement of students or school climate were not the direct targets of the activities of this set
of actions.

3

Executive summary

Policy context

One of the main objectives of the Ministry of Education, Youth and Sports of Czech Republic is to shift the
country towards an economy based on an educated, motivated and creative workforce, and on the production
of high-quality research results. To achieve this objective, the Operational Programme Research, Development
and Education (OP RDE) was established during the programming period 2014-2020.
It is co-(cid:28)nanced by the
European Social Fund and European Regional Development Fund, and managed by the Ministry of Education.

Out of the main actions (cid:28)nanced by this programme, the (cid:16)Support to schools in the form of simpli(cid:28)ed reporting
projects - Templates for nursery schools and primary schools(cid:17) is the focus of this report, which aims to bring
preliminary evidence on the effectiveness of this intervention, using Counterfactual Impact Evaluation Methods,
which allow to determine causal effects of the intervention on the outcomes of interest. This intervention
is a call for projects intended for schools, launched in 2016. The number of participating schools exceeded
5,100 and the original budget allocated was 4,5 billion CZK (between 160 and 180 million euros depending
on the exchange rate throughout the period). The objective of the call was to improve the inclusiveness at
school level ((cid:16)Promote social integration of children and pupils including the integration of Roma children into
education(cid:17)) and to enhance pupil's key competencies ((cid:16)Improving the quality of education and achievement
of students in key competencies(cid:17)). The supported activities to reach these goals, were: professional support
(mentoring and supervision) and systematic training to teachers in key areas (mathematics and reading literacy,
foreign language, mentoring and inclusive education);
increasing the number of auxiliary school personnel;
extracurricular activities for pupils; and promoting cooperation between parents and the school.

Scientific approach

In order to understand the short-term causal effects of the call, we would like to compare the outcome of a
school who participates to the intervention, to the outcome of the same school had it not participated to the
if a school participate to the intervention, we only
intervention. Clearly the latter outcome is not observable:
observe its outcome conditional on the fact of having participated. One possible way out of this problem is
to use the outcome of the schools who did not participate to the intervention as the counterfactual. However,
since the schools selected for the intervention may be different from the schools not selected, we cannot simply
retrieve the impact of the intervention by comparing the outcomes of the two groups, because the results would
suffer from a bias related to the mechanism of selection of participating schools (selection bias). We rely on
propensity score matching as a way to solve the selection bias arising from the fact that schools self-select
themselves, as participation to the call is voluntary. Analyses are performed at the school level and are based on
a combination of administrative data coming from schools' registries, and survey data collected on a relevant
sample of schools. The main outcomes considered are average schools' performances in mathematics and
language test scores and several indicators capturing the level of the school's climate (measuring features like
inclusiveness, discrimination, etc.).

Main conclusion

Preliminary results do not show any consistent pattern of effects on test scores. Some mixed evidence is found
for the school climate measures, where only some dimensions seem to be affected by the programme, but
not in a clear and consistent pattern. An explanation for these results could be that very little time passed
between the intervention and the date when the outcomes were measured, which can be problematic if one
expects academic achievements and school climate to slowly change in response to such interventions. Also,
the academic achievements of students or school climate were not the direct targets of the activities of this set
of actions. These were chosen rather because the theory of change of the intervention suggested there might
be some effect expected, and for data availability.

Looking ahead

Further research is needed in the coming years to understand if this set of actions can have an impact on
changing the school climate and possibly improving students' achievement. This is even more challenging
given the current COVID-19 pandemic, which has caused schools to be closed for a long time during 2020 and
2021. Despite these limitations, it is good to underline the enormous effort put in place by the Czech managing
authority to put together administrative data about the universe of schools in the country, and to carry out
surveys aiming at measuring a dimension (school climate) which is almost impossible to measure with available
registry data. A number of suggestions can be proposed in view of the next programming period:

4

1. Collecting baseline data on relevant outcomes before the start of the intervention on the treated and

control schools can help better measure the impact of the intervention.

2. Finding a measurable outcome that is directly related to the expected results of the intervention can help

identify relevant effects.

3. Having better control of the design of the intervention, and of the selection of the units to treat, can help

in (cid:28)nding the most adequate evaluation method.

5

1 Introduction

The purpose of this report is to evaluate an intervention (cid:28)nanced by the European Social Fund in Czech Republic,
during the Programming period 2014-2020: (cid:16)Support to schools in the form of simpli(cid:28)ed reporting projects -
Templates for nursery schools and primary school(cid:17). The evaluated intervention is a call for projects intended for
schools and issued by the Operational Programme Research, Development and Education (hereinafter referred
to as the OP RDE). The OP RDE is managed by the Ministry of Education, Youth, and Sports of Czech Republic
and the aim of the programme is to contribute to Czech Republic's structural shift towards an economy based
on an educated, motivated and creative workforce and on the production of high-quality research results and
their use to increase competitiveness.

This call was launched in the academic year 2016-17 and participation for the schools was voluntary.
Some schools joined in 2016, while others joined in the following academic year and the intervention generally
lasted 24 months. The main objectives of the intervention were to improve the inclusiveness at school level
((cid:16)Promote social integration of children and pupils including the integration of Roma children into education(cid:17))
and to enhance pupil's key competencies ((cid:16)Improving the quality of education and achievement of students in
key competencies(cid:17)), however these objectives are dif(cid:28)cult to measure given the considerable time that shall
pass before one could expect changes in these dimensions.

Nevertheless, since the intervention could also equally well improve short term measurable outcomes, like
the school climate and children test scores, this study examines short effects of the policy that could have
immediate impact on these two sets of outcomes: academic achievements in mathematics and Czech language
at the end of primary schools, and some indicators of school climate (measuring features like inclusiveness,
discrimination, etc.). Both set of outcomes are measured at the end of the academic year 2018-19 (June 2019).
All outcomes are measured at the school level, as no more granular information is available. The (cid:28)rst results
point to inconclusive evidence about the effectiveness of this intervention, which could be due to the very short
time elapsed between the project and the measured outcomes.
It is necessary to stress that the intervention
wasn't planned to affect these sets of outcomes directly and the outcomes were chosen as the theory of change
of the intervention suggested there might be some expected effects, and due to data availability.

The report is structured as follow: Section 2 provides information about the context and the details of the
intervention, Section 4 describes the methodology and Section 5 the data used for this evaluation. Results and
Conclusions are reported in Sections 6, 7 and 8.

2 Context and details of the intervention

2.1 The Czech educational system

This evaluation report deals with an intervention focused on the Czech primary schools. Arguably the biggest
intervention of this kind in recent years. For this reason, it might be bene(cid:28)cial to describe the education system
the intervention took place in and to clarify the purpose of the intervention. Czech primary education system took
its shape in 1774, when compulsory school attendance was instituted, and it has retained many key attributes
ever since. Nowadays, the education system consists of preschools, primary schools, secondary schools and
tertiary (post-secondary) institutions providing higher education. The system is predominantly (cid:28)nanced by tax
money. And although the number of private educational institutions is growing, a great majority of pupils
are educated in state-funded schools where no tuition is required. For preschool (ISCED 0), there are two
main institutions a child may attend. For the youngest children there are childcare establishments where they
are taken care of in small groups. Besides taking care of the children and socializing them, childcare may
have an educational function as well. For older children there are kindergartens. Age groups may overlap,
since the childcare establishments accept children between 1 and 6 years of age (usually younger ones) while
kindergartens accept children between two and six years of age (usually older ones). It is mandatory for children
to spend at least one year in a kindergarten before they join primary schools. Kindergartens are obliged to follow
the national educational curriculum and they are places of early education, socialization and preparation for
school. Primary schools are mandatory as well and they usually accommodate pupils between 6 and 15 of age.
They consist of two levels: grades 1 - 5 (ISCED 1) and grades 6 - 9 (ISCED 2). The higher level thus corresponds
to lower secondary education in some countries. The transfer between these levels is automatic unless the pupils
intend to change primary school for a grammar school that combines primary and secondary education (4+4 for
pupils joining after 5th grade or 2+4 for pupils joining after 7th grade). Since most kindergartens and primary
schools are established by municipalities, and since Czech Republic has a high density of small municipalities,
there is a high number of kindergartens (over 5000) and primary schools (over 4000) for a country of ten
millions. There is no mandatory, standardized examination at the end of primary schools. However, when the
pupils intend to go to a secondary school (and most of them do), they need to pass a standardized entrance
exam in Czech language and mathematics. The achievements in these exams, aggregate at the school level,
provided kindly by the Centre for the Evaluation of Educational Achievement (CERMAT), are one of the outcomes

6

considered in this evaluation. After primary school, a pupil may choose from a wide range of high schools
(ISCED 3): vocational schools, grammar schools, lycea, specialized high schools, conservatories and so on. Their
duration may vary between 2 and 4 years (most common) and they may be concluded by a vocational or a
standardized state exam. The state exam is a condition for application to an institution of a tertiary education.
These may be vocational colleges (ISCED 6) or universities and colleges with their bachelor (ISCED 6), master
(ISCED 7) and doctoral programmes (ISCED 8).

2.2 The evaluated intervention

The intervention chosen for the evaluation is a part of a complex system of interventions taking place in
preschools, primary schools and secondary schools. The interventions were (and are) part of the effort to make
the Czech education a more inclusive place. A place, where each pupil is educated amidst their peers and where
each pupil's competencies are developed according to their needs. These interventions were prepared as a part
of the support of an important legislation that came into force in 2016 and that tackles the issue of inclusive
education. The aim of the intervention was, in broad terms, to support the inclusive conditions on school level
and to enhance pupil's key competencies. The three speci(cid:28)c objectives, as de(cid:28)ned in the operational programme,
were:

1. Improving the quality of pre-school education, including facilitating the transition of children to primary

school.

2. Improving the quality of education and achievement of students in key competencies.

3. Social integration of children and pupils including the integration of Roma children into education.

The evaluated call (cid:16)Support to schools in form of simpli(cid:28)ed reporting projects - Templates for nursery schools
and primary schools(cid:17) was chosen because of its size, importance for the Czech education system, and relative
homogeneity of the supported activities allowing comparability to a certain degree. The size is signi(cid:28)cant both
in allocation and number of participating schools. The (cid:28)nal budget was 3.035 billion CZK, approximately 124
million euro according to the average exchange rate in 2016. The number of the participating nursery schools
(ISCED 0) and primary (ISCED 1 and 2) schools exceeded 5100. That means almost 64% of nursery schools
and some 73% of primary schools took part in the intervention. There were subsequent calls similar to this one
and targeting the same schools. These will be a subject of a further evaluation. This evaluation focuses only on
primary schools, since it was the (cid:28)rst call and since no relevant outcome is available for nursery schools yet. The
complicated name of the call refers to the simpli(cid:28)ed method of reporting (templates or units) that signi(cid:28)cantly
decreases the administrative burden for the bene(cid:28)ciaries and increases the comparability of the projects. While
other calls specify the goals and supported activities rather broadly, leaving the bene(cid:28)ciaries much freedom in
how to choose, shape, and implement the activities, calls based on (cid:16)template(cid:17) specify in detail what activity is
supported, what form it should take, what should be its goal and how it's proven it took place as it should. The
supported activities then can be divided into coherent units. The supported activities were:

1. Auxiliary school personnel (support in the form of additional specialised educational staff like school
assistants, special educators, school psychologists, social educators, or nannies in case of nursery schools)

2. Personal and social development of teachers (mostly courses of various length on mathematical liter-
acy/numeracy, reading literacy, foreign languages, inclusive education, mentoring and personal develop-
ment)

3. Extracurricular activities (e.g. reading clubs, board game clubs, tutoring)

4. Cooperation with parents of children and pupils (involving the parents in the education of their children,

mostly via thematic meetings)

An additional activity was foreseen for nursery schools: (cid:16)Facilitating the transition of children from nursery
schools to primary schools (prevention of speech disorders and communication problems of children, cooperation
with parents, nursery schools, and primary schools)(cid:17).

Examples of how the template system works are the following: if a school wanted to try additional support
personnel in the form of a school psychologist, the unit was de(cid:28)ned as 0.5 FTE a month. Therefore, if they
wanted to keep a half-time psychologist for twelve months, they chose 12 units.
If they wanted a full-time
In the case of education or training for the school staff, the
psychologist for 12 months they chose 24 units.
unit was de(cid:28)ned by the topic and number of hours, e.g. 8 hours on a course on the improvement of inclusive
education. Extracurricular activities were usually de(cid:28)ned by the number of participants, number of meetings,
and number of months. For example, the reading club unit consisted of 16 club meetings of 90-minute duration
for at least six pupils in one semester. Each unit thus had de(cid:28)ned content, outputs, costs, and was easy to apply

7

for and to report. This simplicity based on experience from the previous programming period translated into
huge popularity among schools and resulted in three consecutive calls for projects of similar scope and size.
However, this evaluation is interested only in the (cid:28)rst call that took place between September 2016 and August
2019. The objective of the call was to provide professional support (mentoring, supervision) of teachers and
systemic training of teachers in the key areas (mathematical and reading literacy, foreign languages, inclusive
education). The interventions are supposed to focus on personal and social development of teachers and the
other educators and their competencies for developing key competencies of children, and provide special staff
support to nursery schools and primary schools.

The effects of these interventions are supposed to be indirect in most cases. The additional professional staff
is supposed to promote a suitable school climate and to create a better environment for learning for all pupils.
The courses for teachers are supposed to help them improve their own teaching by using innovative methods
and by creating an inclusive environment for all pupils. Although some templates were focused on speci(cid:28)c areas
like enhancing numeracy or literacy, these shouldn't be perceived as merely connected to better teaching of the
Czech language and mathematics. They were supposed to be as horizontal as inclusive principles, and teachers
took part in them regardless of the subject they taught. The extracurricular activities are supposed to help both
with the development of key competencies and with inclusion of pupils endangered with school failure. The
intervention's effect should be thus observable on the changing school climate, signs indicating inclusiveness
of the school and the pupil's competences. Considering the age of the treated pupils, the intervention had no
ambition to have an effect on their standing on the labour market.

The duration of the absolute majority of the projects was, according to the condition of the call, 24 months.
Most of the projects also respected the cycle of the school year and started either in September 2016 or
September 2017. Although a signi(cid:28)cant number started also at the beginning of the spring semester 2017, in
January or February. The selection process was one round evaluation of the applications and as a matter of
rule, if the applicants matched the formal criteria (size of the project was within the limits given by the size of
the school, eligibility of the applicant), they were awarded the project. The participation was voluntary but the
choice of the (cid:16)template(cid:17) was not.

In 2015, nearly all primary schools went through a thorough needs assessment survey. One of the results of
this survey was the identi(cid:28)cation of the weaknesses and the strengths of the schools. Schools, as applicants, were
encouraged to choose at least those templates that could improve their situation in the identi(cid:28)ed weaknesses.
Otherwise, the schools could choose the templates as they saw (cid:28)t. Therefore, the participation wasn't random
and the treated schools were more likely to be supported on areas of weakness compared to the rest of the
schools.

3 Literature review

The intervention consists of four different actions. The (cid:28)rst action concerns the provision of auxiliary staff within
the schools. More speci(cid:28)cally, this action aims at increasing the support to schools in the form of specialized
educational staff (like school assistants, special educators, school psychologists, social educators, or nannies
in case of nursery schools). Using auxiliary school personnel in schools has some advantages (Bowman and
Klopf (1966)): 1) more attention to students; 2) improved teaching conditions, allowing teachers to dedicate
more time to professional duties; 3) easing the shortage of professionals. Few studies in the literature try to
assess the role of auxiliary school personnel in students' and teachers' achievements. See Navarro (2015) for
a review on the topic, where he concludes that (cid:16)the work of these staff can reduce pressure on the teacher in
relation to classroom management by helping the teacher cope with student misbehaviour, thus creating a more
productive classroom atmosphere. Furthermore, learning support staff can pair with teachers during class in a
complementary way, potentially providing the most effective teaching in every different context.(cid:17)

The second action aims at favoring the personal and social development of teachers. It consists of courses
of various length, on mathematical literacy/numeracy, reading literacy, foreign languages, inclusive education,
mentoring and personal development. These courses allow the teachers not only to improve their knowledge in
different subjects, but also to develop new skills more related to the personal and social training. For example,
they could learn how to create a better teaching environment, how to develop student-teacher relations and
how to inspire their students in a very delicate age for their development. The literature has tried to evaluate
the effects of teachers' trainings on students' outcomes, in particular students' academic achievement. See
for example Basma and Savage (2018) and Didion et al. (2020) for a meta-analysis on the effect of teacher
professional development on students' reading achievement, Blank and de Las Alas (2009) for effect on
achievement in mathematics and science, and Yoon et al. (2007) for another collection of examples on the
effects on other achievements domain (mathematics, sciences and art). All studies point to a general positive
impacts of teachers' development on students' outcomes. Few others scholars studied the effect of teachers'
professional development on other outcomes, such as peer relationships (Yee Mikami et al., 2011), creativity
(Hosseini and Watt, 2010) on student behavioral engagement in their classrooms (Gregory et al., 2014), all

8

suggesting positive impact on the studied outcomes. Finally, other studies have tried to identify what features
make teachers' development effective (Van Veen et al., 2012).

The third action, concerns the offer of extracurricular activities to students (e.g. reading clubs, board game
clubs, tutoring, sport, music or personal development programs). There are a whole range of bene(cid:28)ts that come
from involvement in these activities, such as: 1) they can help children to develop skills outside the classrooms,
for example team-building or time management; 2) they can open a child's mind to new interests and hobbies
and (cid:28)nding an area in which children excel can boost self-con(cid:28)dence; 3) they provide social opportunities
and a way to have a productive break from study. Extracurricular activities, indeed, give the opportunity to
interact with other children and build friendship.
In the recent literature we (cid:28)nd studies related to effect of
extracurricular activities on students' achievement and personal development. Susan A. Dumais (2006), for
example, (cid:28)nds that the number of activities in which students participate during kindergarten and (cid:28)rst grade
affects their gains in reading achievement test scores between (cid:28)rst and third grade, but does not affect gains
in math achievements. Moreover,
it seems that less-privileged children bene(cid:28)t more from participation in
extracurricular activities than do more-privileged children. The relationship between extracurricular activities
and academic achievements is also studied by Meadows (2019). Her results suggest a statistically signi(cid:28)cant
positive correlation between hours spent participating in extracurricular activities and cumulative GPA. Another
outcome correlated to extracurricular activities is personal development. Findings show that students who
participate in extracurricular activities reduce behavior problems and have a more positive attitude towards
school. Moreover, they are less likely to drop out and more likely to have more social connections (Massoni,
2011).1

The last action is related to the role of families, in particular parents, in a child's education. This treatment,
cooperation with parents, aims to involve the parents in the education of their children, mostly via thematic
meetings. Developing skills to manage children's behavior and gaining con(cid:28)dence and empathy are just two
bene(cid:28)ts coming from parental engagement. Working together, teachers and parents can create an educational
program that meets unique needs and foster a caring and sensitive school climate that respects and responds
to students' differences as well as their similarities. The literature on this theme is mostly qualitative and
nonempirical. Fan and Chen (2001), conducted a meta-analysis to synthesize the quantitative literature about
the relationship between parental involvement and students' academic achievement. Findings reveal that there
is a moderate but meaningful positive relationship between parental involvement and academic achievement.
This relationship is stronger when the outcome is a global academic indicator (e.g. GPA) than by a subject-
speci(cid:28)c indicator (e.g. math grade). Also social development seems to be affected by parents involvement; more
speci(cid:28)cally, it emerges that children with highly involved parents had enhanced social functioning and fewer
behavior problems (El Nokali et al., 2010).

4 Method

Our aim is to assess the effectiveness of the various activities. To do so we would like to compare the outcome
of a school who receives the treatment, to the outcome of the same school had it not received the treatment.
Clearly the latter outcome is not observable: if a school is treated we only observe its outcome conditional on
the fact of having received the treatment. This is de(cid:28)ned as the (cid:16)fundamental problem of causal inference(cid:17) in
the economic literature: one cannot observe the status of a treated unit in the scenario where it did not receive
the treatment (the counterfactual).

One possible way out of this problem is to use the outcome of the schools who did not receive any treatment
as the counterfactual. However, since the schools selected for the treatment may be different from the schools
not selected, we cannot simply retrieve the impact of the intervention by comparing the outcomes of the two
groups, because the results would suffer from a bias related to the mechanism of selection for the treatment
(selection bias).

This issue is further exacerbated by the fact that very little is known about the selection process of eligible
schools. If good schools are selected, the intervention effects will be over-estimated. Conversely, intervention
effects may be underestimated if the programme is targeting the (cid:16)worse(cid:17) schools. As this process cannot be
observed by the evaluator, the resulting estimates of the impact will suffer from some estimation bias. Taking
into account some features of the selection process and controlling for some observable differences between
the two groups can help mitigate this bias. Among the several econometric techniques that can help us in
accomplishing the goal of reducing selection bias and perform a valid evaluation, the (cid:28)nal choice of the most
suitable counterfactual method is strongly related to institutional background and to data availability.
In this
particular setting, the only method which we could apply is matching. The idea behind matching is to (cid:28)nd for

1If we consider some activities in detail, like playing an instrument or sport activities, there are a lot of studies, we report some of them
here: bene(cid:28)cial effects of play music are found in Hille and Schupp (2015); Eccles et al. (2003); mixed results about the effects of sport
(positive or no effect) are found in Lechner (2009); Pfeifer and Corneliÿen (2010); Rees and Sabia (2010); Cuffe et al. (2017); Felfe et al.
(2016); Ransom and Ransom (2018).

9

each treated school, a control school (or a set of control schools) which is as similar as possible to the treated
one in terms of the observable characteristics. This would guarantee that the comparison of the outcomes in
the treated and control school will not be affected by the bias mentioned above.

The matching approach is based on the assumption that the selection into treatment is solely based
on observable characteristics and that all variables that simultaneously in(cid:29)uence treatment assignment and
potential outcomes are observed. This is referred to in the literature as Conditional Independence Assumption
(CIA) or Unconfoundedness assumption and implies that, given a set of observable covariates which are not
affected by the treatment, potential outcomes are independent of treatment assignment. The validity of this
assumption depends on the amount of variables which can be observed in the data, i.e. on the richness of the
data used in the matching procedures. However, it must be said, that controlling for differences in observable
characteristics does nothing to alleviate the bias due to unobservable variables, such as principal's or students'
motivation. Nevertheless, it is worth mentioning that judicious use of observable characteristics can go some
way towards minimising the bias associated with unobservables.

Hence, using the matching approach, we aim at maximising the balance of school characteristics in the
two groups, so as to select as controls those schools who are very similar to the treated. schools. This allows
us to tackle the possible pre-existing differences between treated and controls, and to address the potential
estimation bias resulting from the selection process.

That matching works well when the pool of non-treated units, from which to select potential controls is very
large. In the current setting, we face an additional limitation: the sample of non-treated schools is very small
compared to the sample of treated schools. This may result in not being able to (cid:28)nd a proper control unit for all
the treated schools.

Our analysis relies on one main matching technique, namely, Propensity Score Matching (PSM). The PSM
method (Rosenbaum and Rubin, 1983) is based on the estimation of the so-called balancing score. This is
estimated as a function of the relevant observed covariates, such that the conditional distribution of covariates
given the balancing score is independent of assignment into treatment. The use of a balancing score allows to
solve the (cid:16)curse of dimensionality" arising from the need to condition on a high dimensional vector of relevant
covariates. The Propensity Score is estimated as the probability of participating in a programme given observed
In addition to the CIA assumption, the PSM is also based on the Common Support
schools' characteristics.
assumption. This requires the overlap between the estimated probability of participating for treated and control
units.
It ensures that persons with the same values of covariates have a positive probability of being both
participants and non-participants.

5 Data

The Czech managing authorities provided the JRC-CRIE with four main anonymised sets of data, which are all
aggregated at the school level:

1. A survey administered to teachers and principals in primary schools at the end of academic year 2018-19,
(June 2019) asking questions about the school climate, and about perception of disadvantaged students.
This is used to build a (cid:28)rst set of outcomes, which we will call along the report (cid:16)school climate outcomes(cid:17).

2. Administrative data on all schools in Czech Republic containing information about schools (for both treated
and control), and details of the treatment, for the period 2016- 2019.This dataset is used to build some
of the variables used as a predictor of the probability of being a treated school, in the propensity score
estimation (data collected in 2016) .The data set also contained the results (aggregated at the school level)
of the standardised high school entry exams by the Centre for the Evaluation of Educational Achievement
(CERMAT). The wave taken in June 2019, was used as a second set of outcomes, which we will call (cid:16)Cermat
exams(cid:17).

3. Data from a survey on schools' needs, administered in 2015. This information is used to build the

remaining variables used as a predictor of the probability of being a treated school.

4. Data on proportion of teachers and educational staff actually treated in the sample of treated schools.

5.1 Description of the sample

The original sample of primary schools is 4,201. Once this sample is merged with the (cid:28)le with information
on school needs a total of 140 are lost (130 treated and 10 controls). Of the 4,061 schools retained, 3,058
received some form of treatment over the years 2016-2018, and 1,003 were never treated. 1,676 started the
intervention in 2016, 1,376 in 2017 and only 2 in 2018. However, some of the variables used in the matching,
are not available for all the schools.
If we drop those schools with missing information, we end up with 979
control schools and 3,049 treated schools.

10

The survey administered in 2019 was on a voluntary basis, and therefore it doesn't cover the whole sample
of schools.
Information is available only for 1,587 schools, of which 1,340 treated schools and 247 control
schools. Therefore the effect of the intervention on the (cid:16)school climate outcomes(cid:17) will be estimated only on
this limited sample of schools. Similarly, results of the cermat exams are not available for all schools, but only
for a sample of schools, in particular for roughly 2,000 treated schools and 400 control schools (number varies
slightly according to type of exam - mathematics or language), and the effect of the intervention on the (cid:16)cermat
exams(cid:17) will be estimated only using this sample of schools. Finally, also the data set containing information
about the share of participating teachers is not available for 5 schools.

To understand how the two samples of schools used to estimate the effect of the intervention differ from the
original full sample we run two probit regressions, estimating the probability of being in the two samples, using
the same variables that will be used as controls in the matching (see the following Section). As shown in Table
B1, there are some differences between the universe of the schools, and the two samples for which outcomes
are available: differences for the (cid:16)school climate outcomes(cid:17) sample are mostly in the distribution of the schools
in the various regions, and in a few other variables (school size, share of girls, and reading literacy need); while
differences for the (cid:16)cermat exams(cid:17) sample are present in most of the variables. This is not a problem per se,
but once we interpret the results we should keep in mind that the results funds may not be extended to the
original sample of all the schools.

5.2 Description of variables used as controls

Variables used as controls in the matching come from two main sources: the survey on school needs, done in
2015, and the administrative database collecting information on number of students, number of teachers, etc.,
referring to the beginning of the academic year 2016. As the intervention started in the academic year 2016, all
the variables used refer to the pre-intervention period. The survey on school needs contains several questions
about the areas in which the schools perceive they can improve, and about the planning for development. Replies
are provided by school principals. The dimensions we consider include: reading literacy, mathematics literacy,
entrepreneurship, ICT competence of teachers, support of social and civic competence, polytechnic education,
language literacy. For each dimension a number of questions were asked, and principals could reply about the
current state as for 2015 (1: Not at all or barely applied; 2: Developing area; 3: Implemented; 4: Ideal state)
and about the plan for the future (1: We don't plan to develop this area; 2: We plan to develop this area in
2016-2018; 3: We plan to develop this area in 2018-2020). We rely only on the replies about the current state,
and we build an indicator per dimension, which is the sum of all the replies given. The details of each dimensions
and questions are reported in Annex 1.

In Table 1 we report the average values of each variable in the groups of control and treated schools, and
their difference (with indication of whether the difference is signi(cid:28)cant). The sample used in the table comprises
all schools, independently of whether or not information on the outcome is available.

From the information coming from the school need survey, we see that treated schools are quite similar to
the controls in all dimensions, with the exception of ICT competence of teachers and of language literacy. The
same tables, estimated only in the samples where (cid:16)school climate outcomes(cid:17) and (cid:16)cermat exams(cid:17) are available
in the Annex (Table B2 and B3). In addition, we also control for region (cid:28)xed effect and for the population of the
municipality where the school is located.

11

Table 1: Descriptive statistics in treated and control schools

Share of pupils with adjusted educational plan - 2016

Control schools
0.036

Treated schools
0.014

Share of pupils with special educational needs - 2016

0.217

Share of gifted pupils - 2016

Total number of pupils - 2016

Proportion of pupils repeating the school year - 2016

Number of teachers FTE - 2016

Share of girls - 2016

School need: Inclusive education

School need: Reading literacy

School need: Mathematics literacy

School need: Entrepreneurship

School need: ICT competence of teachers

School need: Support of social and civic competence

School need: Polytechnic education

School need: Language literacy

Region: Hlavní m¥sto Praha

Region: Jiho£eský

Region: Jihomoravský

Region: Karlovarský

Region: Vyso£ina

Region: Královéhradecký

Region: Liberecký

Region: Moravskoslezský

Region: Olomoucký

Region: Pardubický

Region: Plze¬ský

Region: St°edo£eský

Region: Ústecký

Region: Zlínský

Logarithm of population size

Observations

0.001

144.4

0.011

10.88

0.457

49.81

34.72

25.59

23.50

17.50

34.34

43.41

36.97

0.048

0.073

0.117

0.029

0.063

0.070

0.041

0.057

0.057

0.054

0.099

0.153

0.064

0.075

8.234

979

0.124

0.001

248.6

0.008

16.43

0.478

48.70

34.68

25.62

23.25

17.07

33.97

43.94

38.33

0.069

0.059

0.117

0.026

0.065

0.064

0.052

0.122

0.075

0.064

0.036

0.123

0.069

0.059

8.783

3049

Difference
0.022∗∗∗
(0.003)
0.093∗∗∗
(0.009)
-0.000
(0.000)
-104.1∗∗∗
(7.575)
0.003∗∗
(0.001)
-5.550∗∗∗
(0.427)
-0.021∗∗∗
(0.003)
1.114∗∗∗
(0.262)
0.041
(0.218)
-0.026
(0.170)
0.248
(0.156)
0.430∗∗∗
(0.102)
0.361∗
(0.170)
-0.531
(0.307)
-1.362∗∗∗
(0.267)
-0.021∗
(0.009)
0.013
(0.009)
0.000
(0.012)
0.003
(0.006)
-0.001
(0.009)
0.006
(0.009)
-0.011
(0.008)
-0.065∗∗∗
(0.011)
-0.018
(0.009)
-0.009
(0.009)
0.063∗∗∗
(0.008)
0.030∗
(0.012)
-0.005
(0.009)
0.016
(0.009)
-0.549∗∗∗
(0.084)

Note. The table reports the mean values of the outcomes in control (column (1)) and treated (column(2)) schools,
and their difference (column(3)). Standard errors in parentheses. ∗ p < 0.05, ∗∗ p < 0.01, ∗∗∗ p < 0.001

12

5.3 Description of the outcomes considered

5.3.1 ``School climate outcomes''

The survey on school outcomes was built taking inspiration by different sources: few items are taken form the
TALIS survey (Teaching and Learning international Survey)2, others re(cid:29)ects the index of Inclusion by Mooth and
Ainscow3, and others from some recent work done in the country to study the inclusion of Roma and socially
disadvantage students.4.

The survey was administered in June 2019. The number of teachers replying to the survey varies substantially
between schools, from a minimum of only 1 teacher replying, to a maximum of 54.5 The survey asks a battery
of questions related to various topics, and respondents usually have a scale of agreement (1: de(cid:28)nitely disagree,
2: rather disagree, 3: rather agree, 4: de(cid:28)nitely agree), on a set of items referring to the same latent dimension.
Details of the questions referring to each item are reported in Table A1 in Annex 2.

For each dimension, we run a factor analysis, using the principal-component factor method to analyse the
correlation matrix. The idea behind this methodology is to reduce the number of variables in an analysis by
describing linear combinations of the variables that contain most of the information and that admit meaningful
interpretations. So, for each of the groups of variables presented in Table A1 we run a factor analysis, with
the aim of reducing the number of variables to be used as outcomes. For most of the dimensions, the factor
analysis led to just one factor, summarising the latent dimensions underneath.

In the rest of the cases, the factor analysis identi(cid:28)ed 2 or 3 latent dimensions. The division of the items
in the various factors is identi(cid:28)ed by colours in Table A1: items ending up in the same factor, within the same
set of questions, are identi(cid:28)ed with the same color (Factor 1: black, Factor 2: blue, and Factor 3: red). So
for example, if we take the questions about (cid:16)School delinquency and violence" the factor analysis identi(cid:28)ed
3 factors. The red one, clearly related to use of substances (alcohol, drugs and smoking), the blue one to
aggression towards teachers and physical aggression between students, and the black one other relatively less
serious bad behaviours of students at school. Details on this are reported in Annex 2. These outcome variables
were averaged at the school level. The mean of the outcome variables in the treated and control schools, and
their difference is reported in Table 2. The factors, have been standardised to have 0 mean, and 1 standard
deviation and higher values of the factors represent positive behaviors, or climate.6 The differences between
treated and control groups are signi(cid:28)cant in half of the outcomes, and in the treated schools the values are
lower.

As outcomes are measured in June 2019, it should be underlined that some schools did not completed
the intervention when the outcomes were measured: all schools who started the intervention after June 2017,
which represent around the 40% of the treated schools in this sample. However by September 2017, all the
school considered started the intervention, so in this set of schools, the intervention was almost done: they
were surveyed in June 2019, and the intervention would be over by September 2019. To take this into account,
in Section 6.3 we replicate the analysis on the sample of schools which had completed the intervention by June
2019.

5.3.2 ``Cermat exams''

The cermat exams collect information on children test scores in mathematics and Czech language. This is a
standardised exam, administered to children at the end of grade 9 (end of primary school) who want to continue
in secondary education (it is an admission exam, compulsory only for pupils who will go on to secondary
education). The available information is the average test score at the school level. The same test can be
administered also at the end of grade 7, but it is taken by far fewer students, therefore we rely on the test
of children in grade 9, as the info is available for a broader sample of schools. The mean of the outcomes in
mathematics and language, at the end of grade 9, are reported in Table 3 for academic year 2018-19, taken
in June 2019. Scores have been standardised so to have 0 mean and 1 standard deviation. Even if the scores
are standardised and so comparable between schools, in the same year they are not comparable over time and
they are not made to be used in a longitudinal manner. As mentioned above, as outcomes are measured in June

2http://www.oecd.org/education/talis/http://www.oecd.org/education/talis/
3source: https://www.eenet.org.uk/resources/docs/Index%20English.pdf
4https://theses.cz/id/mrd2ot/SAFRANKOVA_Anna_disertacni_prace_autoreferat.pdf
5The correlation between number of full time equivalent teachers in the school and number of responses to the survey is 0.43.

If we
run a regression using as dependent variable the number of teachers responding and as independent variables the number of full time
equivalent teachers in the school and whether the school is treated, we (cid:28)nd that being a treated school on average increases the number
of teachers responding by 1.8, and that 1 more FTE teacher increases the number of teachers responding by 0.22.

6Some of the factors have been reversed, as in principle, higher values were associated to negative behavior and climate. This was done
to make interpretation of the Tables easier: a positive sign of the coef(cid:28)cient will always indicate a positive effects on the outcomes, and
viceversa with the negative sign. The factors that were reversed are: all 3 factors of questions on (cid:16)School delinquency and violence(cid:17), (cid:16)Socially
disadvantaged children(cid:17) and (cid:16)Roma Children(cid:17) and factor 1 of question (cid:16)Socially disadvantaged and Roma Children (teachers' feelings)".

13

Table 2: Descriptive of the outcome variables: School climate outcomes

Participation among stakeholders

Teacher-students relationship

Mutual respect

Control
0.119

Treated
-0.035

0.181

-0.047

0.167

-0.048

School delinquency and violence 1 - minor issues (absenteeism, cheating, vandalism)

0.173

-0.061

School delinquency and violence 2 - violence against other students and teachers

0.110

-0.002

School delinquency and violence 3 - substance abuse

Building community (Index of social inclusion)

Establishing inclusive values (Index of social inclusion)

Developing the school for all (Index of social inclusion)

Organizing support for diversity (Index of social inclusion)

Orchestrating learning (Index of social inclusion)

Mobilizing resources (Index of social inclusion)

0.131

-0.031

0.164

-0.047

0.153

-0.041

0.197

-0.039

0.169

-0.045

0.195

-0.051

0.205

-0.054

Socially disadvantaged and Roma children 1 - negative feelings toward children

-0.066

0.014

Socially disadvantaged and Roma children 2 - teachers feel prepared

0.087

-0.028

Socially disadvantaged 1 - are bad for the class

Socially disadvantaged 2 - are discriminated

Socially disadvantaged 3 - create more work for the teachers

Roma 1 - are bad for the class

Roma 2 - are discriminated

Roma 3 - create more work for the teachers

Teacher's con(cid:28)dence.

-0.024

0.016

0.054

-0.010

-0.078

0.014

-0.033

0.014

0.073

-0.014

-0.106

0.028

0.022

-0.003

Difference
0.154∗
(0.068)
0.228∗∗∗
(0.069)
0.215∗∗
(0.069)
0.234∗∗∗
(0.068)
0.112
(0.065)
0.162∗
(0.069)
0.212∗∗
(0.069)
0.194∗∗
(0.069)
0.236∗∗∗
(0.069)
0.214∗∗
(0.069)
0.246∗∗∗
(0.070)
0.259∗∗∗
(0.070)
-0.080
(0.070)
0.115
(0.071)
-0.040
(0.070)
0.064
(0.071)
-0.092
(0.071)
-0.046
(0.072)
0.087
(0.071)
-0.133
(0.072)
0.025
(0.072)

Note. The table reports the mean values of the outcomes in control (column (1)) and treated (column(2)) schools, and
their difference (column(3)). Standard errors in parentheses. ∗ p < 0.05, ∗∗ p < 0.01, ∗∗∗ p < 0.001

14

2019, it should be underlined that some schools did not completed the intervention when the outcomes were
measured, again around the 40% of the treated schools in this sample.

Table 3: Descriptive of the outcome variables: cermat exams

Standardized values of result language 2019

Control
-0.102

Treated
0.020

Standardized values of result mathematics 2019

-0.071

0.014

Difference
-0.122∗
(0.055)
-0.085
(0.055)

Note. The table reports the mean values of the outcomes in control (column (1))
and treated (column(2)) schools, and their difference (column(3)). Standard errors in
parentheses. ∗ p < 0.05, ∗∗ p < 0.01, ∗∗∗ p < 0.001

15

6 Binary treatment

6.1 Definition of the treatment variables

The treatment proposed to schools is made up of 4 different typologies of actions:

(cid:22) T1: Auxiliary school personnel

(cid:22) T2: Personal and social development of teachers

(cid:22) T3: Extra-curricular activities

(cid:22) T4: Cooperation with parents

The number of schools engaged in the various actions vary substantially between actions. In particular, out
of the 3049 treated schools, only 255 were engaged in all the 4 actions. Looking at the single actions we see
that: 2,002 schools participated in the (cid:28)rst action, 2,557 in the second, 2,653 in the third, and 674 in the fourth.
The greatest overlap is between action 2 and action 3 (2,209 doing both).

We will use different de(cid:28)nitions of treatment:

1. Binary treatment, any kind of action (T). All schools who received any sort of intervention are considered as
treated, independently of the actions or the intensity. In binary treatment we distinguish between treated
and non-treated schools, creating a variable that takes value 1 if the school is treated and 0 otherwise.
It means that if a school has received just one of the 4 different typologies of action, that school is
considered as treated. As already mentioned, the number of treated schools under this de(cid:28)nition is 3049.

2. Binary treatment, considering the four different actions - Version a

In this case, we analyse each action one at a time. For example, if we start from the auxiliary school
personnel action, we create a treatment variable that takes value 1 if the school has received this speci(cid:28)c
treatment and 0 otherwise (so schools who do not do the auxiliary school actions, but do other actions,
are considered as potential controls).
In the end we have 4 different treatment variables, each one
representing one of the 4 different actions. (We call these variables: T1a, T2a, T3a, T4a).

3. Binary treatment, considering the four different actions - Version b

In this case, we analyse each action one at a time, using as control groups always the group of schools
who did not receive any treatment. For example, if we start from the auxiliary school personnel action,
we create a treatment variable that takes value 1 if the school has received this speci(cid:28)c treatment and
0 otherwise only for never treated schools (so schools who do not do the auxiliary school actions, but
In the end we have
do other actions, are not considered in the analysis, nor as treated nor as control).
4 different treatment variables, each one representing one of the four different actions, and the control
groups is always composed by the group of schools who did not receive any treatment. (We call these
variables: T1b, T2b, T3b, T4b).

4. Binary treatment: bundles of treatments

In this case we analyse different combination of actions, using as control group always the group of
schools which did not receive any treatment. There are 11 possible combinations of the various activities,
for example T123 will take value 1 for all schools who do actions 1, 2 , and 3; T12 for all schools who do
actions 1, and 2. (We call these variables: T123, T124, T134, T234, T12, T13, T14, T23, T24, T34, T1234).

Table 4 reports the number of treated and control schools in each de(cid:28)nition of treatment, both for the school

climate and the cermat outcomes.

16

Table 4: Number of treated and control schools in each definition of treatment

School climate
Control
Treated
247
1340
678
909
431
1156
423
1164
1131
276
247
909
247
1156
247
1164
247
276
247
616
247
129
247
129
247
233
247
478
247
760
247
147
247
997
247
255
247
250
247
115

Cermat

Treated
2027
1490
1766
1762
414
1490
1766
1762
414
1058
242
245
353
1260
1258
272
1531
381
381
220

Control
426
963
687
691
2039
426
426
426
426
426
426
426
426
426
426
426
426
426
426
426

T
T1a
T2a
T3a
T4a
T1b
T2b
T3b
T4b
T123
T124
T134
T234
T12
T13
T14
T23
T24
T34
T1234

17

6.2 Results

We start by reporting the results of the binary treatment case, considering as outcomes the school climates
outcomes (as reported in Table 2) and the cermat exams (as reported in Table 3). Some of the schools had
missing values in some of the outcomes. However, due to the already limited sample size, we kept all the
schools for which there is information of at least 1 outcome. When we run a probit regression, estimating the
probability of being treated on the control variable (Table B4) we see that most of the control variables are
not good predictors of the probability of being a treated school. In terms of the choice of matching estimator,
we impose a common support, and try several possibilities: nearest neighbor, with and without caliper, with
different caliper option, radius, and kernel. The choice of the estimator to use is based on the best bias reduction
obtained. This ended up in using a nearest neighbor, with caliper.7 The covariates used in the matching are all
the ones presented in Table 1, with the addition of region (cid:28)xed effect, and of some interactions between the
various variables. The inclusion of the interaction serves the purpose of (cid:28)nding a good balance between the
treated and the control groups. The goodness of the matching is reported in Table 5: both the mean and median
absolute standardised biases are below the established threshold of 5%, the pseudo R2 from probit estimation
of the conditional treatment probability (propensity score) on all the variables, are low, the P-values of the
likelihood-ratio test of the joint insigni(cid:28)cance of all the regressors are high, the B is below 25, R is between 0.5
and 28. Overall, we are able to (cid:28)nd a set of control schools, whose observable characteristics re(cid:29)ect those of
the treated schools, so - in terms of these observable characteristics - the two sample are comparable.

In tables 6 and 7 we report the average treatment effect, for the different binary treatment de(cid:28)nitions, that
is the difference between the outcomes in the treated schools and the matched control schools. We see that
overall there is no signi(cid:28)cant impact of any of the treatments on most of the outcomes. An exception is the
negative effect on variable "School delinquency and violence" (2 - violence against other students and teachers),
meaning that almost all treatments increase the violence in schools. As for the cermat exams, we (cid:28)nd that there
is no signi(cid:28)cant effect of any of the treatments on the outcomes. (We only see a negative effect of treatment
3 (extra curricular activities) on mathematics outcomes in year 2019.)

Tables 8 and 9 report the effects of combinations of treatments on both school and cermat outcomes, by
creating bundles of 2, 3 and 4 treatments. We don't (cid:28)nd signi(cid:28)cant effects on the majority of the outcomes.
Regarding school outcomes, we notice two exceptions: the (cid:28)rst one is once again the negative effect on "School
delinquency and violence" (2 - violence against other students and teachers); the second exception is the positive
effect on "Socially disadvantaged and Roma children" (1 - Negative feelings toward children). We (cid:28)nd it especially
when treatment 4 (cooperation with parents) is present in the bundle (T124, T14, T23, T34) and it means that
when families are involved there is a decrease of negative feelings toward children.

7The number of neighbors and the size of the caliper varies according to the various treatment, and was chosen to achieve the best

balance between the two groups.

8For few cases the B is slightly above 25, but all the other statistics are (cid:28)ne. We could not (cid:28)nd a good matching for the treatment

T1234, in the sample of (cid:16)School climate(cid:17), so we will not report results of this treatment's de(cid:28)nition.

18

A. School outcomes
T
T1a
T2a
T3a
T4a
T1b
T2b
T3b
T4b
T123
T124
T134
T234
T12
T13
T14
T23
T24
T34
B. Cermat exam
T
T1a
T2b
T3c
T4d
T1b
T2b
T3b
T4b
T123
T124
T134
T234
T12
T13
T14
T23
T24
T34
T1234

Table 5: Matching statistics

(1)
R2

(2)

(3)

(4)

p chi2 Med Bias Mean Bias

(5)
R

(6)
B

0.010
0.010
0.005
0.008
0.006
0.017
0.010
0.011
0.018
0.017
0.016
0.018
0.012
0.019
0.016
0.020
0.010
0.010
0.011

0.213
0.676
0.978
0.598
1.000
0.192
0.583
0.392
0.999
0.637
1.000
1.000
1.000
0.282
0.426
1.000
0.610
1.000
1.000

3.251
1.897
1.894
1.982
1.987
2.235
2.742
3.168
3.037
1.692
2.931
3.572
3.064
2.925
1.591
3.234
2.429
2.492
2.354

3.732
2.404
1.776
2.668
2.535
2.728
2.844
2.929
4.024
2.566
3.619
3.925
3.024
3.049
2.649
3.909
2.942
2.801
3.016

1.027
0.925
1.234
1.102
0.789
0.903
1.016
0.966
0.952
0.756
0.711
0.830
1.189
0.953
1.258
1.666
1.025
1.339
1.199

23.560
24.136
16.766
21.509
18.921
30.663
23.447
24.713
31.640
31.405
29.652
31.899
25.448
32.740
29.462
33.296
23.743
23.867
24.626

0.959
0.972
1.057
1.031
0.998
1.529
0.939
1.075
1.037
0.904
0.895
0.703
0.911
0.923
0.586
0.907
1.011
0.845
0.737
1.088

0.747
0.609
0.988
1.000
1.000
0.405
0.826
0.759
1.000
0.728
1.000
1.000
1.000
0.560
0.623
1.000
0.933
1.000
1.000
1.000

1.622
1.401
1.598
1.447
1.065
1.224
1.844
2.013
2.138
1.263
2.210
2.738
3.571
1.021
1.640
2.895
1.619
2.478
2.868
2.437

15.376
18.930
12.948
9.713
11.286
20.279
15.852
16.387
21.254
21.765
24.853
24.693
23.902
21.173
20.313
21.927
15.603
23.252
21.198
25.097

2.027
0.004
2.093
0.006
1.789
0.003
1.515
0.002
1.361
0.002
1.439
0.008
2.442
0.005
2.452
0.005
2.660
0.008
1.699
0.008
2.553
0.011
3.288
0.011
3.524
0.010
1.599
0.008
2.189
0.008
3.116
0.009
2.293
0.004
2.945
0.010
3.376
0.008
2.763
0.011
Note. The table reports the matching statistics in the two samples:
in panel A, for schools for
which the (cid:16)school outcome(cid:17) is available, and in panel B, for the schools for which the (cid:16)cermat exam(cid:17)
is available. Column (1) reports the pseudo R square from a probit estimation of the conditional
treatment probability on all the variables; column(2) reports the P-values of the likelihood ratio test
of the joint insigni(cid:28)cance of all the regressors; column(3) and (4) the median and mean absolute
standardized biases respectively; column (5) Rubin's R (the ratio of treated to (matched) non-treated
variances of the propensity score index); and column (6) the Rubins' B (the absolute standardized
difference of the means of the linear index of the propensity score in the treated and (matched) non-
treated group). The statistics are calculated in the two samples, and for the 9 different de(cid:28)nitions
of treatment (T, T1a,etc..).

19

Table 6: Results: School climate outcomes

Participation among stakeholders

Teacher-students relationship

Mutual respect

School delinquency and violence 1 - minor issues (absenteeism, cheating, vandalism)

School delinquency and violence 2 - violence against other students and teachers

School delinquency and violence 3 - substance abuse

Building community

Establishing inclusive values

2
0

Developing the school for all

Organizing support for diversity

Orchestrating learning

Mobilizing resources

Socially disadvantaged and Roma children 1 - negative feelings toward children

Socially disadvantaged and Roma children 2 - teachers feel prepared

Socially disadvantaged 1 - are bad for the class

Socially disadvantaged 2 - are discriminated

Socially disadvantaged 3 - create more work for the teachers

Roma 1 - are bad for the class

Roma 2 - are discriminated

Roma 3 - create more work for the teachers

Teacher's con(cid:28)dence

T
0.020
(0.067)
0.018
(0.073)
0.014
(0.070)
0.103
(0.070)
-0.113*
(0.056)
0.024
(0.065)
0.062
(0.074)
0.025
(0.059)
-0.011
(0.065)
0.034
(0.062)
0.026
(0.075)
-0.010
(0.069)
0.102
(0.065)
-0.032
(0.073)
0.118
(0.087)
-0.074
(0.079)
0.127
(0.072)
0.052
(0.073)
-0.023
(0.069)
0.109
(0.071)
0.009
(0.071)

T1a
-0.060
(0.054)
-0.065
(0.045)
-0.074
(0.048)
-0.041
(0.041)
-0.054
(0.044)
-0.082*
(0.040)
-0.066
(0.049)
-0.038
(0.042)
-0.118***
(0.043)
-0.101*
(0.047)
-0.026
(0.048)
-0.127***
(0.045)
0.020
(0.046)
-0.062
(0.052)
-0.028
(0.054)
-0.015
(0.056)
0.016
(0.062)
-0.024
(0.048)
-0.081
(0.050)
0.065
(0.044)
-0.059
(0.055)

T2a
-0.053
(0.045)
-0.002
(0.046)
-0.041
(0.049)
0.096
(0.055)
-0.115***
(0.044)
0.094*
(0.045)
-0.038
(0.046)
-0.066
(0.038)
-0.040
(0.044)
-0.001
(0.049)
0.013
(0.050)
-0.032
(0.050)
0.024
(0.047)
-0.027
(0.048)
0.048
(0.055)
-0.063
(0.055)
0.031
(0.055)
0.027
(0.051)
0.032
(0.048)
0.089
(0.047)
-0.062
(0.053)

T3a
-0.087
(0.055)
-0.046
(0.057)
0.005
(0.047)
-0.017
(0.058)
-0.083
(0.046)
0.053
(0.048)
-0.030
(0.056)
0.007
(0.049)
-0.025
(0.051)
-0.034
(0.050)
-0.044
(0.059)
-0.073
(0.055)
0.009
(0.050)
-0.015
(0.048)
0.073
(0.049)
-0.060
(0.050)
0.113*
(0.057)
0.037
(0.052)
-0.067
(0.056)
0.055
(0.063)
-0.028
(0.058)

T4a
-0.043
(0.058)
-0.064
(0.057)
-0.107
(0.061)
0.006
(0.064)
-0.074
(0.051)
0.023
(0.047)
-0.025
(0.061)
-0.025
(0.048)
0.032
(0.048)
-0.054
(0.056)
-0.022
(0.054)
0.011
(0.059)
0.064
(0.049)
-0.061
(0.058)
0.030
(0.052)
0.005
(0.059)
-0.012
(0.058)
0.019
(0.057)
-0.014
(0.055)
0.042
(0.060)
-0.080
(0.059)

T1b
-0.003
(0.075)
-0.016
(0.077)
0.007
(0.075)
0.092
(0.079)
-0.133***
(0.051)
-0.057
(0.066)
0.002
(0.082)
-0.022
(0.063)
-0.066
(0.073)
-0.010
(0.068)
-0.011
(0.086)
-0.081
(0.079)
0.103
(0.073)
-0.080
(0.081)
0.115
(0.094)
-0.056
(0.080)
0.091
(0.077)
0.053
(0.080)
-0.029
(0.081)
0.107
(0.083)
-0.059
(0.078)

T2b
0.013
(0.072)
0.020
(0.072)
-0.006
(0.076)
0.126
(0.078)
-0.099
(0.054)
0.037
(0.078)
0.020
(0.080)
-0.000
(0.066)
0.009
(0.067)
0.045
(0.068)
0.003
(0.072)
-0.027
(0.066)
0.095
(0.072)
-0.021
(0.079)
0.150
(0.095)
-0.084
(0.087)
0.122
(0.081)
0.040
(0.078)
0.005
(0.081)
0.107
(0.092)
0.028
(0.074)

T3b
0.003
(0.077)
0.032
(0.075)
0.053
(0.074)
0.118
(0.079)
-0.111**
(0.047)
0.003
(0.066)
0.062
(0.079)
0.028
(0.064)
0.004
(0.069)
0.032
(0.064)
0.018
(0.077)
-0.030
(0.069)
0.107
(0.073)
-0.013
(0.072)
0.099
(0.090)
-0.063
(0.078)
0.120
(0.069)
0.055
(0.080)
-0.005
(0.080)
0.152*
(0.073)
-0.020
(0.079)

T4b
0.005
(0.131)
-0.004
(0.119)
-0.011
(0.106)
0.148
(0.114)
-0.205**
(0.083)
0.001
(0.107)
0.022
(0.107)
0.006
(0.088)
0.039
(0.093)
0.034
(0.099)
-0.078
(0.110)
-0.125
(0.098)
0.159
(0.160)
-0.126
(0.155)
0.262
(0.149)
-0.140
(0.131)
0.127
(0.132)
0.056
(0.136)
0.066
(0.100)
0.085
(0.113)
0.011
(0.121)

Note. The table reports the effects of the 9 treatment de(cid:28)nitions on the school climate outcomes. Standard errors in parentheses. ∗ p < 0.05, ∗∗ p < 0.01, ∗∗∗ p < 0.001

Table 7: Results: cermat exam

Standardized values of results in language 2019

Standardized values of results in mathematics 2019

T
-0.085
(0.058)
-0.114
(0.060)

T1a
-0.054
(0.044)
-0.026
(0.047)

T2a
-0.021
(0.054)
-0.064
(0.055)

T3a
-0.078
(0.046)
-0.098*
(0.049)

T4a
0.050
(0.058)
0.045
(0.058)

T1b
-0.122*
(0.058)
-0.127*
(0.057)

T2b
-0.083
(0.062)
-0.112
(0.060)

T3b
-0.069
(0.063)
-0.104
(0.061)

T4b
-0.034
(0.081)
0.004
(0.082)

Note. The table reports the effects of the 9 treatment de(cid:28)nitions on the Cermat outcomes. Standard errors in parentheses. ∗ p < 0.05, ∗∗ p < 0.01, ∗∗∗ p < 0.001

2
1

Table 8: Results: Cermat outcomes- bundle treatments

Standardized values of results in language 2019

T34
-0.041
(0.097)
-0.023
(0.092)
Note. The table reports the effects of the 11 treatment de(cid:28)nitions on the school cermat exam. Standard errors in parentheses. ∗ p < 0.05, ∗∗ p < 0.01, ∗∗∗ p < 0.001

Standardized values of results in mathematics 2019

T12
-0.097
(0.075)
-0.112
(0.070)

T234
-0.109
(0.100)
-0.120
(0.094)

T134
-0.113
(0.100)
-0.103
(0.097)

T123
-0.108
(0.073)
-0.117
(0.070)

T24
-0.046
(0.096)
-0.025
(0.092)

T23
-0.113
(0.078)
-0.116
(0.073)

T14
-0.058
(0.100)
-0.040
(0.097)

T13
-0.111
(0.075)
-0.098
(0.071)

T124
-0.106
(0.102)
-0.088
(0.097)

T1234
-0.134
(0.101)
-0.082
(0.098)

Table 9: Results: School climate outcomes- bundle treatment

Participation among stakeholders

Teacher-students relationship

Mutual respect

School delinquency and violence 1 - minor issues (absenteeism, cheating, vandalism)

School delinquency and violence 2 - violence against other students and teachers

School delinquency and violence 3 - substance abuse

Building community

Establishing inclusive values

2
2

Developing the school for all

Organizing support for diversity

Orchestrating learning

Mobilizing resources

Socially disadvantaged and Roma children 1 - negative feelings toward children

Socially disadvantaged and Roma children 2 - teachers feel prepared

Socially disadvantaged 1 - are bad for the class

Socially disadvantaged 2 - are discriminated

Socially disadvantaged 3 - create more work for the teachers

Roma 1 - are bad for the class

Roma 2 - are discriminated

Roma 3 - create more work for the teachers

Teacher's con(cid:28)dence.

T123
-0.057
(0.083)
-0.054
(0.085)
-0.001
(0.091)
0.058
(0.088)
-0.140*
(0.067)
0.035
(0.071)
-0.021
(0.091)
-0.055
(0.073)
-0.104
(0.075)
-0.022
(0.079)
-0.010
(0.089)
-0.113
(0.087)
0.074
(0.079)
-0.007
(0.082)
0.012
(0.084)
-0.047
(0.083)
0.155
(0.089)
0.036
(0.086)
-0.036
(0.081)
0.120
(0.087)
-0.059
(0.089)

T124
-0.019
(0.123)
-0.191
(0.127)
-0.120
(0.134)
0.175
(0.133)
-0.225
(0.116)
0.027
(0.113)
-0.016
(0.128)
-0.087
(0.108)
-0.073
(0.108)
-0.057
(0.115)
0.061
(0.125)
-0.094
(0.121)
0.275**
(0.114)
-0.110
(0.126)
0.142
(0.129)
-0.004
(0.122)
0.294*
(0.129)
-0.006
(0.136)
0.048
(0.121)
0.153
(0.127)
-0.139
(0.133)

T134
-0.072
(0.121)
-0.231
(0.129)
-0.164
(0.135)
0.006
(0.130)
-0.210
(0.111)
0.086
(0.119)
-0.100
(0.130)
-0.116
(0.105)
-0.065
(0.111)
-0.058
(0.114)
-0.170
(0.123)
-0.173
(0.120)
0.128
(0.120)
-0.175
(0.129)
0.101
(0.134)
-0.037
(0.128)
0.227
(0.126)
0.067
(0.135)
-0.134
(0.132)
0.156
(0.130)
-0.165
(0.133)

T234
-0.106
(0.112)
-0.031
(0.118)
-0.043
(0.123)
0.108
(0.118)
-0.199*
(0.097)
0.048
(0.097)
-0.006
(0.118)
0.007
(0.095)
-0.010
(0.099)
0.050
(0.107)
0.008
(0.120)
-0.081
(0.119)
0.183
(0.102)
-0.058
(0.109)
0.129
(0.109)
-0.052
(0.107)
0.095
(0.113)
0.040
(0.107)
0.023
(0.106)
0.053
(0.111)
-0.088
(0.114)

T12
-0.017
(0.082)
-0.031
(0.083)
-0.023
(0.088)
0.090
(0.085)
-0.121*
(0.058)
-0.054
(0.071)
-0.009
(0.087)
-0.065
(0.072)
-0.125
(0.071)
-0.055
(0.076)
-0.059
(0.086)
-0.121
(0.085)
0.100
(0.076)
-0.040
(0.078)
0.067
(0.080)
-0.083
(0.081)
0.108
(0.083)
0.030
(0.081)
-0.035
(0.080)
0.111
(0.083)
-0.036
(0.086)

T13
-0.022
(0.079)
-0.025
(0.082)
0.004
(0.087)
0.051
(0.088)
-0.139**
(0.057)
-0.029
(0.070)
-0.015
(0.086)
-0.057
(0.070)
-0.087
(0.072)
-0.038
(0.075)
-0.038
(0.086)
-0.120
(0.084)
0.062
(0.074)
-0.015
(0.077)
0.042
(0.079)
-0.083
(0.081)
0.160
(0.083)
0.039
(0.080)
-0.073
(0.078)
0.119
(0.084)
-0.064
(0.085)

T14
-0.076
(0.112)
-0.107
(0.118)
-0.070
(0.125)
0.136
(0.124)
-0.253**
(0.102)
0.034
(0.109)
0.035
(0.129)
-0.047
(0.102)
-0.149
(0.102)
-0.081
(0.111)
-0.025
(0.122)
-0.110
(0.120)
0.227*
(0.109)
-0.155
(0.118)
0.113
(0.118)
-0.089
(0.119)
0.160
(0.122)
0.122
(0.126)
-0.019
(0.119)
0.148
(0.120)
-0.164
(0.122)

T23
-0.020
(0.073)
-0.003
(0.077)
0.033
(0.081)
0.081
(0.078)
-0.110
(0.065)
0.054
(0.066)
0.035
(0.079)
0.006
(0.064)
-0.030
(0.067)
0.022
(0.070)
0.005
(0.080)
-0.037
(0.079)
0.073
(0.068)
-0.033
(0.071)
0.076
(0.076)
-0.092
(0.073)
0.152*
(0.077)
0.048
(0.073)
-0.015
(0.072)
0.100
(0.074)
-0.020
(0.076)

T24
0.000
(0.098)
0.006
(0.103)
-0.003
(0.108)
0.080
(0.106)
-0.205**
(0.087)
0.018
(0.088)
0.062
(0.104)
0.040
(0.083)
0.069
(0.087)
0.068
(0.095)
0.083
(0.104)
-0.002
(0.104)
0.203*
(0.088)
-0.019
(0.093)
0.168
(0.096)
-0.115
(0.096)
0.123
(0.100)
0.049
(0.094)
-0.013
(0.094)
0.108
(0.098)
0.003
(0.100)

T34
-0.053
(0.103)
-0.015
(0.107)
-0.021
(0.111)
0.067
(0.107)
-0.220**
(0.088)
0.072
(0.091)
0.041
(0.108)
0.036
(0.087)
0.023
(0.091)
0.041
(0.098)
0.013
(0.109)
-0.029
(0.107)
0.229**
(0.092)
-0.043
(0.096)
0.186
(0.101)
-0.033
(0.099)
0.146
(0.104)
0.090
(0.098)
0.026
(0.098)
0.134
(0.102)
0.017
(0.105)

Note. The table reports the effects of the 10 treatment de(cid:28)nitions on the school climate outcomes. Standard errors in parentheses. ∗ p < 0.05, ∗∗ p < 0.01, ∗∗∗ p < 0.001

6.3 Sample of schools starting in academic year 2016-2017

As mentioned above, the treated schools started the project both in academic year 2016-17 and in academic
year 2017-2018. Since the duration of the intervention is 24 months, this means that when the outcomes are
measured (end of school year 2019), some of the schools starting the intervention in academic year 2017-18
were not done yet with the project. So outcomes are measured while intervention is still ongoing. This is not
a problem if we can expect some immediate results, however, we replicate the analysis only for schools who
started the intervention in academic year 2016-17, so that by end of academic year 2018-19 the intervention
is for sure over.
If we focus only on this sample, the sample size of the treated group roughly halves in both
set of outcomes considered. We report in the table below the results considering only the (cid:28)rst three de(cid:28)nitions
Indeed for the bundle treatment de(cid:28)nition the matching procedure was not successful in most
of treatment.
of the cases to reach a good balance, probably due to the small sample size of the treated group in some of
the bundles. Table 10 reports the matching statistics and Tables 11 and 12 report the results. The results are
identical to the results obtained with the full sample. A persistent negative effect is found on the index of (cid:16)School
delinquency and violence, violence against students and teachers(cid:17), for many of the treatment de(cid:28)nitions, and
not other effects are detected.

Table 10: Matching statistics -Intervention start in 2016-17

A. School outcomes
T
T1a
T2a
T3a
T4a
T1b
T2b
T3b
T4b
B. Cermat exam

T
T1a
T2a
T3a
T4a
T1b
T2b
T3b
T4b

(1)
R2

(2)

(3)

(4)

p chi2 Med Bias Mean Bias

(5)
R

(6)
B

0.012
0.010
0.009
0.010
0.011
0.012
0.012
0.011
0.014

R2
0.004
0.010
0.004
0.002
0.005
0.008
0.007
0.006
0.010

0.836
0.994
0.990
0.959
1.000
0.981
0.939
0.962
1.000

2.311
2.237
3.039
2.320
2.830
3.359
2.906
3.269
2.277

3.174
2.621
3.306
3.006
3.238
3.758
3.394
3.776
3.029

p chi2 MedBias
0.996
0.773
1.000
1.000
1.000
0.909
0.955
0.980
1.000

1.712
1.604
2.211
1.150
1.877
2.024
2.422
2.065
2.304

MeanBias
2.080
2.145
2.127
1.392
2.224
2.288
2.485
2.089
3.109

0.879
0.949
0.668
0.671
0.935
0.793
1.045
0.764
1.259

R
0.946
0.834
0.803
0.743
1.129
1.075
0.838
1.105
1.090

25.325
24.216
21.728
23.708
24.751
25.930
26.189
24.195
27.901

B
15.633
23.671
14.054
11.340
17.235
21.786
19.410
18.328
23.754

Note. The table reports the matching statistics for schools who started the intervention in academic
year 2016-17, in the two samples: in panel A, for schools for which the (cid:16)school outcome(cid:17) is available,
and in panel B, for the schools for which the (cid:16)cermat exam(cid:17) is available. Column (1) reports the
pseudo R square from a probit estimation of the conditional treatment probability on all the
variables; column(2) reports the P-values of the likelihood ratio test of the joint insigni(cid:28)cance of all
the regressors; column(3) and (4) the median and mean absolute standardized biases respectively;
column (5) Rubin's R (the ratio of treated to (matched) non-treated variances of the propensity score
index); and column (6) the Rubins' B (the absolute standardized difference of the means of the linear
index of the propensity score in the treated and (matched) non-treated group). The statistics are
calculated in the two samples, and for the 9 different de(cid:28)nitions of treatment (T, T1a,etc..).

23

Table 11: Results: School climate outcomes - 2016 schools

Participation among stakeholders

Teacher-students relationship

Mutual respect

School delinquency and violence 1 - minor issues (absenteeism, cheating, vandalism)

School delinquency and violence 2 - violence against other students and teachers

School delinquency and violence 3 - substance abuse

Building community

Establishing inclusive values

2
4

Developing the school for all

Organizing support for diversity

Orchestrating learning

Mobilizing resources

Socially disadvantaged and Roma children 1 - negative feelings toward children

Socially disadvantaged and Roma children 2 - teachers feel prepared

Socially disadvantaged 1 - are bad for the class

Socially disadvantaged 2 - are discriminated

Socially disadvantaged 3 - create more work for the teachers

Roma 1 - are bad for the class

Roma 2 - are discriminated

Roma 3 - create more work for the teachers

Teacher's con(cid:28)dence

T
0.036
(0.090)
0.043
(0.077)
0.025
(0.080)
0.112
(0.081)
-0.104
(0.055)
-0.034
(0.062)
0.096
(0.084)
0.042
(0.062)
0.003
(0.079)
0.056
(0.074)
0.059
(0.082)
-0.003
(0.076)
0.146
(0.089)
-0.032
(0.091)
0.110
(0.092)
-0.066
(0.085)
0.120
(0.075)
0.065
(0.081)
0.007
(0.072)
0.130
(0.081)
0.000
(0.074)

T1a
-0.020
(0.058)
-0.055
(0.057)
-0.057
(0.058)
-0.045
(0.061)
-0.089
(0.049)
-0.076
(0.065)
-0.050
(0.059)
-0.043
(0.055)
-0.070
(0.066)
-0.046
(0.061)
-0.027
(0.059)
-0.115*
(0.053)
0.089
(0.070)
-0.021
(0.100)
-0.046
(0.073)
-0.035
(0.078)
0.141
(0.086)
0.053
(0.065)
-0.053
(0.054)
0.089
(0.063)
0.001
(0.070)

T2a
0.021
(0.060)
0.049
(0.064)
0.048
(0.064)
0.195***
(0.071)
-0.132***
(0.050)
0.058
(0.066)
0.120
(0.066)
0.040
(0.051)
0.038
(0.062)
0.089
(0.061)
0.053
(0.070)
0.019
(0.064)
0.072
(0.056)
0.025
(0.063)
0.052
(0.070)
-0.103
(0.065)
0.064
(0.063)
0.033
(0.062)
-0.016
(0.057)
0.094
(0.059)
-0.072
(0.066)

T3a
-0.042
(0.063)
-0.032
(0.066)
0.028
(0.061)
-0.025
(0.069)
-0.048
(0.057)
-0.011
(0.059)
0.017
(0.068)
0.020
(0.057)
-0.031
(0.060)
-0.049
(0.057)
-0.045
(0.065)
-0.067
(0.066)
0.036
(0.057)
-0.001
(0.071)
0.059
(0.065)
-0.048
(0.067)
0.151*
(0.069)
0.018
(0.062)
-0.045
(0.064)
0.083
(0.069)
-0.042
(0.066)

T4a
0.008
(0.072)
0.009
(0.065)
0.089
(0.082)
-0.068
(0.078)
-0.056
(0.072)
0.075
(0.065)
0.081
(0.084)
0.060
(0.064)
0.075
(0.067)
0.045
(0.071)
0.012
(0.074)
0.039
(0.071)
0.094
(0.071)
-0.034
(0.082)
0.117
(0.076)
-0.032
(0.081)
0.067
(0.074)
0.022
(0.070)
-0.009
(0.069)
0.100
(0.081)
-0.113
(0.075)

T1b
0.001
(0.078)
-0.059
(0.076)
-0.065
(0.073)
0.063
(0.079)
-0.158***
(0.059)
-0.100
(0.067)
-0.022
(0.079)
-0.031
(0.062)
-0.067
(0.077)
-0.046
(0.069)
-0.037
(0.078)
-0.105
(0.072)
0.135
(0.088)
-0.031
(0.101)
0.061
(0.105)
-0.132
(0.085)
0.069
(0.087)
0.078
(0.088)
-0.052
(0.086)
0.145
(0.087)
-0.061
(0.087)

T2b
0.034
(0.083)
-0.012
(0.070)
0.013
(0.070)
0.131
(0.089)
-0.120**
(0.050)
-0.005
(0.075)
0.035
(0.081)
0.002
(0.061)
-0.054
(0.075)
0.043
(0.069)
0.006
(0.076)
-0.056
(0.066)
0.095
(0.091)
-0.064
(0.093)
0.058
(0.103)
-0.061
(0.082)
0.057
(0.079)
0.003
(0.096)
-0.004
(0.105)
0.135
(0.079)
-0.076
(0.095)

T3b
0.003
(0.082)
0.004
(0.079)
0.040
(0.081)
0.122
(0.074)
-0.119*
(0.054)
-0.060
(0.065)
0.093
(0.085)
0.034
(0.061)
-0.002
(0.078)
0.041
(0.073)
0.035
(0.080)
-0.012
(0.072)
0.135
(0.083)
0.020
(0.091)
0.046
(0.094)
-0.091
(0.090)
0.165*
(0.077)
0.062
(0.082)
-0.020
(0.079)
0.142
(0.082)
-0.003
(0.080)

T4b
0.145
(0.110)
0.090
(0.098)
0.122
(0.104)
0.075
(0.106)
-0.209***
(0.076)
0.001
(0.088)
0.126
(0.105)
0.078
(0.088)
0.056
(0.086)
0.129
(0.092)
0.064
(0.098)
0.027
(0.098)
0.102
(0.084)
-0.082
(0.112)
0.139
(0.102)
-0.135
(0.107)
0.076
(0.092)
0.012
(0.084)
-0.021
(0.099)
0.105
(0.105)
-0.108
(0.116)

Note. The table reports the effects of the 9 treatment de(cid:28)nitions on the school climate outcomes. Standard errors in parentheses. ∗ p < 0.05, ∗∗ p < 0.01, ∗∗∗ p < 0.001

Table 12: Results: cermat exams

2
5

Standardized values of results in language 2019

T1a
-0.049
(0.056)
-0.062
(0.057)
Note. The table reports the effects of the 9 treatment de(cid:28)nitions on the Cermat outcomes. Standard errors in parentheses. ∗ p < 0.05, ∗∗ p < 0.01, ∗∗∗ p < 0.001

Standardized values of results in mathematics 2019

T2a
-0.018
(0.059)
-0.071
(0.059)

T
-0.101
(0.065)
-0.095
(0.065)

T3a
-0.049
(0.061)
-0.047
(0.063)

T4b
0.016
(0.093)
0.022
(0.094)

T4a
-0.017
(0.072)
0.009
(0.076)

T3b
-0.091
(0.067)
-0.115
(0.064)

T2b
-0.056
(0.065)
-0.104
(0.064)

T1b
-0.090
(0.063)
-0.093
(0.061)

7 Continuous treatment

7.1 Definition of the treatment

In this section we explore if exposure to different treatment intensity can have a differential impact on the
outcomes of interest. Indeed we can combine different available data in order to obtain additional information
about the intensity of the treatment variables.

As already mentioned, the treatment proposed to schools is made up of 4 different typologies of actions:

(cid:22) T1: Auxiliary school personnel

(cid:22) T2: Personal and social development of teachers

(cid:22) T3: Extra-curricular activities

(cid:22) T4: Cooperation with parents

Each of the 4 actions contains a number of modules, indicating the intensity. For the auxiliary school personnel
we know the number of months the schools were assisted by 4 different (cid:28)gures of the educational staff (school
assistants, special educators, psychologists and social educators). For the personal and social development of
teachers, we know the number of hours of training devoted to the further education of teachers in 5 different
(cid:28)elds: Mathematics, Reading Literacy, Foreign Language, Inclusive Education and Mentoring. The extra-curricular
school activities consist in the number of hours devoted to pupils in different activities, such as clubs (reading
clubs or puzzles and games clubs), tutoring, supervision and help for pupils with educational needs. Lastly, for
the cooperation with parents we know the number of hours spent by teachers in thematic meetings with the
parents.

Concerning the auxiliary school personnel we have information on the number of school assistants, special
educators, psychologists and social educators employed in the schools. Summing up these variables provides
an estimate of the total number of auxiliary school staff. We then obtain an average of the school auxiliary
staff per student, by dividing it by school size. We refer to this variable as (cid:16)Total number of school auxiliary
personnel per student(cid:17).

The available variables related to personal and social development of teachers contain the number of hours
of training of teachers in 5 (cid:28)elds: Mathematics, Reading Literacy, Foreign Language, Inclusive Education and
Mentoring. Different templates indicate different lengths of the training in terms of hours: 8, 16, 32, 56, 80
hours. The overall information is summarised in 5 newly created variables, which indicate the total number
of hours of training in a given subject undertaken in each school.9 This information is combined with the
information about the number of treated pedagogical staff and share of treated pedagogical staff to total
number of teachers, to create 5 new variables (one per each (cid:28)eld), which provide an information about the
intensity of the treatment which takes into account the proportion of teachers trained and the number of hours
of training.10

The third type of treatment is related to extra-curricular activities. Two of the available variables refer
to the preparation in reading and games clubs, but they are dif(cid:28)cult to exploit, while a third one counts the
number of hours of tutoring and the supervision of children at risk: we use this variables divided by the school
size, to capture the hours of tutoring per student. Finally, there is information on the hours of meeting with
parents, however, this activity concern a small sample of schools, and so we decided not to use it. We report
the descriptive statistics of these variables in Table 13: Panel A, refers to the sample of schools for which the
school climate outcome is available, and Panel B, refers to the sample of schools for which the cermat exams
are available. Column (1) reports the share of schools which are considered as treated, and from column (2)
to (5) we report the main descriptive statistics in the sample of treated schools. From the Table we can see
that extra hours in reading and mathematics were very common (around 40 to 50% of schools engage in these
activities in both samples), while training in foreign language or about inclusive policy regards smaller fraction
of schools.

Our (cid:28)rst approach was to apply generalised propensity score matching, and estimate a dose-response model.
However, as better explained in Annex 3 this was not possible with the data available. So, we adopted another
strategy. Each of the variables described in Table 13, takes value 0 when schools are in the control group
and takes positive values, depending on intensity, when schools receive that speci(cid:28)c treatment. (The proportion
of (cid:16)treated(cid:17) schools for each of the 6 variables is reported in Column (1) of the two panels, corresponding to

9 Hours devoted to mentoring were not used so much (fewer than 7% of schools engaged in this activity). For this reason, we decided
not to estimate the effect on the different intensities of mentoring activity, as the sample of treated schools would be too small to carry
out meaningful analysis.

10Take as an example 2 schools, both doing a total of 100 hours of training in mathematics, and both involving 20 teachers. Assume
also that in the (cid:28)rst schools, these 20 teachers represent the 30% of the total teachers, and that in the second schools, these 20 teachers
represent the 70 % of the total teachers (i.e the (cid:28)rst school has a total of 66 teachers, while the second school has a total of 28 teachers).
This variable will take values (100/20) *0.3 =1.5 in the (cid:28)rst school and (100/20)*0.70 = 3.5 in the second school.

26

Table 13: Descriptive statistics of treatment variables

A. School climate outcomes
Reading
Mathematics
Foreign language
Inclusive education
School auxiliary personnel
Tutoring

B. Cermat outcomes
Reading
Mathematics
Foreign language
Inclusive education
School auxiliary personnel
Tutoring

0.412
0.385
0.273
0.248
0.529
0.626

Share of treated schools Mean
3.850
3.321
5.167
4.609
0.006
1.001
Share of treated schools Mean
3.677
2.842
4.314
4.422
0 .004
1.315

0.540
0.488
0.366
0.340
0.735
0.620

sd
4.155
3.365
7.072
6.111
0.007
1.054
sd
4.2571
2.869
6.279
6.122
0.003
1.512

Min
0 .114
0.089
0.144
0.120
0.001
0.021
Min
0 .114
0.075
0.144
0.119
0.001
0.021

Max
34.46
26.60
40.63
53.33
0.071
16
Max
51.61
31.66
68.57
80
0.071
12.235

Note. The table reports the descriptive statistics of the various de(cid:28)nitions of treatment.
The table reports in Column (1) (cid:16)Share of treated schools(cid:17), the share of schools engaged
in each of the activities reported in the rows, and in the other columns Mean, standard
deviation, minimum and maximum value of the activities reported in the rows.

the two sets of outcomes. So for example, if we consider the variable (cid:16)Reading(cid:17) in the sample of schools for
which we investigate the effect on the (cid:16)school climate outcome(cid:17), we see that 41.2% of schools had a positive
value of this variable, and are considered treated, the rest has a 0 value of this variable and are considered as
controls).11 For some schools, the information on the intensity was missing, and we dropped them as we did
not know whether those were real missing values or were instead not treated schools. (The number of schools
dropped is 132 out of 1587 in the (cid:16)school outcome(cid:17) sample and 186 out of 2453 in the (cid:16)cermat exams(cid:17) sample).
We then consider only the positive values of intensity variables and we cut them at the 33th and 66th
percentiles. This is done to have homogeneous groups in terms of sample size. (So, in the example above, we
focus on this 41.2% treated schools, and estimate the percentiles of the distribution of the variable only among
those with a positive value). We exploit percentiles to generate an additional variable that takes value 0 when
the schools are in the control group, value 1 when the intensity of treatment in the school is low (below the
threshold of the 33th percentile), value 2 when the value of the original variable is between the 33th and the
66th percentile, and value 3 when the intensity of the treatment is high (above the 66th percentile). Eventually,
we generate a dummy for each intensity, obtaining 4 dummy variables:
intensity 1 (control group), intensity
2 (treated, low intensity), intensity 3 (treated, medium intensity) and intensity 4 (treated, high intensity). We
repeat this process for all the 6 variables considered (the ones in Table 13).

Now it is possible to perform 2 sets of analyses for each of the 6 variables:

1. We (cid:28)rst perform a binary propensity score matching and match all treated schools (independently of
the intensity of the treatment) with schools which are not treated. (So, always referring to the example
above, we would match the 41.2% treated schools, with the schools which are not treated). We use in the
matching the same set of variables used in the binary case, as reported in Section 6.2.12 We then use the
weights obtained by the matching procedure to run an OLS regression including the 3 dummy variables
with positive intensity (intensity 2, intensity 3 and intensity 4). This regression is useful to identify if
there is an intensity of treatment that gives better outcomes than not receiving the treatment, taking into
accounts differences in observable variables between the two groups of schools.

2. We then perform three pairwise propensity score matchings in order to compare the control group with
each treated group represented by the intensity dummies. For example, we can start keeping only intensity
2, in order to compare the control group with the group that receives low intensity of treatment. We repeat
this analysis for all the intensities and for all the treatment variables.

11This correspond to what in the binary treatment we de(cid:28)ned as version a. We also tried to replicate the analysis using as controls only
schools who were never treated- as in the binary case version b - however, due to small sample size it was very hard to (cid:28)nd a good balance
between treated and control schools in most of cases, therefore we only focus on this version)

12We don't report again the matching statistics, but in most of the cases all the statistics were good, and in all of the cases the mean and
median bias were below 5%, and the p-values of the likelihood ratio test and the pseudo R squared were (cid:28)ne. For some of the matching
the values of B or R were slightly outside the recommended values.

27

7.2 Results

in the (cid:28)rst set of
Estimates are shown in Tables B5-B27. The two methodologies have both pros and cons:
analyses, we are using all the information available, keeping all the schools in the estimations, but we are only
sure that the covariates are balanced between the group of treated and the group of controls, as we cannot say
anything about the balancing between the control groups and the groups of treated schools belonging to the
different percentile groups. In the second analysis, we are using sub-samples of schools, which may been seen
as sub-optimal, as not all the information is used and sample sizes are smaller, but at least we are sure that
the comparison between the control and the three different groups of treated schools , based on the intensity,
is meaningful as covariates are balanced. If an estimate is signi(cid:28)cant only in the (cid:28)rst part, it means that maybe
this result is due to different characteristics between samples. On the contrary, we can rely on estimates that
are signi(cid:28)cant only in the second half of the tables, but with the caveat that everything is based on smaller
sample size. Finally, we will only comment on results which are coherent between the two sets of estimates.

With regard to the results for Cermat exams no coherent effects are found on the language or mathematics
test scores. For most of the school outcomes variables there are no signi(cid:28)cant effects of any of the treatment
variables, but we noticed that for some outcomes (Participation among stakeholders, Mutual respect, Building
community, Establishing inclusive values, Developing a school for all, and Organizing support for diversity) doing
limited (low intensity) training in Inclusiveness, rather than not doing anything, has a negative impact, while no
signi(cid:28)cant effect is found for those schools doing medium and high intensity.

Those results are to be interpreted with caution:

indeed the measurement used is based on the reply of
teachers or principals, on a very sensitive set of questions, and replies maybe very subjective. In particular, we
(cid:28)nd it very strange that there is a negative effect of doing some training (rather than nothing) on inclusiveness
on some of the variables measuring exactly inclusiveness. An explanation could be that those teachers who did
some training in inclusiveness, realise more than teachers who were not treated at all, that some behaviours
may be a problem, and tend to give a more negative reply than they would have in the absence of the training.
It could be that the training made them more aware of the problem, and so their replies are negatively affected.
This may not happen with the ones who did more training, as we expected that those teachers maybe also found
a solution to those problems. This can be tested by directly comparing schools who received a high intensity
of training, to the schools receiving medium or low intensity and schools who received medium intensity to
the ones who received low intensity. So not including at all the schools not treated. We perform this analysis
only focusing on the treatment about inclusiveness and on the outcomes for which we found counter-intuitive
results (doing the training with respect to not doing anything had a negative effect on the outcome variables:
Participation among stakeholders, Mutual respect, Building community, Establishing inclusive values, Developing
a school for all, and Organizing support for diversity). We directly compared schools doing high intensity to the
ones doing medium or low intensity, and the ones doing medium intensity to the ones doing low intensity.
In
the matching, we control for the usual set of variables, but we excluded the region (cid:28)xed effect, due to the very
small sample size of our samples. The results we (cid:28)nd indeed show that doing higher intensity training is always
associated with a positive coef(cid:28)cient (both doing high with respect to medium or low, and doing medium with
respect to low). The effects found are not always signi(cid:28)cantly different from zero, but at least the sign of the
coef(cid:28)cients point to the right direction one would expect: doing more is associated with higher scores in the
factors.

28

8 Conclusions

The aim of this report is to carry out the evaluation of an ESF funded intervention in the Czech Republic.
This intervention targets primary schools in the country and offers several types of actions to treated schools.
Participation in the treatment is voluntary and the schools decided whether to apply to the programme or not.
However, the choice of the templates by the schools is not random. Based on the needs assessment, the
schools were obliged to choose at least the templates that target the biggest weakness identi(cid:28)ed in the needs
assessments survey. We estimate the effect of participation in the programme on two sets of outcomes: 1) the
outcomes measuring school climate and 2) the outcomes measuring students' performances in the test scores
in mathematics and Czech language from standardised high school entry exams.

To do so we employ propensity score matching techniques. The two main issues emerging from this

evaluation are:

1.

little is known about the selection process of the schools, and

2. that the pool of potential control schools is small compared to the number of treated schools (especially

when considering the sample of schools for which the outcomes are available).

Preliminary results do not show any consistent pattern of effects on the test scores: treated schools do not
show any differential results compared to non-treated schools. Some mixed evidence is found for the school
climate measure, but again only some dimensions seem to be affected by the programme, but not in a clear
and consistent pattern among the different de(cid:28)nitions of treatment and methods used. When reading these
results, we have to keep in mind that:

1. academic achievement is not a direct target of the activities of the intervention and in addition, the
intervention was not directly targeted to the teachers of the classes taking those tests (9th graders) but
the whole schools, and we do not have information about which teachers were exactly treated within the
treated schools; and

2. very short time has passed between the start of the intervention and the date on which outcomes are
measured, this is particularly true for the school climate outcomes, which are expected to change in a
long time frame.

Further research is needed in the coming years, to understand if this set of actions can have an impact on
changing the school climate. This is even more challenging given the current COVID-19 pandemic, which has
caused schools to be closed for a long period of time in the years 2020 and 2021. A great part of the items
in regular surveys is bound to in-class teaching or the school environment and these surveys are therefore
currently conducted in limited form or not at all. Despite the limitations of this study, underlined above, it is good
to underline that enormous effort was put in place by the Czech managing authority to put together existing
administrative data about the universe of schools in the country, and to carry out surveys aiming at gathering
information about a dimension (school climate of inclusion) which is hardly measurable with the administrative
data in hands. This is a good starting point for any future evaluation involving schools. Finally, there are a
number of suggestions that can be proposed in view of the next programming period:

1. Collecting baseline data on relevant outcomes before the start of intervention on the treated and control
schools can help better measure the impact of the intervention using methods like difference-in-difference.

2. Finding a measurable outcome that is directly related to the expected results of the intervention can help

identify relevant effects.

3. Having better control of the design of the intervention, and of the selection of the units to treat, can help

in (cid:28)nding the most adequate evaluation method.

29

References

Basma, B. and Savage, R., `Teacher professional development and student literacy growth: A systematic review

and meta-analysis', Educational Psychology Review, Vol. 30, No 2, 2018, pp. 457(cid:21)481.

Blank, R. and de Las Alas, N., `Effects of teacher professional development on gains in student achievement.

washington, dc: Council of chief state school of(cid:28)cers'. 2009.

Blatchford, P., Bassett, P., Brown, P. and Webster, R.,

`The effect of support staff on pupil engagement and

individual attention', British Educational Research Journal, Vol. 35, No 5, 2009, pp. 661(cid:21)686.

Bowman, G. W. and Klopf, G. J., `Auxiliary school personnel: Their roles, training, and institutionalization, based
on a nationwide study of teacher-aides, teacher-assistants, family workers, and other auxiliary personnel in
education, conducted for the of(cid:28)ce of economic opportunity.', 1966.

Bressoux, P., `The effects of teachersâ€™ training on pupilsâ€™ achievement: the case of elementary schools in

france', School effectiveness and school improvement, Vol. 7, No 3, 1996, pp. 252(cid:21)279.

Caliendo, M. and Kopeinig, S., `Some practical guidance for the implementation of propensity score matching',

Journal of economic surveys, Vol. 22, No 1, 2008, pp. 31(cid:21)72.

Comer, J. and Haynes, N., `The home-school team: An emphasis on parent involvement'. 1997.

Cuffe, H. E., Waddell, G. R. and Bignell, W., `Can school sports reduce racial gaps in truancy and achievement?',

Economic Inquiry, Vol. 55, No 4, 2017, pp. 1966(cid:21)1985.

Didion, L., Toste, J. R. and Filderman, M. J., `Teacher professional development and student reading achievement:
A meta-analytic review of the effects', Journal of Research on Educational Effectiveness, Vol. 13, No 1, 2020,
pp. 29(cid:21)66.

Dumais, S. A., `Elementary school students'extracurricular activities: The effects of participation on achievement

and teachers'evaluations', Sociological Spectrum, Vol. 26, No 2, 2006, pp. 117(cid:21)147.

Eccles, J. S., Barber, B. L., Stone, M. and Hunt, J., `Extracurricular activities and adolescent development', Journal

of social issues, Vol. 59, No 4, 2003, pp. 865(cid:21)889.

El Nokali, N. E., Bachman, H. J. and Votruba-Drzal, E., `Parent involvement and childrenâ€™s academic and social

development in elementary school', Child development, Vol. 81, No 3, 2010, pp. 988(cid:21)1005.

Fan, X. and Chen, M., `Parental involvement and students' academic achievement: A meta-analysis', Educational

psychology review, Vol. 13, No 1, 2001, pp. 1(cid:21)22.

Felfe, C., Lechner, M. and Steinmayr, A.,

`Sports and child development', PloS one, Vol. 11, No 5, 2016, p.

e0151729.

Gregory, A., Allen, J. P., Mikami, A. Y., Hafen, C. A. and Pianta, R. C., `Effects of a professional development program
on behavioral engagement of students in middle and high school', Psychology in the Schools, Vol. 51, No 2,
2014, pp. 143(cid:21)163.

Grif(cid:28)th, J., `Relation of parental involvement, empowerment, and school traits to student academic performance',

The Journal of educational research, Vol. 90, No 1, 1996, pp. 33(cid:21)41.

Heckman, J. J., LaLonde, R. J. and Smith, J. A., `The economics and econometrics of active labor market programs'.

In `Handbook of labor economics', Vol. 3. Elsevier, 1999. pp. 1865(cid:21)2097.

Hille, A. and Schupp, J., `How learning a musical instrument affects the development of skills', Economics of

Education Review, Vol. 44, 2015, pp. 56(cid:21)82.

Hirano, K. and Imbens, G. W., `The propensity score with continuous treatments', Applied Bayesian modeling and

causal inference from incomplete-data perspectives, Vol. 226164, 2004, pp. 73(cid:21)84.

Hosseini, A. S. and Watt, A. P., `The effect of a teacher professional development in facilitating students creativity',

Educational Research and Reviews, Vol. 5, No 8, 2010, pp. 432(cid:21)438.

Lechner, M., `Long-run labour market and health effects of individual sports activities', Journal of health eco-

nomics, Vol. 28, No 4, 2009, pp. 839(cid:21)854.

Massoni, E., `Positive effects of extra curricular activities on students', Essai, Vol. 9, No 1, 2011, p. 27.

30

Meadows, A., `The impact of participation in extracurricular activities on elementary school students', Journal of

Interdisciplinary Undergraduate Research, Vol. 11, No 1, 2019, p. 2.

Navarro, F. M., `Learning support staff', , No 125, 2015.

. URL https://www.oecd-ilibrary.org/content/

paper/5jrnzm39w45l-en.

Pfeifer, C. and Corneliÿen, T., `The impact of participation in sports on educational attainmentâ€”new evidence

from germany', Economics of education review, Vol. 29, No 1, 2010, pp. 94(cid:21)103.

Pozo-Rico, T. and Sandoval, I.,

`Can academic achievement in primary school students be improved through
teacher training on emotional intelligence as a key academic competency?', Frontiers in psychology, Vol. 10,
2020, p. 2976.

Ransom, M. R. and Ransom, T., `Do high school sports build or reveal character? bounding causal estimates of

sports participation', Economics of Education Review, Vol. 64, 2018, pp. 75(cid:21)89.

Rees, D. I. and Sabia, J. J., `Sports participation and academic performance: Evidence from the national longitu-

dinal study of adolescent health', Economics of Education Review, Vol. 29, No 5, 2010, pp. 751(cid:21)759.

Rosenbaum, P. R. and Rubin, D. B., `The central role of the propensity score in observational studies for causal

effects', Biometrika, Vol. 70, No 1, 1983, pp. 41(cid:21)55.

Van Veen, K., Zwart, R. and Meirink, J., `What makes teacher professional development effective? a literature

review', Teacher learning that matters, 2012, pp. 23(cid:21)41.

Yee Mikami, A., Gregory, A., Allen, J. P., Pianta, R. C. and Lun, J., `Effects of a teacher professional development
intervention on peer relationships in secondary classrooms', School Psychology Review, Vol. 40, No 3, 2011,
pp. 367(cid:21)385.

Yoon, K. S., Duncan, T., Lee, S. W.-Y., Scarloss, B. and Shapley, K. L., `Reviewing the evidence on how teacher

professional development affects student achievement'. 2007.

31

Annexes

1 School need data

The list below reports all the questions related to each of the 8 relevant dimensions on school needs, measure
in 2015.

1. Inclusive education

(cid:22) The school can enroll all pupils without differentiation (including pupils with different cultural back-

grounds, social disadvantages, foreigners, pupils with SEN, etc.)

(cid:22) The school is barrier-free (it is barrier-free both external, ie access to the school and internal, ie

adaptation and equipment of classrooms and other school premises)

(cid:22) School can communicate with pupils, parents and educators, perceives their needs and systematically

(cid:22) develops school culture, safe and open school climate

(cid:22) The school allows teachers to establish relationships with local and regional schools of different
levels (joint discussions, sharing good practice, events for other schools or with other schools, etc.)

(cid:22) School management creates conditions for the implementation of inclusive principles in school
education (providing professional, material and (cid:28)nancial support, further education of pedagogical
staff; regular methodical meetings of members of the teaching staff, etc.)

(cid:22) The school adjusts the organization and course of teaching according to the needs of pupils with
special educational needs (eg provides group instruction for gifted pupils, groups may consist of
pupils of different grades, tutoring, etc.)

(cid:22) The school has a support system for pupils with special educational needs (it is equipped with

compensation / special aid, it uses the services of a teacher assistant, etc.)

(cid:22) Teachers can use special textbooks, aid and compensatory aids

(cid:22) The school can prepare all pupils for a smooth transition to the next level of education (between

ISCED stages)

(cid:22) Teachers work together to meet pupils' educational needs (eg through joint counseling on pupil

education, etc.)

(cid:22) Teachers are able to adapt the content of education appropriately, adjust forms and methods of
education and set different levels of dif(cid:28)culty in accordance with the pupils' speci(cid:28)cities and needs
so that the maximum potential of the pupil is achieved and exploited eg the curriculum for gifted
pupils is extended and deepened in accordance with their assumptions, creation of pupil portfolios,
etc.)

(cid:22) Teachers can cooperate in teaching with other pedagogical staff (pedagogical assistant, other ped-

agogue) and non-pedagogical staff (translator into Czech sign language, personal assistant)

(cid:22) School provides pupils with special educational needs with participation in activities beyond the scope

of school work aimed at developing pupils' skills, abilities and attitudes

(cid:22) Teachers implement pedagogical diagnostics of pupils, evaluate its results and in accordance with

them choose forms and methods of teaching, respectively. steps of further care of pupils

(cid:22) School provides Czech language lessons for foreigners

(cid:22) Teachers use descriptive verbal feedback in communication with the pupil, creating space for pupil's

self-evaluation and development of his / her motivation for education

(cid:22) The school emphasizes not only the building of the pupil's own success, but also the elimination of

barriers between people, it leads to belonging to classmates and other people, etc.

(cid:22) Teachers see creatively the differences between pupils as a source of experience and an opportunity

for self-development

(cid:22) The school teaches all pupils to be aware of their rights and obligations (guilt, punishment, justice,

the Convention on the Rights of the Child, etc.)

2. Reading literacy

(cid:22) School supports reading literacy development in its school curriculum (there are clearly stated set

goals)

32

(cid:22) Teachers of both lower and higher primary school (ISCED 1 and 2) develop their knowledge in reading

literacy teaching and use them in practice (further education courses, literature study etc)

(cid:22) Teachers of both lower and higher primary school (ISCED 1 and 2) use their knowledge in practice and
share examples of best practice in reading literacy teaching among themselves and with teachers
from other schools.

(cid:22) School supports elementary knowledge and skills, elementary work with text (from text understanding

to search for literature in school library according to needs of their pupils)

(cid:22) School supports higher level of complex reading literacy (from understanding the context and drawing

conclusions to sources comparison)

(cid:22) School supports individual work with pupils with extraordinary interest in literature, creative writing

etc

(cid:22) In the school there is a library (or a local library is used for this purpose) and it's available according

to needs and possibilities of the pupils

(cid:22) There are reading clubs in the school or other regular extracurricular activities to support and develop

reading literacy

(cid:22) There are other irregular extracurricular activities for pupils to support reading literacy and to increase

their motivation (project days, author readings, book fairs)

(cid:22) School informs about and collaborate on reading literacy development with parents (presentation of

school/local library services, reading clubs, open days, Christmas fairs etc)

(cid:22) There is supportive and challenging environment (book corners, notice-boards etc)

(cid:22) School possess enough of technical and other material equipment regarding reading literacy (audio-

visual equipment, tools for displaying pupils' success etc)

(cid:22) There are interactive media used in the school and ITC in developing reading literacy

(cid:22) School regularly buys up-to-date literature and multimedia for development of pupils on both lower

and higher primary level (ISCED 1 and 2) in reading literacy

3. Mathematics literacy

(cid:22) School supports mathematical literacy (numeracy) development in its school curriculum (there are

clearly stated set goals)

(cid:22) Teachers of both lower and higher primary school (ISCED 1 and 2) develop their knowledge in
mathematical literacy (numeracy) teaching and use them in practice (further education courses,
literature study etc)

(cid:22) Teachers of both lower and higher primary school (ISCED 1 and 2) use their knowledge in practice and
share examples of best practice in mathematical literacy (numeracy) teaching among themselves
and with teachers from other schools.

(cid:22) In the school, mathematical thinking of the pupils is supported (examples ofdaily use, situations

related to future professions or connected to natural laws etc)

(cid:22) School supports individual support to pupils with extraordinary interest in math

(cid:22) There are regular extracurricular activities in order to support development of mathematical liter-

acy/numeracy

(cid:22) There are irregular extracurricular activities to support mathematical literacy/numeracy and to in-

crease pupils' motivation in the school (project days etc)

(cid:22) School informs about and collaborate on reading mathematical literacy/numeracy development with

parents (presentation of extracurricular clubs, project days, open days etc)

(cid:22) School possess enough of technical and other material equipment regarding mathematical liter-

acy/numeracy

(cid:22) There are interactive media used in the school and ITC in developing mathematical literacy/numeracy

(cid:22) School regularly buys up-to-date literature and multimedia for development of pupils on both lower

and higher primary level (ISCED 1 and 2) in mathematical literacy/numeracy

4. Language literacy

(cid:22) School supports language literacy development in its school curriculum (there are clearly stated set

goals)

33

(cid:22) Teachers of both lower and higher primary school (ISCED 1 and 2) develop their knowledge in
language literacy teaching and use them in practice (further education courses, literature study etc)

(cid:22) Teachers of both lower and higher primary school (ISCED 1 and 2) use their knowledge in practice and
share examples of best practice in language literacy teaching among themselves and with teachers
from other schools.

(cid:22) Development of language literacy is included in other subjects (CLIL, use of authentic foreign texts

and audio tracks etc)

(cid:22) School develops pupils' understanding of life in different cultures and mediate the experience from

these cultures (using of movies, pictures, study stay abroad)

(cid:22) School supports language diversity in form of wide range of foreign languages study options

(cid:22) School creates enough of opportunities for development of language literacy (eTwinning, exchange

study stays etc)

(cid:22) In the school there is a library (or a local library is used for this purpose) containing literature in

foreign languages and it's available according to needs and possibilities of the pupils

(cid:22) School uses textbooks, literature in foreign languages and multimedia for development of pupils on

both lower and higher primary level (ISCED 1 and 2) for development of language literacy

(cid:22) There are regular extracurricular activities in order to support language literacy

(cid:22) There are irregular activities and events to support pupils in development of language literacy

)(interactive book fairs etc)

(cid:22) School informs about and collaborate on language literacy development with parents (presentation

of school/local library services, language clubs, open days, Christmas fairs etc)

(cid:22) There is supportive and challenging environment (language corners, notice-boards etc)

(cid:22) School possess enough of technical and other material equipment regarding language literacy

(cid:22) There are interactive media used in the school and ITC in developing language literacy

(cid:22) School regularly buys up-to-date literature and multimedia for development of pupils on both lower

and higher primary level (ISCED 1 and 2) in language literacy

(cid:22) School collaborates with native speaker(s) of foreign languages

5. Entrepreneurship

(cid:22) School supports key competences according to the national curriculum for primary schools.

(cid:22) Teachers increase their knowledge of how to support creativity development and they use the

knowledge in teaching (further education courses, literature study etc)

(cid:22) Teachers use their knowledge in practice and share examples of best practice in initiative and

creativity development teaching among themselves and with teachers from other schools.

(cid:22) School systematically teaches elements of initiative and creativity. The environment and teachers'

attitude is supportive in development of pupils' fantasy and initiative.

(cid:22) School teaches the pupils how to think critically, perceive the problems in their environment and how
to (cid:28)nd innovative solutions, bear the risks and plan and implement projects (project days, tasks with
multiple possible solutions etc)

(cid:22) Pupils actively take part in activities like (cid:28)ctive (cid:28)rms, Junior Achievement School of Entrepreneurship

etc or they actively participate in preparation and implementation of school projects.

(cid:22) School organizes consultations, debates and excursions as a form of support of entrepreneurship for

pupil and teachers.

(cid:22) School develops (cid:28)nancial literacy of pupils (they are taught the value of money, work with saving,

work with risks etc)

(cid:22) There is space dedicated for sharing of at school (idea workshops etc)

(cid:22) School supports key competences according to the national curriculum for primary schools.

6. Polytechnic education

(cid:22) Technical education is implemented according to the national curriculum for primary schools. Natural
sciences and environmental education are implemented according to the national curriculum for
primary schools.

34

(cid:22) Teachers increase their knowledge of how to support polytechnic education and they use the knowl-

edge in teaching (further education courses, literature study etc)

(cid:22) Teachers of both lower and higher primary school (ISCED 1 and 2) use their knowledge in practice
and share examples of best practice in polytechnic education teaching among themselves and with
teachers from other schools.

(cid:22) School has education plans concerning teaching of polytechnic education subjects (math, natural

and technical sciences etc)

(cid:22) School supports interest of pupils in polytechnic education and link it with daily life and future work

life

(cid:22) School possess enough teaching materials for polytechnic education

(cid:22) As a part of polytechnic education subjects, there are laboratory workshops, experiments, projects
etc supporting practical side of polytechnic education and developing manual dexterity of pupils

(cid:22) School cooperates with technical high schools and universities or colleges

(cid:22) Polytechnic subjects are taught also in foreign languages - CLIL

(cid:22) School supports individual/autonomous work of pupils in polytechnic education

(cid:22) School supports individual work with pupils with extraordinary interest in polytechnic education

(cid:22) There are clubs/regular workshops or other regular extracurricular activities to support development

of polytechnic education

(cid:22) There are other irregular extracurricular activities for pupils to support polytechnic education and to

increase their motivation (project days, excursions, discussions)

(cid:22) School actively support pre-school polytechnic education (cooperation with preschools, kindergartens)

(cid:22) School collaborates/cooperates with local companies/entrepreneurs

(cid:22) School informs parents about polytechnic education (presentation of school events, school clubs,

projectd ays, open days etc)

(cid:22) There is supportive and challenging environment at school regarding education for both pupils and
teachers (also physical places like notice borads or place for displaying information or products,
outcomes of projects etc)

(cid:22) School possess technical and material resources for support and development of polytechnic educa-

tion )classrooms for chemistry, physics, biology etc)

(cid:22) School uses ICT for development of polytechnic education

7. ICT competence of teachers

(cid:22) Teaching staff use ICT classroom or school computers in teaching (not only teaching of ICT)

(cid:22) Teaching staff use mobile ICT in teaching (laptops, netbooks, tablets, smartphones etc)

(cid:22) Teaching staff use BYOD for pupils' projects (allowing the pupils to use their own ICT devices)

(cid:22) Teaching staff use mobile ICT in teaching in the (cid:28)eld and in project education

(cid:22) Teaching staff use free, safe and open internet sources

(cid:22) Teaching staff know free sources on the internet concerning their subjects

(cid:22) Teaching staff can systematically develop awareness about internet security and safety and critical

thinking concerning the internet content

8. Support of social and civic competences

(cid:22) In the school there is cultivated culture of open communication between all stakeholders

(cid:22) In the school there is cultivated cultural awareness and communication (creative expression of ideas,

experience using theatre, music, literature and visual arts)

(cid:22) School builds ethical values, there are rules of social behaviour and these rules are kept

(cid:22) Pupils are taught constructive discussion

(cid:22) Within school, mutual cooperation and collaboration of teaching staff, parents and pupils is cultivated

(cid:22) School develops pupils' skills in self-evaluation and self-re(cid:29)ection

(cid:22) School motivates pupils to life long learning

(cid:22) School develops ability to ask for help and to offer help to others

35

(cid:22) School teaches how to safely use ICT

(cid:22) School develops ability of pupils for learning (autonomously, in groups etc)

(cid:22) Pupils are taught how to takÃ© part in social and work life

(cid:22) School prepares pupils for active life in democratic society, develops civic competences (student

councils, elections, self-government etc)

36

2 Details on the construction of the school outcome

In this section we report the details of the construction of the factors derived from the survey on school climate
outcomes. Table A1 reports all the questions used to build the relevant outcomes. For each of the set of
questions we run a factor analysis using principal components factors. The results of the various factor analysis
are reported in Table A2. While in most of the questions, we obtain only one factor, other questions are split into
2 or 3 factors. In these cases, we assigned each item to the factor with high factor loading (highlighted in red
in the Table). After the factor analysis, we checked using Cronbach's Alpha the reliability of the factors found.
Higher Alpha are associated with higher reliability, meaning that it makes sense to keep those items together
since they really capture the same latent component.

The (cid:28)rst 4 sets of questions were inspired by TALIS, and they refer to 4 different dimensions: (cid:16)Participation
among stakeholders(cid:17), (cid:16)Teacher-students relationship(cid:17), (cid:16)Mutual respect(cid:17) and (cid:16)School delinquency and violence(cid:17).
Each question is composed of a number of items. So for example the question related to the (cid:16)Participation
among stakeholders(cid:17) is composed of 5 different items, to which the teachers have to reply using a liker scale of
agreement, ranging from de(cid:28)nitely disagree to de(cid:28)nitely agree. As reported in Table A2, the factor analysis of the
(cid:28)rst 3 variables, lead to only one factor, while 3 factors where found for the dimension (cid:16)School delinquency and
violence(cid:17). We call the three factors as follows: (cid:16)School delinquency and violence 1 - minor issues (absenteeism,
cheating, vandalism)(cid:17), (cid:16)School delinquency and violence 2 - violence against other students and teachers(cid:17),
(cid:16)School delinquency and violence 3 - substance abuse(cid:17).

The second sets of questions are normally used to build the (cid:16)Index of social inclusion(cid:17), which is composed

by 3 dimensions, each of them split in to two sub dimensions.:

(A) Creating inclusive cultures

(1) Building community

(2) Establishing inclusive values

(B) Producing inclusive policies

(a) Developing the school for all

(b) Organizing support for diversity

(C) Evolving inclusive practices

(a) Orchestrating learning

(b) Mobilizing resources

The factor analysis of the various item of the 6 sets of questions, con(cid:28)rm the 6 underlying dimensions of

the social inclusion index.

The factor analysis for the set of questions related to (cid:16)Socially disadvantaged and Roma children: teachers'
feelings(cid:17) led to 3 factors. For factor 3, composed of items 7 and 8, the Alpha was very low (Alpha 0.2622). This
meaning that it is pointless trying to use these items to capture 1 underlying dimension, so we exclude the last
factor associated with this question and focus only on the (cid:28)rst two, which we called: (cid:16)Socially disadvantaged and
Roma children: teachers' feelings 1 - negative feelings toward children(cid:17), and (cid:16)Socially disadvantaged and Roma
children: teachers' feelings 2 - teachers feel prepared(cid:17). The two sets of questions on Socially disadvantaged
children and Roma children both led to 3 factors each, which we called (cid:16)1 - are bad for the class(cid:17); (cid:16)2 - are
discriminated(cid:17) and "3 - create more work for the teachers(cid:17). Finally, the last set of questions refers to teachers'
con(cid:28)dence in working with children form different backgrounds and only one factor was identi(cid:28)ed.

The (cid:28)nal list of outcome variables used in the analysis is the following:

1. Participation among stakeholders

2. Teacher students relationship

3. Mutual respect

4. School delinquency and violence 1 - minor issues (absenteeism, cheating, vandalism)

5. School delinquency and violence 2 - violence against other students and teachers

6. School delinquency and violence 3 - substance abuse

7. Building community (Index of social inclusion)

8. Establishing inclusive values (Index of social inclusion)

37

9. Developing the school for all (Index of social inclusion)

10. Organizing support for diversity (Index of social inclusion)

11. Orchestrating learning (Index of social inclusion)

12. Mobilizing resources (Index of social inclusion)

13. Socially disadvantaged and Roma children 1 - negative feelings toward children

14. Socially disadvantaged and Roma children 2 - teacher feel prepared

15. Socially disadvantaged 1 - are bad for the class

16. Socially disadvantaged 2 - are discriminated

17. Socially disadvantaged 3 - create more work for the teachers

18. Roma 1 - are bad for the class

19. Roma 2 - are discriminated

20. Roma 3 - create more work for the teachers

21. Teacher's con(cid:28)dence.

Higher values of variables 4,5,6,13, 15,16,17,18,19 and 20 indicates a negative (cid:16)climate(cid:17), so in order to make
easier the interpretation of the results and descriptive tables, these variables have been reversed, so that higher
values indicate positive (cid:16)climate(cid:17) for all the 21 variables.

38

Table A1: Items used to build the outcomes related to school climate

Participation among stakeholders: source TALIS 2013

To what extent do you agree or disagree with
the following statements for your school?

1. de(cid:28)nitely disagree

2. rather disagree

3. rather agree

4. de(cid:28)nitely agree

1. This school provides staff with opportunities to actively
participate in school decisions
2. This school provides parents or guardians with opportunities
to actively participate in school decisions
3. This school provides pupils with opportunities to actively
participate in school decisions
4. This school has a culture of shared responsibility for school
issues
5. There is a collaborative school culture which is characterized
by mutual support

To what extent do you agree or disagree with
the following statements about what is
happening at your school?

Teacher student relationship: source TALIS 2013
1.
each other
2. Most teachers in this school believe that the pupilsâ€™ well-
being is important
3. Most teachers in this school are interested in what pupils
have to say
4. If a pupil from this school needs extra assistance, the school
provides it

In this school, teachers and pupils usually get on well with

1. de(cid:28)nitely disagree

2. rather disagree

3. rather agree

4. de(cid:28)nitely agree

Mutual respect: source TALIS 2013

To what extent do you agree or disagree with
the following statements for your school?

1. de(cid:28)nitely disagree

2. rather disagree

3. rather agree

4. de(cid:28)nitely agree

The school staff share a common set of beliefs about

1.
schooling/learning
2. There is a high level of co-operation between the school and
the local community
3. School staff have an open discussion about dif(cid:28)culties
4. There is mutual respect for colleagues' ideas
5. There is a culture of sharing success
6. The relationships between teachers and pupils are good

School delinquency and violence: source TALIS 2013

How often do the following situations occur
with pupils in your school?

1. Never

2. Rarely

3. Monthly

4. Weekly

5. Daily

Intimidation or verbal abuse among pupils (or other forms

1. Arriving late at school
2. Absenteeism (i.e. unjusti(cid:28)ed absences)
3. Cheating
4. Vandalism and theft
5.
of non-physical bullying)
6. Physical injury caused by violence among pupils
7. Intimidation or verbal abuse of teachers or staff
8. Use/possession of drugs
9. Use/possession of alcohol
10. Smoking
11. Cyberbullying
12. Verbal aggression against the teacher
13. Physical aggression against the teacher

39

Index of social Inclusion
Indicate to what extent do you agree or disagree with the following statements if you relate them to your school:
1) de(cid:28)nitely disagree
2) rather disagree
3) rather agree
4)de(cid:28)nitely agree

A) Creating inclusive cultures
1) BUILDING COMMUNITY

A) Creating inclusive cultures
2) ESTABLISHING INCLUSIVE
VALUES

B) Producing inclusive policies
1)DEVELOPING THE SCHOOL
FOR ALL

B) Producing inclusive policies
2)ORGANISING SUPPORT FOR
DIVERSITY

1. Everyone is made to feel welcome
2. Pupils help each other
3. Staff collaborate with each other
4. Staff and pupils treat one another with respect
5. There is partnership between staff and parents/carers
6. Staff and governors work well together
7. The school programmatically, actively cooperates with sub-
jects in the neighborhood, meetings take place, joint events are
organized, the school participates in life in the neighborhood, it
is open in the (cid:28)eld of social relations
1. There are high expectations for all pupils
2. Staff, governors pupils and parents/carers share a philosophy
of inclusion
3. At school, equal approach to all pupils is applied regardless
of race, religion, gender, disability, etc.
4 School staff and pupils treat each other with respect and
respect each other's role in the school
5. School staff strive to remove barriers to learning and in-
volvement in all aspects of school life
6. The school strives to minimize all forms of discrimination
1. The appointment and promotion of school staff is fair
2. All new school staff are helped to settle into the school
3. The school seeks to admit all pupils from the locality
4. The school makes its buildings physically accessible to all
people
5. All new pupils are helped to to settle into the school
6. The school promotes diversity, pupils in the school are not
divided into classes or groups according to achievement or
discipline problems
1. All forms of support (to pupils and to teachers) are coordi-
nated
2. Staff development activities help school staff to respond to
pupils' diversity
3. The organisational structure of the school enables changes
towards enhancing quality of the school's culture/environment
4. The most open information strategies are developed and
operated at school, i.e. all those who live in the school have
complete and comprehensible information concerning them in
time
5. Support for those for whom Czech is not a mother language
is coordinated with support for learning
6. Communication at school is conducted in a manner accept-
able to all communicating parties
7. Bullying is minimised

40

C) Evolving inclusive practices
1) ORCHESTRATING LEARNING

C) Evolving inclusive practices
2) MOBILISING RESOURCES

1. All pupils are motivated and encouraged to expect to achieve
very good results
2. All pupils are encouraged to participate in the lessons
3. Lessons develop an understanding of differences
4. Pupils are actively involved in their own learning process
5. Pupils learn collaboratively
6. Assessment contributes to achievements of all pupils
7. The discipline in the class is based on mutual respect
8. Teachers teach, plan and review in partnership
9. Teacher assistants support the learning and participation of
all pupils
10. Homework contributes to the learning of all pupils
11. All pupils take part in extracurricular activities organized by
the school, such as trips
1. Diversity is seen as an enriching learning opportunity, not as
a pupil problem
2. The expertise of school staff is fully utilised
3. Teachers are ready to explain why they use certain teaching
methods and how these methods contribute most effectively
to achieving educational goals
4. Material equipment, exhibited materials and aids allow self-
learning and are used in teaching according to the needs of the
pupils

Socially disadvantaged children and Roma: Teachers' feelings

Read the statement carefully and indicate the
degree of agreement or disagreement with the
statement on the appropriate scale.

1. De(cid:28)nitely disagree

2. Rather disagree

3. Don't know / can't judge

4. Rather agree

5. De(cid:28)nitely agree

I believe that the presence of a socially disadvantaged

1.
pupil(s) signi(cid:28)cantly increases my workload
2.
I am convinced that my preparation for the education of
socially disadvantaged pupils is suf(cid:28)cient to achieve the nec-
essary results
3. I am convinced that my preparation for communication with
families of socially disadvantaged pupils is suf(cid:28)cient to achieve
the necessary results
4. Separate classes should be set up for Roma because they
do not manage to teach in mainstream classes
5. Most socially disadvantaged pupils are not interested in
studying
6.
socially disadvantaged pupils
7. Most parents of socially disadvantaged pupils know how to
help their children with schoolwork at home
8. Working with socially disadvantaged pupils is a challenge
for me
9. Socially disadvantaged pupils are less independent in their
lesson than their peers without social disadvantage

I am concerned about direct contact with the families of

41

Socially disadvantaged children
Some teachers say that the presence of a
socially disadvantaged pupil makes their work
more dif(cid:28)cult. What are the pitfalls of the
socially disadvantaged pupils in your classroom?
Indicate if you agree or disagree with the
following statements.

1. De(cid:28)nitely disagree

2. Rather disagree

3. Don't know / can't judge

4. Rather agree

5. De(cid:28)nitely agree

Roma children
Some teachers say that the presence of a
Roma pupil makes their work more dif(cid:28)cult. In
your opinion, what are the pitfalls of Roma
pupils in the classroom? Indicate if you agree or
disagree with the following statements.

1. De(cid:28)nitely disagree

2. Rather disagree

3. Don't know / can't judge

4. Rather agree

5. De(cid:28)nitely agree

Teacher con(cid:28)dence
How con(cid:28)dent do you feel when working with
children who:

1. Very unsure

2. Rather unsure

3. Rather sure

4. Very sure

1. Socially disadvantaged pupils disrupt classes
2. Socially disadvantaged pupils delay classes
3. Socially disadvantaged pupils have a bad in(cid:29)uence on their
classmates
4. Higher workload for the teacher
5. Speci(cid:28)c teacher training required
6. Necessary individual approach of the teacher to socially
disadvantaged pupils
7. Disturbed class climate
8. Not accepting of socially disadvantaged pupils by their class-
mates
9. Higher rate of discipline problems in the classroom
10. Exclusion of socially disadvantaged pupils from the collec-
tive
11. Discrimination by classmates without social disadvantage
and the resulting problems
12. Demotivation of pupils without social disadvantage
13. Higher rate of bullying in the classroom

1. Roma pupils disrupt classes
2. Roma pupils delay classes
3. Roma pupils have a bad in(cid:29)uence on their classmates
4. Higher workload for the teacher
5. Speci(cid:28)c teacher training required
6. Necessary individual approach of the teacher to Roma pupils
7. Disturbed class climate
8. Not accepting of Roma pupils by their classmates
9. Higher rate of discipline problems in the classroom
10. Exclusion of Roma pupils from the collective
11. Discrimination by non-Roma classmates and the resulting
problems
12. Demotivation of non-Roma pupils
13. Higher rate of bullying in the classroom

1. are not disadvantaged
2. are socially disadvantaged
3. are Roma
4. are foreigners and speak Czech
5. are foreigners and do not speak Czech
6. are lightly mentally handicapped
7. are lightly physically handicapped
8. are extremely gifted

42

Participation among stakeholders
Factor 1 Uniqueness
Questions
0.8265
1
0.7872
2
0.7718
3
0.7981
4
0.7619
5
0.8486
Alhpa

0.3169
0.3803
0.4043
0.3630
0.4195

Table A2: Factor loadings

Index of inclusion: school for all
Questions
1
2
3
4
5
6
Alpha

Factor 1 Uniqueness
0.7026
0.7887
0.6639
0.5805
0.8116
0.6898
0.7897

0.5064
0.378
0.5593
0.6631
0.3412
0.5242

Teacher student relationship
Questions
1
2
3
4
Alpha

Factor1
0.7763
0.8765
0.8594
0.8107
0.8511

Uniqueness
0.3974
0.2318
0.2614
0.3428

Mutual respect
Questions
1
2
3
4
5
6
Alpha

Factor1
0.7602
0.6500
0.8133
0.8537
0.8263
0.7225
0.8635

Teacher con(cid:28)dence
Questions
1
2
3
4
5
6
7
8
Alpha

Factor1
0.6755
0.8088
0.7468
0.7791
0.6106
0.6332
0.7417
0.6539
0.8517

Uniqueness
0.4222
0.5775
0.3385
0.2713
0.3173
0.4780

Uniqueness
0.5437
0.3458
0.4423
0.393
0.6271
0.599
0.4498
0.5724

Index of inclusion: building community
Questions
1
2
3
4
5
6
7
Alpha

Uniqueness
0.4657
0.501
0.4445
0.3983
0.4856
0.5293
0.6039

Factor1
0.731
0.7064
0.7453
0.7757
0.7172
0.6861
0.6294
0.8355

Index of inclusion: organizing support for diversity
Questions
1
2
3
4
5
6
7
Alpha

Uniqueness
0.3505
0.3291
0.2959
0.3853
0.6446
0.357
0.6103

Factor1
0.8059
0.8191
0.8391
0.784
0.5962
0.8019
0.6242
0.871

Index of inclusion: Orchestrating learning
Questions
1
2
3
4
5
6
7
8
9
10
11
Alpha

Uniqueness
0.4939
0.4033
0.3908
0.3597
0.4292
0.4137
0.508
0.5194
0.643
0.7035
0.7344

Factor1
0.7114
0.7724
0.7805
0.8002
0.7555
0.7657
0.7014
0.6933
0.5975
0.5446
0.5153
0.8871

Index of inclusion: mobilizing resources
Questions
1
2
3
4
Alpha

Uniqueness
0.4506
0.324
0.3031
0.3779

Factor1
0.7412
0.8222
0.8348
0.7887
0.8068

Index of inclusion: Establishing inclusive values
Variable
1
2
3
4
5
6
Alpha

Uniqueness
0.9045
0.8941
0.526
0.5785
0.4587
0.4538

Factor1
0.3091
0.3255
0.6884
0.6492
0.7357
0.7391
0.8071

43

School delinquency and violence
Factor1
Questions
0.6887
1
0.6843
2
0.7255
3
0.5917
4
0.5748
5
0.3509
6
0.2257
7
0.0898
8
0.1021
9
0.3850
10
0.4481
11
0.3377
12
-0.1266
13
0.7245
Alpha

Factor2
0.0117
0.1719
0.1936
0.3193
0.4584
0.5832
0.7665
0.2276
0.1527
0.1144
0.2157
0.6873
0.7042
0.7184

Factor3 Uniqueness
0.1115
0.2059
0.1692
0.2212
0.0844
0.1438
0.2071
0.8339
0.8615
0.6452
0.4176
0.2130
0.2548
0.7402

0.5131
0.4598
0.4075
0.4989
0.4524
0.5161
0.3187
0.2448
0.2240
0.4224
0.5783
0.3682
0.4232

Socially disadvantaged children and Roma: Teachers' feelings
Questions
1
2
3
4
5
6
7
8
9
Alpha

Factor3 Uniqueness
-0.1792
0.0277
0.0376
0.2436
-0.1413
0.2378
0.8641
0.4927
-0.1287
0.262

Factor1
0.5958
-0.0049
-0.0535
0.7023
0.7292
0.6408
0.0691
-0.3089
0.596
0.675

Factor2
-0.03
0.8902
0.8986
-0.0741
0.034
-0.2617
0.0314
0.2136
0.107
0.7937

0.612
0.2067
0.1882
0.442
0.4472
0.4644
0.2477
0.6162
0.6167

Roma children
Questions
1
2
3
4
5
6
7
8
9
10
11
12
13
Alpha

Factor1
0.8078
0.6905
0.8109
0.4138
0.2277
0.2008
0.7555
0.264
0.7062
0.1817
0.1447
0.4754
0.5777
0.907

Factor2
0.1485
0.1837
0.2354
0.1368
0.2125
0.1149
0.3103
0.7952
0.2278
0.8662
0.8651
0.5258
0.5344
0.8487

Socially disadvantaged children
Factor1
Questions
0.8277
1
0.6738
2
0.8177
3
0.3162
4
0.2288
5
0.0656
6
0.74
7
0.2509
8
0.6964
9
0.1842
10
0.1832
11
0.534
12
0.5358
13
0.898
Alpha

Factor2
0.1571
0.1401
0.2452
0.1015
0.104
0.1345
0.3718
0.8197
0.333
0.8791
0.8736
0.4854
0.5928
0.8614

Factor3 Uniqueness
0.2757
0.4044
0.1964
0.7625
0.8408
0.8432
0.2916
0.1892
0.3389
0.1546
0.1102
0.1749
0.1506
0.853

0.2494
0.3258
0.2484
0.2287
0.1961
0.2355
0.2479
0.2622
0.3345
0.1928
0.2185
0.4669
0.3579

Factor3 Uniqueness
0.2326
0.4284
0.1385
0.8304
0.8544
0.8486
0.2318
0.1433
0.2556
0.1241
0.0808
0.1415
0.0943
0.853

0.2361
0.3429
0.2521
0.2002
0.2069
0.2574
0.2604
0.2446
0.3388
0.1779
0.1968
0.4592
0.3525
44

3 Generalized propensity score matching

Usually, propensity score analysis focuses on the cases in which the treatment is binary but we decided to
implement this method in a setting with a continuous treatment. It becomes possible by generalising the binary
treatment propensity score and building the generalised propensity score (GPS).

It maintains some of the attractive properties of the usual propensity score.

The Generalized Propensity Score can be de(cid:28)ned as the conditional density of the treatment given the
Indeed, the key
covariates.
assumption is the assumption of weak unconfoundedness, which is a generalisation of the unconfoundedness
assumption of the binary treatment. This assumption is (cid:16)weak (cid:17)because we do not require joint independence of
all potential outcomes. Also the balancing property of the GPS is similar to the one of the standard propensity
score: within strata with the same generalised propensity score, the selection into treatment does not depend
on the value of the covariates. In combination with the unconfoundedness assumption, it implies that, given the
GPS, assignment into treatment is unconfounded. Under these circumstances, GPS can be used to eliminate any
biases associated with differences in the covariates.

With the Generalised Propensity Score Method, we are interested in (cid:28)nding the average dose-response
function. Estimating the dose-response function is just the last step of the implementation of this method. It
must be preceded by two steps: (cid:28)rst, we need to estimate the score, and then we estimate the conditional
expectation of the outcome as a function of the continuous treatment variable and of the GPS. By implementing
this (cid:28)rst step, we verify if each treatment variable satis(cid:28)es two assumptions: the (cid:28)rst one is the assumption
of Normality of the distribution conditional on the covariates, while the second one is the standard balancing
property of the PSM. For what concerns the Normality assumption, our aim is to turn the distribution of the
variable into a Normal distribution, so (cid:28)rst we make a transposition. Then we assess the validity of the Normal
distribution model by using a goodness-of-(cid:28)t test. Only if the Normality assumption is satis(cid:28)ed we can go on
verifying the balancing property. We test for it using an algorithm suggested by Hirano and Imbens (2004). This
test assesses whether and to what extent the balancing property is supported by the data.

We perform this analysis for some of the variables described in subsection "Description of the treatment
variable". Out of 6 variables tested 13, only 1 of them satisfy both the Normality assumption and the balancing
property. (Average hours of training in reading done by the pedagogical staff). Therefore we decided to adopt a
coherent approach for all the 6 variables, doing what it is described in Section 7.

13The details of how we built these variables is reported in Section 3.4 "Description of the treatment variable"

45

B Additional tables

46

Table B1: Sample selection

Share of pupils with adjusted educational plan - 2016

Share of pupils with special educational needs - 2016

Share of gifted pupils - 2016

Total number of pupupils - 2016

Proportion of pupils repeating the school year - 2016

Number of teachers FTE - 2016

Share of girls - 2016

School need: Inclusive education

School need: Reading literacy

School need: Mathematics literacy

School need: Entrepreneurship

School need: ICT competence of teachers

School need: Support of social and civic competence

School need: Polytechnic education

School need: Language literacy

Logarithm of population size

Region: Hlavní m¥sto Praha- ref category
Region: Jiho£eský

Region: Jihomoravský

Region: Karlovarský

Region: Vyso£ina

Region: Královéhradecký

Region: Liberecký

Region: Moravskoslezský

Region: Olomoucký

Region: Pardubický

Region: Plze¬ský

Region: St°edo£eský

Region: Ústecký

Region: Zlínský

Constant

Observations

(1)
School outcomes
-0.035
(0.166)
0.063
(0.059)
0.307
(1.149)
0.001∗
(0.000)
0.248
(0.338)
-0.002
(0.003)
0.218∗
(0.105)
0.000
(0.002)
-0.004∗
(0.002)
-0.000
(0.003)
0.002
(0.003)
0.000
(0.003)
0.000
(0.002)
0.003
(0.001)
-0.002
(0.002)
0.020∗∗∗
(0.006)

(2)
Cermat exams
-0.739∗∗∗
(0.109)
-0.297∗∗∗
(0.039)
-0.938
(0.758)
-0.002∗∗∗
(0.000)
-0.022
(0.223)
0.067∗∗∗
(0.002)
-0.102
(0.069)
-0.002∗
(0.001)
-0.006∗∗∗
(0.001)
-0.002
(0.002)
-0.006∗∗
(0.002)
-0.002
(0.002)
-0.004∗
(0.002)
0.009∗∗∗
(0.001)
0.005∗∗∗
(0.001)
0.005
(0.004)

0.156∗∗
(0.052)
0.103∗
(0.046)
0.176∗∗
(0.063)
0.171∗∗
(0.053)
0.125∗
(0.052)
0.185∗∗∗
(0.054)
0.144∗∗
(0.045)
0.139∗∗
(0.051)
0.186∗∗∗
(0.053)
0.231∗∗∗
(0.054)
0.156∗∗
(0.048)
0.122∗
(0.050)
0.239∗∗∗
(0.052)
-0.062
(0.113)
4028

0.060
(0.035)
0.007
(0.030)
0.156∗∗∗
(0.041)
0.059
(0.035)
0.032
(0.034)
0.037
(0.035)
0.032
(0.030)
0.055
(0.034)
0.048
(0.035)
0.056
(0.035)
0.048
(0.032)
0.078∗
(0.033)
0.014
(0.034)
0.201∗∗
(0.074)
4028

Note. The table reports the differences between the original sample and the two samples kept for
47
the analysis. The coef(cid:28)cients are from two linear probability models estimating the probability of
begin in the two samples. Standard errors in parentheses. ∗ p < 0.05, ∗∗ p < 0.01, ∗∗∗ p < 0.001

Table B2: Descriptive statistics in treated and control schools- School climate outcomes sample

Share of pupils with adjusted educational plan - 2016

Control schools
0.035

Treated schools
0.016

Share of pupils with special educational needs - 2016

0.210

Share of gifted pupils - 2016

Total number of pupupils - 2016

Proportion of pupils repeating the school year - 2016

Number of teachers FTE - 2016

Share of girls - 2016

School need: Inclusive education

School need: Reading literacy

School need: Mathematics literacy

School need: Entrepreneurship

School need: ICT competence of teachers

School need: Support of social and civic competence

School need: Polytechnic education

School need: Language literacy

Region: Hlavní m¥sto Praha

Region: Jiho£eský

Region: Jihomoravský

Region: Karlovarský

Region: Vyso£ina

Region: Královéhradecký

Region: Liberecký

Region: Moravskoslezský

Region: Olomoucký

Region: Pardubický

Region: Plze¬ský

Region: St°edo£eský

Region: Ústecký

Region: Zlínský

Logarithm of population size

Observations

0.002

179.0

0.008

12.67

0.462

49.39

34.32

25.55

23.45

17.45

34.22

43.87

37.12

0.045

0.061

0.093

0.032

0.081

0.077

0.028

0.069

0.041

0.057

0.130

0.138

0.057

0.093

8.522

247

0.135

0.001

296.8

0.009

19.21

0.481

48.79

34.61

25.77

23.33

17.10

33.99

44.72

38.63

0.069

0.062

0.101

0.029

0.057

0.054

0.058

0.125

0.072

0.062

0.049

0.129

0.069

0.065

9.123

1340

Difference
0.019∗∗∗
(0.005)
0.074∗∗∗
(0.017)
0.001
(0.001)
-117.88∗∗∗
(14.960)
-0.001
(0.002)
-6.547∗∗∗
(0.819)
-0.018∗∗∗
(0.005)
0.594
(0.487)
-0.290
(0.403)
-0.220
(0.309)
0.120
(0.284)
0.344
(0.183)
0.234
(0.309)
-0.849
(0.563)
-1.509∗∗
(0.489)
-0.025
(0.017)
-0.001
(0.017)
-0.008
(0.021)
0.003
(0.012)
0.024
(0.017)
0.023
(0.016)
-0.029
(0.015)
-0.056∗
(0.022)
-0.032
(0.017)
-0.005
(0.017)
0.080∗∗∗
(0.017)
0.009
(0.023)
-0.012
(0.017)
0.028
(0.018)
-0.601∗∗∗
(0.155)

Note. The table reports the mean values of the outcomes in control (column (1)) and treated (column(2)) schools,
and their difference (column(3)). Standard errors in parentheses. ∗ p < 0.05, ∗∗ p < 0.01, ∗∗∗ p < 0.001

48

Table B3: Descriptive statistics in treated and control schools- Cermat sample

Share of pupils with adjusted educational plan - 2016

Control schools
0.021

Treated schools
0.007

Share of pupils with special educational needs - 2016

0.169

Share of gifted pupils - 2016

Total number of pupupils - 2016

Proportion of pupils repeating the school year - 2016

Number of teachers FTE - 2016

Share of girls - 2016

School need: Inclusive education

School need: Reading literacy

School need: Mathematics literacy

School need: Entrepreneurship

School need: ICT competence of teachers

School need: Support of social and civic competence

School need: Polytechnic education

School need: Language literacy

Region: Hlavní m¥sto Praha

Region: Jiho£eský

Region: Jihomoravský

Region: Karlovarský

Region: Vyso£ina

Region: Královéhradecký

Region: Liberecký

Region: Moravskoslezský

Region: Olomoucký

Region: Pardubický

Region: Plze¬ský

Region: St°edo£eský

Region: Ústecký

Region: Zlínský

Logarithm of population size

Observations

0.001

278.6

0.009

19.30

0.462

49.43

34.79

26.14

23.56

17.36

34.11

45.75

39.40

0.080

0.088

0.106

0.040

0.054

0.047

0.033

0.083

0.047

0.038

0.113

0.121

0.097

0.052

9.206

426

0.107

0.001

347.9

0.007

22.37

0.482

48.34

34.68

25.94

23.26

17.10

33.90

45.51

39.66

0.090

0.061

0.108

0.029

0.057

0.059

0.048

0.124

0.072

0.057

0.041

0.118

0.078

0.058

9.420

2027

Difference
0.014∗∗∗
(0.002)
0.062∗∗∗
(0.010)
0.000
(0.000)
-69.32∗∗∗
(10.595)
0.002
(0.001)
-3.065∗∗∗
(0.550)
-0.020∗∗∗
(0.003)
1.083∗∗
(0.343)
0.102
(0.296)
0.199
(0.235)
0.294
(0.212)
0.258
(0.137)
0.212
(0.224)
0.236
(0.396)
-0.259
(0.360)
-0.009
(0.015)
0.027∗
(0.013)
-0.001
(0.017)
0.011
(0.009)
-0.003
(0.012)
-0.012
(0.012)
-0.015
(0.011)
-0.041∗
(0.017)
-0.024
(0.013)
-0.019
(0.012)
0.072∗∗∗
(0.012)
0.002
(0.017)
0.019
(0.015)
-0.006
(0.012)
-0.214
(0.121)

Note. The table reports the mean values of the outcomes in control (column (1)) and treated (column(2)) schools,
and their difference (column(3)). Standard errors in parentheses. ∗ p < 0.05, ∗∗ p < 0.01, ∗∗∗ p < 0.001

49

Table B4: Probability of being a treated school

Share of pupils with adjusted educational plan - 2016

Share of pupils with special educational needs - 2016

Share of gifted pupils - 2016

Total number of pupupils - 2016

Proportion of pupils repeating the school year - 2016

Number of teachers FTE - 2016

Share of girls - 2016

School need: Inclusive education

School need: Reading literacy

School need: Mathematics literacy

School need: Entrepreneurship

School need: ICT competence of teachers

School need: Support of social and civic competence

School need: Polytechnic education

School need: Language literacy

Region: Hlavní m¥sto Praha- ref category
Region: Jiho£eský

Region: Jihomoravský

Region: Karlovarský

Region: Vyso£ina

Region: Královéhradecký

Region: Liberecký

Region: Moravskoslezský

Region: Olomoucký

Region: Pardubický

Region: Plze¬ský

Region: St°edo£eský

Region : Ústecký

Region: Zlínský

Logarithm of population size

Constant

Observations

(1)
Whole sample
-0.529
(0.445)
-0.509∗∗
(0.162)
-0.092
(3.192)
0.000
(0.001)
0.846
(0.935)
0.024∗
(0.010)
0.673∗
(0.289)
0.003
(0.005)
0.005
(0.006)
-0.014
(0.008)
-0.001
(0.008)
-0.030∗∗
(0.010)
0.000
(0.007)
0.001
(0.004)
0.002
(0.005)

(2)
School outcomes
-0.946
(0.827)
-0.470
(0.311)
-5.894
(4.600)
-0.002
(0.001)
5.023∗
(2.151)
0.059∗∗
(0.021)
1.143∗
(0.570)
0.009
(0.008)
0.014
(0.011)
-0.011
(0.014)
-0.008
(0.015)
-0.031
(0.018)
0.003
(0.014)
0.001
(0.007)
-0.004
(0.009)

(3)
Cermat outcomes
-1.314
(0.823)
-0.085
(0.260)
-1.906
(3.927)
0.001
(0.001)
0.031
(1.782)
-0.007
(0.014)
1.344∗∗
(0.500)
-0.004
(0.007)
0.010
(0.008)
-0.016
(0.011)
-0.004
(0.012)
-0.011
(0.014)
0.007
(0.011)
0.001
(0.006)
-0.006
(0.007)

-0.183
(0.158)
-0.027
(0.143)
-0.106
(0.187)
0.033
(0.163)
-0.007
(0.158)
0.173
(0.166)
0.381∗∗
(0.144)
0.169
(0.158)
0.113
(0.163)
-0.695∗∗∗
(0.159)
-0.135
(0.148)
-0.047
(0.154)
-0.138
(0.157)
-0.000
(0.018)
0.581
(0.331)
4028

-0.190
(0.285)
-0.118
(0.259)
-0.232
(0.319)
-0.372
(0.282)
-0.210
(0.283)
0.214
(0.302)
0.083
(0.248)
0.127
(0.292)
-0.143
(0.287)
-0.837∗∗
(0.270)
-0.147
(0.266)
-0.169
(0.276)
-0.369
(0.272)
-0.021
(0.031)
0.400
(0.605)
1587

-0.359
(0.191)
-0.094
(0.171)
-0.294
(0.221)
-0.062
(0.205)
0.020
(0.202)
0.079
(0.213)
0.126
(0.167)
0.104
(0.198)
0.111
(0.212)
-0.741∗∗∗
(0.191)
-0.175
(0.183)
-0.259
(0.180)
-0.049
(0.201)
-0.012
(0.023)
0.784
(0.471)
2450

The table reports the estimates of the probability of being a treated school, estimated trough a probit regression in the 3 samples: (1) the
whole sample of schools, (2) the sample of schools for which information on the school outcomes is available, (3) the sample of schools for
which information about the cermat exam is available. Standard errors in parentheses. ∗ p < 0.05, ∗∗ p < 0.01, ∗∗∗ p < 0.001
50

Table B5: Mathematics 2019

Treatment 0-33

Treatment 33-66

Treatment 66-100

Observations
T2

Observations
T3

Observations
T4

Observations

(1)

(2)

Reading Mathematics
-0.113
(0.063)
0.092
(0.058)
-0.023
(0.062)
2116
-0.074
(0.068)
1111
0.076
(0.062)
1097
-0.065
(0.069)
1055

0.090
(0.060)
0.013
(0.062)
-0.077
(0.068)
2093
0.079
(0.065)
1106
-0.068
(0.066)
1108
-0.117
(0.073)
1082

(3)
Foreign
0.007
(0.066)
0.009
(0.070)
0.117
(0.068)
1962
-0.018
(0.068)
989
0.022
(0.077)
963
0.018
(0.074)
935

(4)
Inclusive
0.034
(0.065)
-0.127
(0.067)
-0.091
(0.068)
1935
0.034
(0.069)
939
-0.094
(0.074)
898
-0.105
(0.071)
947

(5)
Personnel
0.053
(0.055)
-0.074
(0.057)
-0.155∗
(0.063)
2133
0.002
(0.065)
897
-0.117
(0.060)
1022
-0.017
(0.067)
1142

(6)
Tutor
-0.037
(0.057)
-0.125∗
(0.058)
-0.252∗∗∗
(0.065)
2167
-0.232∗∗∗
(0.057)
1109
-0.065
(0.063)
1099
-0.031
(0.071)
1162

Table B6: Language 2019

Treatment 0-33

Treatment 33-66

Treatment 66-100

Observations
T2

Observations
T3

Observations
T4

Observations

(1)

(2)

Reading Mathematics
-0.090
(0.062)
0.079
(0.057)
-0.027
(0.060)
2116
-0.062
(0.067)
1111
0.056
(0.062)
1097
-0.037
(0.065)
1055

0.146∗
(0.059)
0.018
(0.064)
-0.037
(0.068)
2093
0.122
(0.065)
1106
-0.079
(0.066)
1108
-0.064
(0.072)
1082

(3)
Foreign
0.000
(0.066)
0.022
(0.066)
0.020
(0.068)
1962
-0.052
(0.068)
989
0.054
(0.073)
963
-0.035
(0.073)
935

(4)
Inclusive
0.085
(0.062)
-0.073
(0.066)
-0.030
(0.065)
1935
0.082
(0.065)
939
-0.036
(0.074)
898
-0.041
(0.068)
947

(5)
Personnel
0.017
(0.053)
-0.085
(0.054)
-0.179∗∗
(0.063)
2133
-0.047
(0.063)
897
-0.105
(0.057)
1022
-0.066
(0.068)
1142

(6)
Tutor
-0.057
(0.054)
-0.071
(0.059)
-0.206∗∗
(0.063)
2167
-0.241∗∗∗
(0.056)
1109
-0.021
(0.065)
1099
-0.024
(0.069)
1162

51

Table B7: Participation among stakeholders

(1)

(2)

Reading Mathematics
-0.112
(0.079)
0.027
(0.078)
0.137
(0.076)
1390
-0.055
(0.083)
692
-0.064
(0.084)
689
0.110
(0.084)
703

-0.038
(0.075)
0.004
(0.075)
0.157
(0.082)
1375
-0.096
(0.082)
693
-0.089
(0.080)
726
-0.046
(0.090)
699

(3)
Foreign
0.039
(0.075)
-0.073
(0.093)
0.106
(0.087)
1234
0.063
(0.081)
555
-0.031
(0.098)
589
0.046
(0.092)
579

(4)
Inclusive
-0.359∗∗∗
(0.083)
-0.055
(0.097)
-0.102
(0.086)
1177
-0.270∗∗
(0.092)
509
-0.099
(0.107)
548
-0.080
(0.092)
580

(5)
Personnel
-0.147
(0.079)
-0.161∗
(0.075)
0.108
(0.085)
1276
-0.102
(0.091)
454
-0.068
(0.090)
523
-0.058
(0.085)
696

(6)
Tutor
-0.051
(0.079)
0.022
(0.077)
0.062
(0.079)
1392
-0.033
(0.090)
640
0.091
(0.091)
692
0.008
(0.095)
719

Treatment 0-33

Treatment 33-66

Treatment 66-100

Observations
T2

Observations
T3

Observations
T4

Observations

Table B8: Teacher students relationship

(1)

(2)

Reading Mathematics
-0.114
(0.076)
-0.072
(0.076)
0.071
(0.073)
1381
-0.062
(0.082)
684
-0.091
(0.082)
685
0.000
(0.087)
706

-0.079
(0.073)
-0.055
(0.079)
0.091
(0.082)
1360
-0.067
(0.076)
680
-0.060
(0.084)
735
-0.029
(0.092)
707

(3)
Foreign
-0.043
(0.072)
-0.067
(0.085)
0.072
(0.094)
1261
-0.089
(0.080)
572
-0.091
(0.094)
596
-0.069
(0.104)
605

(4)
Inclusive
-0.257∗∗
(0.086)
0.045
(0.094)
0.064
(0.085)
1175
-0.102
(0.096)
508
-0.157
(0.102)
559
0.071
(0.091)
575

(5)
Personnel
-0.178∗
(0.073)
-0.254∗∗∗
(0.072)
0.288∗∗∗
(0.082)
1275
-0.018
(0.085)
454
-0.191
(0.099)
526
-0.001
(0.095)
709

(6)
Tutor
-0.264∗∗∗
(0.078)
-0.043
(0.076)
0.142
(0.082)
1405
-0.170
(0.087)
622
0.057
(0.083)
698
-0.123
(0.093)
729

Treatment 0-33

Treatment 33-66

Treatment 66-100

Observations
T2

Observations
T3

Observations
T4

Observations

52

Table B9: Mutual respect

Treatment 0-33

Treatment 33-66

Treatment 66-100

Observations
T2

Observations
T3

Observations
T4

Observations

(1)

(2)

Reading Mathematics
-0.116
(0.074)
-0.017
(0.072)
0.076
(0.077)
1383
-0.065
(0.077)
704
-0.104
(0.074)
671
0.062
(0.093)
724

-0.112
(0.076)
-0.034
(0.075)
0.052
(0.084)
1348
-0.100
(0.079)
696
-0.033
(0.077)
723
0.030
(0.096)
693

(3)
Foreign
-0.041
(0.076)
-0.016
(0.083)
0.118
(0.090)
1252
-0.045
(0.082)
569
0.008
(0.089)
577
0.061
(0.098)
616

(4)
Inclusive
-0.308∗∗∗
(0.085)
0.012
(0.088)
0.003
(0.085)
1183
-0.215∗
(0.090)
534
-0.149
(0.099)
546
-0.019
(0.091)
586

(5)
Personnel
-0.159∗
(0.072)
-0.214∗∗
(0.071)
0.284∗∗∗
(0.082)
1260
0.007
(0.081)
460
-0.097
(0.095)
522
-0.030
(0.093)
691

(6)
Tutor
-0.134
(0.078)
-0.009
(0.076)
0.207∗
(0.083)
1410
-0.092
(0.084)
620
0.113
(0.082)
692
0.038
(0.090)
740

Table B10: School delinquency and violence 1 - minor issues (absenteeism, cheating, vandalism)

Treatment 0-33

Treatment 33-66

Treatment 66-100

Observations
T2

Observations
T3

Observations
T4

Observations

(1)

(2)

Reading Mathematics
-0.016
(0.070)
0.065
(0.076)
0.115
(0.082)
1382
0.075
(0.073)
714
-0.031
(0.078)
678
0.048
(0.098)
711

-0.000
(0.072)
0.044
(0.078)
0.278∗∗∗
(0.082)
1366
0.020
(0.079)
702
0.059
(0.084)
725
0.110
(0.094)
692

(3)
Foreign
-0.081
(0.073)
-0.066
(0.083)
0.147
(0.085)
1249
-0.089
(0.078)
580
-0.093
(0.091)
601
0.061
(0.093)
616

(4)
Inclusive
-0.077
(0.079)
0.200∗∗
(0.077)
-0.032
(0.081)
1153
-0.002
(0.084)
535
-0.016
(0.088)
552
-0.047
(0.090)
596

(5)
Personnel
-0.206∗∗
(0.063)
-0.261∗∗∗
(0.070)
0.419∗∗∗
(0.074)
1262
0.024
(0.078)
449
-0.075
(0.079)
529
0.054
(0.082)
675

(6)
Tutor
-0.153∗
(0.074)
-0.048
(0.074)
0.202∗
(0.083)
1400
-0.008
(0.076)
638
-0.045
(0.080)
691
-0.075
(0.090)
745

53

Table B11: School delinquency and violence 2 - violence against other students and teachers

Treatment 0-33

Treatment 33-66

Treatment 66-100

Observations
T2

Observations
T3

Observations
T4

Observations

(1)

(2)

Reading Mathematics
0.021
(0.073)
0.036
(0.074)
-0.157
(0.081)
1382
-0.002
(0.079)
714
-0.016
(0.080)
678
-0.073
(0.094)
711

0.132
(0.074)
-0.067
(0.081)
-0.072
(0.086)
1366
0.040
(0.074)
702
-0.071
(0.084)
725
-0.086
(0.102)
692

(3)
Foreign
0.018
(0.074)
-0.117
(0.082)
-0.126
(0.089)
1249
-0.019
(0.079)
580
-0.140
(0.089)
601
-0.136
(0.095)
616

(4)
Inclusive
-0.038
(0.076)
-0.025
(0.081)
-0.103
(0.090)
1153
0.055
(0.085)
535
-0.052
(0.087)
552
-0.110
(0.095)
596

(5)
Personnel
-0.177∗
(0.072)
-0.161∗
(0.074)
0.044
(0.068)
1262
-0.097
(0.083)
449
-0.081
(0.085)
529
-0.075
(0.071)
675

(6)
Tutor
-0.159∗
(0.071)
-0.023
(0.072)
-0.032
(0.079)
1400
-0.085
(0.075)
638
-0.033
(0.068)
691
-0.055
(0.087)
745

Table B12: School delinquency and violence 3 - substance abuse

Treatment 0-33

Treatment 33-66

Treatment 66-100

Observations
T2

Observations
T3

Observations
T4

Observations

(1)

(2)

Reading Mathematics
-0.091
(0.077)
-0.008
(0.080)
-0.006
(0.081)
1382
-0.038
(0.079)
714
-0.075
(0.082)
678
0.046
(0.115)
711

-0.094
(0.075)
0.025
(0.079)
0.041
(0.081)
1366
0.029
(0.080)
702
0.015
(0.084)
725
0.048
(0.105)
692

(3)
Foreign
0.056
(0.084)
0.086
(0.084)
0.099
(0.093)
1249
0.096
(0.087)
580
0.067
(0.093)
601
0.051
(0.097)
616

(4)
Inclusive
-0.011
(0.084)
0.129
(0.081)
-0.137
(0.093)
1153
0.111
(0.099)
535
0.025
(0.091)
552
-0.164
(0.100)
596

(5)
Personnel
-0.296∗∗∗
(0.078)
-0.343∗∗∗
(0.082)
0.279∗∗∗
(0.072)
1262
-0.127
(0.092)
449
-0.207∗
(0.086)
529
0.025
(0.074)
675

(6)
Tutor
-0.086
(0.083)
0.033
(0.076)
0.331∗∗∗
(0.074)
1400
0.057
(0.092)
638
0.005
(0.079)
691
0.126
(0.078)
745

54

Table B13: Building community

Treatment 0-33

Treatment 33-66

Treatment 66-100

Observations
T2

Observations
T3

Observations
T4

Observations

(1)

(2)

Reading Mathematics
-0.093
(0.071)
-0.090
(0.078)
0.003
(0.077)
1385
-0.010
(0.077)
687
-0.115
(0.081)
690
-0.043
(0.082)
696

-0.051
(0.069)
-0.022
(0.078)
0.034
(0.089)
1363
-0.070
(0.072)
716
-0.038
(0.083)
699
-0.086
(0.099)
691

(3)
Foreign
-0.135
(0.081)
-0.049
(0.085)
0.081
(0.087)
1252
-0.056
(0.087)
575
-0.068
(0.090)
598
0.040
(0.093)
602

(4)
Inclusive
-0.334∗∗∗
(0.089)
0.016
(0.098)
0.019
(0.087)
1147
-0.258∗∗
(0.092)
533
-0.078
(0.105)
565
-0.002
(0.090)
583

(5)
Personnel
-0.232∗∗
(0.075)
-0.221∗∗
(0.073)
0.319∗∗∗
(0.082)
1251
-0.129
(0.083)
448
-0.089
(0.083)
528
0.018
(0.095)
682

(6)
Tutor
-0.155∗
(0.077)
-0.062
(0.078)
0.154
(0.085)
1406
0.035
(0.092)
625
0.040
(0.089)
688
0.008
(0.090)
731

Table B14: Establishing inclusive values

Treatment 0-33

Treatment 33-66

Treatment 66-100

Observations
T2

Observations
T3

Observations
T4

Observations

(1)

(2)

Reading Mathematics
-0.056
(0.076)
-0.034
(0.085)
0.103
(0.075)
1385
0.002
(0.078)
687
-0.062
(0.087)
690
0.039
(0.080)
696

-0.004
(0.073)
-0.022
(0.084)
0.060
(0.091)
1363
-0.026
(0.073)
716
-0.022
(0.088)
699
-0.044
(0.099)
691

(3)
Foreign
-0.023
(0.086)
-0.068
(0.091)
-0.015
(0.090)
1252
0.060
(0.093)
575
-0.097
(0.098)
598
-0.057
(0.097)
602

(4)
Inclusive
-0.318∗∗∗
(0.095)
0.005
(0.108)
0.007
(0.085)
1147
-0.286∗∗
(0.095)
533
-0.095
(0.116)
565
0.027
(0.088)
583

(5)
Personnel
-0.251∗∗
(0.082)
-0.158∗
(0.074)
0.200∗
(0.082)
1251
-0.181
(0.093)
448
-0.062
(0.086)
528
-0.032
(0.097)
682

(6)
Tutor
-0.140
(0.079)
-0.019
(0.078)
0.168
(0.086)
1406
0.043
(0.099)
625
0.120
(0.100)
688
0.029
(0.092)
731

55

Table B15: Developing the school for all

(1)

(2)

Reading Mathematics
-0.136
(0.079)
0.044
(0.076)
0.021
(0.079)
1371
-0.122
(0.082)
709
-0.016
(0.087)
691
-0.053
(0.088)
712

-0.113
(0.077)
-0.015
(0.080)
-0.009
(0.085)
1359
-0.162∗
(0.082)
663
-0.036
(0.084)
699
-0.127
(0.086)
718

(3)
Foreign
-0.039
(0.079)
0.006
(0.087)
0.012
(0.094)
1241
-0.065
(0.082)
599
-0.084
(0.090)
600
-0.151
(0.101)
591

(4)
Inclusive
-0.263∗∗
(0.093)
-0.037
(0.094)
0.130
(0.089)
1168
-0.206∗
(0.099)
521
-0.090
(0.102)
548
0.115
(0.094)
586

(5)
Personnel
-0.298∗∗∗
(0.083)
-0.184∗
(0.075)
0.050
(0.082)
1249
-0.216∗
(0.098)
457
-0.075
(0.082)
526
-0.214∗
(0.088)
691

(6)
Tutor
-0.098
(0.080)
-0.084
(0.079)
0.101
(0.080)
1396
-0.008
(0.091)
633
0.015
(0.093)
677
-0.044
(0.083)
725

Treatment 0-33

Treatment 33-66

Treatment 66-100

Observations
T2

Observations
T3

Observations
T4

Observations

Table B16: Organizing support for diversity

(1)

(2)

Reading Mathematics
-0.197∗
(0.083)
-0.064
(0.080)
0.071
(0.081)
1371
-0.167∗
(0.082)
679
-0.071
(0.080)
691
0.005
(0.085)
712

-0.083
(0.077)
-0.076
(0.083)
0.099
(0.084)
1352
-0.102
(0.081)
666
-0.060
(0.087)
718
-0.053
(0.089)
711

(3)
Foreign
-0.036
(0.082)
-0.029
(0.091)
0.051
(0.094)
1255
0.001
(0.087)
595
-0.070
(0.094)
593
-0.100
(0.098)
589

(4)
Inclusive
-0.283∗∗
(0.092)
-0.047
(0.099)
0.215∗
(0.090)
1161
-0.200∗
(0.098)
534
-0.160
(0.110)
547
0.161
(0.096)
581

(5)
Personnel
-0.173∗
(0.079)
-0.186∗
(0.075)
0.078
(0.081)
1251
-0.146
(0.103)
457
-0.088
(0.087)
525
-0.140
(0.092)
691

(6)
Tutor
-0.114
(0.079)
-0.032
(0.076)
0.117
(0.080)
1399
0.007
(0.086)
634
0.036
(0.087)
677
0.027
(0.086)
725

Treatment 0-33

Treatment 33-66

Treatment 66-100

Observations
T2

Observations
T3

Observations
T4

Observations

56

Table B17: Orchestrating learning

Treatment 0-33

Treatment 33-66

Treatment 66-100

Observations
T2

Observations
T3

Observations
T4

Observations

(1)

(2)

Reading Mathematics
-0.126
(0.079)
-0.074
(0.079)
0.053
(0.078)
1355
-0.130
(0.080)
699
-0.126
(0.083)
669
0.026
(0.086)
707

-0.098
(0.072)
-0.089
(0.081)
0.067
(0.078)
1350
0.005
(0.076)
676
-0.084
(0.083)
704
-0.055
(0.090)
707

(3)
Foreign
-0.168∗
(0.075)
-0.088
(0.087)
0.005
(0.090)
1223
-0.144
(0.081)
569
-0.062
(0.092)
597
-0.037
(0.095)
616

(4)
Inclusive
-0.269∗∗
(0.090)
0.030
(0.087)
0.044
(0.085)
1148
-0.102
(0.093)
541
-0.052
(0.094)
524
0.031
(0.090)
569

(5)
Personnel
-0.107
(0.075)
-0.173∗
(0.073)
0.255∗∗
(0.079)
1238
0.003
(0.091)
447
-0.090
(0.086)
524
-0.103
(0.088)
695

(6)
Tutor
-0.122
(0.079)
-0.085
(0.080)
0.209∗
(0.082)
1398
-0.005
(0.080)
635
-0.114
(0.085)
678
0.010
(0.086)
709

Table B18: Mobilizing resources

Treatment 0-33

Treatment 33-66

Treatment 66-100

Observations
T2

Observations
T3

Observations
T4

Observations

(1)

(2)

Reading Mathematics
-0.102
(0.082)
-0.024
(0.076)
0.119
(0.079)
1364
-0.130
(0.083)
706
-0.046
(0.079)
675
0.029
(0.089)
713

-0.063
(0.075)
-0.100
(0.080)
0.052
(0.081)
1353
0.022
(0.079)
684
-0.078
(0.081)
704
-0.127
(0.090)
697

(3)
Foreign
-0.028
(0.084)
-0.036
(0.088)
0.124
(0.089)
1231
0.002
(0.089)
570
-0.033
(0.091)
582
0.062
(0.094)
633

(4)
Inclusive
-0.181∗
(0.092)
0.104
(0.090)
0.147
(0.088)
1139
-0.156
(0.097)
532
-0.050
(0.099)
526
0.083
(0.095)
566

(5)
Personnel
-0.261∗∗∗
(0.077)
-0.341∗∗∗
(0.076)
0.100
(0.078)
1238
-0.184∗
(0.088)
447
-0.260∗∗
(0.085)
524
-0.196∗
(0.090)
693

(6)
Tutor
-0.151∗
(0.076)
-0.089
(0.076)
0.137
(0.079)
1390
-0.053
(0.080)
629
-0.079
(0.083)
674
-0.023
(0.088)
706

57

Table B19: Socially disadvantaged and Roma children 1 - negative feelings toward children

Treatment 0-33

Treatment 33-66

Treatment 66-100

Observations
T2

Observations
T3

Observations
T4

Observations

(1)

(2)

Reading Mathematics
0.051
(0.077)
-0.021
(0.078)
0.095
(0.075)
1353
0.071
(0.083)
685
-0.066
(0.089)
656
0.010
(0.083)
694

0.008
(0.073)
0.023
(0.081)
0.079
(0.083)
1347
0.011
(0.078)
680
0.040
(0.080)
681
0.071
(0.086)
678

(3)
Foreign
-0.001
(0.080)
-0.133
(0.089)
-0.046
(0.094)
1214
-0.009
(0.089)
558
-0.096
(0.093)
591
-0.007
(0.106)
568

(4)
Inclusive
-0.066
(0.086)
0.169
(0.087)
0.049
(0.098)
1107
-0.047
(0.092)
513
0.139
(0.096)
543
0.004
(0.106)
554

(5)
Personnel
-0.033
(0.077)
-0.002
(0.074)
-0.018
(0.083)
1227
-0.164
(0.115)
445
-0.022
(0.086)
528
0.019
(0.090)
667

(6)
Tutor
-0.030
(0.074)
-0.149∗
(0.075)
-0.114
(0.077)
1358
0.107
(0.083)
607
-0.056
(0.092)
654
-0.119
(0.097)
692

Table B20: Socially disadvantaged and Roma children 2 - Teachers feel prepared

Treatment 0-33

Treatment 33-66

Treatment 66-100

Observations
T2

Observations
T3

Observations
T4

Observations

(1)

(2)

Reading Mathematics
-0.131
(0.078)
-0.127
(0.075)
0.036
(0.081)
1353
-0.102
(0.087)
685
-0.013
(0.088)
656
-0.086
(0.091)
694

-0.078
(0.078)
-0.102
(0.083)
-0.063
(0.084)
1347
-0.020
(0.082)
680
-0.061
(0.083)
681
-0.105
(0.091)
678

(3)
Foreign
0.106
(0.078)
-0.010
(0.093)
0.100
(0.095)
1214
0.108
(0.086)
558
-0.005
(0.097)
591
0.079
(0.103)
568

(4)
Inclusive
-0.049
(0.090)
-0.208∗
(0.098)
0.166
(0.085)
1107
0.016
(0.098)
513
-0.258∗
(0.102)
543
0.134
(0.096)
554

(5)
Personnel
0.022
(0.082)
-0.036
(0.079)
-0.105
(0.084)
1227
-0.020
(0.115)
445
-0.030
(0.095)
528
-0.062
(0.090)
667

(6)
Tutor
-0.168∗
(0.079)
-0.022
(0.078)
0.100
(0.084)
1358
-0.059
(0.090)
607
0.009
(0.082)
654
0.141
(0.090)
692

58

Table B21: Socially disadvantaged 1 - are bad for the class

Treatment 0-33

Treatment 33-66

Treatment 66-100

Observations
T2

Observations
T3

Observations
T4

Observations

(1)

(2)

Reading Mathematics
-0.137
(0.070)
0.034
(0.080)
0.052
(0.078)
1343
-0.073
(0.077)
685
0.131
(0.090)
676
0.019
(0.090)
671

0.041
(0.077)
0.113
(0.077)
0.051
(0.085)
1307
0.070
(0.079)
668
0.033
(0.080)
679
0.044
(0.092)
689

(3)
Foreign
-0.015
(0.082)
-0.019
(0.090)
-0.048
(0.098)
1228
0.055
(0.090)
554
-0.043
(0.095)
592
-0.071
(0.102)
594

(4)
Inclusive
-0.126
(0.088)
0.141
(0.090)
-0.049
(0.098)
1112
-0.009
(0.096)
521
0.161
(0.098)
532
-0.060
(0.110)
546

(5)
Personnel
0.096
(0.080)
-0.003
(0.083)
0.062
(0.083)
1216
0.013
(0.096)
435
-0.007
(0.087)
523
0.070
(0.095)
674

(6)
Tutor
-0.048
(0.082)
-0.085
(0.087)
-0.047
(0.086)
1361
0.103
(0.094)
622
-0.029
(0.093)
666
-0.067
(0.097)
689

Table B22: Socially disadvantaged 2 - are discriminated

Treatment 0-33

Treatment 33-66

Treatment 66-100

Observations
T2

Observations
T3

Observations
T4

Observations

(1)

(2)

Reading Mathematics
0.059
(0.071)
-0.099
(0.080)
0.077
(0.077)
1343
0.011
(0.076)
685
-0.076
(0.090)
676
-0.026
(0.090)
671

-0.031
(0.076)
-0.101
(0.080)
0.000
(0.082)
1307
0.007
(0.080)
668
-0.111
(0.085)
679
0.027
(0.086)
689

(3)
Foreign
0.018
(0.078)
-0.037
(0.097)
-0.081
(0.089)
1228
0.048
(0.084)
554
-0.043
(0.103)
592
-0.088
(0.094)
594

(4)
Inclusive
0.074
(0.091)
-0.045
(0.093)
0.010
(0.094)
1112
0.098
(0.095)
521
0.036
(0.103)
532
-0.022
(0.098)
546

(5)
Personnel
-0.122
(0.079)
-0.104
(0.078)
-0.150
(0.084)
1216
-0.073
(0.102)
435
-0.103
(0.086)
523
-0.057
(0.092)
674

(6)
Tutor
0.043
(0.080)
0.050
(0.081)
0.101
(0.083)
1361
0.110
(0.086)
622
0.046
(0.082)
666
0.010
(0.102)
689

59

Table B23: Socially disadvantaged 3 - create more work for the teachers

Treatment 0-33

Treatment 33-66

Treatment 66-100

Observations
T2

Observations
T3

Observations
T4

Observations

(1)

(2)

Reading Mathematics
0.103
(0.073)
-0.036
(0.078)
-0.055
(0.080)
1343
0.118
(0.084)
685
-0.033
(0.086)
676
-0.015
(0.092)
671

0.033
(0.076)
-0.040
(0.075)
0.037
(0.091)
1307
0.021
(0.080)
668
-0.031
(0.077)
679
0.031
(0.096)
689

(3)
Foreign
-0.057
(0.085)
-0.044
(0.083)
0.023
(0.091)
1228
-0.055
(0.094)
554
-0.067
(0.087)
592
0.030
(0.097)
594

(4)
Inclusive
-0.127
(0.083)
-0.098
(0.091)
0.097
(0.097)
1112
-0.155
(0.089)
521
-0.150
(0.098)
532
0.152
(0.103)
546

(5)
Personnel
0.047
(0.077)
0.099
(0.078)
0.140
(0.083)
1216
-0.031
(0.095)
435
-0.044
(0.089)
523
0.186∗
(0.088)
674

(6)
Tutor
0.177∗
(0.081)
0.097
(0.085)
0.036
(0.085)
1361
0.058
(0.088)
622
0.012
(0.092)
666
0.029
(0.105)
689

Table B24: Roma 1 - are bad for the class

Treatment 0-33

Treatment 33-66

Treatment 66-100

Observations
T2

Observations
T3

Observations
T4

Observations

(1)

(2)

Reading Mathematics
-0.002
(0.078)
0.006
(0.083)
0.070
(0.079)
1316
-0.027
(0.082)
651
-0.024
(0.094)
639
0.087
(0.087)
689

0.072
(0.082)
0.062
(0.083)
0.099
(0.080)
1319
0.044
(0.088)
677
0.040
(0.086)
706
0.119
(0.089)
689

(3)
Foreign
0.051
(0.083)
-0.152
(0.092)
-0.039
(0.099)
1196
0.070
(0.088)
562
-0.158
(0.097)
553
-0.038
(0.102)
576

(4)
Inclusive
-0.144
(0.080)
0.114
(0.090)
-0.023
(0.103)
1122
-0.122
(0.104)
512
0.123
(0.100)
531
-0.011
(0.113)
541

(5)
Personnel
0.001
(0.085)
0.002
(0.084)
0.080
(0.082)
1201
-0.048
(0.109)
434
-0.026
(0.097)
509
0.118
(0.096)
661

(6)
Tutor
0.035
(0.084)
0.054
(0.085)
0.014
(0.084)
1355
0.093
(0.093)
615
0.070
(0.087)
668
-0.025
(0.088)
705

60

Table B25: Roma 2 - are discriminated

Treatment 0-33

Treatment 33-66

Treatment 66-100

Observations
T2

Observations
T3

Observations
T4

Observations

(1)

(2)

Reading Mathematics
-0.127
(0.077)
-0.189∗
(0.079)
-0.026
(0.078)
1316
-0.113
(0.084)
651
-0.211∗
(0.085)
639
-0.122
(0.084)
689

-0.186∗
(0.082)
-0.103
(0.081)
-0.111
(0.082)
1319
-0.177∗
(0.083)
677
-0.155
(0.084)
706
-0.146
(0.087)
689

(3)
Foreign
-0.016
(0.088)
-0.094
(0.091)
-0.093
(0.089)
1196
-0.019
(0.094)
562
-0.086
(0.095)
553
-0.051
(0.093)
576

(4)
Inclusive
0.024
(0.088)
-0.204∗
(0.095)
0.062
(0.086)
1122
-0.051
(0.092)
512
-0.098
(0.103)
531
0.039
(0.092)
541

(5)
Personnel
0.091
(0.081)
0.050
(0.082)
-0.161
(0.088)
1201
-0.155
(0.113)
434
0.091
(0.089)
509
0.067
(0.088)
661

(6)
Tutor
0.007
(0.080)
-0.007
(0.083)
-0.122
(0.085)
1355
0.017
(0.084)
615
-0.036
(0.085)
668
-0.067
(0.092)
705

Table B26: Roma 3 - create more work for the teachers

Treatment 0-33

Treatment 33-66

Treatment 66-100

Observations
T2

Observations
T3

Observations
T4

Observations

(1)

(2)

Reading Mathematics
0.125
(0.080)
0.032
(0.080)
0.068
(0.080)
1316
0.077
(0.080)
651
0.031
(0.088)
639
0.147
(0.088)
689

0.052
(0.080)
0.036
(0.077)
0.004
(0.085)
1319
0.049
(0.084)
677
-0.018
(0.082)
706
0.032
(0.092)
689

(3)
Foreign
0.081
(0.091)
-0.015
(0.084)
0.025
(0.090)
1196
0.074
(0.099)
562
-0.096
(0.091)
553
-0.026
(0.096)
576

(4)
Inclusive
0.041
(0.086)
-0.024
(0.092)
0.072
(0.094)
1122
0.006
(0.095)
512
-0.024
(0.101)
531
0.068
(0.099)
541

(5)
Personnel
0.063
(0.085)
0.105
(0.085)
0.048
(0.088)
1201
0.035
(0.102)
434
0.067
(0.095)
509
-0.034
(0.087)
661

(6)
Tutor
0.160
(0.083)
0.084
(0.084)
0.158
(0.085)
1355
0.007
(0.094)
615
0.044
(0.081)
668
0.171
(0.091)
705

61

Table B27: Teacher's confidence

Treatment 0-33

Treatment 33-66

Treatment 66-100

Observations
T2

Observations
T3

Observations
T4

Observations

(1)

(2)

Reading Mathematics
-0.036
(0.077)
-0.114
(0.075)
0.080
(0.077)
1284
-0.043
(0.083)
633
-0.075
(0.086)
654
0.036
(0.085)
661

-0.024
(0.078)
-0.037
(0.074)
0.022
(0.083)
1262
-0.038
(0.086)
650
0.034
(0.082)
667
-0.021
(0.091)
657

(3)
Foreign
0.082
(0.076)
-0.243∗∗
(0.089)
0.033
(0.087)
1141
0.094
(0.083)
546
-0.170
(0.095)
550
0.084
(0.095)
587

(4)
Inclusive
0.025
(0.085)
-0.162
(0.085)
0.039
(0.101)
1081
0.018
(0.093)
505
-0.175
(0.097)
516
-0.076
(0.110)
560

(5)
Personnel
-0.053
(0.089)
-0.012
(0.081)
-0.170
(0.088)
1164
-0.112
(0.106)
426
0.030
(0.086)
513
-0.083
(0.097)
625

(6)
Tutor
-0.003
(0.080)
-0.018
(0.083)
0.038
(0.084)
1296
0.035
(0.080)
608
-0.020
(0.087)
609
0.063
(0.093)
665

62

GETTING IN TOUCH WITH THE EU In person All over the European Union there are hundreds of Europe Direct information centres. You can find the address of the centre nearest you at: https://europa.eu/european-union/contact_en On the phone or by email Europe Direct is a service that answers your questions about the European Union. You can contact this service: -by freephone: 00 800 6 7 8 9 10 11 (certain operators may charge for these calls),-at the following standard number: +32 22999696, or-by electronic mail via: https://europa.eu/european-union/contact_enFINDING INFORMATION ABOUT THE EU Online Information about the European Union in all the official languages of the EU is available on the Europa website at: https://europa.eu/european-union/index_en EU publications You can download or order free and priced EU publications from EU Bookshop at: https://publications.europa.eu/en/publications. Multiple copies of free publications may be obtained by contacting Europe Direct or your local information centre (see https://europa.eu/european-union/contact_en). K
J
-
N
A
-
3
0
8
6
0
-
E
N
-
N

doi:10.2760/293062

ISBN 978-92-76-42311-9