metadata: authors: 0: author: Ponte Barbosa,Marcelo
1: author: Oliveira Costa,Leandro
count: Brazil
docna: 0: docna: Requirements to Be a Teacher in Brazil :
            Effective or Not ?
docty: Policy Research Working Paper
abstracts: cdata!: Policy makers in Brazil attempted to
            improve human capital through changes in the legislated
            requirements for teacher education in 1996. They passed a
            national Law of Guidelines and Standards of Education that
            established 2007 as the deadline for all Brazilian basic
            education teachers to have tertiary education–level
            qualifications. This implied a significant change in the
            profile of teachers in basic education and in the provision
            of pre-service training. The objective of this study is to
            investigate the effects of the increase in the share of
            public upper secondary school teachers with higher education
            on students’ performance in math and Portuguese and analyze
            the role of the pre-service training framework in the
            quality of teachers in recent years. The study carried out
            an empirical analysis to estimate the average treatment
            effect on the treated on public upper secondary students
            through the combination of difference-in-difference and
            propensity score matching methods. The analysis found no
            evidence of positive effects on Portuguese scores, and
            despite the statistically significant positive effect of the
            rise in teachers with higher education on math scores, there
            was no effect from specific math training. Finally, the
            paper discusses the possible reason for the ineffectiveness
            of teacher pre-service training, such as the quality of the
            training delivered by distance learning modalities and the
            low performance of the secondary students who enter the
            teacher schools.
geo_reg_key: 119228,517191
repnme: repnme: Requirements to Be a Teacher in Brazil :
            Effective or Not ?
display_title: Requirements to Be a Teacher in
            Brazil : Effective or Not
listing_relative_url: /research/2019/09/31400283/requirements-teacher-brazil-effective-or-not
publishtoextweb_dt: 2019-09-11T00:00:00Z
no_of_pages: 23
author_keys: 0: author_key: 1518396
1: author_key: 1173721
model_extractions: Policy Research Working Paper 9006 Requirements to Be a Teacher in Brazil Effective or Not?

_Marcelo Ponte Barbosa Leandro Oliveira Costa_ Education Global Practice September 2019
Policy Research Working Paper 9006

# **Abstract**

Policy makers in Brazil attempted to improve human capital through changes in the legislated requirements for teacher education in 1996. They passed a national Law of Guidelines and Standards of Education that established 2007 as the deadline for all Brazilian basic education teachers to have tertiary education–level qualifications. This implied a significant change in the profile of teachers in basic education and in the provision of pre-service training. The objective of this study is to investigate the effects of the increase in the share of public upper secondary school teachers with higher education on students’ performance in math and Portuguese and analyze the role of the pre-service training framework in the quality of teachers in recent years. The study carried out an empirical analysis to estimate the average treatment effect on the treated on public upper secondary students through the combination of difference-in-difference and propensity score matching methods. The analysis found no evidence of positive effects on Portuguese scores, and despite the statistically significant positive effect of the rise in teachers with higher education on math scores, there was no effect from specific math training. Finally, the paper discusses the possible reason for the ineffectiveness of teacher pre-service training, such as the quality of the training delivered by distance learning modalities and the low performance of the secondary students who enter the teacher schools.

This paper is a product of the Education Global Practice. It is part of a larger effort by the World Bank to provide open access to its research and make a contribution to development policy discussions around the world. Policy Research Working Papers are also posted on the Web at http://www.worldbank.org/prwp. The authors may be contacted at lcosta@worldbank.org.

_The Policy Research Working Paper Series disseminates the findings of work in progress to encourage the exchange of ideas about development issues. An objective of the series is to get the findings out quickly, even if the presentations are less than fully polished. The papers carry the names of the authors and should be cited accordingly. The findings, interpretations, and conclusions expressed in this paper are entirely those of the authors. They do not necessarily represent the views of the International Bank for Reconstruction and Development/World Bank and its affiliated organizations, or those of the Executive Directors of the World Bank or the governments they represent._ Produced by the Research Support Team
# **_Requirements to Be a Teacher in Brazil_ : Effective or Not?**

Marcelo Ponte Barbosa (UFC)<sup>*</sup> Leandro Oliveira Costa (World Bank)<sup>**</sup>

**Keywords:** teacher quality; teacher education requirements; secondary education. **JEL: I21, I28, H43, C52.**

> * Professor at Universidade Federal do Ceará (UFC/DA). E-mail: marceloponte@ufc.br. ** Senior Economist at The World Bank. E-mail: <u>lcosta@worldbank.org.</u>
# **1. Introduction**

There have been many attempts of policy makers to improve human capital through changes in the legislation requirements for teacher education in developing countries. In the case of Brazil, the national government strived to improve the profile of teachers through the requirement of tertiary education for teaching in basic education in 1996. They passed the Law of Guidelines and Standards of Education (Law No. 9394/96, LDB) that stablished the year 2007 as deadline for all Brazilian basic education teachers to achieve tertiary education level.<sup>1</sup> School census data of the Ministry of Education (MEC) suggest that, although this goal has not been fully achieved so far, this law appears to have influenced profoundly pre-service teachers’ education. From 1995 to 2007, the percentage of teachers of basic education with higher education increased from 49% to 68%, then reached 83% in 2015. For upper secondary school teachers, this percentage increased from 82% to 93% in the period between 1995 and 2007, remaining stable until 2015.

At the same time, there was a significant increase in coverage of upper secondary education of 5.4 million to 8.1 million students from 1995 to 2015, which was achieved in detriment of improvements in quality. National Assessment of Basic Education (SAEB) results show a drop in the percentage of graduates of basic education with adequate levels of learning in both mathematics, from 11.6% to 7.3% of students, and in Portuguese, from 45.4% to 27.5% of students, from 1995 to 2015. It is possible that the massive incorporation into the educational system of out-of-school children from lower background families has negatively influenced the quality of elementary education at first and of secondary education years later. This seems consistent with the trajectory of the Portuguese learning indicator, which fell sharply up to 2005, showed a slight improvement until 2011 and stagnation afterwards, but does not seem to have had the same influence on the math learning indicator. Thus, it is not clear whether the variation in enrollments was able to counterbalance the positive effects expected from the improvement in teachers' schooling.

Figure 1 - Percentage of graduates of secondary education with adequate levels of learning in math and Portuguese and Enrollment in secondary education

|11.6<br>45.4<br>5.4|17.9<br>39.7<br>6.4|11.9<br>27.6<br>7.8|11.6<br>25.8<br>8.4|12.8<br>26.9<br>9.1|10.9<br>22.6<br>9.0|9.8<br>24.5<br>8.4|11.0<br>28.9<br>8.3|10.3<br>29.2<br>8.4|9.3<br>27.2<br>8.3|7.3<br>27.5<br>8.1|
|---|---|---|---|---|---|---|---|---|---|---|
|1995|1997|1999|2001|2003|2005|2007|2009|2011|2013|2015|
||||Math|Portu|guese|Enrol|lment (mil|lion)|||

_Note._ Minimum score on the SAEB scale that characterize adequate learning for students in 3<sup>rd</sup> grade of secondary education was 300 points for Portuguese and 350 points for math. _Source._ School Census and SAEB – INEP/MEC.

The literature has established the central role of teachers in student learning, but there is still no consensus on how teachers’ pre-service training affects teacher performance. Studies that measure the value-added of teachers in a school year find that students with a good teacher can achieve an average gain of one year; while students with great teachers might gain 1.5 grade levels (Hanushek and Rivkin 2010). In addition, teachers critically impact not only children’s immediate learning progress, but also their longterm development and life choices (Bruns and Luque, 2015). However, studies based on the educational production function, estimated from cross-sectional data and on regression analysis of aggregated levels of student performance, fail to find statistically significant effects of teacher education measures (Hanusheck, 2003). Other series of studies developed since the 2000s, based on student-level longitudinal data and involving broader sets of variables, and more recent research that addresses non-observed heterogeneity

> 1 Repealed by Law No. 12796/13, which reintroduced the secondary level Normal course as minimum level of training for teaching in early childhood education and in the first years of elementary school.

2
and selection bias problems, find positive effects for some disciplines and grades and specific types of teacher pre-service education (Wayne and Youngs, 2003; Harris and Sass, 2011).

The objectives of this study are (i) to investigate the effects of the increase in the share of public upper secondary school teachers with higher education on students learning in math and Portuguese and (ii) to analyze the role of the pre-service training institutional framework on the quality of teachers in recent years. To this end, we first present the recent changes on teachers’ pre-service training in Brazil and an overview of the empirical literature on the subject. Subsequently, we carry out an empirical analysis in which we estimate the Average Treatment Effect on the performance of public upper secondary students. We conclude with a discussion on how the current policies affect the quality of teachers in Brazil.

# **2. Key issues on teachers’ pre-service training in Brazil**

A new regulation on pre-service training was established since the publishing of the LDB in 1996, whose article 62 defined the minimum requirements for teaching in basic education. LDB first determined that basic education teachers should be trained at the tertiary level, in _licenciatura_ courses. The exception was a specific upper secondary level training, called _Normal_ course, for teaching exclusively in early childhood and early grades of primary education. In 1999, the decree number 3276/99 established that the training for teaching in specific fields of knowledge, such as math, biology, physics, etc., would be required for one to hold a teaching position for a specific subject. After that, the resolution CNE number 02 of 2015, which defines the National Curriculum Guidelines for Teacher Training, established three pre-service training options that certify teachers for basic education: (i) _licenciatura_ degree, with a duration of four academic years and a minimum of 3,200 hours of academic work; (ii) _second licenciatura_ degree, for _licenciatura_ degree holders that want to teach another subject, with a reduced minimum workload of 1,200 hours; and (iii) _pedagogical supplementation_ , for bachelor degree holders, which require a minimum of 1,000 hours for the completion depending on the equivalence between graduation already obtained and the field of the intended pedagogical training.<sup>2</sup> There are many questions about the quality of teachers in Brazil related to the framework of preservice training and the governance of the system. For example, what was the effect of the increase in teachers ́ education requirements on basic education students' learning? Is the public education system not able to attract top performance secondary schools’ graduates? Is there a problem of quality in teacher training courses? Regarding pre-service training, the literature points to the existence of two major vectors that may affect the quality of teachers: the first considers aspects of the educational background of those interested in a teaching career; the second is regarded to the profile and structure of tertiary education (Bruns and Luque, 2015).<sup>3</sup> The significant increase in coverage of primary and secondary education observed until the middle of the 2000s in Brazil required hiring new teachers, which resulted in a change in the profile of the newcomers in this career (Lerch et al, 2010). In response to the rapid increase in demand for teachers with degrees, there was a boom in the provision of tertiary education programs. As a result, secondary education graduates from lower socioeconomic background joined the teaching career.

The percentage of public upper secondary education teachers who hold a tertiary degree increased from 82% to 93% in more than 20 years, mainly due to _licenciatura_ degrees (Figure 2). However, there are still teachers without higher education or with a pedagogy degree which is not focused on secondary education, whose share corresponds to 7.6%. Also, states and municipalities selected teachers with distinct profiles, since there was not a full compliance with the national standards for teacher admission. All these teachers were trained in different types of institutions that are practically autonomous to decide about the instructional design of their curriculum.

Louzano et al. (2010) investigated the profile and preferences of secondary school graduates who declared themselves interested in a teaching career in 2005.<sup>4</sup> Only 10% of this group belonged to the top

> 2 _Pedagogical supplementation_ was instituted in 1997 as a provisional act and originally required a workload of 540 hours. Its provisional nature is reversed 2017, and the certification is now recognized in LDB as a valid requisite for teaching. 3 The authors find that Latin American countries are trapped in a low-level equilibrium with low standards for entry into teacher training programs, which are not prepared to produce the set of competences found in high performance educational systems. 4 Based on socioeconomic questionnaire data of the National Examination of High School (ENEM).

3
performance students, and about a third were among the bottom performance students. This evidence suggests that the teaching career in Brazil is attracting a high share of lower performing students. In fact, the cut-off ENEM score required for admission to public higher education institutions is much lower for _licenciatura_ courses in these universities. In year 2014 the mean cut-off scores for _licenciatura_ in specific disciplines and in pedagogy were, respectively, the 5th and the 2nd lowest in a classification of 75 courses.<sup>5</sup> In addition, _licenciatura_ offered by private institutions are among courses with less competition for admission and therefore might be the choice of students with weaker educational background.<sup>6</sup> Figure 2 - Percentage of Public Upper Secondary Schools Teachers with Higher Education <!-- Start of picture text --> 82.0 87.6 88.5 78.9 89.8 82.3 95.3 89.6 93.2 87.5 91.6 87.7 92.8 93.2 81.0 93.5 83.6<br>74.4 75.4 76.9<br>52.2 54.5 54.4<br>47.4<br>42.2<br>38.0 36.2 39.0<br>11.4<br>7.4 7.6<br>5.7 5.2 3.3<br>1995 1999 2001 2003 2005 2007 2009 2011 2013 2015<br>Higher Education Licenciatura Postgraduate<br>Pedagogy Pedagogical supplementation Private institution<br><!-- End of picture text -->

* _Private institution_ indicates percentage of higher educated teachers that attended only private higher education Institution. _Source._ School Census, microdata, INEP/MEC.

Other factors that may be affecting the quality of trained teachers are related to the nature and quality of teacher pre-service training programs. As pointed out by Bruns et al. (2012) and Carnoy et al. (2008), teachers pre-service training, in general, gives low relative importance to didactics and techniques of teaching and classroom management. In order to shed light on these aspects we explore the changes in teacher pre-service training programs in the last 10 years.

Table 1 – Graduates from higher education programs

|**Areas and Programs**||**Year**<br>||**Dife**<br>|**rence**<br>|
|---|---|---|---|---|---|
||**2005**|**2010**|**2015**|**2005- 2015**|**2010- 2015**|
|_Training of specific subjects* _<br>|_77,795_|_95,550_|_82,158_|_6%_|_-14%_|
|Mathematics|10,194|11,915|10,896|7%|-9%|
|Portuguese|12,609|23,864|17,772|41%|-26%|
|_Other teacher training_**|_118,570_|_108,041_|_123,867_|_4%_|_15%_|
|Pedagogy|74,508|107,808|122,835|65%|14%|
|Normal Superior|44,062|233|1,032|-98%|343%|
|_Bachelor in specific subjects***_|_31,365_|_12,197_|_13,022_|_-58%_|_7%_|
|Mathematics|2,502|237|719|-71%|203%|
|Portuguese|7,146|1,347|1,213|-83%|-10%|
|**_All higher education programs_**|**_730,484_**|**_973,839_**|**_1,150,067_**|**_57%_**|**_18%_**|

* _Licenciatura_ courses in typical secondary education fields, like math, Portuguese, physics, biology, etc. ** E arly childhood education and early grades of primary education training. ***Bachelor's degree courses in typical secondary education fields. Source. Synopsis of the Census of Higher Education - INEP / MEC.

In the aggregate, the number of graduates in pre-service training courses practically stagnated and the bachelor's degree in typical secondary education fields dropped sharply in the period (Table 1). There was a decrease in the presence of potential teachers with a bachelor’s degree in favor of the _licenciatura_

> 5 Data from the Unified Selection System (SISU) of the Ministry of Education. The most recent data available in a complete and systematic way for all public higher education institutions that participated in SISU were from year 2014.

> 6 Dada from Synopsis of Tertiary Education Census - INEP/MEC shows that the average number of candidates per offered place in _licenciatura_ face-to-face courses is 0.68 and in bachelor’s degree face-to-face courses is 1.45.

4
above 90% since 2010 (Figure 4). Despite this protagonist, the share of private institutions in distance learning reduced among _licenciatura_ in specific subjects, with exception to Portuguese, in which private institutions showed a small increase. On the other hand, federal institutions increased their share in distance learning in other specific subjects’ programs. In particular, their participation in the total number of graduates in math rose from 12.6% to 36.8% in the period, while private institutions reduced their share. Among the graduates in Portuguese, the share of federal institutions went from none to 16% in the period.

To shed light on the potential effects of these recent changes on the quality of future teachers, Tables 2 and 3 present the mean performance of graduates in ENADE<sup>8</sup> exam and a measure of value-added<sup>9</sup> from public and private _licenciatura_ programs in year 2014. In general, ENADE’s results show that top performance students are enrolled in face-to-face federal university and private non-for-profit institutions programs and the bottom performance are in state and private for-profit institutions. Also, federal universities graduates performed better in both training modalities and graduates from private for-profit institution programs showed the lowest results. Finally, besides the better performance of _licenciatura_ graduates in face-to-face programs as a whole, when restricting to distance learning modality this difference almost disappears for graduates in math and pedagogy. In particular, in private institutions, the performance of the distance learning programs was superior to face-to-face programs.

Table 2 – Performance in ENADE by type of higher education institution (2014)

||Peda<br>Face-to-<br>face|gogy<br>Distance|Ma<br><sup>Face-to-</sup><br>face|th<br>Distance|Portu<br><sup>Face-to-</sup><br>face|guese<br>Distance|Oth<br><sup>Face-to-</sup><br>face|er*<br>Distance|T<br><sup>Face-to-</sup><br>face|otal<br>Distance|
|---|---|---|---|---|---|---|---|---|---|---|
|Private|2.42|2.00|1.96|2.07|2.20|1.90|2.29|1.88|2.30|1.94|
|_Profit_|_2.29_|_1.77_|_1.70_|_NA_|_1.92_|_1.77_|_2.09_|_1.68_|_2.13_|_1.75_|
|_Nonprofit_|_2.53_|_NA_|_2.11_|_NA_|_2.36_|_NA_|_2.36_|_1.97_|_2.41_|_2.11_|
|Federal|3.02|NA|2.58|2.36|2.91|NA|2.72|2.25|2.76|2.39|
|State|2.59|NA|1.97|NA|2.08|NA|2.24|NA|2.23|1.92|
|**Total**|**2.50**|**2.48**|**2.20**|**2.17**|**2.32**|**1.79**|**2.44**|**1.96**|**2.41**|**2.08**|

<mark>*</mark> _<mark>Licenciaturas</mark>_ <mark>in physics, chemistry, biology, history and geography. NA: Not available (less than five observations).</mark> _<mark>Note.</mark>_ <mark>Performance in ENADE: The indicator is a result of the weighted average of the standardized scores of the graduates in the General Training test (common for all areas evaluated, weighing 25% of the grade) and in the Specific Knowledge test (75% of the grade).</mark> _<mark>Source.</mark>_ <mark>ENADE/IDD microdata, INEP/MEC.</mark> Table 3 – Value-added by type of higher education institution (2014)

||Peda<br>Face-to-<br>face|gogy<br>Distance|M<br><sup>Face-to-</sup><br>face|ath<br>Distance|Portu<br><sup>Face-to-</sup><br>face|guese<br>Distance|Oth<br><sup>Face-to-</sup><br>face|er*<br>Distance|T<br><sup>Face-to-</sup><br>face|otal<br>Distance|
|---|---|---|---|---|---|---|---|---|---|---|
|Private|2.53|2.35|2.18|2.24|2.68|2.38|2.55|2.37|2.53|2.36|
|_Profit_|_2.50_|_2.23_|_1.94_|_NA_|_2.61_|_2.08_|_2.53_|_2.49_|_2.48_|_2.21_|
|_Nonprofit_|_2.55_|_NA_|_2.32_|_NA_|_2.72_|_NA_|_2.55_|_2.32_|_2.56_|_2.48_|
|Federal|2.34|NA|2.58|2.40|2.59|NA|2.46|2.27|2.49|2.35|
|State|2.30|NA|2.27|NA|2.52|NA|2.44|NA|2.41|2.28|
|**Total**|**2.47**|**2.36**|**2.36**|**2.31**|**2.62**|**2.39**|**2.49**|**2.34**|**2.50**|**2.35**|

<mark>*</mark> _<mark>Licenciaturas</mark>_ <mark>in physics, chemistry, biology, history and geography. NA: Not available (less than five observations).</mark> _<mark>Note.</mark>_ <mark>Value Added: Difference Between Observed and Expected Performances Indicator (Indicador da Diferença entre os Desempenhos Observado e Esperado [IDD]).</mark> _<mark>Source.</mark>_ <mark>ENADE/IDD microdata, INEP/MEC.</mark> In aggregate, private institutions tend to add as much value as federal institutions, and non-for-profit institutions are more capable of adding more value in either training modalities. This advantage is even higher for face-to-face _licenciatura_ programs in pedagogy and Portuguese. On the other hand, public institutions perform better in math programs, but nonprofit private institutions add more value that state institutions. Finally, distance learning tends to add more value only for _licenciatura_ in math for private

> 8 The National Student Performance Test (ENADE) evaluates the skills and competences acquired by the graduates of all the undergraduate courses in Brazil. The first application of ENADE occurred in 2004 and each area of knowledge is evaluated triennially. The most recent edition that evaluated the _licenciatura_ programs occurred in 2014.

> 9 Indicator of Difference Between Observed and Expected Performances (IDD), a value-added indicator calculated by INEP/MEC used to control the performance in the ENADE by pre-university learning level of the incoming students shows the details on the calculation formula are available in INEP technical note No 33/2017/CGCQES/DAES.

6
institutions. Clearly, higher average scores in ENADE achieved by federal institutions programs are partly due to the better learning background of their incoming students. Therefore, the higher value-added of private programs may be due to decreasing returns to the educational production function, whereby it would be easier to obtain return from individuals with poor average educational background.

The above findings allow us to point out a set of factors that may explain why the raise in teacher tertiary education does not seem to have affect students learning in the last decade. Beyond the low educational background of incoming students and the raise in distance learning modality, the present analysis raises other key aspects. First, the preponderance of teachers with _licenciatura_ and the sharp fall in bachelor’s degree in specific areas of knowledge may not have benefited teacher quality. It is well documented that the majority of _licenciatura_ programs assigns little importance in developing core teaching competencies, like teaching techniques and planning and organizing classroom activities (Bruns et al, 2011). But it is also possible that they fail in providing the necessary knowledge on the specific subjects taught too. Second, lack of effects may lie in the mismatch between the specific subject training of the teacher and the discipline he teachers (e.g.: math teachers without the certification of _licenciatura_ or bachelor’s degree in math). The data presently available allow us to test this mismatch hypothesis, as described later.

Finally, the leading role of private institutions and of distance learning modality in the provision of pedagogy courses, while not directly affecting the quality of teaching in secondary education, can have effects on the quality of elementary and early childhood schools and, therefore, compromise the learning ability of students entering high school. This hypothesis will be investigated in an upcoming study.

# **3. Empirical literature on teacher training and student learning**

The quality of teachers and their effect on students' learning is a central theme of educational policy. The international literature indicates that a student with a low performance teacher might end the school year accomplishing less than half of the curriculum planned for the grade, while students with an excellent teacher may accomplish 50 percent more than expected curriculum (Farr, 2010; Hanushek and Rivkin, 2010). Thus, exposure to low-quality teachers for years running can lead to insurmountable deficits of students' learning.

Despite the recent evidence confirming the central role of teacher quality in students’ learning, there is still no consensus on how much schooling and teacher training contribute to a greater or lesser degree, to raise the quality of education. The first works on the productivity of teachers were developed based on the educational production function estimates from cross-sectional data, and regression analysis of aggregated levels of student performance in teacher training measures and several other controls.

In studies conducted until the mid-1990s for public schools in the United States, Hanushek (2003) identifies 170 estimates of the effect of teacher training measures on the performance of students, of which only 14% reported statistically significant results, 9% of which showing positive effects. This percentage drops to zero when the author considers only high-quality studies (based on the estimates of the "value added" by a student of the same state). The author suggests that the difficulty in capturing the effect of teacher education may be in part due to low variability of the indicator between American schools. In another compilation, spanning 63 studies applied to developing countries, Hanushek (2003) reports that 56% of estimates of the effect of teacher's education school performance were positive and significant.

Wayne and Youngs (2003) conducted an extensive review of the existing empirical literature until the early 2000s, when most of the evaluation studies of the effects of the education of teacher passes using student-level longitudinal data and involve broader sets of variables. Most of the work considered indicates that secondary school students learn more math when their teachers have higher training in mathematics, and no effect in relation to the lower grades and to other areas of knowledge. Moreover, the survey indicated that the quality of teacher education (measured by the raking of the institution) has positive relationship on the performance of the students.

Based on a review of a wide range of empirical studies, Rice (2003) reports positive effects on high school mathematics and science achievement of students whose teachers have obtained advanced degrees in these subjects, and reports mixed evidence regarding the impact of advanced degrees at the elementary 7
level. Also, there is a positive effect on high school mathematics achievement from certified teachers only when the certification is in mathematics. Finally, reports that the training institution attended by the teacher has a positive effect on student achievement, particularly at the secondary level, but points out that this can be a reflection of the cognitive ability of the teacher.

Harris and Sass (2011) carry out a survey of the researches developed up to 2010, highlighting the fragility of previous work to control the heterogeneity among students only by their characteristics observed (covariates of students). The authors cite evidence that students who present greater learning capacity and fewer disciplinary problems tend to be allocated in classes with more experienced and qualified teachers, so that the lack of appropriate controls for the unobservable characteristics tends to generate biased estimates of the effects associated with the training of teachers.

Regarding the studies that have addressed the problem of selection bias, Harris and Sass (2011) point out eight studies that used fixed effects at the student level to control for the heterogeneity not observed, and five that explored random assignment experiments between students and teachers or that use natural experiments. None of the studies finds positive and statistically significant effect of the nature of higher education on the future teacher performance and the majority finds no relationship between the selectivity of higher education institution and teacher performance. Finally, with the exception of two studies that have identified significant and positive effects of master’s degree education of teachers on students’ math scores, all other studies either indicate absence of statistically significant effect or even negative effect of holding a postgrad degree on math and language students’ performance.

Since unobservable characteristics of teachers (like their IQ) can influence their pre-service training and also affect their future performance in the classroom, Harris and Sass (2011) incorporate in their study the teachers grade in the college admission exam as a way of controlling the effect of higher education for the pre-college skills, thus avoiding a major source of bias.<sup>10</sup> The authors found no robust evidence that pre-service teachers training affects the productivity of the future teacher.

For Brazil, Louzano (2010) did not find a significant correlation between pre-service training of teachers and the performance of students. Menezes Filho (2007), on the other hand, reported significant impact of teacher's education only on third grade secondary education students, and for teachers with a degree in mathematics. The author also reports that teachers with 50 years or more of age positively affect students' learning. Such studies, however, do not use methods capable of dealing with various issues, in particular with the existing selection bias in the allocation between students, teachers and schools.

# **4. Data and Descriptive Analysis**

In this study we use data from the National Examination of Secondary School (ENEM) and from the School Census for the years 2009 and 2015, which covers proficiency indicators and a broad set of covariates at student and school levels, for potentially all Brazilian schools.<sup>11</sup> In total, there were 20,979 public secondary schools in ENEM microdata, of which 17,214 appeared in the year 2009, 19,847 appeared in the year 2015, and 16,082 occurred in both years. Following the MEC criteria for disseminating the ENEM results, schools with fewer than 10 students from the 3rd grade participating in the exam were excluded, which resulted in 11,506 schools remaining in the sample. Of these, 151 schools with a participation rate above 100% were also excluded, leaving 11,355 schools at the base in both years.

Based on the School Census data, two school-level variables were constructed to characterize teachers’ education for each field or discipline (math and Portuguese): (i) _Percentage of classes taught by teacher with higher education_ , and (ii) _Percentage of classes taught by teacher with higher education in the field_ . The calculation was restricted to the schools with at least three upper secondary school classes for each discipline, so that the indicators reflect at least one class for each series of this teaching stage.<sup>12</sup>

> 10 Harris and Sass (2011) investigate the effects of the specific areas of higher education of teachers ( _Education Major, Math Education Major, English Education Major, Math Major_ e _English Major_ ) so as to take advantage of the greater variability of these variables to try to capture some effects that are statistically significant on student performance.

> 11 ENEM incorporate the Item Response Theory for scores calculation in 2009, when a proficiency scale with an average of 500 points and standard deviation of 100 points was defined, which allowed comparing results of different editions of the exam. Moreover, since this year students’ performance in ENEM has become the main access route to public university, which has significantly enlarged the number participants. The year of 2015 was the last year in which microdata are available. 12 Resulted in 11,336 and 10,857 schools with, respectively, at least three high school math and Portuguese classes in 2009, and in 11,317 and 11,323 schools with, respectively, at least three high school math and Portuguese classes in 2015.

8
Table 4 shows statistics for outcome and teacher education variables, and relevant available covariates in this sample. The average math score dropped from 475.6 to 450.8 points in this period, a reduction of 0.31 standard deviation from the average obtained in 2009. This decrease came together with the increase of the dispersion of results among students, captured by the rise in the standard deviation. Portuguese average score, on the other hand, rose from 477.1 to 490.2 points, an increase of 0.15 standard deviation from the average obtained in 2009, and there was a reduction in the dispersion of scores.

Table 4 – Summary Statistics for Student and School Variables, by year, 2009 and 2011.

||||2009<br>|||||2015<br>|||
|---|---|---|---|---|---|---|---|---|---|---|
|Covariate|obs.|mean|sd.|min.|max.|obs.|mean|sd.|min.|max.|
|||_S_|_tudent le_|_vel varia_|_bles_||||||
|Math score|523,043|475.6|79.6|345.2|939.5|793,728|450.8|85.8|285.4|999.5|
|Portuguese score|523,043|477.1|90.0|224.3|835.6|793,728|490.2|66.4|303.3|795.4|
|Female|523,043|62.5%|48.4%|0%|100%|793,728|59.3%|49.1%|0%|100%|
|Black|466,681|11.2%|31.5%|0%|100%|784,540|12.5%|33.1%|0%|100%|
|Mother secondary|456,105|38.2%|48.6%|0%|100%|755,439|46.2%|49.9%|0%|100%|
|Car|463,789|44.7%|49.7%|0%|100%|793,621|42.0%|49.4%|0%|100%|
|Computer & Internet|463,685|41.7%|49.3%|0%|100%|793,614|56.3%|49.6%|0%|100%|
|Work|446,886|47.3%|49.9%|0%|100%|793,614|34.7%|47.6%|0%|100%|
|Private primary|466,374|14.0%|34.7%|0%|100%|793,614|15.5%|36.2%|0%|100%|
|Evening classes|459,708|40.2%|49.0%|0%|100%|793,613|25.6%|43.7%|0%|100%|
|||_S_|_chool lev_|_el varia_|_bles_||||||
|Infra PNE|11,355|0.193|0.395|0.000|1.000|11,355|0.305|0.460|0.000|1.000|
|Class hours|11,355|4.4|0.5|3.0|8.0<br>|11,355|4.8|1.1|2.7|12.9|
|Class size|11,355|34|6|11|214<br>|11,355|31|6|7|222|
|Participation ENEM|11,355|35.9%|17.4%|1.3%|100%|11,355|54.5%|19.4%|3.8%|100%|
|Math classes higher<sup>*</sup>|11,336|93.0%|18.9%|0%|100%|11,317|95.1%|13.7%|0%|100%|
|Math classes higher field<sup>**</sup>|11,336|71.8%|32.7%|0%|100%|11,317|76.4%|27.9%|0%|100%|
|Portuguese classes higher<sup>*</sup>|10,857|94.9%|15.5%|0%|100%|11,323|96.9%|10.3%|0%|100%|
|Portuguese classes higher field<sup>**</sup>|10,857|84.3%|26.1%|0%|100%|11,323|82.1%|25.4%|0%|100%|

_Note._ Students in the 3rd year of public regular secondary school that participate in ENEM. _Infra PNE_ : _Indicator for school infrastructure_ from Observatory of theNational Education Plan (Observatório PNE). *% classes taught by teacher with higher education. **% classes taught by teacher with higher education in the field. _Source._ ENEM,and School Census microdata.

In the same period, the percentage of classes taught by teachers with higher education increased by about 2 percentage points for both math and Portuguese. This result was accompanied by the increase of 4.6 percentage points in the percentage of math classes taught by teacher with higher education in this field, and the decline of almost 2.2 percentage points in the indicator for Portuguese classes. That is, the growth of teacher-specific training for the discipline, from an aggregate point of view, does not seem to have had a positive relationship with student performance.

Some important changes occurred in the period that may be correlated with students’ results. Regarding the educational offer conditions, the percentage of students attending evening classes decreased from 40.2% to 25.6%, the average duration of classes increased from 4.4 to 4.8 hours, the average number of students per class decreased from 34 to 31, and the school infrastructure indicator increased from 0.193 to 0.305. In relation to students’ profile, the percentage of students who have already worked decreased from 47.3% to 34.7%, the percentage of students with mothers with complete secondary education rose from 38.2% to 46.2%, and the percentage of students with computer and Internet access at home increased from 41.7% to 56.3%. The other covariates showed minor variation.

Table 5 presents the percentiles of the _percentage of classes taught by teachers with higher education_ and _with higher education in the field_ in years 2009 and 2015. While more than 80% of the schools already had all math and Portuguese classes taught by higher-educated teachers in 2009, big changes occurred in the lowest percentiles. For example, 95% of the schools had more than half of its math classes taught by teachers with higher education in 2009 and six year later the same proportion of schools had more of 66.7% of its math classes taught by teachers with that degree. For _higher education in the field_ indicator, more than half of schools already had all Portuguese classes with teachers trained in this discipline but the proportion of math classes with teachers trained in math is much far from being complete. In general, 9
while the changes in the _percentage of classes taught by teachers with higher education_ were concentrated to the very lower percentiles of the distribution, for _higher education in the field_ indicator the changes spread also to higher order percentiles. So schools seem to be converging to universalization of teachers training at the tertiary level, excepting for field training of Portuguese teachers. Table 5 – Percentage of Classes Taught by Teachers with Higher Education

||||||<br>|Percentile<br>|<br>|||||
|---|---|---|---|---|---|---|---|---|---|---|---|
||p5|p10|p15|p20|p30|p40|p50|p60|p70|p80|p90|
|_Math classes h_|_igher_|||||||||||
|2009|50.0%|73.3%|87.5%|100%|100%|100%|100%|100%|100%|100%|100%|
|2015|66.7%|81.0%|90.9%|100%|100%|100%|100%|100%|100%|100%|100%|
|_Portuguese cl_|_asses higher_|||||||||||
|2009|63.6%|82.4%|94.0%|100%|100%|100%|100%|100%|100%|100%|100%|
|2015|76.9%|90.3%|100%|100%|100%|100%|100%|100%|100%|100%|100%|
|_Math classes h_|_igher in the_|_field_||||||||||
|2009|0.0%|16.0%|31.3%|41.7%|60.0%|72.7%|83.3%|95.3%|100%|100%|100%|
|2015|14.3%|33.3%|45.7%|55.0%|66.7%|76.9%|85.7%|95.2%|100%|100%|100%|
|_Portuguese cl_|_asses higher_|_in the fi_|_eld_|||||||||
|2009|20.0%|42.9%|59.4%|69.2%|83.3%|94.4%|100%|100%|100%|100%|100%|
|2015|25.0%|44.4%|56.3%|64.3%|76.9%|86.7%|95.7%|100%|100%|100%|100%|

It would be more interesting to see how many schools increased the indicator and to what degree this happened. Tables 6 and 7 show the transition of the schools between percentile strata shown in Table 5, based on the percentile intervals in 2009. The blue cells show how many schools managed to raise the indicators in the period and the red cells show how many schools faced a decrease in indicators.

Table 6 – Number of schools by _Percentage of Classes Taught by Teachers with Higher_ _<u>Education</u>_ – Transition between percentile strata, 2009 and 2015.

|_Math c_|_lasses_<br>|<br>(87.5%, 100%]|<br>(73.3%, 87.5%]|<br>2015<br>(50.0%, 73.3%]|<br>(0%, 50.0%]|Total|
|---|---|---|---|---|---|---|
||(87.5%, 100%]|<br>**8,545**|**508**|**370**|**162**|9,585|
||(73.3%, 87.5%]|<br>**454**|65|36|27|582|
|2009|(50.0%, 73.3%]|<br>**414**|55|45|24|538|
||[0%, 50.0%]|**386**|62|72|75|595|
||Total|9,799|690|523|288|11,300|
|_Portug_|_uese classes_|||2015|||
|||(94.0%, 100%]|(82.4%, 94.0%]|(63.6%, 82.4%]|(0%, 63.6%]|Total|
||(94.0%, 100%]|<br>**8,265**|**400**|**350**|**185**|9,200|
||(82.4%, 94.0%]|<br>**441**|55|28|17|541|
|2009|(63.6%, 82.4%]|<br>**424**|48|41|22|535|
||[0%, 63.6%]|**397**|55|58|41|551|
||Total|9,527|558|477|265|10,827|

_Note._ Strata were constructed based on 5th, 10th, and 15th percentiles. Blue (red) color indicates schools that moved the indicator to a superior (inferior) stratus between years 2009 and 2015.

Table 7 – Number of schools by _Percentage of Classes Taught by Teachers with Higher Education in_ _<u>the Field</u>_ – Transition between percentile strata, 2009 and 2015.

|_Math c_|_lasses_|<br>(72.7%, 100%]|<br>(60.0%, 72.7%]|<br>2015<br>(41.7%, 60.0%]|<br>(16.0%, 41.7%]|<br>(0%, 16.0%]|Total|
|---|---|---|---|---|---|---|---|
||(72.7%, 100%]|<br>**4,986**|**674**|**576**|**322**|**179**|6,737|
||(60.0%, 72.7%]|<br>**630**|163|155|82|40|1,070|
|2009|(41.7%, 60.0%]|<br>**684**|185|209|105|53|1,236|
||(16.0%, 41.7%]|<br>**519**|144|196|179|95|1,133|
||[0%, 16.0%]|**445**|106|171|173|229|1,124|
||Total|7,264|1,272|1,307|861|596|11,300|
|_Portug_|_uese classes_|||2015||||
|||(94.4%,100%]|(83.3%, 94.4%]|(69.2%, 83.3%]|(42.9%, 69.2%]|(0%, 42.9%]|Total|
||(94.4%,100%]|<br>**3,812**|**743**|**790**|**702**|**444**|6,491|
|2009|(83.3%, 94.4%]|<br>**474**|168|185|149|78|1,054|
||(69.2%, 83.3%]|<br>**477**|148|197|179|114|1,115|

10
|(42.9%, 69.2%]|**374**|136|174|228|169|1,081|
|---|---|---|---|---|---|---|
|[0%, 42.9%]|**304**|97|154|251|280|1,086|
|Total|5,441|1,292|1,500|1,509|1,085|10,827|

_Note._ Strata were constructed based on 10th, 20th, 30th, and 40th percentiles. Blue (red) color indicates schools that moved the indicator to a superior (inferior) stratus between years 2009 and 2015.

In Table 6, for example, note that some schools that had high _percentage of classes taught by teachers with higher education_ in 2015 (above 87.5% for math classes and above 94% for Portuguese classes) was in lower strata of the distribution in 2009 (below 87.5% for math classes and below 94% for Portuguese classes): 1,254 of 9,799 schools and 1,262 of 9,527 schools, respectively, considering the indicator for math and Portuguese classes. On the other hand, other schools that had a high _percentage of classes taught by teachers with higher education_ in 2009 passed to lower levels of distribution in 2015: 1,040 from 9,585 schools and 935 from 9,200 schools, respectively for math and Portuguese classes.

These transitions from lower strata to the highest stratum of the distribution can be used to characterize school treatment condition in the following sense: schools that went from a lower strata to the greater stratum of the distribution are considered to have being treated, since they showed improvement in the indicator in the period; and schools that went from the highest stratum to a lower strata of the distribution are considered to have undergone a _negative treatment_ . Due to its large number, schools that remained in the highest stratum of the distribution are considered as control group.

Based on the above strategy we constructed a set of variables that characterize different degrees of exposure to treatment, here understood as the increase (or decrease, for _negative_ treatment) in the _percentage of classes taught by teachers with higher education_ and _by teachers with higher education in the field_ . We subdivide the exposition to treatment into three degrees (and four degrees, for the _higher education in the field_ indicator) according to the stratum in which the school was in year 2009: treatment group 1 ( _Treated_ 1) is composed of schools whose indicator was in the stratum immediately below the highest, treatment group 2 ( _Treated_ 2) is composed of schools whose indicator was in the next stratum, and so on; so that the greater the order of treatment, the greater its intensity is. Similarly, the degree of _negative_ treatment can also be characterized in degrees, according to the stratum of the school in 2015.

We see in Table 8 that for both years and disciplines the average scores are higher among students from untreated schools, and are smaller for more intensively treated schools. This is consistent with the hypothesis of positive correlated between teachers’ education and students learning. It would be also expected a higher increase in outcome among treated schools, especially among the most intensively treated ones. Data on Portuguese scores corroborate with this hypothesis, but not for math, as students from treated and untreated schools perceived similar reduction in the average scores.

<u>Table 8 – Mean score on ENEM by condition of exposure to treatment</u>

||||Math<br>||||<br>|Portuguese<br>|<br>||
|---|---|---|---|---|---|---|---|---|---|---|
||20|09|20|15|∆ Mean|20|09|20|15|∆ Mean|
|Treatment|Mean|SD|Mean|SD||Mean|SD|Mean|SD||
||||_Te_<br> <br>|_achers w_<br>|_ith higher edu_<br>|_cation_<br>||<br>|||
|Control|479.1|(81.0)|<br>454.4|(87.0)|-24.7|480.7|(90.2)|<br>493.3|(66.2)|12.6|
|Treated 1|463.5|(72.2)|<br>439.0|(77.5)|-24.5|463.7|(87.5)|<br>478.6|(65.8)|14.9|
|Treated 2|461.1|(72.2)|<br>437.2|(80.1)|-23.9|460.8|(88.6)|<br>477.5|(66.8)|16.7|
|Treated 3|455.6|(69.9)|<br>431.1|(74.2)|-24.5|448.5|(85.0)|<br>469.7|(64.6)|21.2|
|Treated 1,2,3|461.0|(71.8)|<br>436.7|(77.8)|-24.3|459.4|(87.5)|<br>476.3|(66.0)|16.9|
|Treated 2,3|458.9|(71.4)|<br>434.9|(78.0)|-24.0|456.0|(87.4)|<br>474.6|(66.1)|18.6|
||||_Teache_<br> <br>|_rs with hi_<br>|_gher educatio_<br>|_n in the field_<br>|<br>|<br>|||
|Control|481.8|(82.6)|<br>457.5|(89.0)|-24.3|487.1|(90.6)|<br>498.4|(65.8)|11.3|
|Treated 1|472.2|(76.8)|<br>446.8|(83.2)|-25.4|477.8|(90.4)|<br>491.4|(67.6)|13.6|
|Treated 2|470.4|(76.1)|<br>444.6|(81.4)|-25.8|472.1|(90.2)|<br>486.4|(67.1)|14.3|
|Treated 3|466.5|(73.4)|<br>441.9|(80.0)|-24.6|465.4|(88.3)|<br>481.2|(66.0)|15.8|
|Treated 4|460.8|(72.7)|<br>437.2|(78.5)|-23.6|448.1|(85.9)|<br>469.5|(65.9)|21.4|
|Treated 1,2,3,4|468.7|(75.3)|<br>443.6|(81.3)|-25.1|469.5|(89.8)|<br>484.5|(67.3)|15.0|
|Treated 2,3,4|467.0|(74.5)|<br>442.0|(80.4)|-25.0|464.7|(89.2)|<br>480.8|(66.8)|16.1|

_Note._ Students in the 3rd year of public regular secondary school that participate in ENEM in the years 2009 or 2015. _Source. ENEM_ , 2009 and 2015, microdata.

11
From the perspective of the _negative_ treatment, it would be expected that the mean increase (decrease) in the outcome indicator would be smaller (greater) among treated schools, the most intensively treated schools performing relatively worse. However, with the exception of the exposition of the students of math classes to higher educated teachers, there is no clear pattern indicating adherence to the hypothesis (Table 9). Average scores in math decreased most for treated students, as expected. Also, average scores in Portuguese increased the least for treated students, except for the most intensively treated school ( _Treated_ 3), which showed similar advance in scores. Similar results are found when analyzing the treatment in terms of _percentage of classes taught by teacher with higher education in Portuguese_ ( _Treated_ 3 and 4), which may reflect schools’ strategy of exchanging higher educated but lower performance teachers by newer teachers, who perform better despite the lower schooling.

<u>Table 9 – Mean score on ENEM by condition of exposure to</u> _<u>negative</u>_ <u>treatment</u>

||2009||Math<br>2015|||2009||Portugues<br>2015|e||
|---|---|---|---|---|---|---|---|---|---|---|
||||||||||||
|Treatment|Mean|SD|Mean|SD|ean|Mean|SD|Mean|SD|ean|
||||<br> <br>|_Teachers w_<br>|_ith higher ed_<br>|_ucation_<br>||<br>|||
|Control|479.0|(81.0)|<br>454.4|(87.0)|-24.7|480.7|(90.2)|<br>493.3|(66.2)|12.6|
|Treated 1|474.6|(78.1)|<br>448.3|(85.1)|-26.3|477.8|(88.5)|<br>488.2|(65.1)|10.4|
|Treated 2|472.9|(75.0)|<br>445.0|(81.6)|-27.9|475.8|(91.4)|<br>486.5|(67.7)|10.7|
|Treated 3|466.5|(74.4)|<br>440.1|(78.3)|-26.4|468.7|(87.3)|<br>481.1|(64.4)|12.4|
|Treated 1,2,3|473.1|(76.7)|<br>446.3|(83.2)|-26.8|475.8|(89.4)|<br>486.6|(66.0)|10.8|
|Treated 2,3|471.4|(74.9)|<br>443.9|(80.8)|-27.5|473.8|(90.4)|<br>485.1|(66.9)|11.3|
||||_Teach_<br> <br>|_ers with hi_<br>|_gher educatio_<br>|_n in the fiel_<br>|_d_<br>|<br>|||
|Control|481.8|(82.6)|<br>457.5|(89.0)|-24.3|487.1|(90.6)|<br>498.4|(65.8)|11.3|
|Treated 1|479.4|(80.3)|<br>454.9|(87.4)|-24.5|487.4|(89.3)|<br>496.6|(65.2)|9.2|
|Treated 2|478.1|(79.3)|<br>452.6|(84.8)|-25.4|486.2|(91.1)|<br>496.6|(65.7)|10.4|
|Treated 3|471.1|(75.6)|<br>446.7|(81.3)|-24.4|480.8|(89.0)|<br>492.4|(65.7)|11.6|
|Treated 4|475.9|(86.8)|<br>446.6|(81.1)|-29.3|467.5|(87.1)|<br>487.5|(64.6)|19.9|
|Treated 1,2,3,4|477.4|(79.7)|<br>452.4|(85.3)|-25.0|482.8|(89.7)|<br>494.5|(65.5)|11.7|
|Treated 2,3,4|475.7|(79.2)|<br>450.3|(83.5)|-25.4|480.5|(89.9)|<br>493.5|(65.6)|13.0|

_Note._ Students in the 3rd year of public regular secondary school that participate in ENEM in the years 2009 or 2015. _Source. ENEM_ , 2009 and 2015, microdata.

# **5. Causal inference strategy**

What effect did the growth in higher education schooling of upper secondary school teachers have on students’ learning? One way to answer this question would be to compare the performance of the students in schools with high percentage of teachers with a degree (treated group) with the performance in schools that did not raise this percentage (comparison group). The main challenge we confront by doing this, however, is to address the problem of selection bias that may affect the estimation of this effect. This bias occurs when schools with certain characteristics, like the presence of more motivated and intelligent students, which have higher chances to achieve higher performance, are the most attractive ones for teachers with higher education. This entails in an overestimation of the treatment effects due to reverse causality.<sup>13</sup> Does being enrolled in treated schools imply better student performance? Or do characteristics related to the performance of students imply more treatment schools?

In this sense, it is necessary to compare the performance of treated schools with a group of schools that have similar chances to be treated, even if they were not. We do this by applying difference-indifference estimation on matched samples obtained by propensity score matching, which allows us to control for several observed and unobserved school characteristics that may contribute to selection bias. Adapted from Cameron and Trivedi (2005), in a model of differences-in-differences, considering a school panel of two periods, the _ATT_ can be obtained from the  parameter in equation (1):

> 13 It is expected that better schools attract or select higher performance teachers. As an example, Hanushek et al (2004) find that teachers in Texas tend to move to schools with more proficient students, especially the more experienced ones, who often have the option of going through districts and choose the school within the district in which they teach. Thus, it is necessary to isolate the characteristics of students and schools in order to guarantee that the odds of attracting/retaining teachers with higher education are similar for both treatment and control schools. In this sense, the treatment status between schools with same characteristics would be random (conditional independence assumption).

12
ist = Ap +ayt + yD, + d(tD,) + Xj,8 + Wo F ise 0 1 st st E(Yo,-1| D; = 1) — E(Ys,-1| D, = 0) = E(¥,| D, = 1) — E(Y2,| D, = 0) = E(YS; ~~ Yo.-1| D, = 1) = E(Y¢, ~~ Ye.-41| Ds = 0) > E(AY?,) 1 D, E(AY2,) 1 Ds

ATT = E(AY,,| D, = 1) — E(AY,,| D, = 0)

E(D; = 1| X;)

(YP LD: i i
null hypothesis that the means of control and treatment groups covariates are equal (balancing property). If null hypothesis is not rejected, use the weighted sub-sample in common support to estimate the _ATT_ . In the end, it is expected that the estimated propensity scores densities are overlapping, what indicates the two groups of schools in the common support present similar pre-treatment characteristics.

In summary, to measure the effects of teacher education on the performance of schools, we first match the schools based on propensity scores, obtained by estimating a _logit_ model of _ex ante_ probability to be treated. Then, we apply, as a benchmark, the method of Radius matching, with maximum distance ( _radius_ ) of 0.01. Then, the parameters of equation (1) are measured by fixed effects estimators at the school level. To evaluate the sensitivity of the results to the matching method, the _ATT_ is also estimated on the matched sample by Kernel method (bandwidth calculated by Silverman rule). The procedures discussed here will apply to public upper secondary schools, observed at years 2009 and 2015.

The implementation of this strategy requires that exposure to treatment be characterized by binary variables. We have already defined them as the three (or _four_ ) dummy variables constructed from the percentiles of the teachers’ education indicators in the previous section. Such characterization allows the estimation of effects for different degrees of exposure to the treatment, with a sufficiently large number of schools in each of the _n_ treated groups and a higher number of schools in the control group.

# **6. Results**

In order to provide a basis of comparison for the causal inference strategy, Table A1 in the Appendix shows the results of the fixed effects regressions of the students scores on the _percentage of classes taught by teachers with higher education_ and _by teachers with higher education in the field_ , controlling by a set of students and schools characteristics. The coefficients related to these two explanatory variables were statistically significant only for students’ Portuguese scores.

Although statistically significant, the magnitude of the coefficients of our two explanatory variables imply very low effects, when compared to the magnitude of other variables that may affect students’ scores. For example, if the schools in 5th percentile (as presented in Table 7 of the _Descriptive Analysis_ section) increased the Percentages of Classes Taught by Teachers with Higher Education and by Teachers with Higher Education in Portuguese to 100% we would expect an increase of only 1.5 and 2.4 points in Portuguese scores, respectively, less than 0.03 standard deviations from the year 2009’s mean.

The estimated coefficients suggest strong positive effects on students’ scores from the following attributes: mother’s education, presence of computer and Internet access at home, and having attended private primary education. Having a car at home ( _proxy_ for income) is positively correlated to math scores but negatively correlated to Portuguese scores. Having worked or being working is negatively correlated and having attended evening classes during secondary school is highly negatively correlated to students’ math and Portuguese scores. Students’ access to the Internet at home is positively correlated with performance in the exam. Also, a higher proportion of female students in school positively affects Portuguese scores and negatively affects math scores, the duration of classes is positively correlated with students’ math scores and the class size is negatively correlated with Portuguese scores, and the school infrastructure indicator did not capture any effect. Finally, the addition the _participation rate in_ ENEM and the _number of secondary school classes_ does not affect the results for the explanatory variables. In fact, the participation rate in the ENEM is negatively correlated to the Portuguese scores, which may indicate the presence of self-selection in the exam, as explained below.

The use of ENEM scores as output indicator may present a selection bias problem, since student enrollment is voluntary and the lower performance students have less incentive to participate in the exam. Thus, the lower performance schools are expected to have lower participation rates in the exam. On the other hand, a low participation rate may result from a deliberate action by the school administration to encourage only the top performance students (or discourage the lower performance) to attend the exam, which may artificially raise the school's average grade. To verify the sensitivity of the results in relation to this potential selection bias problem, we re-estimate the fixed effects regressions for a restricted sample, 14
which excludes schools whose participation rates in 2009 were below the median of the distribution.<sup>17</sup> After carrying out the estimation, the coefficients for our two explanatory variables related to teachers education become statistically non-significant also for Portuguese scores, and do not change significantly the estimated coefficients of the other covariates.<sup>18</sup> As we have seen in last section the validity of such estimates can be compromised by the presence of teacher selection bias, since chances to be employed in better schools are higher for those teachers with higher education. To deal with this problem, following the strategy of propensity score matching, we first estimate _logit_ models of probability of treatment on a set of pre-treatment schools characteristics for each of the binary treatment variables defined in Section 4. The set of covariates used in this model and the criteria for their inclusion can be seen in Table 10.

Table 10 – Covariates of the _logit_ model of _ex ante_ probability to be treated

|**Variable**|**Description**|**Justification**|
|---|---|---|
|Metropolitan Area|Dummy for school located in metropolitan area|Controls for the effects of|
|Population|Natural logarithm of the population of schools municipality.|the attractiveness of the|
|Per capita income|Per capita income of schools municipality population.|municipality.|
|University campus|Dummy for university campus in the municipality.|Controls for the effects of|
|University centre campus|Dummy for university centre campus in the municipality.|the offer of higher education|
|College campus|Dummyfor college campus in the municipality.|in municipality.|
|Federal School|Dummy for Federal Secondary School.||
|Public water network|Dummy for school connected to public water network.||
|Public sewerage system<br>Science lab<br>Computer lab<br>Library|Dummy for school connected to public sewerage system.<br>Dummy for school that has science lab.<br>Dummy for school that has computer lab.<br>Dummy for school that has library.|Control for schools<br>infrastructure that may<br>affect the admission of<br>teachers with better|
|Internet connection|Dummy for school that has internet connection.|<br>education|
|Sports court|Dummy for school that has sports court.|.|
|High School students<br>High School classes|Natural logarithm of the number of students in high school.<br>Number of high school classes.||
|Age-grade distortion<br>|Age-grade distortion of 1st grade high schools students.<br>|Control for the influence of<br>|
|Class size|Average number of students per class.|the teaching effort on the|
|Class hours|Average number of hours of tuition.|school's chances of|
|Eveningclasses|% of students studyingat eveningclasses .|attractingteachers.|
|Female|% female students.|Control for students and|
|Black|% black students.|families socioeconomic|
|Mother low schooling|% students whose mother schooling is bellow the elementary.|status, which may influence|
|Car<br>|% students whose familyown a car(proxyfor familyincome).|teachers’ decisions.<br>|

We then apply the Radius matching method to generate weights for the schools, in order to guarantee that an appropriate (that attend to both _selection on observables_ and _common support_ assumptions) matching between treated and control schools. Once paired the samples for all treatment conditions, differences in means between control and treatment groups of all covariates become statistically non significant, which support the validity of _selection on observables_ assumption. Also, for all paired samples the estimated propensity scores densities are overlapping, which indicates that the schools in the common support present similar pre-treatment characteristics ( _balancing property_ ).<sup>19</sup> Table 11 presents a summary of the Treatment Effect on Treated Schools, estimated from Equation (1) controlling by the same set of student and school characteristics considered in previous regressions, for both no-matched and matched samples.<sup>20</sup> The coefficients for the no-matched sample [column (1)] are positive and statistically significant only for the schools that experienced the greatest increase in percentage of Portuguese classes taught by teachers with higher education ( _Treated_ 3) and with Higher Education in

> 17 MEC established a minimum participation rate in ENEM of 50% of the students for a school to have its results reported. As this filter would restrict the sample size to only 1,907 schools, we chose to restrict the sample of schools based on the median of the 2009 participation rate distribution, which was 33.3%, which resulted in 5,349 schools remained in the base.

> 18 Output for restricted sample estimation provided upon request.

> 19 Full _logit_ regressions outputs, tables containing mean comparisons between groups for all covariates, and graphics of the densities are provided upon request.

> 20  Full regression outputs provided upon request.

15
Portuguese ( _Treated_ 4). However, for matched samples [Columns (2) and (3)], the first coefficient becomes statistically non-significant and the coefficient of _Treated_ 3 related to Higher Education in Portuguese becomes statistically significant at 10% of significance.

Thus, the initial results indicate that the only change in teachers’ education indicators capable of affecting students’ scores was that related to the increase in Percentage of Portuguese Classes Taught by Teachers with Higher Education in Portuguese, especially among the schools that had the highest increases ( _Treaded_ 4). For these schools, whose indicator was below 43% [or 20.1%, on average] in the year 2009 and has risen above 94% [or 99.9%, on average] in the year 2015, the estimated _ATT_ is between 6.9 and 7.5 points in Portuguese exam, what implies in an mean increase between 0.086 and 0.094 points for every 1 percentage point increase in teachers education indicator. This estimated effect is about 3 times the one previously estimated in the regressions without characterizing the treatment from binary variables, which considers all schools [see Table 11, Columns (2) and (3)].

Table 11 – Estimates of Treatment Effects on Mathematics and Portuguese Scores

||||All schools<br>||Participati<br>|on restriction<br>|on schools<br>|
|---|---|---|---|---|---|---|---|
|Teachers<br>education|Treatment<br>group|No<br>matching<br>(1)|Radius<br>matching<br>(2)|Kernel<br>matching<br>(3)|No<br>matching<br>(4)|Radius<br>matching<br>(5)|Kernel<br>matching<br>(6)|
||||_Math sc_<br>|_ores_<br>||||
||Treated1|1.496|2.386|2.454|-1.082|-0.568|-0.620|
|Hih|Treated 2|1.116|1.684|1.672|1.215|0.057|0.684|
|ger<br>education|Treated 3|0.731|0.838|1.692|3.716|4.148|5.185*|
||Treated 2,3|0.368|1.146|1.395|3.653*|4.533**|4.191*|
||Treated 1,2,3|0.602|0.970|1.141|1.780|2.371|2.336|
||Treated 1|0.033|-0.442|-0.349|-0.352|-0.217|-0.251|
|Hih|Treated 2|-0.048|0.289|0.390|-0.283|0.101|0.236|
|ger<br>education in|Treated 3|0.277|-0.177|-0.106|-2.688|-1.682|-1.285|
|<br>Math|Treated 4|-0.302|1.398|1.375|2.065|2.658|3.088|
||Treated 2,3,4|-0.471|0.710|0.640|0.0639|0.426|0.146|
||Treated 1,2,3,4|0.0670|0.628|0.614|0.648|0.886|0.793|
||||_Portugues_<br>|_e scores_<br>||||
||Treated 1|0.808|-0.403|-0.385|1.358|1.005|0.932|
|Hih|Treated 2|1.633|0.359|0.385|1.725|1.652|1.666|
|ger<br>education|Treated 3|5.017**|2.066|2.114|4.365|4.044|4.185|
||Treated 2,3|2.533|0.707|0.672|1.391|1.099|0.967|
||Treated 1,2,3|1.831|0.307|0.545|2.321|1.324|1.212|
||Treated 1|2.909|2.972|2.357|1.917|1.651|1.745|
||Treated 2|-1.189|-0.882|-1.433|0.211|-0.601|-0.836|
|Higher<br>datin in|Treated 3|3.544|4.543*|4.367*|2.483|2.335|1.805|
|euco<br>Portuguese|Treated 4|7.598***|7.529***|6.908**|7.221**|3.980|4.275|
||Treated 2,3,4|1.778|2.643|2.595|2.302|1.189|1.442|
||Treated 1,2,3,4|1.724|2.736*|2.506|2.583|1.947|2.047|

_Note._ School level Fixed Effects Model with linear trends by UF (omitted). Covariates were omitted. Cluster-robust standard errors in parentheses. *Statistically significant at 10% significance level. **Statistically significant at 5% significance level. ***Statistically significant at 1% significance level.

Again, in order to verify the robustness of the results in relation to the presence in the sample of schools with low student participation rates in ENEM, we re-estimate the model for the restricted sample, which excludes schools whose participation rate in 2009 was below the median of the distribution. As we can see, the estimated coefficients for the non-matched sample are all statistically non-significant, with the exception of the coefficients for _Treated_ 2,3 related to the effect of the increase in the Teachers Higher Education on math scores and for _Treated_ 4 related to the effect of the increase in the Teachers Higher Education in Portuguese on Portuguese scores.

In regressions on paired samples [Columns (5) and (6)], however, the only coefficients that remains statistically significant are those for _Treated_ 2,3 related to the effect of the increase in the Teachers Higher Education on math scores, and the coefficient of _Treated_ 3 related to the effect of the increase in the Teacher Higher Education on math scores becomes statistically significant at 10% significance for the sample 16
matched by the Kernel method. In special, the coefficients related to Higher Education in Portuguese on Portuguese scores become all statistically non-significant.

In summary, the results indicate that the increase in Teacher Higher Education in Portuguese that occurred in _Treated_ 3 and _Treated_ 4 schools (most intensively treated schools) may have positively affected students’ Portuguese scores, but that this result is not robust to schools’ participation restriction on ENEM. Also, the increase in Teacher Higher Education seems to have positively affected students’ math scores of _Treated_ 2,3 schools (those whose indicator was below 73.3% in 2009 and raised above 87.5% in 2015). For these schools, whose indicator was below 73.3% [or 45.8%, on average] in the year 2009 and has risen above 87.5% [or 99.2%, on average] in the year 2015, the estimated _ATT_ is between 4.2 and 4.5 points in math exam, what implies in an mean increase between 0.079 and 0.084 points for every 1 percentage point in Percentage of Math Classes Taught by Teachers with Higher Education.

Table 12 presents a summary of the so called _negative_ Treatment Effect, also estimated from Equation (1) controlling by the same set of characteristics for both the whole sample and matched samples. Again, the estimated results on the paired samples are statistically non-significant, except for the effect on Portuguese scores of school in Treatment group 2 in relation to the Percentage of Portuguese Classes Taught by Teachers with Higher Education in Portuguese, whose coefficient is positive and significant at 10% significance only when considering the sample matched by Radius method.

Table 12 – Estimates of _Negative_ Treatment Effects on Mathematics and Portuguese Scores

||||All schools<br>||Participati|on restriction<br>|on schools<br>|
|---|---|---|---|---|---|---|---|
|Teachers<br>education|Treatment<br>group|No<br>matching<br>(1)|Radius<br>matching<br>(2)|Kernel<br>matching<br>(3)|No<br>matching<br>(4)|Radius<br>matching<br>(5)|Kernel<br>matching<br>(6)|
|||<br>|<br>_Math sc_<br>|<br>_ores_<br>|<br>|<br>|<br>|
||Treated 1|-2.025|-1.400|-1.427|1.740|1.719|1.943|
|Hih|Treated 2|-1.078|-0.539|-0.713|-0.625|-0.436|-0.397|
|ger<br>education|Treated 3|2.305|2.539|2.211|3.053|4.491|4.342|
||Treated 2,3|0.770|0.566|0.609|1.026|0.634|0.660|
||Treated 1,2,3|-0.835|-0.163|-0.102|0.492|1.026|0.891|
||Treated 1|-1.135|-1.635|-1.557|0.495|0.213|0.219|
||Treated 2|-0.732|-0.439|-0.508|0.371|0.628|0.663|
|Higher<br>dtin in|Treated 3|0.594|1.891|1.786|0.465|1.798|1.899|
|eucao<br>Math|Treated 4|-2.893|-2.424|-2.254|1.251|1.074|0.690|
||Treated 2,3,4|0.121|0.786|0.791|1.276|1.760|1.712|
||Treated 1,2,3,4|-0.483|-0.487|-0.533|1.211|1.148|1.225|
||||_Portugues_|_e scores_||||
||Treated 1|-0.0589|0.295|0.0774|0.894|1.324|1.181|
|Hih|Treated 2|1.019|0.129|-0.0246|2.362|2.961|2.773|
|ger<br>education|Treated 3|0.732|0.166|0.563|3.774|1.229|1.254|
||Treated 2,3|0.128|-1.100|-1.049|4.026|2.037|2.016|
||Treated 1,2,3|0.378|-0.650|-0.605|4.244**|3.630*|3.523*|
||Treated 1|-2.463*|-1.198|-1.251|-2.070|-2.225|-1.714|
||Treated 2|1.661|2.398*|2.079|0.359|0.686|0.763|
|Higher<br>di i|Treated 3|0.0466|-0.303|-0.387|1.544|1.498|1.840|
|eucaton n<br>Portuuese|Treated 4|4.105**|1.246|2.310|3.090|2.542|2.011|
|g|Treated 2,3,4|1.008|0.292|0.274|0.233|-0.227|-0.375|
||Treated 1,2,3,4|-0.136|-0.0840|-0.355|-0.123|-0.595|-0.537|

_Note._ School level Fixed Effects Model with linear trends by UF (omitted). Covariates were omitted. Cluster-robust standard errors in parentheses. *Statistically significant at 10% significance level. **Statistically significant at 5% significance level. ***Statistically significant at 1% significance level.

After restricting the sample by participation in ENEM exam, the only statistically significant coefficients are those related to the effect of the decrease in Teacher Higher Education on _Treated_ 1,2,3 schools scores in Portuguese (schools whose indicator was above 72.7% in 2009 and dropped to below 72.7% in 2015). These schools seem to have benefited from the reduction in the percentage of higher educated teachers in the period, which may reflect a strategy of exchanging higher educated but lower performance teachers by newer teachers, who perform better despite the lower schooling.

17
The results do not allow us to reject the null hypothesis of no effect on Portuguese scores, and despite the statistically significant positive effects on math scores from the increase in _percentage of math classes taught by teachers with higher education_ , we find no statistically significant effect from _teachers with higher education in math_ . The lack of effect on Portuguese scores is consistent with the several studies that find no correlation between teachers’ education and student learning (Hanushek and Rivkin (2006), Harris and Sass (2011)), and the positive effect on math scores partly contradicts the findings of Wayne and Youngs (2003), that reports that upper secondary school students learn more math only when their teachers have higher training in mathematics, and no effect in relation to other areas of knowledge.

# **7. Conclusions**

There are only few rigorous studies that relate teachers’ educational background and training to the students' performance. Given the total amount of public and private investments to provide higher education training and the central role of teachers in student learning, understanding the relations between pre-service training and teachers’ quality is essential to shed light on the changes capable of reversing the learning crisis in Brazil. The present study aimed at contributing to fill this gap in applied research.

We estimated the _ATT_ effect of the change in the _percentage of secondary school classes taught by teachers with higher education_ and _with a higher education degree in the field_ on the achievement in math and Portuguese of 12th grade students of public schools at ENEM, through a combination of differencesin-differences and propensity score matching methodologies. In summary, we find no evidence of positive effects on Portuguese scores, and despite the statistically significant positive effect of the rise in teachers with higher education on math scores, we find no effect from specific math training.

The incapacity of teacher education to influence students’ learning can have diverse origins. First, the quality of the teacher may be mostly determined by characteristics other than pre-service training, such as school performance records and innate skills. In this sense, the admission of weak educational background individuals in pre-service training could explain the absence of impact on the quality of the teaching. This seems consistent with the fact that tertiary teacher training courses are among the easiest ones to access in Brazil. Low competition to enroll in these courses reflects the low attractiveness to higher performance graduates of basic education.

A second explanation is based on the idea that lack of effect may be due to the inadequacy and/or poor quality of teacher training programs themselves, assuming that teacher quality is a function of preservice training.<sup>21</sup> In this regard, Wayne and Youngs (2003) and Rice (2003) find positive effects of the quality of teachers’ pre-service education on students’ learning. Section 2 shed some light on the issue for the Brazilian case, where there was a significant change in the profile of teacher training courses, with the rapid expansion of distance learning and other recent changes in the higher education provision structure.

<mark>Our results point to a fragility in the training of teachers in the field of both math and Portuguese. It is likely that</mark> _<mark>licenciatura</mark>_ <mark>courses are failing to compensate for the learning deficits accumulated by their graduates during basic education. It is also possible that the incorporation of bachelor’s degree holders with stronger knowledge in these fields may contribute to raise the quality of upper secondary education. In this regard, the</mark> _pedagogical supplementation_ certification <mark>may contribute to raise the quality of teaching. However, t</mark> he percentage of these certified teachers from 11.4% to 3.3% between 2011 and 2015 and <mark>only 1.2% of math classes and 0.6% of Portuguese classes in public upper secondary education were taught by those teachers in 2015. This low share may be due to a lack of supply in</mark> _<mark>pedagogical supplementation</mark>_ <mark>certification courses, the excessive increase of required duration for certification, selection barriers in the official exams, or simply the absence of interest of bachelor’s degree holders in joining a teaching career.</mark> <mark>The convergence of teacher wages of public upper secondary school towards the average wages received by other professionals with higher education degrees and the reform of the national basic education core curriculum may influence professionals with non-education degrees to enter a teaching career and the universities to change their pre-service curriculums. Incentives could be created to promote both changes through: (i) a reduction of the duration of</mark> _<mark>pedagogical supplementation</mark>_ <mark>for a specialization of 360h to allow join the public system top performance professionals with higher education (it requires a change in the</mark>

> 21 An extensive analysis on the key issues involving teacher quality in Latin American and Caribbean countries can be seen in Bruns and Luque (2015).

18
<mark>resolution of the CNE); (ii) promoting colleges or universities to restructure their curriculum of</mark> _<mark>licenciatura</mark>_ <mark>degrees in accordance with the new national core curriculum through requirements in students’ loans in private institutions and scholarship programs in public universities.</mark> <mark>It is still necessary, however, to deepen the analysis in search of understanding the reasons for the low effectiveness of training teachers in leveraging students’ learning in public schools.</mark> 19