metadata: authors: 0: author: De Hoyos Navarro,Rafael E.
1: author: Estrada,Ricardo
2: author: Vargas,Maria Jose
count: Mexico
docna: 0: docna: Predicting individual wellbeing through test
            scores : evidence from a national assessment in Mexico
docty: Policy Research Working Paper
subtopic: Educational Sciences,Rural Labor Markets,Labor Markets,Educational Institutions & Facilities,Effective Schools and Teachers,Health Care Services Industry
teratopic: Industry,Social Protections and Labor,Education,Rural Development
abstracts: cdata!: This paper constructs two longitudinal
            datasets that record students' test scores in a
            national standardized exam in Mexico and track students from
            the end of primary (Grade 6) to the end of lower (Grade 9)
            and upper (Grade 12) secondary school, then to university
            and labor market participation up to two years after
            graduation from upper secondary. The results show that test
            scores are a strong predictor of future education and labor
            market outcomes. Using a large sample of twins in the data,
            the paper shows that the relationship between Grade 6 test
            scores and future education outcomes goes beyond family
            background. Finally, the paper exploits the
            within-individual correlation between subject test scores
            and finds evidence that this standardized assessment
            captures in a meaningful way the specific skills that it is
            designed to measure. These results show that, despite their
            limitations, large-scale standardized tests can capture
            skills that are important for future individual wellbeing.
geo_reg_key: 119228,517191
repnme: repnme: Predicting individual wellbeing through
            test scores : evidence from a national assessment in Mexico
display_title: Predicting individual wellbeing
            through test scores : evidence from a national assessment in Mexico
listing_relative_url: /research/2018/05/29939101/predicting-individual-wellbeing-through-test-scores-evidence-national-assessment-mexico
topicv3: Teachers Management m1326119 1523,Education, Skills Development and Labor
            Market m1326135 1484,Labor Markets m1326488 855,Education m1327048 470,Health m1504661 3480
publishtoextweb_dt: 2018-05-30T00:00:00Z
historic_topic: Industry,Social Protections and Labor,Education,Rural Development
no_of_pages: 48
author_keys: 0: author_key: 900926
model_extractions: WPS8459 Policy Research Working Paper 8459 Predicting Individual Wellbeing Through Test Scores Evidence from a National Assessment in Mexico _Rafael de Hoyos Ricardo Estrada María José Vargas_ Education Global Practice May 2018
Policy Research Working Paper 8459

# **Abstract**

This paper constructs two longitudinal datasets that record students’ test scores in a national standardized exam in Mexico and track students from the end of primary (Grade 6) to the end of lower (Grade 9) and upper (Grade 12) secondary school, then to university and labor market participation up to two years after graduation from upper secondary. The results show that test scores are a strong predictor of future education and labor market outcomes. Using a large sample of twins in the data, the paper shows that the relationship between Grade 6 test scores and future education outcomes goes beyond family background. Finally, the paper exploits the within-individual correlation between subject test scores and finds evidence that this standardized assessment captures in a meaningful way the specific skills that it is designed to measure. These results show that, despite their limitations, large-scale standardized tests can capture skills that are important for future individual wellbeing.

This paper is a product of the Education Global Practice. It is part of a larger effort by the World Bank to provide open access to its research and make a contribution to development policy discussions around the world. Policy Research Working Papers are also posted on the Web at http://www.worldbank.org/research. The authors may be contacted at rdehoyos@ worldbank.org.

_The Policy Research Working Paper Series disseminates the findings of work in progress to encourage the exchange of ideas about development issues. An objective of the series is to get the findings out quickly, even if the presentations are less than fully polished. The papers carry the names of the authors and should be cited accordingly. The findings, interpretations, and conclusions expressed in this paper are entirely those of the authors. They do not necessarily represent the views of the International Bank for Reconstruction and Development/World Bank and its affiliated organizations, or those of the Executive Directors of the World Bank or the governments they represent._ Produced by the Research Support Team
Predicting Individual Wellbeing Through Test Scores: Evidence from a National Assessment in Mexico<sup>_∗_</sup> Rafael de Hoyos<sup>_†_</sup> Ricardo Estrada<sup>_‡_</sup> María José Vargas<sup>§</sup>

# **1 Introduction**

There is increasing recognition that education brings about individual and society-wide benefits if students acquire a set of relevant skills, including literacy and numeracy, during their formative years. Literacy and numeracy are seen as the foundation for the acquisition of other skills and a direct determinant of labor market outcomes, health conditions, and democratic participation, among other benefits ( World B ank, 2 018; CAF, 2016). However, there is less agreement about whether and how we can measure these foundational skills and track their progress in an education system in a reliable way.

Standardized tests are one way to directly measure foundational skills, and, in fact, many countries have in recent years adopted standardized testing in large-scale student assessments (Ganimian and Koretz, 2013). The increased availability of test scores in developing countries has helped build public awareness of the importance of numeracy

> _∗_ Acknowledgements: the authors would not have been able to write this paper without the support of the Mexican Secretariat of Public Education (SEP), we are particularly grateful to Ana María Aceves and Proceso Silva for granting us access to the micro data. The paper benefited from comments made by Jérémie Gignoux, Juan Vargas, Paula Villaseñor, Lucila Berniell, and participants at seminars at CAF and the Jobs and Development Conference at Universidad Javeriana. We are grateful to Elizabeth Monroy for putting together the datasets used in this paper. The findings, interpretations, and conclusions expressed in this paper are entirely those of the authors.

> _†_ The World Bank, 1818 H St. NW, Washington DC 20433, rdehoyos@worldbank.org

> _‡_ CAF-Development Bank of Latin America, Avenida Madero 900, Piso 15, Torre Catalinas Plaza, Ciudad de Buenos Aires, Argentina, restrada@caf.com

> §The World Bank, 1818 H St. NW, Washington DC 20433, mvargasmancera@worldbank.org
and literacy and their too-often low levels, especially among disadvantaged children and youth. This public awareness can stimulate demand for change and provide governments with the necessary legitimacy to design and enact education reforms. However, critics of standardized tests argue that they promote a “reductionist” approach to education, emphasizing literacy and numeracy in detriment of other equally important subject areas. Moreover, critics point out that the reliability of standardized test scores is ruined by either the presence or absence of incentives. On the one hand, high-stakes test can create perverse incentives that lead to “gaming” or “teaching to the test,” without true improvements in learning.<sup>1</sup> On the other hand, in low-stakes tests students might not put in enough effort to pass the test if no consequences are attached to the results (Gneezy et al., 2017). The key questions, then, are: Despite their potentially limited scope and imperfect set of incentives, can standardized test scores in early grades be used to predict future education trajectories and labor market outcomes? Can standardized tests capture foundational skills—or a relevant subset of them-–in a meaningful way?

To address these questions, we construct two longitudinal datasets that track students along their education trajectories and through school to labor market transition. These datasets are among the first of their kind in a developing country. We use information from ENLACE, a census-based standardized test that primary and secondary school students in Mexico took from 2006 to 2013. Our dataset tracks students who took the test at the end of primary school (Grade 6) in 2007 through the end of lower secondary (Grade 9) and upper secondary (Grade 12) in 2010 and 2013, respectively. We take advantage of the large size of this student cohort (1.986 million individuals) and identify about 10,000 pairs of twins in our dataset (individuals with identical last names, birthdays and Grade 6 schools). A second dataset, merges Grade 12 test scores into a special module of the Mexican labor force survey (ENOE) applied to individuals aged 18 to 20 years old in 2010.

Our results show that, despite their limitations, large-scale standardized tests have a strong predictive power for future education and labor market outcomes and capture the

> 1For a discussion on incentives and strategic behavior in testing, see Koretz and Barron (1998), Deere and Strayer (2001), Koretz (2017), Figlio and Loeb (2011) and Neal (2011).

2
skills that they are designed to measure in a meaningful way. We first document that sixth-grade learning outcomes are an important predictor of lower- and upper-secondary on-time graduation and test scores. A one standard-deviation (SD) increase in Grade 6 test scores is associated with an increase in the probability of on-time lower secondary graduation by 10 percentage points and (conditional on this) with an increase in learning outcomes by 0.6 SD. The association between Grade 6 test scores and Grade 12 outcomes is even larger: Grade 6 students with 1 SD higher scores have a probability of graduating on time that is 12 percentage points higher and, among those graduating on-time, they get 0.62 SD higher test scores. We observe that Grade 6 test scores explain a larger share of the variation in future test scores than of the variation in on-time graduation (the R-squared in the graduation regressions is 0.059-0.069, while the R-squared in the test score regressions is 0.362-342).

Second, we find that Grade 6 test scores and future outcomes have a strong relationship even when we compare individuals with an identical family background. To do so, we introduce a family fixed effect in our basic specification and focus the estimation on the sample of twins that we identify, controlling in this way for all observed and unobserved household characteristics. Qualitatively, the results outlined in the previous paragraph hold. As expected, including the family fixed effects decreases the magnitude of the coefficients of interest by 59 percent and 48 percent in the case of the relationship between Grade 6 test scores and on-time graduation from Grades 9 and 12, and by 22 percent and 14 percent in the case of test scores in the same grades. The larger reduction in the coefficient of interest in the dropout regressions is consistent with the hypothesis that family background plays a larger role in explaining on-time graduation from secondary school than future test scores. Importantly, the estimated relationship between test scores and future outcomes is still statistically and economically relevant. A 1-SD higher test score in Grade 6 is correlated with a 3.4 percentage points higher probability of on-time graduation from Grade 9 and 5.5 percentage points in Grade 12, and with 0.49 and 0.53 SD higher test scores in Grades 9 and 12. We interpret these estimated coefficients as the composite of the effect of skills at Grade 6, as captured by ENLACE, on future outcomes 3
and the within-family individual-level variation in factors that produce better outcomes both in Grade 6 and later grades (e.g. intelligence, motivation, and health).

Furthermore, we find evidence that strongly supports the case that ENLACE captures the subject-specific skills that the test is designed to measure. To do so, we adapt our main specification and run a regression of ENLACE test scores in a particular subject area at Grades 9 and 12 on the score for the same subject area in Grade 6 controlling for the individual’s Grade 6 score in the other subject area, plus a vector of twin fixed effects. The intuition behind this econometric model is that we can control for factors that have a common effect on math and language test scores by including in the equation scores on the other subject area. We find that Grade 6 test scores in mathematics (or Spanish) have a strong relationship to test scores in the same subject in Grades 9 and 12, conditional on grade 6 test scores in the other subject. An increase of 1 SD in math (or Spanish) test scores in Grade 6 is associated with increases of 0.26 (0.31) and 0.33 (0.32) SD in test scores in Grades 9 and 12. All results are statistically significant at the one percent level.

In addition, our results show that Grade 12 test scores are a good predictor of college enrollment and, conditional on being employed, hourly wages. A 1 SD increase in grade 12 test scores is correlated with a 10 percentage points increase in the likelihood of being enrolled in university. Conditional on being employed, individuals with 1 SD higher test scores in Grade 12 enjoy wages that are 6 percent higher one or two years after graduating, even after controlling for upper secondary school GPA and other socio-economic variables. Also, we observe that students enrolled in majors that require higher math (Spanish) skills, e.g. engineering (Social Sciences), have higher test scores in this subject.

Evidence from high-income countries shows that achievement levels in test scores are good predictors of future education and labor market trajectories. For instance, using longitudinal data from the United States, Rose (2006) and Jencks and Phillips (2011) show that high school test scores are positively correlated with earnings both for seven and 10 years in the future. Rivera-Batiz (1992) finds a correlation between math test scores and the likelihood of securing a full-time job. A positive correlation between test 4
scores, particularly math, and future labor market outcomes has been documented also for the United Kingdom and Sweden (Lindqvist and Vestman, 2011; Meghir et al., 2013). But high school test scores, are, in turn, predicted by test scores at earlier grades, meaning that early grade test scores can, in principle, predict labor market outcomes. Chetty et al. (2011) show that test scores at kindergarten are a good predictor of college attendance, earnings at age 27, home ownership and savings for retirement. Although evidence on the importance of early test scores for future education outcomes is very scarce, data for the United Kingdom shows that higher test scores at age seven relate to higher test scores at age 16 (Lindqvist and Vestman, 2011).

To the best of our knowledge, our paper is the first exploiting individual-level information on the association between test scores and future education and labor market trajectories in a developing country. In addition, the present paper contributes to the literature on the relevance of standardized tests showing that, despite their limitations, test scores from large-scale, census-based, assessments have a strong relationship with future outcomes, and that this relationship goes beyond socio-economic status. Furthermore, we find evidence that supports the case that these assessments can accurately capture the foundational skills that they are designed to measure, and that these skills are persistent over the years of a young person’s life. Our results are particularly interesting because they come from a national education system where people were subject to types of incentives that can result in score inflation. The results presented here have two important policy implications. First, they support the design and implementation of this type of assessments on a broad scale. Second, and importantly for the income inequality and social mobility agendas, results from large-scale assessments, especially those of early grades, could be used to target resources towards schools serving students with the lowest learning levels—which tend to be the poorest. In a country like Mexico, with a large concentration of wealth and high income disparities, building a public education system that guarantees foundational skills to all children and youth can be an effective strategy to reduce income inequality and promote social mobility.

The paper is organized in the following way. After this introduction, Section 2 de5
scribes the ENLACE national examination and the steps followed to construct the longitudinal data. Section 3 of the paper describes the empirical approach followed to study the relationship between test scores and future education and labor market trajectories. Section 4 presents the results and, finally, Section 5 presents policy implications and recommendations.

# **2 Constructing education trajectories and the transition to the labor market**

We construct two longitudinal datasets for this paper: (1) the ENLACE panel and (2) the ENILEMS-ENLACE panel. The first panel is formed by a cohort of students’ test scores at the end of primary school (Grade 6) in 2007, end of lower-secondary school (Grade 9) in 2010 and end of upper-secondary school (Grade 12) in 2013. The second panel merges a special module of the Mexican labor survey, _Encuesta Nacional de Empleo y Ocupaciones_ , ENOE, applied to individuals aged 18, 19 and 20 years during the third quarter of 2010, with students that sat for the ENLACE test in Grade 12 in 2008, 2009, and 2010.

## **2.1 The ENLACE test**

From 2006 to 2013, the Mexican Secretariat of Public Education (SEP) applied the census-based standardized test _Evaluación Nacional de Logro Académico en Centros Escolares_ , ENLACE. The test gathered information at the end of each academic year on student performance in math, Spanish, and a rotating subject for all third to ninth graders in private and public schools. Starting in 2008, ENLACE was also applied to students finishing upper secondary school (Grade 12). ENLACE was originally designed as a lowstakes assessment and had no bearing for students on GPA, graduation, or admission. Nonetheless, the take-up of the test was consistently close to 90 percent (Figure 1). A total of 15.1 million students in 136,000 schools took the examination in 2013, the last year when ENLACE was applied.

6
The purpose of ENLACE was to strengthen school accountability by informing parents and society at large about the levels and evolution of learning outcomes. In this regard, the results from ENLACE received wide attention from the Mexican public. Every year, ENLACE results made it to the front page of most newspapers in Mexico; civil society organizations were empowered thanks to ENLACE results. In an effort to use all the information produced by ENLACE to guide school improvement plans, SEP produced school report cards and a related online platform. Still, few schools viewed ENLACE as a diagnostic tool, limiting its effectiveness in improvement strategies (de Hoyos et al., 2017).

By design, ENLACE had a national mean score of 500 and a standard deviation of 100 for every subject area and grade in its first year of implementation. In addition, the test’s designers categorized scores into four achievement levels: insufficient, elementary, good, and excellent. ENLACE’s methodology followed item response theory (IRT), allowing horizontal comparability of results (same grade over time), but not vertical comparisons (between grades). Implementation of the test involved parent volunteer monitors, external test coordinators hired by SEP, and close to a million teachers.

In 2008, SEP decided to use ENLACE scores to measure teacher performance in _Carrera Magisterial_ (CM), a national teacher incentive program that operated in primary and lower secondary schools from 1992 to 2015. CM offered salary bonuses to primary and lower secondary teachers who took professional development courses and agreed to be subject to yearly evaluations. In 2008, ENLACE scores among their students were given a weight of 20 percent in the program’s total score, and this was increased to 50 percent in 2011. Almost all eligible teachers chose to take part in CM (Santibáñez et al., 2007). The decision to link it with CM made ENLACE a de facto high-stakes test encouraging “strategic behavior” among teachers and, without the proper controls, it probably resulted in grade inflation (Contreras and Backoff, 2014). Anomalies in ENLACE and the creation of a national autonomous evaluation institute (INEE) helped cause the test to be replaced by a new one called PLANEA, implemented for the first time in 2015.

7
# **2.2 The ENLACE Panel**

ENLACE created a unique personal identifier ( _Clave Única de Registro Poblacional_ or CURP) for all test takers, enabling the construction of a panel of students with learning outcomes at different points in their education trajectory. In addition to these outcomes, the ENLACE dataset included a school identifier, socioeconomic information for each school based on its geographical location,<sup>2</sup> and age and sex of the student. Using the CURP, we merge the information from all Grade 6 students who took the exam in 2007 with their results in the 2010 (Grade 9) and 2013 (Grade 12) exams. We begin the panel in 2007 because of the relatively low take-up in 2006 and the lack of availability of the unique identifier in some states during that year. Among the 1,986,190 students who sat for the exam in 2007, we were able to identify 71 percent three years later in ENLACE Grade 9 of 2010 and 32 percent in the Grade 12 exam in 2013 (Figure 2).

The large attrition observed in the panel is caused by (1) grade repetition, (2) school drop-out, (3) exam take-up rates of less than 100 percent and (4) imperfect matching due to administrative data or coding errors. If a large share of the attrition in the panel is caused by a low test take-up or imperfect matching, we would not be able to identify accurately the effects of Grade 6 test scores on lower- and upper-secondary on-time graduation rates. To quantify the magnitude of the causes behind attrition, we use administrative data from the annual school census ( _Formato 911_ ) to estimate the expected survival rates given state-level repetition and drop-out rates in lower and uppersecondary. Given the school trajectories implied by administrative data, 77 percent of the student population that completed Grade 6 in 2007 was expected to finish Grade 9 in 2010 and 39 percent to graduate from upper secondary in 2013 (Figure 2). Therefore, our ENLACE panel has a survival rate that is 6 and 7 percentage points lower vis-a-vis the survival rate implied by administrative data, in lower- and upper-secondary, respectively. This difference is the result of both less than 100 percent test take-up and imperfect

> 2The National Population Council ( _Consejo Nacional de Población_ , CONAPO) ranks all of Mexico’s localities (an administrative and/or geographic entity according to a marginality index, a weighted average of literacy, access to basic public utilities, household infrastructure and average wages. Rankings range from very high marginalization to high marginalization, medium marginalization, low marginalization, and very low marginalization. For methodological details regarding Mexico’s marginality index, see www.conapo.gob.mx.

8
matching. Take-up rate in the ENLACE test is high, 87.3 percent of the population finishing lower secondary in 2010 and 89.4 percent of the population finishing upper secondary in 2013. The difference in graduation or survival rates between the ENLACE panel and administrative data is not trivial (6 or 7 percentage points, depending on the grade), but it is reassuring that most of the attrition observed in the panel is explained by grade repetition and school dropouts, subjects we want to examine in this paper.

Two additional issues should be addressed. First, the states of Oaxaca and Michoacán are excluded from the panel because they had very low test take-up rates in 2007 (65 percent and 49 percent, respectively) and even lower in 2010 (34 percent and 0.7 percent, respectively). Second, the states of Nuevo Leon and to a lesser extent Coahuila and San Luis de Potosi have two-year upper secondary schools (as opposed to three-years), reducing substantially the matching between lower- and upper-secondary students in these states. Despite the significant share of two-year upper secondary schools in these states, they were kept in the sample.

ENLACE included a context questionnaire which was applied to a random sample of students (and parents in the case of primary and lower-secondary students) to gather information on socioeconomic characteristics, schooling aspirations, and studying habits. Although the sample size was relatively large,<sup>3</sup> including controls from the context questionnaire reduces considerably the number of observations in the regression analysis, as we will show in the results section.

We use student identification variables to identify 20,187 twins in the ENLACE panel. We define twins as students with identical first and second last names and birthdays, and who attend the same school in Grade 6.<sup>4</sup> After the matching, the twins identified account for 1.02 percent of the ENLACE panel in 2007, a level close to the prevalence of multiple pregnancies in Mexico.<sup>5</sup> Table 1 presents summary statistics for the ENLACE panel dataset. Columns 1-3

> 3There were 7,557 observations for students and 13,766 for parents.

> 4The CURPs as well as the students’ names and dates of birth are confidential data protected by Mexico’s personal information laws. We were able to identify the sample of twins due to a close collaboration with SEP for the present study.

> 5This corresponds to 10,350 pairs of twins, 93 groups of triplets, and four of quadruplets.

9
report statistics for test takers in Grades 6, 9, and 12, respectively. Columns 4 and 5 report statistics for the ENLACE survey sample and the twins sample, respectively, in Grade 6. A simple comparison of means across the first three columns in Table 1 indicates that students who graduate on time from lower- and upper-secondary were more likely to have had higher test scores at Grade 6, be female, and enrolled in a private school. Table A.1 in the Online Appendix reports differences in means between the sample of twins and non-twins. Although statistically significant at conventional levels, the differences are very small and support the idea that multiple (versus single) pregnancies are mostly a random event.

# **2.3 ENILEMS-ENLACE**

Every quarter, the Mexican statistics office, INEGI, collects labor market information through the _Encuesta Nacional de Ocupación y Empleo_ (ENOE), a nationally- and staterepresentative rotating household survey.<sup>6</sup> In most quarters ENOE’s core survey is complemented by a thematic module usually capturing information sought by different secretariats and government agencies. During the third quarter of 2010 (July to September), ENOE’s special module was the _Encuesta Nacional de Inserción Laboral de los Egresados de Educación Media Superior_ (ENILEMS), a survey targeting upper secondary school graduates aged 18, 19, and 20. The objective of ENILEMS was to provide information on the transition between the end of mandatory schooling (Grade 12) and higher education or the labor market.<sup>7</sup> The ENILEMS-ENLACE panel merges information from the respondents of the ENILEMS 2010 survey with their results in the ENLACE Grade 12 taken in May of 2008, 2009, or 2010. Although ENILEMS 2010 did not capture the CURP, it included all the necessary information to create a pseudo-CURP formed by name, sex, age, and state where the student was born. The difference between the pseudo-CURP and the CURP is that the former does not have the last three digits of the student identification code that the Mexican government generates. We created the pseudo-CURP for 7,105 observations included

> 6For more information on ENOE, see http://www.beta.inegi.org.mx/proyectos/enchogares/regulares/enoe/.

> 7For more information on ENILEMS, see http://www.beta.inegi.org.mx/proyectos/enchogares/modulos/enilems/.

10
in ENILEMS 2010 using the official algorithm for generating the CURP.<sup>8</sup> A simple merge of ENILEMS and ENLACE Grade 12 using the pseudo-CURP and CURP, respectively, was able to match 2,820 observations (40 percent of the ENILEMS sample). This relatively low matching rate is almost entirely explained by the lack of the last three digits in the pseudo-CURP. An additional 18 percent of the sample was recovered manually by identifying coding or registration errors in the CURP generation process (i.e. errors in birth date or misspelled names). Overall, 58 percent of the individuals in the ENILEMS sample were matched to their ENLACE Grade 12 test scores. After eliminating missing observations in the ENLACE score, the panel reaches a total of 3,714 observations. The ENILEMS-ENLACE panel also includes the information from ENOE for all 3,714 matched observations plus information from ENLACE’s context questionnaire for 13 percent of the panel. Table A.2 in the Online Appendix reports differences in means between the observations of ENILEMS that were merged with ENLACE scores and the ones that were not. Statistically differences are very small and hard to interpret, as they relate to the survey sampling, design, or implementation.

Table 2 presents summary statistics for the ENILEMS-ENLACE panel dataset. Columns 2 and 3 in Table 2 decompose the sample into those who are enrolled in college and those who are not. Individuals enrolled in college have on average considerably higher Grade 12 ENLACE scores than individuals out of college (by around 0.38 SD), but they are also more likely to have a higher GPA in upper secondary, to have graduated from a private secondary school, and to live in an urban area. Interestingly, about 26 percent of college students are employed versus 58 percent of those out of college being employed.

> 8The CURP is an 18-digit unique personal identifier formed by a combination of letters and numbers taken from the individual’s full name, date of birth, sex, and state of birth, plus a three-digit code assigned by RENAPO, the Mexican population council. For more information, see https://renapo.gob.mx/swb/.

11
# **3 Methods**

## **3.1 Econometric model**

We are interested in the predictive power of ENLACE test scores over future schooling and labor market outcomes. In particular, we are interested in the link between test scores and three outcome variables: future educational attainment, student achievement, and labor market outcomes including employment and wages. With this purpose in mind, we write the following model: in which _yi_ denotes an education or labor market outcome of individual _i_ , _enlacei_ is the individual’s ENLACE test score in Grade 6 as a predictor of future education outcomes or test scores at Grade 12 as a predictor for labor market outcomes or university enrollment. and _εi_ is a disturbance term. Our main parameter of interest is _β_ 1 _._ We use the ENLACE panel to study the relationship between Grade 6 test scores and future education outcomes. First we estimate the relationship between Grade 6 test scores and the probability of on-time graduation from lower- and upper-secondary school, proxied by observing student “i” sitting for ENLACE in Grades 9 and 12, respectively. We also use the ENLACE panel to examine the relationship between Grade 6 test scores and test scores three and six years later, conditional on sitting for the tests.

The ENILEMS-ENLACE panel is used to examine the relationship between Grade 12 test scores and university enrollment and, conditional on not being enrolled in university, labor market outcomes. The labor market outcomes included in our analysis are the probability of being employed–conditional on not being enrolled in college–, and, among those employed, the logarithm of the hourly wage, and the probability of having a formal job.

Learning achievement is measured using an aggregated ENLACE test score, the simple average of the math and Spanish scores, in our main specification. However, we also run math and Spanish test scores in separate regressions (the results are presented in the 12
online Appendix 5).

We estimate Equation 1 using OLS. Because several of our outcomes are binary variables (e.g. graduating on time from high school, enrolling in university, etc.), one could be worried about potential biases introduced in the estimation of _β_ 1 due to the linear projection of test scores. As a robustness check, we run a Probit model for the binary outcomes, which does not impose linear parameters as the OLS model and limit the predicted values of the dependent variable to the interval [0,1]. Results are presented in the online Appendix 5.

As already stated, in this paper we aim, first, to study the predictive power of ENLACE test scores over future schooling and labor market outcomes, and second, to understand how much of ENLACE’s predictive power is related to the skills it captures as opposed to just family background. In this second endeavor we face a couple of empirical challenges, due to relevant variables being omitted from the analysis.

The main empirical challenge behind the identification of the relationship between test scores and future outcomes is related to the correlation between test scores, future outcomes, and family background. Test scores at Grade 6 have a positive relationship with both school dropouts in subsequent grades and family income. Now, suppose, in an extreme scenario, that school dropout is only explained by family income irrespective of 6th grade student achievement. In this case, since family background and test scores are highly correlated, we will then find that 6th grade test scores predict schooling trajectories —in line with our first research question—but this would not be due to the effects of skills, which is important for our second research question. We use the richness of our datasets to understand the contribution of family background to the simple correlation between test scores at Grade 6 and future outcomes. To clarify ideas, we modify Equation 1 as where _τf_ denotes a family fixed effect. Including _τf_ in the estimation and looking at changes in _β_<sup>ˆ</sup> 1 helps to understand the role of families in the simple correlation. This requires, of course, having individuals who share the same family and who ideally were born 13
on the same day, to avoid changes in family circumstances or birth rank that bias the estimation. By identifying twins in our large data set, we are able to estimate 2. To account for the effects of all observable and unobservable family characteristics on education and labor market outcomes, we run our model on a sample of twins with family fixed effects. The fixed effects specification on the sample exploits just the within-family (between twins) variation on 6th grade ENLACE, identifying, therefore, the relationship between test scores and education and labor market outcomes net of family characteristics. We interpret these estimated coefficients as the composite of the effect of skills at Grade 6, as captured by ENLACE, on future outcomes and the (within-family) individual-level variation in factors that produce better outcomes both in Grade 6 and later grades (e.g. intelligence, motivation, and health). We also follow a more conventional approach in which we substitute the vector _τf_ for a vector _Xi_<sup>_′_ofkstudent,andhouseholdobservable</sup> characteristics. In the ENLACE panel, this last specification uses the sample of students who answered the ENLACE context questionnaire to control for differences in household characteristics with respect to both maternal and paternal education (graduation from lower secondary school or not) and parents’ occupation (blue- versus white-collar). In the ENILEMS-ENLACE panel regressions, _Xi_<sup>_′_includesuppersecondaryschoolgradepoint</sup> average (GPA), a dummy variable indicating graduation from a private upper secondary school, residence in an urban area, sex, and age.

ENLACE test scores are likely not independent of other student characteristics which are, in turn, important determinants of education and labor market trajectories. For instance, math scores might be correlated with socio-emotional skills such as motivation or perseverance that produce higher test scores today and better attainment, achievement and labor market outcomes in the future. This type of omitted variable bias is not necessarily a limitation of our empirical strategy since the objective of our exercise is not to identify the effect of an exogenous change in, say, math skills at the end of primary school holding constant all other student characteristics. Instead we want to understand what drives the predictive power of test scores for future wellbeing. Test score outcomes will surely be the result of a combination of previous subject-specific knowledge and 14
other factors such as socio-emotional skills. In Section 4 we discuss what are the skills that ENLACE does capture and propose an empirical strategy to identify the impacts of changes in subject-specific skills versus other characteristics such as personality traits or socio-emotional skills.

# **4 Results**

## **4.1 Grade 6 ENLACE and Secondary School Outcomes**

Figure 3 plots local means of on-time graduation and test scores in Grades 9 and 12 by ENLACE score percentile in Grade 6. We find a clear and positive relationship between these variables. Higher test scores in Grade 6 are associated with a higher probability of on-time graduation in Grades 9 and 12, and, conditional on this, with higher test scores in Grades 9 and 12, as presented in panels (a) and (b) respectively. The differences in outcomes between students in the top and bottom of the test score distribution in sixth grade are startling. For example, less than 20 percent of students in the bottom decile of the test score distribution in Grade 6 are enrolled in Grade 12 six years later, compared to more than 50 percent of students in the top decile.

Table 3 quantifies the relationship depicted in Figure 3 by regressing graduation and test scores in Grades 9 and 12 on ENLACE test scores at Grade 6, and a dummy for whether the student is female, using the individual-level data. The results confirm the strong association between ENLACE test scores at Grade 6 and secondary school outcomes.

A one standard deviation (SD) increase in ENLACE test scores at Grade 6 is associated with an increase in the probability of on-time graduation from lower and upper secondary school of 10.3 percentage points and 11.9 percentage points, respectively. Both results are statistically significant at the one-percent level (see columns 1 and 2). A similar story goes for future test scores. A one SD increase in ENLACE test scores at Grade 6 is correlated with a 0.61-0.62 SD increase in test scores in Grades 9 and 12, conditional on taking the ENLACE exam. Again, these results are statistically significant at the 15
one-percent level (see columns 3 and 4).

Note that Grade 6 test scores explain a larger share of the variation in future test scores than of the variation in on-time graduation. The R-squared in the graduation regressions is 0.059 and 0.069 (in Grades 9 and 12, respectively) while the R-squared in the test score regressions is 0.362 and 0.342 (in the same grades). This implies that the variables not included in this regression (such as family background) are relatively more important explaining on-time graduation than future test scores.

We move now to study the degree to which the relationship between test scores and future outcomes is driven by family background, which, very likely, affects both 6th grade test scores and future education outcomes. To do so, we exploit the sample of twins in the ENLACE Panel and run a specification including a vector of twin (family) fixed effects. This allows us to study how variation in test scores at Grade 6 predicts variation in future student outcomes holding constant all family characteristics (observed and unobserved). We first estimate the same model used in Table 3 to verify that any change in the coefficients of interest is due to the introduction of the family fixed effects and not due to sample variation between the population of twins and the total population of ENLACE takers. Columns 1-4 in Table 4 report the results from a model restricted to the sample of twins and not including family fixed effects. Reassuringly, the estimated coefficients are similar to those reported before. A 1 SD increase in Grade 6 test scores is associated with a 8.2 and 10.5 percentage points higher probability of on-time graduation from Grades 9 and 12 (versus 10.3 and 11.9 in the full sample), and with 0.63 and 0.61 SD higher test scores in Grades 9 and 12 (versus 0.61 and 0.62 in the full sample). For details, see columns 1-2 and 3-4. Columns 5-8 in Table 4 report the results of the estimations that include the family fixed effects. Qualitatively, the findings are the same as the ones from previous specifications: test scores at Grade 6 predict on-time graduation from Grades 9 and 12, and conditional on this, test scores in Grades 9 and 12. In the four cases, the results have a high statistical significance. However, as expected, the coefficients from the family fixed-effects specification are smaller than those estimated using the main specification 16
(see columns 1-4 in Table 3).<sup>9</sup> A 1 SD higher test score in Grade 6 is correlated with a 3.4 percentage points higher probability of on-time graduation from Grade 9 and 5.5 percentage points in Grade 12 (see columns 5-6), and with 0.49 and 0.53 SD, respectively, higher test scores in Grades 9 and 12. In other words, taking into account differences in family background reduces the magnitude of the estimated correlation between Grade 6 test scores and secondary school outcomes by 59 percent and 48 percent in the case of on-time graduation from Grades 9 and 12, and by 22 percent and 14 percent in the case of test scores in the same grades. The larger reduction in the coefficient of interest in the dropout regressions is consistent with the hypothesis that family background plays a larger role in explaining on-time graduation from secondary school than future test scores. The R-squared in the four regressions estimated with family fixed effects increases considerably (to about 0.81 and 0.86-0.89 in the graduation and test scores regressions, respectively). This indicates the importance of family background in shaping education outcomes.

Our results show that there are significant gaps in outcomes by gender. Conditional on the Grade 6 score, girls are about 4 percentage points more likely than boys to follow an education trajectory free of age-grade distortions, a difference statistically significant at the one percent level (see row 2 in columns 5-6). When one looks at test scores, a surprising pattern emerges: the gender gap reverses by Grade 12. Conditional on initial test scores (and staying in school), girls do better on average than boys in Grade 9 (by 0.11 SD), but worse by Grade 12 (0.10 SD). The switch in the sign of the gender coefficient is explained by the girls’ lower performance in mathematics in Grade 12. See results in online Appendix 5 and Avitabile and De Hoyos (2015) for a discussion on this issue.

Finally, we compare the results from the twin fixed effects model—which can accurately control for family characteristics– with a more standard approach based on the inclusion of socio-economic controls. We estimate Equation 1 including baseline covari-

> 9The simple correlation between test scores and future outcomes is a composite of family, school and individual characteristics and circumstances. Given that the positive correlation between family resources, test scores, and other schooling outcomes is an empirical regularity, it makes sense that estimates that accurately control for family background have a smaller magnitude than the simple correlation. Even though these estimates are smaller than those reported in previous specifications, they are still significant and economically large.

17
ates measured in the ENLACE context questionnaire, which allows us to also control for parental education and occupation. Specifically, the model includes a set of dummy variables equal to one when the mother (or father) completed lower secondary and a dummy capturing if she (or he) is a white-collar worker. As before, we first estimate the simple equation to verify that any change in the coefficients of interest is due to the inclusion of covariates and not due to sample variation. Table 5 (columns 1 to 4) reports the results without controls for parental characteristics but restricted to the sample included in the ENLACE context questionnaire, which are similar to those already reported. Columns 5-8 in the same table report the results from the regressions that include family characteristics. As expected, ENLACE test scores at Grade 6 are strong predictors of on-time graduation and test scores at Grades 9 and 12, even after including in the model variables that directly control for family background at the individual level. The inclusion of these covariates reduces the point-estimator of the association between Grade 6 test scores and the probability of on-time graduation in Grade 9 and 12 by 22 percent and 13 percent, respectively; and by 3 percent for test scores for both Grades 9 and 12, respectively. This reduction in the estimated parameter is considerably smaller than the reduction observed by the inclusion of twin fixed effects, which shows the importance of unobserved family characteristics, orthogonal to the conventional parental control, that are important in determining future education outcomes.

The specification with controls for family background shows that there are significant gaps in outcomes by parental background. Students with more educated parents are more likely to finish lower- and upper-secondary on time, and conditional on this, have higher test scores. Having a mother who at least completed lower secondary education is associated with an increase of 8 percentage points in the probability of on-time graduation from lower- and upper-secondary (columns 5 -6 in Table 3) and with 0.05 and 0.10 SD higher test scores, respectively (columns 7-8 in Table 3). In three of the four cases, results are statistically significant at conventional levels. The point estimates for the conditional correlation between paternal education and on-time graduation also have high statistical significance, although their magnitude is smaller (about 3.5-6.2 percentage points, as 18
shown in columns 5 -6 on Table 3). In contrast, we do not observe a statistical relationship between paternal education and test scores in Grades 9 and 12 (columns 7-8 in Table 3).

Summing up, we find that test scores at the end of primary school are a large and robust predictor of on-time graduation and test scores in lower- and upper-secondary. Conditional on Grade 6 test scores, there are clear gender and socioeconomic gaps in graduation and achievement in secondary school. Students with lower test scores in Grade 6 face a larger risk of dropping out of school and going through life with low levels of skills. These findings persist when we accurately deal with family background using the sample of twins. We also find that (1) Grade 6 test scores explain a larger share of the variation in future test scores than in on-time graduation, and (2) family background has a more prominent role driving the simple correlation between Grade 6 test scores and on-time graduation than between Grade 6 test scores and future test scores. This finding suggests that family background plays a larger role in explaining on-time graduation from secondary school than future test scores.

# **4.2 ENLACE Test Scores and Labor Market Outcomes**

Are ENLACE test scores at Grade 12 a good predictor of college enrollment and labor market outcomes one or two years after graduation? We investigate the relationship between test scores at the end of upper secondary (Grade 12) and the probabilities of being enrolled in college and, conditional on not being a college student, being employed. Figure 4 (top panel) plots local means of both outcomes by ENLACE score ventiles in Grade 12. We find a strong and monotonic relationship between test scores and college enrollment (see panel A). The gap in outcomes between people at the bottom and top of the test score distribution is startling. Ninety percent of individuals in the top ventile of the test score distribution are enrolled in college, compared to just 45 percent of those in the bottom ventile. In contrast, we do not find that test scores predict employment status among individuals who are not enrolled in college. The conditional expectation function of employment on test scores seems to have a flat profile (though it gets less precise in the top part of the distribution, because there are few people at this level of ENLACE 19
who are out of college). Figure 4 (bottom panels) also depicts the relationship between test scores and employment characteristics among the employed. We observe here that higher test scores predict higher wages, with the exception of the top decile, but because only 20% of individuals in this decile work, we have less precision here. There seems not to be a statistical association between test scores and the probability of being employed in the formal sector of the economy (conditional on being employed).

To estimate if the predictive power of test scores at Grade 12 over college enrollment and wages is robust to the inclusion of controls, we ran an alternative specification. Table 6 reports the results of regressing post-secondary school outcomes on Grade 12 test scores, GPA for upper secondary, indicators for whether the individual is a female, a graduate of a private high school, and an urban resident, plus a vector of state and year of birth fixed-effects. We find a strong relationship between test scores and college enrollment (see column 1). A 1-SD increase in the ENLACE score is associated with a 10 percentage point increase in the probability of college enrollment (a result statistically significant at the one percent level). The GPA in upper secondary school is also positively associated with college enrollment. A 1-SD increase in the former predicts an increase of 7 percentage points in the latter, with statistical significance at the one percent level.

It stands out that conditional on these two measures of ability, females are 5 percentage points less likely to be enrolled in college (statistical significance at the ten percent level) and graduates of private secondary schools are 11 percentage points more likely to be enrolled in college (statistical significance at the one percent level). In other words, holding constant test scores and secondary school GPA, there is a gender and family background gap in college enrollment against women and public school graduates.

Estimates in column 2 confirm that among those individuals out of college there is no association between test scores and employment status. Among those who are employed, we do not find that test scores have predictive power over being employed in a formal firm (see column 4). However, among those employed, test scores have a large and significant relationship with future hourly wages. As reported in column 3 of Table 6, a 1-SD increase in test scores is associated with an increase of about 7 percent in the hourly wage.

20
In summary, we find that test scores at the end of secondary school predict college enrollment and, for individuals who are employed, hourly wages. This relationship is robust even when controlling for upper secondary GPA and graduation from a private senior secondary school (a proxy for family background). We also find that, conditional on test scores and upper secondary GPA, women and graduates from public schools are less likely to be enrolled in college.

# **4.3 Robustness Checks**

As discussed in Section 3, the estimation of a linear probability model, as applied so far, could lead to potential biases in _β_<sup>ˆ</sup> 1 due to the linear projection of test scores. To address this concern, we estimate probit regressions for our binary outcomes, using the same variables as in the OLS estimation. The estimations using this alternative specification are similar to those presented in the previous subsection. We find a positive and strong (statistically and economically) relationship between test scores at Grade 6 and on-time graduation in Grades 9 and 12. The same is true for the relationship between test scores at Grade 12 and college enrollment. The probit results show that test scores at Grade 12 do not predict the conditional probability of being employed nor of being employed in a formal firm, which is consistent with the OLS results reported above. Results are available in online Appendix 5.

# **4.4 What Skills Do Test Scores Capture?**

In Section 3, we discussed a finding that, even holding constant a student’s family background, ENLACE scores could capture both the numeracy and literacy skills that the test is designed to measure, as well as other unobservable skills or cofounders that produce higher test scores, such as health, motivation, or other socio-emotional skills. Given our interest in understanding what drives the relationship between test scores and future outcomes, we turn now to further investigate this issue. To formalize these ideas, we suppose that the production function of test scores can be modeled in the following way: 21
where _enlace_<sup>_s_</sup> _it_<sup>isstudentifromfamily</sup><sup>_f_’stestscoreinsubject</sup><sup>_s_„</sup><sup>_αist_isthespecific</sup> skills to subject _s_ of student i from family _f_ , _git_ is general skills of student i from family _f_ , _τf_ is a vector of family fixed effects, and _eit_ is a random disturbance term.This model makes explicit that test scores are produced by both general and subject-specific skills. Now, since test scores in two different subjects, math and language, are available, we can estimate the following specification: Equation 4 is a modified version of our main specification that allows us to study the relationship between student i’s test score in a particular subject area at Grade 6 and the score for the same area in Grades 9 and 12 _controlling for the Grade 6 score in the other subject area_ . The underlying assumption in Equation 4 is that there are skills that have a common effect on math and language test scores and which can therefore be captured by including them in the equation scores on the other subject area. Our strategy does not deal with the component of skills that produces higher test scores in one subject, nor with other potential cofounders that affect Grade 6 and later test scores and that are subject-specific and within-family specific. In other words, this specification is useful to deal with individual-level factors that affect test scores in math and language in a similar fashion (e.g. student motivation to perform in standardized exams). We interpret _β_ 1 in Equation 4 as the composite effect of subject-specific skills in Grade 6 on future test scores and other (within-family) factors that affect subject-specific test scores in Grade 6 and subsequent grades.

Table 7 reports the results of estimating Equation 4 in the sample of twins. Both math and Spanish scores in Grade 6 predict scores in those subject areas in Grades 9 and 12, even when controlling for Grade 6 scores in the other subject area. The coefficients of interest (see row 1 in columns 1 and 2 and row 2 in columns 3 and 4) have a large statistical 22
and economical significance. Notice that the point estimator of _β_ 1 in specification 4 is significantly smaller than the one presented in Table 4, as one would expect if test scores were driven by both initial subject-specific and general skills. Once the effects of general skills are removed, the association between one SD higher test scores in Grade 6 is about 0.25-0.34 SD on test scores in 9th and 12th grades (as opposed to the 0.49-0.52 SD reported in Table 4). The implied estimates for general skills at Grade 6 on future test scores is about 0.2 SD and highly significant–see row 2 in columns 1 and 2 and row 1 in columns 3 and 4 in Table 7. These results suggest that test scores at Grade 6 are a strong predictor of future education trajectories with the persistence working through the positive link between initial and future subject-specific skills and through the positive effects of initial general skills on future education outcomes.

Finally, we go back to the ENILEMS-ENLACE panel and look at average ENLACE test scores in mathematics and Spanish (at the end of secondary school) by university field of studies (see Figure 5). If ENLACE captures specific skills and students tend to self-select to university degrees that match better their stock of skills, one would expect to observe that students enrolled in majors that require higher math (Spanish) skills, have higher test scores in this subject. Importantly for our story, engineering students tend to have higher test scores in mathematics than in Spanish, while the opposite is true for Social Science students. We interpret this pattern as complementary evidence that ENLACE indeed captures the specific skills it is designed to measure. Furthermore, to our knowledge, this is the first piece of evidence about how university students sort by ability into fields of study in Mexico.

# **5 Conclusions**

Using the Mexican census-based standardized test ENLACE, we construct a longitudinal dataset tracking students’ completion of and test scores in Grades 6, 9, and 12. Our analysis shows that higher test scores at Grade 6 have a large and significant relationship with the student’s likelihood of finishing lower- and upper-secondary school on time and, 23
among those who finish, with their test scores levels. Using a sample of twins to deal with differences by family background, we find that a reduction of 1 SD in test scores in sixth grade reduces by 5.5 percentage points the probability of graduating from upper secondary (the mandatory education level), and, among those who graduated, it reduces their Grade 12 test scores by 0.53 SD. We present compelling evidence that these effects are driven by the levels of subject-specific knowledge at sixth grade (as captured by the standardized test) rather than by unobservables such as socio-emotional skills.

The paper also assesses the short-term relationship between Grade 12 test scores and post-secondary outcomes such as university enrollment, employment in the formal sector, and hourly wages. For this objective, we merge upper secondary ENLACE test scores with a labor market survey to follow students one or two years after graduation. Our results show that test scores are a strong and positive predictor of university enrollment, and, conditional on students not being enrolled and being employed, it also correlates positively with hourly wages. A positive change of one SD in test scores at the end of upper-secondary is associated with a 10 percentage point increase in the likelihood of enrolling in university, and, conditional on being employed, it increases hourly wages by 6 percent. We found no significant effects of Grade 12 test scores on the likelihood of being employed in the formal sector. All these results are robust to the inclusion of socio-economic controls, state, and year of birth fixed-effects and upper secondary grade point average (GPA).

The significant relationship between Grade 12 test scores and wages is somewhat surprising considering that all the individuals in our panel have the same number of years of schooling and many of them are holding their first jobs. Most likely, the relationship between upper secondary test scores and wages is a lower-bound estimate of the mediumand long-run estimates. This would be the case if, for instance, employers have imperfect information about the true skills of high school graduate applicants, and they learn progressively about those skills as the new employees spend more time on the job. A second possible reason for expecting a larger association between test scores and wages in the medium- to long-term is if individuals accumulate human capital in the workplace, 24
enabling those with relatively high test scores to increase their wages.

These results shed light on how disadvantages at early stages in the education trajectories can have important and persistent implications for future education and labor market outcomes. The concept that learning begets learning is corroborated by evidence presented here showing that a low learning outcome in sixth grade can have a negative consequence in personal incomes 10 years later. The lower performance in sixth grade works its way forward in time, signaling lower chances of completing upper secondary and, if the student graduates, reduced learning outcomes which, in turn, diminish the individual’s probability of entering university, and among those who do not enter, reduce their wages. This should make a failing mark at the end of primary school–or earlier–a trigger for strong attention and support for the student. We believe that, despite their limitations, large-scale standardized tests like ENLACE capture skills that are important for wellbeing. Our findings support the rationale for using test score results in combination with information on family background to identify students at risk and set in motion the necessary remediation steps to reduce the path dependency that we present here. By identifying early in their education trajectories students who need more support, large-scale standardized tests can help create an education system that promotes equality and social mobility, rather than one that simply replicates or even exacerbates existing inequalities.

25