metadata: authors: 0: author: Hernandez Zavala,Martha Belem
1: author: Patrinos,Harry Anthony
2: author: Sakellariou,Chris
3: author: Shapiro,Joseph S.
count: Latin America,Peru,Mexico,Guatemala
docna: 0: docna: Quality of schooling and quality of schools
            for indigenous students in Guatemala, Mexico, and Peru
docty: Policy Research Working Paper
subtopic: Public Sector Management and Reform,Educational Sciences,Educational Institutions & Facilities,Educational Populations,Education for Development (superceded),Educational Policy and Planning - Textbook,Effective Schools and Teachers,Education For All,Inequality
teratopic: Education
abstracts: cdata!: A substantial gap in test scores exists
            between indigenous and non-indigenous students in Latin
            America. Using test score data for 3rd and 4th year primary
            school pupils in Guatemala and Peru, and 5th grade pupils in
            Mexico, the authors assess the magnitude of the indigenous
            and non-indigenous test score gap and identify the main
            family and school inputs contributing to the gap. A
            decomposition of the gap into its constituent components
            suggests that the proportion that is explained by family and
            school characteristics is between 41 and 75 percent of the
            overall test-score gap. Furthermore, family variables
            contribute more than school variables to the overall
            explained component.
geo_reg_key: 119228,545133,517191
repnme: repnme: Quality of schooling and quality of schools
            for indigenous students in Guatemala, Mexico, and Peru
display_title: Quality of schooling and quality of
            schools for indigenous students in Guatemala, Mexico, and Peru
listing_relative_url: /research/2006/08/6962214/quality-schooling-quality-schools-indigenous-students-guatemala-mexico-peru
topicv3: Teachers Management m1326119 1523,Human Resources m1326304 1211,Poverty m1326687 695,Textbooks m1326802 2129,Education m1327048 470
publishtoextweb_dt: 2010-07-01T00:00:00Z
historic_topic: Education
no_of_pages: 40
author_keys: 0: author_key: 738040
1: author_key: 95903
2: author_key: 428386
3: author_key: 614684
model_extractions: WPS3982

# **Quality of Schooling and Quality of Schools for Indigenous Students in Guatemala, Mexico and Peru**<sup>∗</sup>

Martha Hernandez-Zavala<sup>1</sup> Harry Anthony Patrinos<sup>1</sup> Chris Sakellariou<sup>2</sup> Joseph Shapiro<sup>3</sup>

## **Abstract**

A substantial gap in test scores exists between indigenous and non-indigenous students in Latin America. Using test score data for 3<sup>rd</sup> and 4<sup>th</sup> year primary school pupils in Guatemala and Peru and 5<sup>th</sup> grade pupils in Mexico, we assess the magnitude of the indigenous/non-indigenous test score gap and identify the main family and school inputs contributing to the gap. A decomposition of the gap into its constituent components suggests that the proportion that is explained by family and school characteristics is between 41 and 75 percent of the overall testscore gap. Furthermore, family variables contribute more than school variables to the overall explained component.

JEL codes: J15, I20 Keywords: Indigenous, education, discrimination, test score

## World Bank Policy Research Working Paper 3982, August 2006

The Policy Research Working Paper Series disseminates the findings of work in progress to encourage the exchange of ideas about development issues. An objective of the series is to get the findings out quickly, even if the presentations are less than fully polished. The papers carry the names of the authors and should be cited accordingly. The findings, interpretations, and conclusions expressed in this paper are entirely those of the authors. They do not necessarily represent the view of the World Bank, its Executive Directors, or the countries they represent. Policy Research Working Papers are available online at http://econ.worldbank.org.

> ∗ Address all correspondence to: Harry Patrinos, World Bank, 1818 H Street NW, Washington DC 20433, email: <u>hpatrinos@worldbank.org.</u>

> 1 World Bank; 2 Nanyang Technological University; 3 Oxford University
# **1. Introduction**

Quality of schooling denotes two concepts. The first contrasts with years of schooling – it refers to the amount that a student learns over time from attending school, an amount often measured with reading and mathematics test scores. The second might better be called quality of schools—it refers to the extent to which a school’s management, teachers, infrastructure, schedule and other factors improve a student’s learning. The two concepts are related but distinct: a student might repeat grades and receive low test scores while studying under extraordinary teachers in an atmosphere conducive to learning. That student, in other words, may have low quality of schooling, but may attend a good quality school.

Recognizing that the quality of a student’s schooling can differ from the quality of a student’s school is critical to understanding the extent to which education policy can improve the learning of disadvantaged students. Indigenous students – children of parents who identify with the first inhabitants or native peoples of the country – in Latin America, who score lower on tests than non-indigenous students do, provide a good example. Indigenous students tend to be poorer than non-indigenous students (Hall and Patrinos 2006), and it is not immediately obvious whether indigenous students score lower on exams because indigenous students have more disadvantaged backgrounds than non-indigenous students have, or whether indigenous students perform worse because indigenous students study in lower-quality schools. If indigenous students learn less than non-indigenous students because indigenous schools have less effective instructors or less appropriate infrastructure than non-indigenous schools have, then policy measures focused on improving school management, teacher quality, schedules and infrastructure have ample potential to improve the learning of indigenous students.

1
Latin America’s indigenous people earn less and complete fewer years of schooling than Latin America’s non-indigenous people do. The two deficits are linked: indigenous people generally receive lower returns to schooling than non-indigenous people do, partly because indigenous children enter school with disadvantaged backgrounds, partly because indigenous students learn in lower-quality schools that reinforce that disadvantage and partly because indigenous adults may face discrimination in the labor market (Psacharopoulos and Patrinos 1994).

Indigenous people in Latin America, who number between 22 and 34 million, are no homogenous group. Approximately 30 percent of Peruvians in urban areas are indigenous. In Peru’s urban areas, most indigenous people speak both Spanish and an indigenous language, while in Peru’s rural areas, few indigenous people speak Spanish. In Guatemala, approximately 40 percent of the population is indigenous and speaks one of over 20 Mayan languages. Three of four indigenous people in Guatemala live in rural areas and about half of employed indigenous people work in agriculture (Shapiro 2006). In absolute terms, Mexico has the largest indigenous population in Latin America, with the states of Oaxaca, Yucatan, Puebla, Veracruz and Chiapas registering the highest indigenous population density (Gonzalez 1994). According to the 1990 census, 23 percent of the indigenous people in Mexico spoke Nahuatl, 14 percent Mayan, and 7 percent spoke Mixteco and Zapoteco. Based on various censuses, the indigenous population has, however, been declining. By the year 2000, the indigenous language speaking population was about 7 percent, down from 8 percent in the 1990s, 10 percent in 1950 and 14 percent in 1930.

2
Despite that heterogeneity, research using household data shows that across the region, the educational achievement of Latin America’s indigenous people lags distantly behind the achievement of Latin America’s non-indigenous people. In Guatemala, indigenous adults have about half the years of schooling that non-indigenous adults have. In Mexico, adults in indigenous municipalities have completed 3 years of schooling while adults in non-indigenous municipalities have completed 8 years of schooling. In Peru, indigenous adults have on average completed 6 years of schooling while non-indigenous had completed 9 years of schooling. Explanations for the lower schooling achievement of indigenous students include the quality of the educational environment at home and interactions of greater poverty with rural residence, neglect of indigenous languages and failure to accommodate linguistic diversity (Herrera Lara 1999; McEwan 2004 **)** .

Using mathematics and Spanish reading test scores from indigenous and non-indigenous students in Guatemala, Mexico and Peru, we attempt to distinguish the quality of indigenous schooling from the quality of indigenous schools. We first use feasible generalized least squares (FGLS) estimate production functions for Spanish and math exam scores that control for various student-, family- and school-level factors. Family- and student-level factors proxy a student’s cognitive ability. We then use Oaxaca-Blinder decompositions to separate the portion of the test score gap between indigenous and non-indigenous students that is due to family and school factors, and the portion that cannot be explained using observed background indicators.

Our results are generally consistent across the three countries. In every country, the test score gap between indigenous and non-indigenous students is about twice as large in Spanish as 3
in mathematics exams. The determinants of learning are similar for indigenous and nonindigenous students, and for both groups, family background factors have the largest impact on student learning. Indigenous students come from less wealthy families, study under less experienced teachers, and attend schools with worse infrastructure and fewer learning materials. Three-quarters of the test score variance in Mexico and Peru and nearly half in Guatemala is due to observable student, family and school-level factors, suggesting a significant opportunity for supply-side interventions to improve student learning. The remaining, unexplained, gap could be due to unobserved factors, to teachers’ bias in instruction, or to other factors.

# **2. Methodology and Estimation**

To what extent do test scores matter? Controlling for relevant background factors, performance on exams – tests of cognitive ability or school learning – can effectively predict school attendance, educational attainment, adult wages and adult employment (Bishop 1989, 1991; O’Neill 1990; Grogger and Eide 1993; Rivkin 1995; Murnane, Willett and Levy 1995; Hanushek, Rivkin and Taylor 1996; Neil and Johnson 1996; Murnane, Wilett, Duhaldeborde and Tyler 2000). Test scores by no means capture all learning, and multiple choice exams in particular may not capture critical thinking ability. Nonetheless, test scores provide some quantifiable indicator of learning. Additionally, to the extent that primary school reading exams measure literacy, primary school test scores directly measure a capacity that itself has important value.

Several literature reviews summarize numerous learning production functions from developing countries, and those show that student and family factors have the largest impact, 4
though school factors matter also. In a summary analysis of 96 studies on learning in developing countries, Hanushek (1995) finds that while most United States studies show that teachers’ education is unimportant, almost half the reviewed studies from developing countries find that improving the education of teachers has a positive and significant effect on student learning; other studies find that teachers’ education has insignificant effects. Similarly, while most studies from industrialized countries show that building additional school facilities has no effect on student learning, the majority of studies from developing countries show that additional facilities have significant and positive effects on learning. Velez, Schiefelbein and Valenzuela (1993) review 18 papers that include 88 separate regressions estimating education production functions for countries in Latin America and the Caribbean. Overall, they find that the size of a student’s school, the school’s location, teacher and student gender, and presence of a secondary school connected to a primary school most often impact learning. The presence of textbooks and reading materials are also shown to impact learning, as does the presence of other materials and quality of school infrastructure. Again, about half the studies find that a teacher’s education and experience have significant impacts on learning. Few studies include data on school principals, but those that do find inconclusive evidence on the effect of a principal’s education, and they generally find a negative association of student learning with a principal’s experience.

Glewwe (2002) emphasizes the empirical problems with many studies of learning in developing countries, but describes in detail four studies from Brazil, Ghana, India, and Jamaica that use representative data and correction for selection bias. Overall, improvements to school infrastructure, teacher education, teacher salaries, teacher training, and use of textbooks had significant and positive effects on school learning. Interestingly, the application of eye 5
examinations had significant and positive effects on reading scores. In India, going from having no physical facilities to having all 17 listed facilities (including toilets, computers and musical instruments) would increase math scores by 0.7 standard deviations and reading scores by 1.0 standard deviations.

Until recently, little test data was comparable between Latin American countries. Casassus and others (2000) present results from the 13 countries across Latin America and the Caribbean, not including Guatemala, which participated in UNESCO’s Laboratorio exam. Overall, they found that achievement in math was worse and more varied across students than achievement in Spanish. Compared to boys, girls showed better achievement in language but worse achievement in math. They found that school-level factors explained about two-thirds of test score variation between students. Student background was also critical in explaining student outcomes (for a survey of related literature, see Todd and Wolpin 2003). Although a child’s achievement is inherently individual in nature, a large body of evidence points to the existence of persistence effects in educational achievement across generations (Fertig 2003; Fertig and Schmidt 2002; Currie and Thomas, 1999). Consequently, one must control for individual pupil characteristics as well as family background. Finally, one needs to control for characteristics on school environment as well as institutional arrangements.

Disagreement persists over whether spending additional resources on schools has greater impact in developing than in industrialized countries. An earlier analysis of the Second International Mathematics and Science Test Study (Heyneman and Loxley 1983) also found that added resources for schools had more impact on learning in developing than in industrialized 6
countries. In a later study, Hanushek and Luque (2003) use data from the Third International Mathematics and Science Study (TIMSS) to compare performance in about 40 countries. They find that additional resources may impact learning, but effect is small and appears to be no greater in developing than in industrialized countries.

# <u>Methodology</u>

We first estimate cognitive achievement production functions that relate students’ achievement to individual, family and school inputs. We then decompose the indigenous/nonindigenous test gap into an explained component, accounting for family characteristics and school characteristics, and an “unexplained” component, using the traditional Oaxaca (1973)Blinder (1973) decomposition method.

The model specification for the estimation of the production function for cognitive achievement is as follows: where Tija is the observed test score of student _i_ in household _j_ at time _a_ (time of the test), **F** ija is a vector of individual and parent inputs, **S** ija is a vector of school-related inputs and єija is an additive error, which includes all the omitted variables including those which relate to the history of past inputs, endowed mental capacity and measurement error. Its linear specification (after dropping subscript _a_ ) is given by: where β0 to β2 are coefficients to be estimated.

7
The standard procedure for analyzing the determinants of the test score differentials is to fit equations between test scores and observed characteristics. The observed test score differential can be decomposed as: where T is the standardized test score (z-score), **X** _i_ is a vector of family and school characteristics for the _i_ th individual, β is a vector of coefficients and NI, I subscripts are identifiers of non-indigenous and indigenous students. Standardized test-scores are used to facilitate the comparison between language and mathematics test results, as the two tests consist of a different number of questions. A student is identified as indigenous if that student grew up speaking an indigenous language at home.

The overall test score gap can, therefore, be decomposed into two components: one is the portion attributed to differences in characteristics (XNI – XI) evaluated with returns to endowments of non-indigenous students ( β NI); the other portion is attributable to differences in effects on performance ( β NI - β I) of non-indigenous and indigenous students derived from the same characteristics.

This second can be assigned a few interpretations. First, the unexplained portion of the test gap may reflect unobserved family characteristics that are correlated with achievement and indigenous status, possibly relating to household wealth. Second, even in the absence of overt discrimination against indigenous students, teachers may view indigenous students as underachievers and, therefore, use different teaching standards. This may be a contributing 8
factor that helps to sustain, and possibly expand, the black-white test-score gap in the United States (Ferguson 1998). Finally, it may be that indigenous and non-indigenous students do not reap the same benefits from equivalent school and classroom resources (McEwan 2004).

# <u>Estimation Method</u>

While test scores and individual and family information are observed at the individual level, school resources and other school-related inputs are at the school level. In choosing the estimation method, we recognize that first, standard errors are heteroskedastic across schools, and second, observations within schools are correlated due to their exposure to the same community factors. Thus the assumption that disturbances are independently and identically distributed with fixed conditional variance does not hold.

To control for the clustered nature of the data, we use feasible generalized least squares (FGLS). General linear models take the form: where y is a vector of _n_ random variables, _X_ and _Z_ are matrices of independent variables with dimensions _n_ × _k_ and _n_ × _q_ , β is a vector of _k_ parameters and σ is a vector of _q_ unobserved random variables with mean zero and variance _D_ , and _v_ is a vector of _n_ random errors with mean zero and variance _R_ (Moulton 1986). The variables _v_ and σ are uncorrelated. For this general model, the variance of y, denoted as _V_ , is Feasible GLS calculates the coefficients and their variance as follows: 9
where _V_<sup>ˆ</sup> is evaluated at estimates of the variance or covariance components (Moulton 1986). In sum, we estimate the effect of school and student-level factors on test scores using equation (1.3), with coefficients estimated according to equation (2.3), and standard errors estimated according to equation (2.4).

# <u>Data</u>

Data used are from three sources: for Peru from the 1997 First Comparative International Study on Language, Mathematics and Associated Factors (“Regional Laboratorio”), overseen by The Regional Education Office for Latin America and the Caribbean (OREALC), a division of the United Nations Economic and Social Council (UNESCO); from Guatemala’s national Laboratorio exam, which was based on the UNESCO test; and from Mexico’s National Standards exam. For the present analysis, each exam score was standardized to have a mean of zero and a standard deviation of one.

The regional Laboratorio study applied the same 32 multiple choice math questions and 19 multiple-choice Spanish language questions to students from 13 countries in 1997. The data were not scaled with item response theory (IRT), so one correct answer on any question translated to one point on a student’s final score. The sample was stratified to ensure sufficient observations of public, private, urban, rural and metropolitan pupils in each country. Data were collected for 100 schools in each country, with 40 children per school surveyed. Half of the students were in the third and half in fourth grade. The survey also gave questionnaires to school principals, teachers, parents or legal guardians of tested pupils and the students themselves. The test and 10
questionnaires were given only to children who were enrolled in school and who attended school the day of the exam. Indigenous students were identified with a question on language spoken.

The second source, for Guatemala, is a year 2002 exam of math and reading in Spanish that we refer to as “Laboratorio Guatemala”. Representatives from Guatemala’s Ministry of Education and Universidad del Valle designed math and Spanish reading exams that combined questions from the regional Laboratorio exam with nationally-developed questions that would include more culturally familiar content for Guatemalan students. In the math exam, 7 questions were nationally developed and 33 were from the regional Laboratorio exam. In the reading exam, 21 questions were nationally developed and 19 questions were from the regional Laboratorio exam. Third and fourth grade students completed the reading and math exams; the same content was tested in each grade. The tests were accompanied by background questionnaires for principals, teachers and students of the schools selected for the exam. Data including test scores and background questions were collected from 179 randomly selected schools. In total, 5,825 students and 249 teachers were surveyed (further description of the Guatemala Laboratorio data appears in Wu, Goldschmidt and Hara 2003).

The Guatemala Laboratorio study uses different questions to identify indigenous students, teachers and principals. Students were asked, “What language do you speak at home most of the time,” with answer choices of Spanish or Other. Teachers were asked, “Do you speak a language besides Spanish,” with answer choices of K’iche, Kaqchikel, Mam and Q’eqchi’. Principals were asked, “Do you belong to one of these ethnic groups,” with answer choices of Ladino, K’iche, Kaqchikel, Mam and Q’eqchi’.

11
The third source is Mexico’s National Standards ( _Estándares Nacionales_ ) exam. These data were collected by Mexico’s Ministry of Education ( _SEP-Secretaría de Educación Publica_ ) through the General Directorate of Evaluation ( _Dirección General de Evaluación_ ). SEP conducted the National Standards exam annually between 1998 and 2002, with different grades and background questionnaires used for each year. The 2001 exam for fifth grade students includes 47,719 students from 3,221 schools and also includes the most extensive background data, hence that is the round of the exam that we use in the present analysis. National Standards was stratified by school type – community courses, indigenous schools, urban public schools, urban private schools and rural schools – and this stratification allows representativeness of each school type in the sample. Background questionnaires were obtained from students taking the exam, from a random sample of fifth grade teachers in the selected sample schools – not necessarily instructors of the students who took the exam – and from principals of the sample schools. National Standards uses the Rasch Correction, a form of IRT scaling, to adjust scores from raw to final values (further description of the National Standards data appears in Shapiro and Moreno 2004).

Officially, Mexico has five types of schools: Communitarian Schools, Public Rural Schools, Public Urban Schools, Private Urban Schools and Indigenous Schools. In this analysis the type of school was the only variable that allowed to identify the indigenous students, however, in Mexico the indigenous population that attends school is not only concentrated in the Indigenous Schools and many of these people go to others schools. So, this measure is certainly under-represented.

12
Three cautions should be considered in comparing results between these three data sources. First, the regional Laboratorio and Guatemala Laboratorio data include third and fourth grade students, while the Mexican National Standards data only represents fifth grade students. More students have dropped out of school by fifth than by third or fourth grade, so the Mexican sample offers a rosier picture of the well-being of indigenous and non-indigenous students than the Guatemalan or Peruvian samples offer. Since the grade difference between the Laboratorio data and the National Standards data is a single year, since we control for a student’s grade in the Laboratorio regressions, and since we are comparing groups within a country, this caveat is not critical.

Second, one could worry that the Mexican data are scaled using IRT while the Regional and Guatemalan Laboratorio data are not. That worry identifies a weakness of the Laboratorio exams – they may overweight easy questions and underweight hard questions, so the final score is an imperfect measure of a student’s ability. To address this concern, we use students’ z-scores in regressions and avoid comparison of raw scores between countries. Additionally, this difference in scaling should affect all students and hence is unlikely to impact the indigenous/non-indigenous test score gap.

Third, many students, teachers and principals gave no response to some background questions, forcing exclusion of some observations from multivariate analysis. The present analysis walks a balancing act: on one hand, adding explanatory variables to the regressions increases explanatory power and more effectively shows the determinants of learning. On the 13
other hand, each added explanatory variable decreases the number of observations that can be used in the regressions, and hence increases possible selection bias. We select explanatory variables for inclusion based on four sometimes conflicting factors: (1) indicators that other research has shown to be significant in explaining test scores; (2) indicators that are similarly available in all four countries of this study; (3) indicators that do not force dropping of many observations; and (4) indicators that give particular interest for comparing indigenous and nonindigenous students. While selection bias may play some role in influencing results, we compare test scores of the complete sample and the sample used in regressions for each country; those comparisons show that, at least for learning outcomes, bias introduced by our selection of observations is rather small.

# **3. Discussion**

In each country, indigenous students are a minority of test-takers, but indigenous students are under-represented in Peru and over-represented in Mexico. In Peru, 11 percent of test-takers are indigenous while 30 percent of the national population is indigenous. In Guatemala, 42 percent of test-takers and about 40 percent of the national population is indigenous. In Mexico, 15 percent of test-takers but only 7 percent of the national population are indigenous (Table 1).

Overall, indigenous students enter school with more disadvantaged backgrounds, study in schools with fewer resources and perform worse on exams. In all three countries, indigenous students performed significantly worse than non-indigenous students on both Spanish and math exams. Indigenous students are also significantly more likely to work. Indigenous students consistently have fewer reading materials at home, less access to electricity and parents with less 14
education. Compared to teachers of non-indigenous students, teachers of indigenous students tend to be more often male, less experienced and less educated. The schools of indigenous students more often lack Spanish and math textbooks and have conditions that students describe as bad. Indigenous students are less likely to attend a private school or study in an urban area (Table 6). In short, in all four countries, when indigenous students begin school, they begin with backgrounds likely to undermine their learning, and the lacking facilities and human resources of indigenous schools may reinforce that early disadvantage.

The background and test score differences between indigenous and non-indigenous students give additional insight into the distinct challenges that indigenous students face. In every country, the test score gap between indigenous and non-indigenous students was greater in Spanish reading exams than in math exams, and the gaps in both subjects ranged between 0.6 and 1.1 standard deviations. Compared to non-indigenous students, indigenous students more often enter school with limited knowledge of Spanish; hence, one might expect that the learning gap would be larger in Spanish exams than in math exams. Those gaps compare to a difference of 1 standard deviation between blacks and whites in the United States (Jencks and Phillips 1998) and 0.3 to 0.5 between indigenous and non-indigenous people in Bolivia’s and Chile’s national exams (McEwan 2004). While internationally-designed tests might generate more learning inequality between indigenous and non-indigenous students, these data do suggest that learning inequality between indigenous and non-indigenous students is more severe in Guatemala than in Bolivia or Chile. Educational achievement overall in Guatemala is among the worst in the Western Hemisphere, excepting Haiti (World Bank 2003), and low achievement may appear along with high inequality.

15
In every country, indigenous students are more likely than non-indigenous students to work. In Guatemala, 24 percent of indigenous students regularly work but only 16 percent of non-indigenous students regularly work. In Mexico, indigenous students are 8 percentage points more likely than non-indigenous students to work in the house and 12 percentage points more likely than non-indigenous students to work outside the home. In Peru, indigenous students are 14 percentage points more likely than non-indigenous students to regularly work (Tables 2-5). The additional poverty of indigenous families may partly explain why indigenous students are more likely than non-indigenous students to work and these descriptive statistics do not control for background differences between indigenous and non-indigenous students. Some evidence, however, suggests that a cultural value or positive stigma associated with children’s work in some indigenous communities may explain part of why children’s work remains widespread in indigenous communities (Shapiro and Patrinos 2004).

Also as one might expect, indigenous students more often repeat a grade. In Mexico, one of three indigenous students repeated a grade but only one of five non-indigenous students did. In Peru, the gap in repetition rates is even wider.

Indigenous students also come from families that have fewer reading materials and less educated parents. In Peru, indigenous students are a sixth as likely as non-indigenous students to come from a family with more than ten books at home. Across the three countries, indigenous students are two to eight times as likely to have a guardian that did not complete primary school.

16
Indigenous students also have different teachers and schools. In Peru, indigenous students are 30 to 45 percentage points less likely than non-indigenous students to have a math or Spanish book. In Peru, practically all indigenous students (over 99 percent) attend public schools, while about 25 percent of non-indigenous students attend private schools. In Guatemala, the distance from a school to a departmental Ministry of Education office is twice as great for indigenous as for non-indigenous schools.

For each country, we estimate regressions, one including only indigenous students and the other only non-indigenous students. At the student level, for all three countries, we find that similar factors have the largest impact on math and Spanish test scores. Controlling for other observable factors, being female is associated with increased performance on language exams but worse performance on math exams. Progressing from third to fourth grade is associated with a 0 to 0.5 standard deviation increase in test scores. Working significantly decreases test scores, though the magnitude of the effect is larger for language than for math exams. In Mexico, all types of work worsen students’ test scores, but working outside the home has a more negative effect than working in the home does.

At the household level, having more than ten books at home or having a family member who reads the newspaper – likely associated with parents’ literacy, wealth and concern for education – are both associated with increased Spanish and math test scores (more so for nonindigenous students). In Peru, the education of test takers’ parents only significantly impacts students’ test scores if parents have some tertiary education, while in Guatemala, non-indigenous students having a father who has completed primary or higher level of education is associated 17
with a significant impact on test scores. In Mexico – perhaps due to the larger Mexican sample size - every increase in parents’ education has a significant positive impact on students’ test scores. In Guatemala, most household goods are associated with improved test scores, but only having electricity, a television and telephone have significant impact.

At the school level, the effect of teachers’ experience varies by country, possibly because experience closely correlates with age—a factor for which most of these regressions do not control. In Guatemala, Mexico, and Peru (non-indigenous only), teachers’ experience has a significant and positive impact on test scores. In Peru, teachers’ declaring themselves to have “freedom” has a significant impact in increasing language test scores only for indigenous students. A higher pupil/teacher ratio, in the case of Peru, is associated with a higher test score, while in the case of Guatemala, a higher pupil/teacher ratio results in lower test scores for indigenous and higher for non-indigenous pupils. The empirical literature provides inconclusive evidence on the effect of the pupil/teacher ratio on student performance. Finally, the evidence on the effect of the availability of textbooks in improving learning is weak.

# <u>Selection issues</u>

Two potential sources of student self-selection may be of importance. First, households choose specific schools based on observed arbitrary differences, such as cost and location. Controlling for various socioeconomic and other student characteristics can greatly reduce this potential bias. We therefore control for student background to the extent permitted by the data. Second, households may choose a school based on unobserved differences, or differences that are hard to measure (Vegas 2002). If for example, more motivated parents systematically tend to 18
choose, say, private schools, results will overestimate any positive effects associated with private schools. A third possible source, namely, students (or their parents) choosing a school based on its resources is, probably, of lesser importance in countries like the ones considered here, as opposed to a country such as the United States, where there are substantial differences in resources per student among schools.

We should also note that, while family income is absent from the set of regressors, variables such as pupils’ work activity and number of books at home are expected to capture – at least partly – the effect of family wealth. Therefore, the estimated coefficient of such variables may partly reflect the effect of household wealth. Furthermore, the _a priori_ expectation of the sign of certain (mostly school) variables may be ambiguous; for example, being given a Mathematics textbook is certainly preferred to nothing at all, however, students who are given a textbook may perform worse compared to, say, those who are given teacher-compiled notes. Likewise, certain questions in the questionnaire may be misleading; for example, teachers having a lot of freedom at work may be associated with worse student performance, as it may reflect bad management and poor supervision of teachers’ performance.

Family variables such as the ones considered here are best interpreted as proxies of family wealth and the home educational environment; they may also reflect the influences of unmeasured school variables that are correlated with certain family characteristics (McEwan 2004).

19
# <u>Decomposition Results</u>

Table 9 reports the results of decomposing the non-indigenous/indigenous achievement gap for Peru, Mexico and Guatemala using the traditional Oaxaca-Blinder decomposition method. Achievement decompositions are based on separate regressions for indigenous and nonindigenous students (Tables 6-8). The gross standardized gap for the language test is 0.77, 0.73 and 1.06 standard deviations for Peru, Mexico and Guatemala respectively, in favor of nonindigenous (Spanish-speaking) students. The gross standardized gap in the mathematics test is about 0.69 standard deviations for Peru as well as Mexico and 0.85 standard deviations for Guatemala.

Two issues worth commenting on before proceeding with the discussion of the test score decomposition results relate to how representative are the survey data available of the indigenous and non-indigenous populations of 3<sup>rd</sup> and 4<sup>th</sup> year pupils in the two countries, as well as the severity of selection bias due to dropping observations with partial background information. With respect to the first, it should be noted that pupils who drop out of school before the 3<sup>rd</sup> grade are not part of the sample, while the probability of a child dropping out of school is higher among the indigenous population. Furthermore, pupils who are more often absent (and hence more likely to be absent the day of the test) are not part of the sample.

The severity of selection bias due to dropping observations with partial background information can be assessed by comparing the average test score gap in the original sample to that after dropping observations with partial background information (that is the one that appears in the test-score decomposition in Table 9). Based on this comparison, the selection bias is 20
probably not significant, as the test-score differentials from the original sample of students who took the tests, are not drastically different from those derived form the decomposition and in the case of Mexico identical.

The proportion of the gap that is explained by the various family and school characteristics varies between 41 percent in Guatemala and 75 percent in Mexico (with Peru in the middle with 70 percent) for the language test, and between 55 and 68 percent for Guatemala and Mexico for the mathematics test (with Peru in the middle with 66 percent). Generally speaking, family variables contribute more than school variables to the overall explained component. The unexplained component varies between 25 and 59 percent of the standardized test score gap for the language tests (Mexico and Guatemala) and for the mathematics test, between 32 and 45 percent (Mexico and Guatemala).

McEwan (2004) decomposed the indigenous/non-indigenous test score gap for Bolivia and Chile using the modified decomposition approach. He finds that the unexplained component in the case of the Bolivia language test for third-grade pupils is about one-quarter of the gross gap, an estimate similar to the one derived in this study. However, the results are not directly comparable, as McEwan attempts to control for peer effects (proportion of indigenous students and father’s education across schools). (McEwan and Trowbridge 2005 present similar results for Guatemala, using a different data source.) In the United States, for the black-white test score gap, Fryer and Levitt (2004a) find that only differences in school quality account for differences in test scores, after controlling for many other observable characteristics. But in more recent 21
work, Fryer and Levitt (2004b) find no explanation, including differences in school quality, convincingly explain divergences.

# **4. Conclusions**

In this study, we assess the magnitude of the indigenous/non-indigenous test score gap for 3<sup>rd</sup> and 4<sup>th</sup> year primary school pupils in Guatemala, Peru and Mexico. We identify the main family and school inputs contributing to the test score gap, and we decompose the gap into its constituent components using the Oaxaca-Blinder decomposition method. The non-indigenousindigenous test score differential is much higher in the language than the mathematics test for all three countries, while test score differentials for both language and mathematics are much higher in Guatemala compared to Peru and Mexico.

Certain variables have an unambiguous effect on student test performance. In particular, in Peru, having more books at home, having a father with post-secondary education, and attending a private paying school increases test scores, while poor classroom conditions decrease test scores. On the other hand, for other variables with a more ambiguous _a priori_ effect, the results vary between indigenous and non-indigenous students and the type of test (language versus mathematics). In Mexico, mother’s education is an important factor for determining achievement for non-indigenous children, and the more educated the mother is, the greater is the impact on achievement of these children. In contrast, for indigenous children this effect does not hold and it seems that basic education for the mother is the relevant level that matters for indigenous student achievement.

22
At the student level, the fact of failing a past school year negatively affects achievement for all students. An interesting phenomenon is present with the labor variables; that is, nonindigenous students are more negatively affected in test scores by working than are indigenous children. In particular, indigenous children do not seem to be affected by working in activities inside the house, and the negative effect for them comes from working outside the house. However, the magnitude of this effect is less than half the negative impact that this variable has for non-indigenous children. Finally, teacher and school impacts are small in magnitude for the achievement of all students and for the non-indigenous-indigenous score gap.

Decomposition results suggest that the proportion of the test score gap that is explained by family and school characteristics varies between 41 percent (Guatemala, language test) and 75 percent (Mexico, language test) of the overall gap. Furthermore, family variables contribute more than school variables to the overall explained component. Therefore, although a sizable gap remains, there is still much that can be done to reduce the gap in the future. Among promising policy interventions one could consider the following non-exhaustive list: (1) effective bilingual education instruction designed to provide indigenous children with mastery of the Spanish language before the completion of primary school; often times, however, bilingual education is poorly implemented even within schools designated as indigenous (see, for example, World Bank 2005); (2) compensatory education programs meant to equalize learning opportunities; these have proven to be very effective in closing learning gaps (see, for example, Shapiro and Moreno 2004); and (3) choice of school and increased autonomy; theoretically, school choice could lead to improved learning outcomes (see, for example, Angrist and others 2002), but there is little evidence on this for indigenous students, presumably since they are 23
mostly rural inhabitants and there are very few opportunities for alternative schools (but see the results for Guatemala that show that private school attendance is not beneficial for indigenous students, whereas it is for non-indigenous). Whatever policy choices are made, it is important that such experiments are rolled out over time with treatment and control groups, and that decisions about scaling up are made on the basis of rigorous impact evaluation results.

24