metadata: authors: 0: author: Fuchs, Thomas
1: author: Woessmann,Ludger
count: Colombia,Argentina
docna: 0: docna: Families, schools, and primary-school
            learning : evidence for Argentina and Colombia in an
            international perspective
docty: Policy Research Working Paper
subtopic: State Owned Enterprise Reform,Public Sector Administrative & Civil
            Service Reform,Economics of Education,Educational Sciences,Educational Institutions & Facilities,Effective Schools and Teachers,Democratic Government,Economics and Finance of Public
            Institution Development,Public Sector Administrative and Civil
            Service Reform,De Facto Governments
teratopic: Gender
abstracts: cdata!: This paper estimates the relationship
            between family background, school characteristics, and
            student achievement in primary school in two Latin American
            countries, Argentina and Colombia, as well as several
            comparison countries. The database used is the student-level
            international achievement data of the Progress in
            International Reading Literacy Study (PIRLS), which tested
            the reading performance of fourth-grade students in 2001.
            The nationally representative samples have 3,300 students in
            Argentina and 5,131 students in Colombia. The emerging
            general pattern of results is that educational performance
            is strongly related to students' family background,
            weakly to some institutional school features, and hardly to
            schools' resource endowments. In an international
            perspective, estimated family background effects are
            relatively large in Argentina, and relatively small in
            Colombia. A specific Argentine feature is the lack of
            performance differences between rural and urban areas. A
            specific Colombian feature is the lack of significant
            differences between gender performance. Nonnative students
            and students not speaking Spanish at home have particularly
            weak performance in both countries. But there are no
            differences by parental occupation and no positive effects
            of kindergarten attendance. In Argentina, students perform
            better in schools with a centralized curriculum and
            ability-based class formation.
geo_reg_key: 119228,517191
repnme: repnme: Families, schools, and primary-school
            learning : evidence for Argentina and Colombia in an
            international perspective
display_title: Families, schools, and
            primary-school learning : evidence for Argentina and
            Colombia in an international perspective
listing_relative_url: /research/2005/03/5780377/families-schools-primary-school-learning-evidence-argentina-colombia-international-perspective
topicv3: Teachers Management m1326119 1523,Education, Skills Development and Labor
            Market m1326135 1484,State Owned Enterprises m1326323 721,Education m1327048 470,Returns to Education m1327513 2174
publishtoextweb_dt: 2010-07-01T00:00:00Z
historic_topic: Gender
no_of_pages: 39
author_keys: 0: author_key: 901330
model_extractions: # **WPS3537**

# **Families, Schools, and Primary-School Learning: Evidence for Argentina and Colombia in an International Perspective**<sup>*****</sup>

**Ludger Wößmann** with the help of Thomas Fuchs

Ifo Institute for Economic Research at the University of Munich and CESifo Poschingerstr. 5 81679 Munich, Germany Phone: (+49) 89-9224 1699 E-mail: woessmann@ifo.de Internet: www.ifo.de/link/hs_e.htm

**JEL Classification:** I21, J24, O54

**Keywords:** Education production function, primary schools, international student achievement test, PIRLS, Argentina, Colombia, Latin America

World Bank Policy Research Working Paper 3537, March 2005 _The Policy Research Working Paper Series disseminates the findings of work in progress to encourage the exchange of ideas about development issues. An objective of the series is to get the findings out quickly, even if the presentations are less than fully polished. The papers carry the names of the authors and should be cited accordingly. The findings, interpretations, and conclusions expressed in this paper are entirely those of the authors. They do not necessarily represent the view of the World Bank, its Executive Directors, or the countries they represent. Policy Research Working Papers are available online at http://econ.worldbank.org._

* We gratefully acknowledge financial support from the World Bank for the project “Using International Achievement Results to Improve Public Sector Management of the Education System”. We also appreciate the constructive comments and discussions of Harry Patrinos, Emiliana Vegas and the other participants at the Argentina Skills Study workshop of the World Bank in Washington, D.C.
# **1. Introduction**

The quality of education is of prime importance for the economic fates of individuals and nations alike. Recent research has emphasized that the returns to early learning are particularly large, both because young people’s time horizons to recoup their educational investments are longer and because skills acquired early on facilitate later learning in a dynamic synergistic process of skill formation (cf. Heckman 2000; Carneiro and Heckman 2003). A particular element in this synergistic process is the ability to read, because basic reading abilities are a crucial means to acquire further knowledge in all other areas. A general pedagogical assessment is that by the end of fourth grade, children should have learned how to read and start to read to learn (cf. Martin et al. 2003). Lacking reading literacy by the end of primary school thus diminishes the future possibilities of students to accumulate human capital in other subjects and harms their career opportunities in the society and the economy. It therefore does not come as a surprise that empirical evidence consistently shows that rates of return to education are highest at the primary level (Psacharopoulos and Patrinos 2004).<sup>1</sup> This gives vital relevance to policy questions of how to improve primary-school learning.

This paper estimates empirically the importance of family features and school policies for learning achievement in primary school in two Latin American countries, Argentina and Colombia. It has long been realized that education is one of the key challenges for government policy in Latin America in the era of government withdrawal due to economic liberalization and privatization (cf., e.g., Edwards and Baer 1993). Furthermore, the supply of skilled workers through widespread high-quality education, rather than the liberalization reforms, can be expected to affect the long-run income distribution in Latin America (Baer and Maloney 1997). Therefore, it is a cause for worry that the educational performance of Latin America over recent decades has been dismal (cf. Mizala and Romaguera 2002; Pritchett 2004). A focus on the basic foundations of learning in Latin American societies thus seems particularly warranted.

While a few empirical studies of the determinants of educational quality in individual Latin American countries exist, a drawback often consists in the lack of a benchmark for estimated parameters. For example, finding a statistically significant relationship between parental education and student performance does not yet answer the question of whether such

> 1 It has also be argued that the basic literacy skills generated by primary schooling may carry particularly widespread positive externalities to the population at large (cf. Psacharopoulos 1996), e.g. in terms of the basic functioning and stability of societies.

1
a relationship is particularly strong or rather weak. In order to address this issue, this paper uses data from an internationally comparable student achievement test which provides data for Argentina, Colombia and selected comparison countries, thereby allowing for a benchmarking of estimated parameters relative to other countries.

Specifically, we use data from the most recent international achievement study, the Progress in International Reading Literacy Study (PIRLS), which tested the reading performance of fourth-grade students in 2001, with data released in 2003. A uniform data generation process for representative samples of students in all participating countries ensures comparability across countries. Latin America’s poor educational performance is well documented in PIRLS: Argentina and Colombia ranked 31<sup>st</sup> and 30<sup>th</sup> , respectively, out of a total of 35 participating countries,<sup>2</sup> while at the same time featuring relatively large inequality among their students.<sup>3</sup> The PIRLS study not only provides comparable information on the reading performance of students, but also extensive information on student and school background to an extent not generally available in student achievement studies. This allows for an analysis of the determinants of learning both in the realm of the family and of the school. The available information stems from background questionnaires answered, respectively, by students, teachers, school principals, and – as a particular advantage over other recent international achievement tests – a home background questionnaire completed by the parents of tested students. The latter questionnaire provides particularly relevant information which primaryschool children themselves could not have provided. The available student background features include information on the students’ pre-school reading performance, their kindergarten attendance, school starting age, language spoken at home, immigration status, parental education, occupation and income and rural versus urban location, among several others. The school features include class size, instruction time, proxies for the material endowment of the classroom, teacher characteristics including gender, education, recent training participation and experience, testing policies, curriculum determination, class formation rules and length of students’ staying with the same teacher, among others.

> 2 The only countries that performed lower were Iran, Kuwait, Morocco and Belize.

> 3 At the secondary level, Colombia participated in the 1995 TIMSS study, where it ranked 40<sup>th</sup> of 41 participating countries both in math and in science, with only South Africa performing lower. Similarly, Argentina participated in the 2002 PISA study, ranking 35<sup>th</sup> of 41 in reading, 34<sup>th</sup> in math and 37<sup>th</sup> in science, with in each case at least half of the few lower-performing countries also being Latin American.

2
Estimating education production functions based on such observational data certainly has its limitations (cf., e.g., Glewwe and Kremer 2005). While the extensive PIRLS data does allow for several important features, including the estimation of a limited value-added specification, it cannot solve all the problems imminent to retrospective studies using observational data. What the presented results provide are informative knowledge on the relationship between students’ reading performance and a host of student, family and school characteristics, conditional on all the other characteristics considered. These are useful descriptive results which can inform policymaking when properly – and thus cautiously – interpreted. It should be clear from the start that they do _not_ necessarily always reflect a causal relationship in the sense that the student or school characteristics causally influenced reading performance by the amount of the estimated parameters. Given that observational data always emanate from individual choices, unbiased causal empirical estimates – if ever possible – can ultimately only come from experimental setups of one form or the other. It is hoped that the descriptive results presented here can serve as a first step that leads towards a better-informed quest for causal identifications in the future.

So far, little research has been done on the determinants of educational quality in Argentina and Colombia, and actually in Latin America more widely. The existing empirical research on educational production in Latin America is ably reviewed in Mizala and Romaguera (2002). For Argentina, the only work they cite is Llach et al. (1999), who analyze a national achievement test. Galiani and Schargrodsky (2002) provide an analysis of secondary school decentralization on educational test performance in Argentina. Research on Colombian educational quality focuses mainly on the relative performance of private and public schools (Cox and Jimenez 1991; Piñeros and Rodríguez 1998) and the Colombian voucher system (King et al. 1999; Angrist et al. 2002). With respect to other Latin American countries, somewhat more research has been done on Chile (cf. Sapelli 2002 and the contributions introduced therein), again with special emphasis on its voucher system (e.g., Mizala and Romaguera 2000; Sapelli and Vial 2002; Vegas 2002; Hsieh and Urquiola 2003) and the relative performance of private and public schools (Mizala et al. 2002). Further examples of the little recent work on educational quality in Latin America include Jimenez and Sawada (1999) on the effect of community involvement in El Salvador, Urquiola (2000) on class-size effects in Bolivia and Lopez-Acevedo (2004) on the effects of teacher development policies in Mexico. Schultz (2004), Coady and Parker (2004) and Bourguignon et al. (2003) analyze the effects on the enrollment of students of cash transfer schemes that are 3
contingent on school participation in Mexico and Brazil, but they do not have educational performance data to estimate effects on educational quality.<sup>4</sup> Research on the determination of educational quality in Latin America would be particularly welcome since Latin America is the region featuring the highest rates of return to education among the studies surveyed in Psacharopoulos and Patrinos (2004). Also, the returns to education typically increased in Latin America (including Argentina and Colombia) during the 1990s (cf. Pritchett 2004), which has been attributed to the economy-wide reforms that affected technological progress and created higher returns to education in the adaptation to the generated disequilibria (Behrman et al. 2000). While economy-wide reforms such as capital-market and trade liberalization, privatization and tax and labor-market reforms influence the profitability of educational investments, particularly during transitions to new equilibria, the same can be expected of the education system itself, which will lower the longrun returns to education if it does not succeed in producing basic skills to a satisfactory degree.

The remainder of the paper is structured as follows. Section 2 describes the database of the PIRLS 2001 international student achievement study and presents descriptive statistics on educational achievement and student and school characteristics in the selected Latin American and comparison countries. Section 3 derives the empirical model and its econometric implementation. Section 4 presents results on the relationship between student background and educational performance in the different countries. Section 5 estimates the relationship between school characteristics, both material and institutional, and educational performance. Section 6 summarizes the main findings and concludes.

# **2. The Database**

## **_2.1 The PIRLS 2001 International Student Achievement Test_**

The dataset used in the analysis is the PIRLS 2001 International Database. The PIRLS assessment was conducted in 2001 by the International Association for the Evaluation of

> 4 Cf. Hanushek (1995), Glewwe (2002), Pritchett (2004) and Glewwe and Kremer (2005) for reviews of research on the determinants of educational quality in developing countries. Velez et al. (1993) review the results of 18 early research reports on primary education in Latin American countries, mostly not published in refereed journals or books. While the evidence from other countries is informative, specific national evidence is indispensable because to some extent, the success of any service delivery depends on specific institutional circumstances that cannot be replicated elsewhere (cf. Pritchett and Woolcock 2004).

4
Educational Achievement (IEA) in 35 developed and emerging countries.<sup>5</sup> The aim was to obtain an internationally comparable database for the reading literacy of students in what is the fourth grade of primary school in most education systems. The assessment was designed to capture the skills and features that are necessary “to understand and use those written language forms required by society and/or valued by the individual” (Campbell et al. 2001, p. 3). This definition leads to a real-life focus of the study. Because most national curricula require that students are able to use reading to learn, the definition also ensures that the international comparability of the measured reading performance should not be severely affected by differing national curricula (cf. Gonzalez and Kennedy 2003).

The Latin American countries that participated in PIRLS are Argentina and Colombia. To be able to compare and benchmark the Latin American results, we also analyze an additional six countries. These are Turkey and Macedonia, which show average performance levels similar to Argentina and Colombia. Together, these four countries form our group of “low performing countries” (LPC). We also look at benchmark countries from the industrialized world. Germany and England have been chosen as two large developed countries representing a Continental-European and Anglo-Saxon background, respectively.<sup>6</sup> In addition, two further Western European countries that might constitute interesting benchmarks for the Latin American countries are Greece and Italy. The two countries are presumably the European countries participating in PIRLS that are most comparable to Argentina and Colombia. Many Argentines are of Italian origin, while Greece is perhaps closest among the European countries to the Latin American countries in terms of its level of economic development. Germany, Greece, Italy and England together form our group of “high performing countries” (HPC).

The specific target population of the PIRLS study was defined as the upper of the two grades with the highest share of nine-year-olds of a country at the time of testing. This led to a student population which was in the fourth grade in the countries considered in this paper, with the exception of England, where it was fifth grade. The mean age of the students in the

> 5 For detailed information on the study and the database, cf. Mullis et al. (2003), Martin et al. (2003), Gonzalez and Kennedy (2003), Campbell et al. (2001) and the PIRLS homepage at http://timss.bc.edu/pirls2001.html.

> 6 The United States could not be used for the analysis because two crucial student background variables, the number of books at home and parental education, were not administered in the US. This would result in a lack of comparability to the other countries, as well as to considerable bias in the estimation of the remaining student background and school effects in the US. Instead of the US, England is included in the analysis as a representative of the Anglo-American country group.

5
considered countries is 10 years and 3 months, with a variation between countries of at most 5 months.

While virtually all children in developed countries are enrolled in fourth grade, this is not necessarily the case in developing countries. If a substantial part of the potential student population of a country is not enrolled in school in fourth grade, this might introduce sample selection bias relative to full-enrollment countries. The most detailed data on this are Filmer’s (2003) enrollment profiles based on national survey data, which include Colombian data for 2000. The data show that while 98.5% of Colombian children have been enrolled in first grade, this declines to 91.6% in fourth grade. While the rich and middle class, as well as the urban population in general, come relatively close to universal enrollment at fourth grade, this is not true for the poorest 40% of the Colombian population, or for the rural population. Even though more than 97% of the poor and of the rural population in Colombia enroll in first grade, early drop-out rates among them are very large. At the age of 15-19, only 83.4% of the poorest 40% of the Colombian student population, and only 81.4% of the rural Colombian population, report that they were enrolled in fourth grade (Filmer 2003; cf. Pritchett 2004). This selective non-attendance at the primary-school level may somewhat distort the Latin American samples of students tested by PIRLS, although the enrollment data show that the effect can be expected to be limited.

The PIRLS sampling procedure ensured that a representative sample of the target population was tested in each country. PIRLS used a two-stage sampling method, sampling in the first stage schools and in the second stage intact class(es) of the school. This let to a sample size of 3,300 students in Argentina and 5,131 students in Colombia. In the comparison countries we analyze, the sample size ranges from 2,494 to 7,633 students.

The performance tests were paper and pencil tests, with the assessment lasting a total of 80 minutes for each student. The PIRLS tests were constructed to test a range of relevant skills and competencies that reflected how well the reading literacy of students was developed. Each subject was tested using a broad sample of tasks with differing levels of difficulty in order to represent a coherent and comprehensive indicator of the continuum of students’ abilities. The test items included multiple-choice items and questions requiring the students to construct their own responses. Using Item Response Theory (IRT), PIRLS mapped performance on a scale with an international mean of 500 test-score points and an international standard deviation of 100 test-score points.

6
In addition to the performance tests, PIRLS provides a rich set of background information. This background information was gathered through a student questionnaire, a home questionnaire (completed by parents), a teacher questionnaire and a school questionnaire (completed by school principals). By merging these data with the student achievement data, the database used in this paper contains extensive information on the family background of students, on the resource endowment and teacher characteristics of the class and school and on schools’ institutional features.

A problem immanent in any empirical work based on survey data is non-response and thus missing data. Table A1 in the appendix reports the share of missing values for each variable used in this paper. While there are only very few missing values on such variables as student age and gender, kindergarten attendance and teacher gender and education, the amount of missing data on some variables in some countries is quite substantial, reaching more than 30 percent in Argentina in the case of books at home and parental education, working status and occupation. In such a situation, dropping any student that misses data on at least one variable in multivariate analyses such as the ones conducted in this paper means that one is left with relatively small samples. Furthermore, since data points are unlikely to be missing at random, such a procedure could introduce severe biases and reduce the representativeness of the results. In effect, when sticking to the original data, there is an inescapable trade-off between controlling for many variables and keeping many observations. Even more, dropping all observations with some missing values always means dropping relevant information, because there is abundant information on other variables for each student that misses data on one or a few of the numerous considered variables.

Thus, to ensure the usage and exploitation of all the information available in the dataset, we imputed missing values in the data using a simple median imputation method. This method assigns the median of the available observations at the lowest possible level, i.e. either the class, school or country median, for any missing observation. In this procedure, we used sampling weights to take account of the survey structure of the PIRLS dataset (see Section 3 below). In the estimated regressions, we include imputation dummies which ensure that none of the substantive results are driven by imputed rather than original data, and we do additional robustness tests of the imputation data (again, see Section 3 below).

7
# **_2.2 Descriptive Statistics on Student Achievement_**

The mean reading performance of the eight considered countries in PIRLS, reported in Table 1, clearly reveals the division into the two groups. The group of relatively “low performing countries” (LPC), Argentina, Colombia, Turkey and Macedonia, shows a mean performance between 418 and 449 achievement points (AP), while the group of relatively “high performing countries” (HPC), Germany, Greece, Italy and England, performs between 525 and 553 AP.<sup>7</sup> As the lowest performer among the selected countries, Argentina scores 4 AP lower than Colombia, 24 AP lower than Macedonia, 31 AP lower than Turkey and 122 AP lower than the HPC group. The size of these performance gaps can be illustrated by the unconditional difference of 42 AP between third graders and fourth graders in Sweden, which was the only country in PIRLS that tested third graders in addition to fourth graders. This difference can be interpreted as the additional value-added of one school year in Sweden, since both grades are tested in representative samples. Thus, the Argentine and Colombian performance is the equivalent of one-half to two-thirds of a (Swedish) school year behind the Macedonian and Turkish performance, and roughly three years behind the HPC group.

The mean performance hides substantial differences in the performance distribution between countries, though. The performance variation is much larger in Argentina than in all the countries of the HPC group. This is independent of the measure of performance variation, which include the standard deviation in national test scores (either absolute or relative to the national mean) and the performance difference between percentiles such as the 90<sup>th</sup> and 10<sup>th</sup> or the 75<sup>th</sup> and 25<sup>th</sup> . Only Macedonia shows a spread in performance that is similarly large as in Argentina, while depending on the measure, Colombia and Turkey show a variation that is slightly larger than in the HPC group, sometimes surpassed by England, but substantially lower than in Argentina and Macedonia. This pattern is also evident in the graphical depiction of Figure 1. The descriptive pattern shows that the Argentine education system has a shortcoming with respect to ensuring educational equity and providing support for the relatively weak students.

Another feature emerging from the descriptive data is that the performance of the 75<sup>th</sup> and 90<sup>th</sup> percentile students in Argentina is actually similar to (or even slightly better than in) Colombia. The lower mean performance and larger variation in Argentina arise from the

> 7 The fact that England is the highest performing among the considered countries may be related to the fact that it is the only country where most students start school at age five, so that England tested fifth-grade rather than fourth-grade students.

8
particularly poor performance of the lower half of the Argentine student population. Still, even the 75<sup>th</sup> percentile of students in Argentina and Colombia only roughly reaches the performance level of the 25<sup>th</sup> percentile in the HPC group, and only the 90<sup>th</sup> percentile in Argentina and Colombia comes close to the median performance in the HPC group.

In sum, both Argentina and Colombia show a relatively low reading performance among their students. But Argentina has a relatively large variation among its student body, with the lower part of its distribution performing particularly weak.

# **_2.3 Definition of Variables and Descriptive Statistics on Student Background_**

Table 2 reports descriptive statistics on the input variables which will be used to estimate education production functions. These variables combine two main categories, student background and school characteristics.<sup>8</sup> In terms of students’ background, we have a measure of pre-school reading performance based on retrospective parental evaluation of how well their children could read when they began primary school. Specifically, parents reported reading performance in five areas, namely recognizing most of the letters of the alphabet, reading some words, reading sentences, writing letters of the alphabet and writing some words. Each of the areas was evaluated on a 4-point scale, with 1 = not at all, 2 = not very well, 3 = moderately well and 4 = very well. We use the mean of these five performance evaluations to control for pre-school reading performance in our value-added specification. It is interesting to note that the mean values of this measure do not vary systematically between the LPC and HPC group, always ranging between not very well and moderately well pre-school reading performance. This suggests that this measure is a relative one, formed by the parents under the perception of other students in their country. This does not undermine its value for a value-added specification within each country, though, and as will be seen in the results below, the measure is very strongly correlated with fourth-grade reading performance in every single country.

We also have data on kindergarten attendance, school commencement age and current age for each tested student. At 94%, kindergarten attendance in Argentina exceeds the rates of other LPCs and is comparable to the kindergarten attendance in HPCs. In Colombia, it is 81%. At 50% and 42%, respectively, most Argentine and Colombian children attend

> 8 Cf. Mullis et al. (2002) for additional background information on the education systems and reading instruction in the countries participating in PIRLS.

9
kindergarten for one to less than two years. The fraction of children entering school at age five or younger is larger in the two Latin American countries than in the other LPCs, at 20% and 39%. A large fraction of 93% of Argentine students attends school before they are seven years old. By contrast, late enrollment after age six is a larger problem in Colombia, at 18%.

At 84%, the share of students who always speak the test language at home is relatively low in Argentina and Colombia, albeit not substantially smaller than in the other LPCs. Both countries also feature relatively large fractions of immigrated students, at 38% and 30%.

A proxy for the socio-economic status of the students’ family is the number of books in the household, which reflects the educational, social and economic background of the home. More of the Argentine and Colombian students grow up in a household possessing very few books relative to the comparison countries. As many as 45% and 47% of students in the two countries have at most 10 books in their home (not counting magazines, newspapers or children’s books), and only 4% and 3% have more than 200 books at home. This compares to a mean among the HPCs of 9% with at most 10 books (31% in the other two LPCs) and 22% with more than 200 books (6% in the other two LPCs).

Another indicator of students’ family background, the educational level of the parents (measured as the highest level reached by either father or mother, whichever is higher), also suggests a relatively weak educational background in Argentina and Colombia. However, the share of parents holding a university degree is larger than in the other two LPCs, indicating a larger educational inequality already at the parental level. The strong difference in the two family background indicators between the LPCs and the HPCs is evident in their graphical depictions in Figures 2 and 3.

In the LPCs, about one third of the parents report that neither of them has work, which is substantially larger than in the HPCs, where the fraction is around 10%. While the fraction of blue-collar workers in Latin America is surprisingly similar to the HPCs, the share of whitecollar workers in the LPCs is 16 percentage points lower than in the HPCs.<sup>9</sup> Parents were also asked to report their before-tax annual household income. However, this question is not

> 9 White-collar workers were defined as major group 1-3 of the International Standard Classification of Occupations (ISCO), encompassing legislators, senior officials and managers; professionals; and technicians and associate professionals. Blue-collar workers were defined as ISCO 8-9, encompassing plant and machine operators and assemblers; and sales and services elementary occupations. The residual category between the two, ranging from ISCO 4-7, encompasses clerks; services workers and shop and market sales workers; skilled agricultural and fishery workers; and craft and related trades workers. The variable was set to white-collar if at least one parent was in ISCO 1-3, and to blue-collar if no parent was in ISCO 1-7.

10
available in Argentina, Turkey and Italy. Furthermore, the categories are scaled to each national currency, so that they are not directly comparable across countries.<sup>10</sup> Finally, at 11%, less tested schools in Argentina were located in villages or rural areas than in the other LPCs, while the share of schools located in cities with more than half a million inhabitants is largest in Turkey (35%), Colombia and Macedonia (both 24%).

# **_2.4 Definition of Variables and Descriptive Statistics on School Characteristics_**

The data on school characteristics entail information both on the resource endowment of classrooms and schools and on their institutional features. In terms of resource endowments, the average class size in Argentina is 27 students per class, which is smaller than in Colombia (30), Turkey (35) and England (29), but larger than in the other three HPCs.

Total instructional time (measured as the product of hours per day and days per year) in Argentina is among the lowest observed, 15% lower even than in Germany’s half-day schooling system.<sup>11</sup> The time that students report spending on reading homework per week in Argentina is the lowest among the LPCs, but not lower on average than in the HPCs.

The equipment of schools with instructional material, instructional staff and qualified teachers is measured by principals’ assessment to what extent the shortage or inadequacy of the respective resource affects the school’s capacity to provide instruction, where “lack” means “a lot”, “no lack” means “not at all” and the residual category is in-between (“a little” or “some”). In general, the thus measured equipment is better in Argentina than in Colombia, which again is better than in the other LPCs.<sup>12</sup> 93% of the Argentine and 82% of the Colombian teaching force are female, which is larger than in most other, particularly LPC, countries. Remarkably few teachers (2%) in the Argentine school system hold a university degree, far less than in the other countries. Most of the teachers in all considered countries also hold a specific teaching certificate. 28% of the

> 10 The national categories are: Colombia (in Minimum Monthly Household Income, where 1 MMHI = 286000 pesos): less than 1, 1-2, 3-4, 5-6, more than 7, do not know; Macedonia (in Denars): less than 3900, 3900-6199, 6200-9299, 9300-12399, 12400-15499, 15500 or more; Germany (in 1000 DM): less than 40, 4059, 60-79, 80-99, 100-119, 120 or more; Greece (in 1,000 drachmas): below 2400, 2400-3599, 3600-4799, 4800-7199, 7200-9599, 9600 or more; England (apparently in 1000 $): less than 20, 20-29, 30-39, 40-49, 50-59, 60 or more.

> 11 This measure of instructional time is reported by the school principal. We also experimented with an alternative measure, the specific time for reading instruction and activities per week reported by the teacher, with similar qualitative results in our estimations.

> 12 As alternative measures of material endowment, we also experimented with information from teacher questionnaires on whether they use or never use textbooks, workbooks, computer software and other materials. These measures did not yield statistically significant results in our estimations.

11
Argentine teaching force, and 17% of the Colombian teaching force, reported having participated in a professional development program (workshop or seminar) on the teaching of reading for more than one week (of 35 hours) in the preceding two years, which is larger than in all comparison countries. The Argentine teaching force has a relatively low mean professional experience, at slightly less than 14 years on average.

In terms of more institutional features, the teacher background questionnaire gives information on student assessment practices. 59% of Argentine teachers and 65% of Colombian teachers report that they place major emphasis on classroom tests (for example, teacher-made or textbook tests) to monitor students’ progress in reading. Only in England do teachers of more than a third of the student population place major emphasis to monitor student progress on national or regional examinations.

In all countries, a huge majority of principals reports that the national or regional curriculum has a lot of influence on their school’s fourth-grade curriculum. About two thirds of students in Argentina and Colombia are taught in schools where fourth-grade classes are formed on the basis of students’ ability, so that all students in a class are about the same ability. This prevalence of student tracking is considerably larger than in the other countries, particularly the HPCs. Staying with the same classroom teacher for four or more years is highly uncommon in Argentine primary schools, but much more so in most other countries. 44% of Argentine students stay with the same teacher for at most one year, which is only surpassed by England.

# **3. The Empirical Model**

This section derives the empirical model and discusses its econometric implementation, including discussions of the employed value-added specification, the implemented estimation equation, imputation controls, the covariance structure and sampling weights.

As derived, e.g., in Hanushek (2002), under standard assumptions the education production function can be estimated in the value-added form:<sup>13</sup>

> 13 Cf. Todd and Wolpin (2003) for a detailed discussion of the relative benefits of different models to estimate education production functions, and of the respective structural and behavioral assumptions underlying the different model specifications. Cf. also Glewwe (2002) and Glewwe and Kremer (2005) for discussions of underlying behavioral models and the limitations of estimating them using retrospective data.

12
where _T_<sup>_t_</sup> _is_<sup>is the test-score performance of student</sup><sup>_i_in school</sup><sup>_s_at time</sup><sup>_t_,</sup><sup>_B_are student</sup> background characteristics and _S_ are school characteristics. This value-added specification is considered preferable to simple contemporaneous specifications because it accounts for the cumulative character of the education process by controlling for prior performance at time _t_ –1 (cf. Hanushek 2002). Given the proxy of pre-school reading performance we derived from the PIRLS questionnaire, we can implement a version of this value-added specification using the PIRLS data. This is not a perfect solution because ideally, we would either have the students tested at the start of fourth grade or cumulative measures of school inputs _S_ from first to fourth grade. However, given relative constancy of school factors over the course of the first four years of schooling, the approximations introduced by this specification should be limited.

Treating the input factors as additively separable and considering the imputed character of some observations, we implement the following microeconometric estimation equation of the education production function: where _T_<sup>_t_</sup> _is_<sup>is the PIRLS test score of student</sup><sup>_i_in school</sup><sup>_s_in fourth grade,</sup><sup>_Tt_–1</sup> _is_<sup>is the measure</sup> of pre-school reading performance, _B_ is a vector of student background data, _S_ is a vector of data on school characteristics and _ε_ is a student-specific error term. The parameter vectors β 1 to β 4 will be estimated in the regressions.

The vector _D_ contains one dummy for each variable in _B_ and _S_ that has missing data. The dummies equal 1 if a value was missing and thus imputed and 0 for original data (cf. Section 2.1 above on the data imputation). By including these imputation controls in the estimation, observations with missing data on each variable are allowed to have their own intercepts. This ensures that none of the results are driven by imputed data. While data imputation introduces measurement error in the explanatory variables, which should generally make it more difficult to observe statistically significant effects, the imputation controls ensure that the results are robust against possible bias arising from data imputation.

As an additional procedure to test for the robustness of our treatment of missing data, we also estimated our models under disregard of any missing data individually for each variable, re-running the regressions as many times as there are variables with missing data, for the two Latin American countries. We did not observe a single qualitative change of our results under these alternative specifications without each variable’s imputed values, so that our presented results do not hinge on the data imputation.

13
Owing to the complex data structure produced by the PIRLS survey design and the multilevel nature of the explanatory variables, the error term ε of the regression has a non-trivial structure. Although we include a considerable amount of school-related variables, we cannot be sure that there are no omitted variables at the school level. Given the possible interdependence of students within the same school, the use of class- and school-level variables and the fact that schools were the primary sampling unit (PSU) in PIRLS, there may be unobservable correlation among the error terms ε _is_ at the school level (cf. Moulton 1986). We correct for potential correlations of the error terms by imposing an adequate structure on the covariance matrix. Thus, we suppose the error term to have the following structure: where _ηs_ is the school-level element of the error term and _υi_ is the student-specific element.

We use clustering-robust linear regressions (CRLR) to estimate standard errors that recognize this clustering of the student-level data within schools. The CRLR method relaxes the standard independence assumption and requires only that the observations be independent across the PSUs, i.e. across schools. By allowing any given amount of correlation within the PSUs, CRLR estimates appropriate standard errors when many observations share the same value on some but not all independent variables (cf. Deaton 1997). To avoid inefficiency due to heteroscedasticity, CRLR imposes a clustered covariance structure on the covariance matrix, allowing within-school correlations of the error term. Thus, the form of the covariance matrix _V_ is with ∑ _i_ ( _i_ =1,..., _l_ ) representing the covariance matrices of the residuals of least-square regressions within each school cluster (PSU). Observations of two different PSUs are thus assumed to be independent and uncorrelated, leading to the block-diagonal matrix _V_ with PSUs as diagonal elements. This method yields consistent and efficient estimates, enabling valid statistical inferences of the obtained estimation results (cf. White 1984).

Finally, the stratification of the PIRLS sampling design within each country produces varying sampling probabilities for different students. To obtain representative estimates from 14
the stratified survey data, we employ weighted least squares (WLS) estimation using the sampling probabilities as weights. WLS estimation ensures that the proportional contribution to the parameter estimates of each stratum in the sample is the same as would have been obtained in a complete census enumeration (cf. DuMouchel and Duncan 1983; Wooldridge 2001).

# **4. Student Background and Educational Achievement**

This section reports the results of regressing students’ educational performance on a host of student characteristics, which mainly measure their family background. Because this section is interested in the total impact of student background on educational performance, including any effect that might work through families’ differential access to schools with different endowments and through their influence on the institutional features of schools, the estimation of the student-background effects does not control for school characteristics, such as resource endowment and institutional characteristics, as in equation (2): While this specification omitting school characteristics should be preferable for the research question considered in this section, we also tested the robustness of our student-background results to estimating equation (2), which includes the school characteristics. None of our qualitative results are sensitive to the alternative specification.

Tables 3a and 3b report the results of estimating equation (5) for each of the considered countries. Fourth-grade reading performance as tested in PIRLS is strongly positively related to reading performance at the start of primary school in each of the countries. The measure of pre-school reading performance has the highest _t_ -statistic of all variables in Argentina and Colombia, and also one of the highest _t_ -statistics in all other countries. This provides some confidence in the value-added specification employed.

Kindergarten attendance is not statistically significantly related to reading performance of Argentine students, and in Colombia, students who attended kindergarten for a considerable time even performed significantly worse. In the other considered countries, there is similarly very little evidence of positive relationships between kindergarten attendance and later student performance. In interpreting this result, it should be borne in mind that children are not randomly assigned to attending kindergarten, so that significant selection patterns may influence this result. However, assuming that students from superior backgrounds are more 15
likely to attend kindergarten, particularly in developing countries, the estimates might generally be expected to be positively biased. If this was the case, the results would suggest that kindergarten attendance alone does not improve primary-school learning. This might be the case where kindergartens are day-care centers rather than learning environments, so that the alternative of spending time at home would prepare students for later learning just as well.

There is no significant performance difference between Argentine children starting school at age five and age six, while in Colombia, children starting at age six perform slightly better. Because the effect of school commencement age controls for linear effects of the current age of the students in months, thereby holding effects of monthly age and possible grade repetition constant (as well as the grade level, which is the same for all students of a country), this result suggests that starting school early at age five is slightly detrimental for students in Colombia. However, later school commencement at age seven or older is related to lower performance in Argentina, but not in Colombia. Thus, starting school later than at age six is detrimental for students in Argentina.

Boys perform significantly worse than girls in all countries, with the sole exception of Colombia, where the performance difference between gender is small and statistically insignificant. In Argentina, the performance difference is 15.1 AP, which is equivalent to 15.1% of an international standard deviation in test scores and to 15.8% of a standard deviation in Argentine test scores.

Students who always speak the test language at home perform statistically significantly better than otherwise in all countries. Similarly, with the sole exception of Greece, students who were not born in the country perform statistically significantly worse than native students. The size of these effects is substantial and broadly similar between the Latin American and comparison countries. Holding the student’s immigration status constant, belonging to the second generation of immigrants does not have a significant additional negative effect in Argentina, while there is an additional negative effect of the father being non-native in Colombia.

The first measure of general family background, the number of books in the household, shows a consistent and strong positive relation to student performance in all countries. This proxy for the social, economic and educational background of the students’ home has a particularly large effect in Argentina, but a relatively small effect in Colombia. In Argentina, the achievement differential between the lowest and highest category of books at home is as large as 53.4 AP, surpassing all comparison countries. By contrast, it is only 14.4 AP in 16
Colombia, smaller than in all comparison countries. A small part of this difference can be attributed to the fact that data on household income are not available in Argentina, so that income is controlled for in Colombia but not in Argentina. However, excluding the income controls in Colombia and the other LDC with income data, Macedonia, does not change the picture significantly. In Macedonia, there is no substantive change at all in the results, and in Colombia, the currently significant estimates on books at home, parental education and working status get a bit larger and statistically significant at the 1% level. Still, the Colombian estimates remain smaller than in the comparison countries, e.g. at 21 AP in the case of the highest category of books at home.

The second measure of general family background, the education level of the parents, shows a very similar pattern. It is statistically significantly and strongly related to student performance in all countries, but the effect is relatively large in Argentina and particularly small in Colombia. The estimated effects of books at home and parental education in the different countries, depicted graphically in Figures 4 and 5, both suggest that the Argentine school system performs worse than other school systems, particularly among the LPCs, in terms of alleviating the effects of differential family backgrounds. By contrast, the Colombian school system achieves an equitable performance for children from different family backgrounds to a considerably larger extent. A tentative conclusion would be that these relatively large family background effects in Argentina might be one important factor underlying the relatively large performance variation detected in Argentina, particularly compared to Colombia.

For Argentina, these results on intergenerational effects on the quality of primary education correspond to findings on intergenerational mobility in the quantity of education between parents and their children. Behrman et al. (2001) and Dahan and Gaviria (2001) report that the intergenerational mobility in educational quantity between parents and their children in Latin America, including Argentina and Colombia, is relatively low compared to other countries (i.e., the effect of parental on children’s educational quantity is relatively large). Our results suggest that while these quantitative effects go hand in hand with an equally differentiated educational performance in terms of the skills that children learn in Argentina, this does not seem to be the case in Colombia.

Both in Argentina and Colombia, students who have one parent working full-time perform statistically significantly higher than students whose parents do not work. However, in both countries there is no significant performance difference between students whose parents have 17
different occupations. This is exceptional, as such performance differences exist in all comparison countries. Household income is significantly and strongly related to student performance in Colombia.<sup>14</sup> The same is true in all other countries with income data. However, due to differing national currencies and thus income categories, the estimates are not directly comparable across countries.

In contrast to all comparison countries, there is no statistically significant difference in student performance between any category of school location in Argentina. In the other LPCs, students in rural areas perform systematically worse than students in urban areas, and even than students in large cities of more than half a million inhabitants.

Not considering the imputation controls, the student background variables considered in the model together account for as much as 27% of the total variation in student performance both in Argentina and in Colombia. This is similar to Greece and England, slightly smaller than in Germany and Macedonia, but substantially larger than in Turkey and Italy.

# **5. School Characteristics and Educational Achievement**

Tables 4a and 4b report the relationship between school characteristics and student achievement, estimated using equation (2) above, which controls for the effects of student background characteristics. The considered school characteristics fall into two broad categories, namely measures of the resource endowment of classrooms and schools and measures of the institutional features of schools.

## **_5.1 Resource Endowment_**

In all considered countries, there is no consistent evidence of significant relationships between resource endowments and student performance. The few statistically significant estimates of a positive relationship tend to be matched by statistically significant estimates of negative relationships. This is true for Argentina and Colombia as much as for the comparison countries in the LPC and HPC group.

In no country are smaller classes related to better performance in a statistically significant way. However, estimates of class-size effects are particularly prone to endogeneity bias. For example, if class size is allocated in a compensatory way, so that weak students are placed in smaller classes, results from observational data will be biased against finding class-size effects (cf., e.g., Wößmann and West 2004). By contrast, if strong performers are placed into

> 14 Note that in Colombia, national category 5 refers to “I do not know”.

18
smaller classes, as may be the case in the more elitist education systems of many developing countries, results will be biased in favor of finding class-size effects. The extent to which either of this is the case in this study has to remain an open question.

Instructional time is also not statistically significantly related to better reading performance in any country at conventional significance levels. Likewise, time spent on homework is not significantly related to student performance in Argentina, and actually significantly negatively in Colombia.

The three groups of variables on lack of instructional material, instructional staff and qualified teachers are mostly not statistically significantly related to student performance in Argentina and Colombia. In Argentina, students in schools whose principals report that their instruction is not at all impeded by lack of instructional material actually perform significantly worse in Argentina. Students in Argentine schools whose principals report that their instruction is not at all impeded by lack of instructional staff perform significantly better, though. In Colombia, the only statistically significant relationship is that students in schools whose principals report that their instruction is strongly impeded by the lack of instructional material perform worse.

The only uniform result on teacher characteristics is that students of female teachers tend to perform better, with the relationship being statistically significantly in Argentina, Colombia, Macedonia and Italy. Teachers’ holding a university degree is statistically significantly positively related to student achievement in Greece, but neither in Argentina nor in Colombia. Similarly, there is no statistically significant relationship between teachers’ holding a teaching certificate and student performance in Argentina and Colombia. Students of teachers who participated in professional development seminars during the preceding two years actually perform statistically significantly lower in Argentina. Teacher’s experience in the teaching profession is not significantly related to student performance in Argentina and Colombia.

This general lack of resource effects resembles much previous research, also in Latin America (cf. Mizala and Romaguera 2002) and in other developing countries (cf., e.g., Hanushek 1995; Kremer 2003; Pritchett 2004). It should be borne in mind, though, that this correlational evidence does not necessarily depict causal effects, as the estimates might be biased by resource endogeneity, e.g. due to omitted variables and placement biases (cf. Glewwe 2002; Wößmann and West 2004; Glewwe and Kremer 2005). Thus, the presented results are suggestive at best, but clearly not definitive.

19
# **_5.2 Institutional Features_**

Recent theoretical contributions have emphasized the important effects that institutional features of the school system can play in determining educational performance, because they determine the incentives that the involved actors have to further student learning (cf. Bishop and Wößmann 2004 and the references therein). Furthermore, evidence based on the crosscountry variation in student performance shows that institutional effects matter empirically (e.g., Wößmann 2002, 2003; Fuchs and Wößmann 2004a, 2004b). However, most of the institutional features generally do not vary substantially within school systems, but only between countries. Empirically, institutional effects should therefore mainly be an issue in cross-country rather than within-country research. That is, altered institutional set-ups of their school systems could potentially shift countries onto production functions that are substantially different in terms of overall efficiency.

As is apparent from Table 2, the PIRLS background data reveal that a few institutional features do also vary within some of the considered school systems. Unfortunately, PIRLS does not provide data on such institutional features as the private or public operation or financing of individual schools, nor on decentralization and school autonomy on budgets, personnel and process decisions.<sup>15</sup> But there is data on testing policies and curriculum determination, as well as on ability-based class formation and on how long primary students stay with the same teacher. While these institutional features seem much more important on a cross-country perspective (cf. Fuchs and Wößmann 2004b for the importance of institutional effects in cross-country PIRLS evidence), it seem worth including them also in the withincountry analyses presented here.

Teachers’ method to monitor students’ progress, in terms of use of classroom tests or centralized examinations, are not statistically significant related to reading performance in Argentina and Colombia, as well as within most other considered countries.<sup>16</sup> In Argentina, as well as in Turkey, students in schools whose curriculum is strongly influenced by the national or regional curriculum perform statistically significantly better than otherwise. This might suggest that external standards are conducive to student performance in these countries. At 36 AP, the effect size is substantial in Argentina.

> 15 Galiani and Schargrodsky (2002) present important results on the effects of decentralization on student performance in Argentina.

> 16 The positive relationship between use of national or regional examinations and student performance in Argentina is statistically significant only at the 20 percent level.

20
In Argentina, as well as in Macedonia, students in schools that have a policy of class formation based on student ability perform statistically significantly better than otherwise. This might indicate that tracking students into relatively homogenous groups helps their learning in Argentina. In Colombia, students in schools where they typically stayed with the same teacher for one year or less in primary school performed statistically significantly lower. However, the opposite is true in Argentina, suggesting that there is no consistent relationship of length of teacher assignments to student performance in the different countries.<sup>17</sup>

# **6. Summary and Conclusion**

The general pattern of results on the relationship between families, schools and primaryschool learning in Argentina and Colombia is similar to what has been found in other regions of the world and mostly at the secondary-school level (cf. Hanushek 2002 for the United States; Wößmann 2004 for Western Europe; Ammermüller et al. 2004 for Eastern Europe; Wößmann 2005 for East Asia). There is strong evidence of a relationship between students’ educational performance and numerous measures of their family background, but results on school characteristics are very inconsistent. There is basically no consistent evidence of a relationship between educational performance and schools’ resource endowment. Within each country, there is only some evidence that the variation in student performance is related to the institutional characteristics of schools which could be measured in this study. For example, a centralized curriculum and ability-based class formation within schools are positively related to student performance in Argentina. Using the same data source, evidence on institutional effects is much stronger between countries (cf. Fuchs and Wößmann 2004b), suggesting that institutional features of the school systems may exert strong systemic effects.

The evidence found in this paper pinpoints the crucial role that families play in determining the reading performance of students in primary school. Consistent with recent US evidence emphasizing basic family background factors in determining college readiness of students (cf. Carneiro and Heckman 2003), we find that learning readiness and family background have very strong effects on educational performance of Argentine and Colombian primary-school students. These effects do not seem to be easily amenable to simple policy interventions such as increased kindergarten attendance, as kindergarten attendance is not

> 17 The large negative coefficient in Argentina on staying with the same teacher for at least four years is due to only one principal reporting so. It may therefore reflect other specific characteristics of this school, for example a remotely located school which is hardly accessible and thus has only a single teacher.

21
found to have significant attenuating effects on performance differences. Given the importance of primary-school reading ability for later learning, these strong familybackground effects may have important consequences for the longer-run intergenerational mobility of Latin American societies.

Comparing the evidence for Argentina and Colombia with the set of comparison countries, the general pattern of results in terms of effects of student background and school characteristics is rather consistent across countries. But benchmarking the different estimated parameters across the countries also reveals characteristic national patterns. In particular, the effects of family background, measured by the number of books at home and by parental education, are relatively strong in Argentina and relatively weak in Colombia. In this sense, the fact that the intergenerational transmission of skills is particularly strong in Argentina can be viewed as one important underlying factor behind the relatively large variation in student performance observed in Argentina, particularly in the comparison within Latin America to Colombia.

Another result specific to Argentina is that once the family-background factors are held constant, no significant performance difference is found between rural and urban areas. A specific result for Colombia is that it is the only country where boys do not score significantly worse than girls. Finally, a result that distinguishes the two Latin American countries from all comparison countries is that they do not show a significant performance difference between parental occupations.

The results presented here using the data from the PIRLS student achievement test provide useful descriptive features of the different primary-school systems. However, the limitations of such retrospective studies, particularly when it comes to causal inferences, are well known (cf., e.g., Glewwe 2002; Todd and Wolpin 2003; Glewwe and Kremer 2005). The need for further research on the underlying behavioral relationships, preferably based on quasiexperimental or truly experimental setups which are able to exploit exogenous variations in important family and school characteristics, is beyond doubt.

22