metadata: authors: 0: author: King, Elizabeth M.
1: author: Orazem, Peter F.
2: author: Paterno, Elizabeth M.
count: Pakistan,World
docna: 0: docna: Promotion with and without learning :
            effects on student enrollment and dropout behavior
docty: Policy Research Working Paper
subtopic: Educational Sciences,Educational Institutions & Facilities,Effective Schools and Teachers
teratopic: Education
abstracts: cdata!: Many educators and policymakers have
            argued for lenient grade promotion policy - even automatic
            promotion - in developing country settings where grade
            retention rates are high. The argument assumes that grade
            retention discourages persistence or continuation in school
            and that the promotion of children with lower achievement
            does not hamper their ability or their peers' ability
            to perform at the next level. Alternatively, promoting
            students into grades for which they are not prepared may
            lead to early dropout behavior. This study shows that in a
            sample of schools from the Northwest Frontier Province of
            Pakistan, students are promoted primarily on the basis of
            merit. An econometric decomposition of promotion decisions
            into a component that is based on merit indicators
            (attendance and achievement in mathematics and language) and
            another that is uncorrelated with those indicators allows a
            test of whether parental decisions to keep their child in
            school is influenced by merit-based or non-merit-based
            promotions. Results suggest that the enrollment decision is
            significantly influenced by whether learning has taken
            place, and that grade promotion that is uncorrelated with
            merit has a negligible impact on school continuation.
geo_reg_key: 80475,80451,80470
repnme: repnme: Promotion with and without learning :
            effects on student enrollment and dropout behavior
display_title: Promotion with and without learning
            : effects on student enrollment and dropout behavior
listing_relative_url: /research/2008/09/9881999/promotion-without-learning-effects-student-enrollment-dropout-behavior
topicv3: Teachers Management m1326119 1523,Education, Skills Development and Labor
            Market m1326135 1484,Teacher Absenteeism m1327035 1528,Education m1327048 470,School-Based Management m1327730 2143
publishtoextweb_dt: 2010-07-01T00:00:00Z
historic_topic: Education
no_of_pages: 35
author_keys: 0: author_key: NA
1: author_key: NA
2: author_key: NA
model_extractions: WPS4722 Policy ReseaRch WoRking PaPeR 4722 Promotion with and without Learning: Effects on Student Enrollment and Dropout Behavior _Elizabeth M. King Peter F. Orazem Elizabeth M. Paterno_ The World Bank Development Research Group Human Development and Public Services Team September 2008
Policy ReseaRch WoRking PaPeR 4722

# **Abstract**

Many educators and policymakers have argued for lenient grade promotion policy—even automatic promotion—in developing country settings where grade retention rates are high. The argument assumes that grade retention discourages persistence or continuation in school and that the promotion of children with lower achievement does not hamper their ability or their peers’ ability to perform at the next level. Alternatively, promoting students into grades for which they are not prepared may lead to early dropout behavior. This study shows that in a sample of schools from the Northwest Frontier Province of Pakistan, students are promoted primarily on the basis of merit. An econometric decomposition of promotion decisions into a component that is based on merit indicators (attendance and achievement in mathematics and language) and another that is uncorrelated with those indicators allows a test of whether parental decisions to keep their child in school is influenced by merit-based or non-merit-based promotions. Results suggest that the enrollment decision is significantly influenced by whether learning has taken place, and that grade promotion that is uncorrelated with merit has a negligible impact on school continuation.

This paper—a product of the Human Development and Public Services Team, Development Research Group—is part of a larger effort in the department to understand how to improve educational outcomes in developing countries, especially among poor people, by examining what goes on inside schools and how specific government policies affect the behaviors and choices of students, their teachers and their parents. Policy Research Working Papers are also posted on the Web at http://econ.worldbank.org. The author may be contacted at eking@worldbank.org.

_The Policy Research Working Paper Series disseminates the findings of work in progress to encourage the exchange of ideas about development issues. An objective of the series is to get the findings out quickly, even if the presentations are less than fully polished. The papers carry the names of the authors and should be cited accordingly. The findings, interpretations, and conclusions expressed in this paper are entirely those of the authors. They do not necessarily represent the views of the International Bank for Reconstruction and Development/World Bank and its affiliated organizations, or those of the Executive Directors of the World Bank or the governments they represent._ Produced by the Research Support Team
# **Promotion with and without Learning: Effects on Student Enrollment and Dropout Behavior**

Elizabeth M. King<sup>a</sup> , Peter F. Orazem<sup>b</sup> , and Elizabeth M. Paterno<sup>c</sup> Keywords: Grade repetition, grade retention, grade promotion, enrollment, achievement, dropout, Pakistan _______________________________________________ a The World Bank. eking@worldbank.org b Iowa State University pfo@iastate.edu c Ministry of Community and Social Services, Government of Ontario. Elizabeth.Paterno@css.gov.on.ca Useful comments on earlier versions of this paper from Felipe Barrera, Halsey Rogers, Guilherme Sedlacek, and participants at meetings of the Population Association of America and the Midwest Economics Association are gratefully acknowledged. The opinions and conclusions expressed in this paper are those of its authors and do not necessarily reflect positions of the World Bank and its member governments.
1

# **1.  Introduction**

Education policy-makers have long debated the relative benefits of social promotion versus grade retention. Social promotion is the policy of promoting students from one grade to the next, irrespective of their performance. Advocates claim that even low-performing students would benefit from staying with and learning from their peer group, whereas grade retention harms students’ selfesteem, does not improve their performance, and increases their likelihood of dropping out of school (Shepard and Smith 1989). To varying degrees, social promotion is practiced in countries such as Denmark, Japan, Korea, Norway, Sweden and many states in the United States. Countries such as France and most developing countries use grade retention extensively as a means to address student performance (Bonvin 2003): the practice of holding back underperforming students in the same grade until they attain minimum grade-appropriate skills. Proponents of grade retention believe that waiting until students have attained mastery of the curriculum will better prepare them for more advanced work at the higher grades whereas social promotion will doom them to falling ever farther behind their classmates.

There is mixed evidence regarding the relative merits of the two policies. Reviewing a large number of studies of U. S. schools, Holmes (1989) and Jimerson (2001) conclude that retained children perform about one-third of a standard deviation below promoted students on achievement measures in subsequent school years.<sup>1</sup> The adverse effect of grade retention appears larger on test scores than on emotional or behavioral outcomes (Jimerson 2001). Using a regression discontinuity design to get a better match between the retained students and comparison groups, Roderick and Nagaoka (2005) find that third-grade students who were retained under Chicago’s high-stakes testing policy do not yield higher language test scores two years after the retention, and that retained sixth graders had lower achievement growth. Focusing instead on dropout rates, Grissom and Shepard (1989) find that grade retention increased dropout rates by as much as 20-30 percent, even after controlling for achievement, socioeconomic status and gender, although Eide and Showalter (2001)
2 conclude that grade retention has a negative but statistically insignificant impact on the probability of dropout in their study.

Other studies report contrary findings – grade retention improves student achievement by allowing students who are ill-prepared for the next grade to catch up academically and emotionally. In impoverished areas such as Brazil’s rural northeast, retained grade-two students performed more than half a standard deviation below average before repetition, but performed slightly above average after repetition (Gomes-Neto and Hanushek 1994).<sup>2</sup> In Burundi, grade repetition at the end of the primary cycle is the accepted way by which sixth-grade students prepare for a very selective entrance examination that would give them a place in a greatly limited secondary school system (Eisemon et al. 1993). In Chicago, the high-stakes testing-based promotion policy that increased the grade retention rate of eighth-graders from 1% to 10% actually lowered later dropouts because fewer students entered high school ill-prepared (Allensworth 2004). Even Holmes (1989), whose metaanalysis has been cited frequently in opposition to grade retention, reports that when retained students are compared with same-grade rather than same-age peers, there are positive effects from repeating a grade, although the gains may be temporary.<sup>3</sup> This study examines a different aspect of the debate about grade retention and promotion. In particular, we explicitly consider how parents process the information that grade promotion or retention provides about student achievement and integrate that information into parental decisions regarding their children's schooling.<sup>4</sup> In developing countries, even at the earliest grades, parents implicitly evaluate whether the value of their schooling dominates the opportunity costs of child time outside of school, and these assessments may be influenced by whether the child is perceived to be learning from school.

We address this question in the context of first-grade and second-grade children in the Northwest Frontier Province (NWFP), the smallest and northernmost of Pakistan’s four main provinces. First, we establish the extent to which promotions are based on student performance in
3 the classroom. The teacher’s decision to retain or promote a child could be based on factors other than academic performance, including policies on automatic promotion, student deportment, and parental preferences. This leads to a decomposition of promotions into two components – one based on observed academic merit and the other based on factors other than merit. We then examine how merit-based and non-merit-based promotions are related to continued enrollment versus student dropout.

Our analysis uses a unique data set collected in the NWFP in 1994. At that time, the NWFP Education Management Information System (NEMIS) was first exploring the use of a student assessment system. Prior to that time, there had not been any formal means by which parents or teachers could compare student performance across schools or even across children within a school. Despite this lack of prior knowledge of relative student learning, we find that teacher promotion decisions in primary schools are based primarily on student attendance and on student skills as measured by performance on tests of mathematics and language. The relationship to test scores is particularly interesting in that the teachers never saw the students' performance on the tests, and so they must know which children have mastered the material even without aid of formal assessments. Even more striking, largely illiterate parents appear to base their decisions of whether to send the child to school for another year largely on merit-based promotions. Promotions that are not correlated with measured student cognitive attainments have a much smaller positive impact on the probability of school continuation. This finding implies that parents make their decisions regarding a child’s continued schooling on the basis of perceived learning in the previous year, rather than on promotion or repetition _per se_ . It would also suggest that if a child's ability to learn in future years is reduced by being placed in a grade for which the child is unprepared, then promotion could lead to increased dropout.

Increasingly, researchers and development agencies have been exploring school-based management reforms and increased parental involvement as mechanisms for improving school
4 quality and student learning (e.g., Gunnarsson et al. 2004, Umansky and Vegas 2007; World Bank 2007). One of the concerns about decentralized control of local schools, however, is that uneducated parents are not able to assess quality and would therefore be unable to advocate for quality schools.<sup>5</sup> Our findings suggest that, even without external sources of information, parents may be better judges of school quality than may have been previously thought.

# **2.  The Model**

This section presents a model of demand for child education in the household, followed by a model of the decisions made by teachers and schools about whether or not to promote students to the next grade. As first proposed by Becker (1967) in his Woytinsky Lecture, households choose levels of human capital investments by equating the (discounted) expected marginal private benefits and the expected marginal private costs of these investments. Applying this framework to the demand for child schooling, parental decisions about child education depend on the direct and indirect costs of schooling, family income, and expected hedonic and/or pecuniary returns to schooling.<sup>6</sup> This model generally assumes that costs, income, and returns are all known at the time of the decisions. In reality, information on returns is revealed over time and parental decisions can better be modeled as a sequential one.

Part of the revelation process for parents is learning more about the child’s ability and aptitude for school by observing the child’s performance in school. Each school year, parents observe the child’s progress in school through two indicators, whether the child is promoted to the next grade and how much the child learns. Conditional on new information, parents update their decision (as well as aspirations) regarding how many years the child will remain in school. Abstracting from any compulsory education law, this model implies that if the child’s progress is below parental expectations, then parents will reduce the additional number of years the child will be in school.
5 Another part of the revelation process is parents discovering more about the quality of the schools their children attend, including the extent to which schools are able to impart new knowledge and skills to their children, and about how schools (teachers) assess student performance and decide on which students to promote or retain at each grade. The model below incorporates grade promotion decisions into parental decision making about schooling.

# **_A.  Length of Time in School_**

The human capital model suggests that parents will choose length of time in school in order to maximize a child’s lifetime income. We modify the model slightly by assuming that parents choose schooling so as to maximize lifetime discounted utility from their children’s schooling and earnings. Parental utility is assumed to be positively influenced by their children’s schooling through two potential avenues: their children’s enhanced earnings potential; and pride in their children’s academic accomplishments. Both types of returns are subject to diminishing marginal utility.

# _<u>Case A1: Only child cognitive attainment raises parental utility</u>_

If parents only care about whether their children are learning in school and not on their years of schooling , then lifetime discounted utility at time t will depend solely on the human capital acquired, _qit_ . This would characterize parental utility if, for example, parents only valued the child's earnings potential and if firms only paid for cognitive attainment and not for years of schooling. In fact, Glewwe (2002) and Hanushek and Kimko (2000) present evidence that it is only cognitive attainment and not years of schooling that raise earnings. The parents' net discounted utility of the _t_<sup>_th_</sup> year of schooling can be written as where _VSt_ is the present value of an additional year of schooling at time _t_ , _Wit_ is the market wage earned by a child of age _t_ and knowledge _qit_ , and _Cit_ is the cost of schooling incurred in year _t_ . As
6 shown in Figure 1, parents interested in schooling only for its effect on cognitive attainment will _t_ 7 _t_ * * * continue to invest as long as _VS_ (.) > 0. Optimal _t*_ occurs where _VS_ ( _qit_ , _Wit_ , _Cit_ ) = 0.

_t_ We assume that _dVS_ / _dt_ < 0. The assumption implies that an optimum length of time in school exists, given a finite life of length T. There are several reasons for this assumption. First is the traditional assumption of diminishing marginal utility. Additionally, as _t_ increases by one year, the length of time the child can earn returns on schooling decreases by one year. Third, as years of schooling increase, a child’s human capital, _Wit_ rises. Because _Wit_ is the opportunity cost of schooling in year _t_ , the opportunity cost of schooling increases in _t_ . Finally, the direct cost of schooling, _Cit_ , may also rise as _t_ rises.

The marginal utility of the _t_<sup>_th_</sup> year of schooling implied by (1) presumes that parents and/or employers can observe _qit_ . In practice, _qit_ will be observed with error and skills are proxied by the number of years completed by the student, which in turn results directly from a series of promotion outcomes from grade to grade. It is possible that schools provide perfect information on student achievement through grade promotion decisions and that employers (and parents) are thus able to assess accurately how much students have learned in school. On the other hand, the information content of grade promotion decisions depends on several factors – whether promotions are, in fact, based on merit, as measured by student effort, achievement, or both; whether they are influenced by pressure from aggressive and powerful parents; and whether they are largely in accordance with a social or automatic promotion policy. That suggests that years of schooling provide only partial information on learning such that employers – as well as parents – will have to rely on their own means to assess student performance.

_<u>Case A2: Both promotion and child cognitive attainment raise parental utility</u>_
7 If the labor market rewards the number of years of schooling attained or if parents get utility from their children's promotion independent of their cognitive attainment, then promotions will also enter the parents discounted utility from the _t_<sup>_th_</sup> year of schooling: where _Pit_ indicates if child _i_ is promoted in year _t_ . We can then compare years of schooling with and without promotions. Parents interested in promotions as well as learning will choose length of time _t_ in school _t**_ such that _Vp_ ( _qit_ **, _Wit_ **, _Cit_ **, _Pit_ **) = 0 . As illustrated in Figure 1, the optimal length of time in school could rise or fall relative to the case where _Pit_ has no value. In other words, parents could raise or lower the amount of time a child is enrolled in school if the number of grades completed is valued, independent of _qit_ .<sup>8</sup> The rationale is that the opportunity cost of schooling rises more rapidly when promotions (independent of _qit_ ) are valued, even as the marginal returns to a year of schooling are increased.

The key conclusion from this section is that if automatic promotions are to have a positive effect on a child’s length of time in school, then parents must derive some utility from promotion, independent of the utility derived from actual learning. That will mean that the probability of continuing in school will differ between promoted and nonpromoted students, holding _qit_ constant. However, we cannot predict whether promoted students will be more or less likely to continue in school.
8

# **_B.  Promotions_**

We require a mechanism to distinguish between a promotion decision that is purely meritbased and a promotion decision based on nonmerit factors. We first characterize the two polar cases for teacher promotion decisions, one based exclusively on student performance and the other based on other factors, and then we describe how actual promotions are a mixture of the two.

# _<u>Case B1. Merit-based promotion</u>_

At one extreme, suppose that student _i_ ’s merit in year _t_ and school _j_ is given by _qijt_ . Merit is determined by a human capital production process where _Aijt_ is student attendance during the year, and _Mijt_ and _Lijt_ are math and language skills at the end of the year. Conditional on _Aijt_ , _Mijt_ , and _Lijt_ , the teacher derives an estimate of merit, _qijte_ which _j_ differs from actual merit, _qijt_ , by a random error, ε _ijt_ . There is also a threshold level of merit, _q_ min necessary to justify promotion to the next grade in school _j_ .<sup>9</sup> Hence, the promotion decision can be written as, _Pijt_ is a discrete variable that takes the value of one if the student is promoted and zero if the student is failed.

The promotion decision (4) can be estimated by rearranging terms such that If ε _ijt_ is distributed normally with zero mean, then (4’) can be estimated as a probit equation with the elements of _qijt_ from (4) and _q_ min _j_ as regressors. If schools hold strictly to a merit-based promotions
9 _j_ policy, observed promotions will fail to sort students correctly into qualified ( _qijt_ ≥ _q_ min ) and _j_ unqualified ( _qijt_ < _q_ min ) groups only due to random errors in measuring merit, ε _ijt_ .<sup>10</sup>

# _<u>Case B2. Non-merit based promotion</u>_

Promotion policy may be based on factors other than merit, as would be the case of a strict social promotion policy. Teachers may also base promotions on parental status in the community or a perceived need for teachers to curry favor with parents. The likelihood that non-merit factors will enter the promotion decision increases, the more that parents value promotion itself, independent of their child's cognitive attainment. In the extreme case, only non-merit factors, _Z_ , affect promotion. We characterize the aggregation of the non-merit factors as The promotion decision based on index I can be written as, which can be rearranged to be

# _<u>Case B3. Hybrid promotions</u>_

Promotion policies are unlikely to follow either of the two polar cases of merit-based and non-merit based promotions, suggesting that the actual promotion policy can be characterized as a convex combination of the two polar cases. Using γ as the weight placed on merit factors, the promotion policy becomes
10 (7) This hybrid promotion decision returns the strict merit-based policy if γ = 1, and the strict non-merit policy if γ = 0. This specification adds another influence on promotion decisions, γ , which might depend on the overall mission of the school, its general policy or practice with respect to promotions and repetitions, or the pressure it receives from parents to promote their children. If there are no reliable nonschool-based standardized tests of student achievement, then teachers might have more leeway to discount merit-based factors and set γ = 0. In fact, absent external validation of student performance, teachers may have an incentive to fool illiterate parents into thinking the students are learning more than they really are.

Two observations must be made about (7). First, it is difficult to determine from data whether non-merit based promotions are due to noisy student evaluations or a low γ _._ A low γ automatically diminishes the value of student performance. Secondly, a school that values merit in general must improve its method of evaluating students if it is to appropriately reward student performance. Even if γ were high, if student assessments are wildly inaccurate, the impression that will be transmitted to parents is that merit is not valued.

In the estimation we derive a statistical decomposition of promotions into merit- versus nonmerit-based components. While we are not able to identify _γ_ structurally, the reduced-form representation does enable us to determine whether unearned grade promotions encourage parents to keep their children in school. Next, we turn to parental choices regarding their child's schooling.

# **_C. Performance, Promotion and Persistence_**
11 We considered two alternative views of child persistence in school: parental decisions can be driven by the child's performance in school as in equation (1), or by both performance and promotion as in equation (2). Note that promotion will be positively correlated with child performance as long as γ > 0 in equation (7), even if promotion is not solely merit-based. Consequently, distinguishing between equations (1) and (2) requires that the incidence of promotion be decomposed into a component correlated with merit and a component that is uncorrelated with merit. This can be accomplished by predicting promotion on the basis of equations (3) and (4’), the strict merit-based model. Then, student performance will be measured by the merit-based component of the promotion, _E_ ( _Pijt_ | _Aijt_ , _M ijt_ , _Lijt_ ) . The difference between actual promotion and the predicted merit-based

> promotion, [ _Pijt_ − _E_ ( _Pijt_ | _Aijt_ , _M ijt_ , _Lijt_ )] , can then be used as the incidence of promotion that is

uncorrelated with _qijt_ . The parents' decision to keep the child in school approximates equation (2) by where δ is a set of parameters to be estimated, υ _tp_ is the observable component of parental marginal utility from an additional year of child schooling and ξ _ijt_ is a random error. Child persistence in school is governed by where _Eijt+1_ =1 if the child is enrolled in school the following year.

# **3.  Data Sources**

The empirical analysis uses data from a series of survey questionnaires designed by the authors and fielded by NEMIS staff under the supervision of Rafiq Jaffer. The survey covered a
12 representative sample of 257 government, mosque and private schools that were first surveyed by Ali and Reed (1994) as part of a textbook study. In each school, we selected one teacher in each of the first three grades; this selection was random if there was more than one teacher in a grade. We collected data on teachers and children in _kachi_ or kindergarten, _pakki_ or grade one, and grade two, but our analysis focuses only on the last two grades as the _kachi_ were too young to take the tests. The survey elicited information on teachers’ socioeconomic backgrounds. Similarly, in each class we randomly selected two students for inclusion in the household survey. Enumerators visited those households to obtain information on the sample child and on the socioeconomic attributes of the child's family. Variable definitions and sample statistics are reported in the Appendix.<sup>11</sup> During the course of the school year, the enumerators conducted two unannounced spot checks on teacher and student absenteeism. The first check occurred in the first two months of the term, and the second occurred in the final two months. Data on monthly student and teacher attendance over the school year were also obtained from the school’s attendance register. The average teacher spot-check absence rate was 19% which is comparable to the absence rates found by Chaudhury et al. (2006) for six developing countries using similar methods to those employed in our study. Students had a lower average spot-check absenteeism rate of 13%.<sup>12</sup> Grade-one level exams in mathematics and languages were administered to the secondgraders at the start of the academic year, and to first-graders at the end of the term. The tests were designed by the late Sar Khan. The exam was based on the official curriculum which all schools, public or private, were expected to follow. The curriculum sets minimum objectives for each grade level, and the exam assessed student attainment of these minimum competencies for grade one or _pakki_ . The language test was in the language of instruction to avoid giving undue advantage to any one language group.<sup>13</sup> Out of a possible score of 20, average scores for the math and language tests were 14.3 and 11.3, respectively.
13 Lastly, enumerators returned to the schools at the end of the school year and at the beginning of the next to collect information on which students had been promoted and which students were continuing in school the following year. If a student was not enrolled again in the same school, enumerators obtained information on whether that student had transferred to another school or had dropped out. Enumerators were able to find the enrollment status of all but 6 of the original sample children; we drop those observations from the analysis of child persistence in school.

Of the sample of 736 children for whom we have full information, 94.4% were promoted and 95% remained in school the following year (Table 1). Of those not promoted, 18% dropped out of school. Of those promoted, only 4% dropped out of school. Clearly, promotion and persistence are positively correlated, but we require further investigation to assess whether the correlation is due to learning or to the state of being promoted.

Because we are interested in estimating the relationship between achievement and promotion, our working sample does not include those children who did not take the mathematics and language exams that were administered in class. When these children are added to the sample, the fraction not promoted rose from 5.6% to 9.2% and the fraction not continuing in school the following year rose from 4.8% to 7.9%. Therefore, the children in our working sample are more likely to be promoted and to continue in school than the universe of students. These differences could alter our conclusions regarding the relative importance of merit-based versus social promotions on the likelihood of continuing in school, a possibility we investigate directly.
14

# **4.  Empirical Analysis and Results**

First, we analyze the determinants of student promotion. We then use those findings to embed predicted promotions into our analysis of student continuation.

# **_A.  Student Promotion_**

We estimate two versions of the promotion equation, the pure merit-based promotion specification (4’) and the hybrid specification (7). Factors reflecting student merit include the test scores for mathematics and language achievement. These tests were designed to establish the extent to which students have learned material required by the nationally approved curriculum. Our other measure of student performance is the student’s official attendance record, on the presumption that daily school attendance is positively correlated with learning. As mentioned above, spot-checks of student attendance confirmed the reliability of the official attendance record.

A school’s required performance for promotion, _q_ min _j_ , is assumed to vary with the school’s average score on the math and language achievement tests. This would imply that it would be harder for a student to pass in a school in which students perform better on average. Since the performance level required for promotion should rise with grade level, a dummy variable for grade two is also added. Anything that raises the performance standard should lower the probability of promotion.

Measures of _Z_ include mother’s and father’s highest grade attained, whether there is only a single parent in the household, and household income. Past studies in various countries have found that grade retention is higher for children who come from poorer homes and schools (Bonvin 2003; Eide and Showalter 2001; Gomes-Neto and Hanushek 1994; Hauser, Pager and Simmons 2004; Mete 2004; Patrinos and Psacharopoulos 1996), a reason why appropriate identification has been a methodological issue for the studies that estimate the impact of grade retention on student performance.

Parents with high education or income would be expected to have more power or ability to influence teachers. Variables that might reflect parental incentives to do so include the number of
15 younger siblings a child has (since parents may have a particular interest in the success of their firstborn child), whether the child is male (since parents may have a stronger desire for their sons to succeed), and whether the child is healthy (since education may be more valuable if the child is expected to be able to reap the benefits of schooling for a longer period of time). Finally, two measures of how well the teacher can assess student performance are used – the teacher’s own attendance in class and class size.<sup>14</sup> The more frequently the teacher is absent and the larger the class, the less the teacher is able to know a given child’s ability, and potentially the lower the weight that the teacher can place on merit, _γ_ .

Results are reported in Table 2. The estimated effects of the variables on promotion probability are reported in Table 3. The first important result is that the pure merit-based promotion specification is rejected in favor of the hybrid promotion model. The test of the null hypothesis that the coefficients on the non-merit elements of the regression are jointly zero is soundly rejected.

That said, the promotion model works well. Superior performance on either the math or language test significantly increases promotion probability, even though there is no systematic testing system in NWFP. In fact, the teachers did not have the results of these exams at the time they made their decisions on whom to promote. To illustrate the magnitude of the impacts of these test scores, we estimate the predicted probability of promotion at the highest and lowest realization of each test score, holding all other regressors at their sample means. As shown in Table 3, the score on the math exam raises the probability of promotion by 7 percentage points as we go from lowest to highest math score. The range of promotion probabilities attributable to the variation in language test scores is wider at 10 percentage points.

Promotions are even more strongly tied to student attendance. The predicted promotion probability ranges from 21 to 99 percent as the proportion of school days attended varies from zero to one. Promotion probability is not strongly tied to a child's performance relative to other children
16 in the school. Average test scores have a small and statistically insignificant effect on promotion probability. This suggests that promotion standards may not differ significantly across schools. Three factors stand out in the vector of non-merit influences on student promotion. First, children with younger siblings are more likely to pass the grade. This is consistent with the presumption that there will be pressure to pass the older of several children in a family. Schools may promote older children so as not to discourage the enrollment of younger children in the family. Nevertheless the effect on promotions is very small: students with the largest number of younger siblings are only 4 percentage points more likely to pass than students with no younger siblings.

Second, student promotion is influenced by teacher attendance. As teacher attendance increases, all else equal, the probability of promotion decreases. Students should perform better when the teacher is present, but these regressions control for student test scores. One plausible explanation for this is that frequently absent teachers have little merit-based information upon which to base promotion decisions and thus cannot defend a decision to hold back a child for another year.<sup>15</sup> However, student promotion varies only modestly with teacher attendance: less than a 1 percentage point difference between classrooms with absent versus present teachers.

Finally, children from wealthier households are more likely to be promoted. It may be that wealthier parents pressure teachers to promote their children, or it may be that wealthier families invest more in their children's schooling outside of school. In any event, again the range of promotion results associated with household wealth is small--less than 5 percentage points separating the promotions of the poorest and wealthiest children.

We conclude that our nonmerit influences on promotions are much less important in explaining the likelihood of promotion than were test scores or child attendance, and so teachers in these NWFP schools based their promotions more on merit than nonmerit factors. Although we cannot isolate a numerical estimate of _γ_ , the implication is that the weight placed on merit promotions is high and so _γ_ is close to 1.
17 Because of gender differences in schooling in Pakistan, we also analyzed promotion separately for boys and girls. We could not reject the null hypothesis that the process governing promotions was the same for boys and girls. The coefficients are very consistent in sign and significance across the boys' and girls' equations. For both, it is test scores and attendance that drive the promotion process with nonmerit factors playing only minor roles in numerical impact, even when they have significant coefficients.

# **_B.  Student Continuation_**

Next, we attempt to identify the extent to which the occurrence of a promotion affects the probability that a student persists in school. We estimate two specifications of the probability that a student who is currently enrolled will remain in school or drop out in the subsequent year. The first includes the dummy variable, _Pijt_ , which indicates whether or not the child was promoted. The second decomposes the observed promotion decision into _ˆPijtm_ , the component predicted by the merit-

> based specification (column 4 in Table 2), and _Pijt_ − _Pˆijtm_ , the component of the promotion that is not explained by merit.  Because the estimations using the decomposed promotions require a two-step process, we correct the standard errors using a bootstrapping procedure.16

The results are reported in Table 4. The first striking result is that only promotion matters; the probability of remaining in school is not affected by household, school or child attributes. The null hypothesis that the coefficients other than promotion are jointly equal to zero could not be rejected at standard confidence levels.<sup>17</sup> The lack of importance of these child, household, and school attributes is less surprising in that the sample is preconditioned on the child being in school the previous year. Variation in household attributes may have an impact on the likelihood that a child ever attends school, but our sample only includes children whose household and school attributes were sufficient to induce them to attend. Nevertheless, even with this selected and thus relatively
18 homogeneous group of households, promotion strongly influences the decision to send the child to school the following year.

The high correlation between promotions and continuation in school would seem to support an automatic or social promotion policy. However, a promotion may signal only that the child is prepared for more advanced schooling, leading parents to continue to invest in their child's time in school. In that case, the promotion itself would have no impact on the child's persistence in school. In column 2, we replace _Pijt_ with our decomposition into merit and non-merit based promotions. It is the merit component of the promotion that drives the continuation effect. While both components have statistically significant effects, the estimated effect is much larger for the merit than for the nonmerit component of the promotion. When we compute the implied probability of continuing in school at the highest and lowest values of _ˆPijtm_ and _Pijt_ − _Pˆijtm_ toward the bottom of Table 4, we find that the child with the lowest merit component is 25 percentage points less likely to enroll in school the following year than is the child with the highest merit component. The range of enrollment probabilities from the bottom to the top of the non-merit based promotion is only 11 percentage points.

One might be concerned that promotion is endogenous, perhaps because teachers are more likely to promote children who have precommitted to staying in school the following year. Our instruments for promotion mitigate against that problem. The two tests were administered by the enumerators and the teachers never saw the test results, so the tests are outside the teacher's influence. Student attendance registers were validated by spot checks of student attendance, and so they also appear to be outside the teacher's control. Consequently, our merit-based promotion measure will not be clouded by possible teacher motives to promote children who intend to return to school.
19 On the other hand, the non-merit promotion component would be clouded by such incentives if teachers had those incentives. If this non-merit component were endogenous and correlated with the other regressors in the model, its inclusion would bias the other coefficients. In our robustness checks, we found that the coefficients were not greatly affected by the inclusion or exclusion of nonmerit based promotion. Consequently, endogeneity does not appear to be a serious problem with the interpretation of the coefficient on either promotion component.

We can further subdivide the non-merit component into unexpected failures (negative surprises on the promotion decision) and unexpected promotions. Presumably it is the latter that most closely fits the social promotion concept. We find that most of the impact of the non-merit component is attributable to the negative effect on continuation of failing a student who should have passed based on test scores or attendance. Promoting an undeserving student has only a very modest positive effect on the likelihood that the child will continue in school.

Dividing the sample into boys and girls, we find that only merit-based promotion retains statistical significance and the size of that effect is much larger than the impact of non-merit based promotion on student persistence. Nevertheless, when we test the null hypothesis of equality of coefficients across the boys' and girls' equations, we cannot reject equality at standard significance levels, and so we consider the estimates in the first two columns as more precise.

As pointed out in section 3, our working sample includes only the children who were in school during the survey visit and were able to take the math and language tests. Those who did not take the tests include a higher fraction of students who were not subsequently promoted and who did not continue in school, and so our conclusions may be affected by sample selection. To address this issue, we replicated the entire analysis without the test scores and used as a proxy indicator of merit the student’s annual attendance rate. The marginal effect of the unpredicted promotion rises from 0.055 to 0.13, suggesting a higher effect of the surprise element of promotion on student continuation. However, the impact of the merit-based component also increased by a similar
20 proportion from 0.14 to 0.31, so our conclusion that student continuation is driven largely by meritbased promotion continues to hold.

# **4.  Conclusions**

Previous studies have estimated the cost burden that grade retention imposes on school systems in developing countries. For example, Schiefelbein and Wolff (1992) estimate that ten million repeaters each year in ten South American countries cost these countries about one billion dollars annually. Gomes-Neto and Hanushek (1994) estimate that the average cost of grade retention as a strategy for raising test scores by one point was about one-fourth to one-third of the average student cost in Brazil’s rural northeast region. From this perspective, grade retention is very costly for countries with grade retention rates at or above 20 percent (e.g. Brazil, Nepal, and Cameroon). But is automatic or social promotion an appropriate and effective remedy for this problem? Increasing the pass-through rates of students in a mechanical way through automatic promotion is not likely to be the right remedy in the difficult situations under which students in low-income settings must study – impoverished home conditions, few educated role models in the home or community, poorly equipped schools, poorly trained teachers.

There is strong evidence that parents make and implement decisions about their children’s schooling even at the early grades of the primary cycle. Primary schooling is now compulsory in nearly all countries, yet not all children in the relevant age group enroll in school. This suggests that in their parents’ eyes the returns to schooling do not outweigh the (direct and opportunity) costs of going to school. In addition, parents seem willing to bypass a low-quality public school in favor of a better-quality but more remote public school (Gertler and Glewwe 1990) or a better-quality private school that charges higher fees (Andrabi et al. 2007). For these reasons, a policy of automatic promotion that assumes away parental decisionmaking would not prevent children from dropping out early if they are not learning what they are expected to learn.
21 Our results indicate that grade promotion raises the probability of a student continuing in school when that promotion is based on student performance, not when promotion is uncorrelated with student achievement. Moreover, contrary to some arguments, illiterate and poorly educated parents appear able to discern their children's academic progress independently of information on test scores and to use this information in deciding whether to keep their children in school for another year. The implication for policymakers is that it would be difficult to prevent early dropouts and raise the numbers of children who complete schooling without improvements in the quality of the basic education system—or without effective supplementary or remedial programs to address the needs of low-performing students.<sup>18</sup> This is an immense challenge in the world’s poorest countries where dramatic expansions in school enrollment have come at the price of quality (World Bank 2007).
22

# **Endnotes**

1 Both reviews (Holmes 1989; Jimerson 2001) raise concerns about the soundness of methodology. Many of the earlier studies Holmes (1989) reviewed do not appropriately control for differences in the characteristics of students. Only 25 of the 63 studies used matched comparison designs; only 16 studies matched students on prior achievement, and only 8 studies matched students on characteristics that are generally found to predict grade repetition. In both reviews, however, those that followed matched retained and promoted students apparently found larger negative effects.

2 The study tracked a sample of schools during the period 1981-85; of 2,619 students in the second grade in 1981, 127 students were still in the second grade in 1985 (Gomes-Neto and Hanushek 1994).

3 Jacob and Lefgren (2004) find that retained third-graders in Chicago had higher test scores one year after retention but, two years after retention, these gains had declined substantially.

4 Eide and Showalter (2001) evaluated parental decisions regarding grade retention in their study of the returns to grade retention for high school students in the United States. They model the parental decision as depending on whether the child’s earnings net of schooling costs are higher or lower from grade retention. Controlling for endogeneity of grade retention, white students do get a benefit from grade retention in lower dropout rates and higher labor market earnings.

5 These concerns, among others, are reviewed by Ahmad et al (2005) and Bardhan (2002, 2005).

6 See, for example, Behrman and Deolalikar (1991); Alderman, Orazem and Paterno (2001); Anderson, King and Wang (2003) for studies on developing countries.

7 A similar model with similar implications can be expressed in terms of discounted lifetime income.

8 Grade repetition, however, can dilute marginal returns to each year spent in school. Behrman and Deolalikar (1991) examined the effect of high repetition rates on returns to primary education in Indonesia and found those rates to be extremely overestimated. Under alternative estimates of repetition rates, these rates are overstated by 82-114 percent for the below-completed-primary level and by 38-78 percent for the completed primary category.

9 Bedi and Marshall (2002) model day-to-day school attendance as an outcome of parental schooling decisions, and like parental decisions about enrollment, they find it is affected by the expected benefits as measured by an associated increase in test scores and by the quality of school facilities.

10 In their study on Brazil, Gomes-Neto and Hanushek (1994) found that promotion rates were directly related to students’ test scores. Each 10 points on the Portuguese achievement test (with a standard deviation of approximately _25_ points) reduced the repetition probabilities by 1 percent, while the effect of the mathematics achievement test is half of this. Since the mean observed repetition rate in the sample was only 4 percent in 1983, these merit effects are significant.

11 Our choice of which school and household measures to use was related to the frequency of response. For example, distance from the home to the school was not answered by 25 households. Our results are not affected by the inclusion or exclusion of the distance to school (the average is under 1 kilometer, and so this is not surprising), and so we only report the results using the larger sample where school distance is excluded.
23 12 Das et al. (2007) also found that in Zambia, spot-check absenteeism rates were lower for students than teachers. In Zambia, the higher teacher absenteeism is due to AIDS illnesses and deaths. In NWFP, the absenteeism is apparently a result of an implicit contract in which present teachers take on the burden of their absent colleagues which allows teachers to be absent more frequently in multiple teacher schools than in single teacher schools.

> 13 Although Pashto is the main language in the NWFP, the province has many other languages spoken, including Hindko, Punjabi, Persian and Urdu.

14 We use the spot check observations on teacher attendance rather than the official teacher attendance. The official attendance was nearly 95 percent, but the spot check attendance averaged only 80 percent.

15 Das et al. (2007) find that a 5-percent increase in teacher’s absence rate in Zambia reduces learning by 4-8 percent of average gains over the school year. They explain this reduction in learning not only being due to a reduction in contact time but also due to a decrease in preparation time for class by teachers.

16 Bootstrapped standard errors were nearly identical to the original standard errors. Estimation using a linear probability model yielded similar results.

> 17 The _Χ2_ (10) statistic was 13.6 which has a marginal significance level of 0.14.  The corresponding test statistics for column 2 in Table 4 was _Χ_<sup>_2_</sup> (10) = 12.6 with a marginal significance level of 0.18.

> 18 Two studies which measure the impact of supplementary programs indicate that remedial programs can help at-risk students.  Jacob and Lefgren (2004) evaluate the combined impact of Chicago’s testing-based grade retention policy and a linked summer remedial program.  They find a modest positive gain in test scores for low-achieving students, although only for younger (third grade) students, not for older (sixth grade) students.  In India, a small experimental project provided in-school support to struggling third- and fourth-graders through a young woman (“balsakhi”) from the community who worked on basic literacy and numeracy skills with the students for two hours out of a four-hour school day (Banerjee et al. 2007).  The program evaluation finds that this remedial class increased average test scores by 0.28 of a standard deviation for all children in treatment schools, mostly due to large gains at the bottom of the testscore distribution.  This positive gain vanished a year after the program ended but the evaluation suggests that extra classroom help for low-performing students has the potential to address learning deficits.