metadata: authors: 0: author: Sharma,Dhiraj
count: Nepal
docna: 0: docna: How liberal is Nepal's liberal grade
            promotion policy ?
docty: Policy Research Working Paper
subtopic: Primary Education,Secondary Education,Educational Sciences,Educational Institutions & Facilities,Educational Populations,Education for Development (superceded),Effective Schools and Teachers,Education For All
teratopic: Education
abstracts: cdata!: This paper estimates the magnitude of
            liberal grade promotion in public schools in Nepal by
            comparing the pass rate in internally administered exams
            with the pass rate in the district-exam whose scores
            determine grade transition. The pass rate in the year-end
            exam is three and a half times as high as the pass rate in
            the internal exams. The difference is not explained by an
            increase in student effort and it is not due to the
            financial incentives for which students in a random subset
            of schools were eligible. The paper concludes with a
            discussion of the policy implications of the findings on the
            student assessment system in Nepal.
geo_reg_key: 517191,517212,517198
repnme: repnme: How liberal is Nepal's liberal grade
            promotion policy ?
display_title: How liberal is Nepal's liberal
            grade promotion policy
listing_relative_url: /research/2016/11/26968471/liberal-nepals-liberal-grade-promotion-policy
topicv3: Teachers Management m1326119 1523,Primary Education m1326139 480,Education m1327048 470,Secondary Education m1327058 481,Learning Assessment m1327283 1555
publishtoextweb_dt: 2016-11-29T00:00:00Z
historic_topic: Education
no_of_pages: 22
author_keys: 0: author_key: 1206675
model_extractions: WPS7896 Policy Research Working Paper 7896 How Liberal Is Nepal’s Liberal Grade Promotion Policy?

_Dhiraj Sharma_ Poverty and Equity Global Practice Group November 2016
Policy Research Working Paper 7896

# **Abstract**

This paper estimates the magnitude of liberal grade promotion in public schools in Nepal by comparing the pass rate in internally administered exams with the pass rate in the district-exam whose scores determine grade transition. The pass rate in the year-end exam is three and a half times as high as the pass rate in the internal exams.

The difference is not explained by an increase in student effort and it is not due to the financial incentives for which students in a random subset of schools were eligible. The paper concludes with a discussion of the policy implications of the findings on the student assessment system in Nepal.

This paper is a product of the Poverty and Equity Global Practice Group. It is part of a larger effort by the World Bank to provide open access to its research and make a contribution to development policy discussions around the world. Policy Research Working Papers are also posted on the Web at http://econ.worldbank.org. The author may be contacted at dsharma5@worldbank.org.

_The Policy Research Working Paper Series disseminates the findings of work in progress to encourage the exchange of ideas about development issues. An objective of the series is to get the findings out quickly, even if the presentations are less than fully polished. The papers carry the names of the authors and should be cited accordingly. The findings, interpretations, and conclusions expressed in this paper are entirely those of the authors. They do not necessarily represent the views of the International Bank for Reconstruction and Development/World Bank and its affiliated organizations, or those of the Executive Directors of the World Bank or the governments they represent._ Produced by the Research Support Team
# **How Liberal Is Nepal’s Liberal Grade Promotion Policy?**<sup>**1**</sup>

Dhiraj Sharma JEL Codes: I20, I21, I28 Keywords: Liberal promotion, Nepal, Student assessment

> 1 I would like to thank Brian Roe for his support and guidance and Yubraj Acharya, Mohal Prasad Aryal, Saurav Dev Bhatta, Dilip Parajuli, and Dhushyanth Raju for valuable comments and suggestions. All errors, omissions, and shortcomings are my own.
**I. Introduction**

To reduce the wastage of resources caused by school repetition and dropout, some educators and policy makers have argued for liberal grade promotion policy in developing countries. Liberal promotion, also known as social promotion, is the practice of promoting students to the higher grade regardless of academic performance.

At the heart of the debate on the pros and cons of lenient promotion policy is the tradeoff between economic efficiency and social well-being of the child on the one hand and student learning and incentives to learn on the other. Grade repetition imposes fiscal burden on the public education system as more resources have to be devoted on the same child to complete a grade level. From the psychological perspective, separating a child from his or her peers may have harmful effects on his or her self-esteem, socialization, and self-expectation. From this point of view, grade retention may be counterproductive, leading to higher dropout rate and lower student achievement (Jimerson, 2001).

Others maintain that it is necessary to hold students back until they master the content before advancing to the higher grade. Automatic promotion may create a gap between student learning level and the syllabus, a gap that only grows over time in absence of remedial education (Pritchett & Beatty, 2015). Progressing through school without learning may only postpone the inevitable, setting students up for failure in school leaving examinations and beyond. Rather than being a burden on students, the “threat” of grade retention is a necessary motivator; without the implicit threat of the penalty, students do not have the incentive to put in the requisite effort to succeed in school.

The empirical evidence on the relative impact of grade retention and automatic promotion on subsequent school outcomes is mixed. Some studies find positive effect of grade retention on student achievement and the probability of academic success (Gomes-Neto & Hanushek, 1994; Eisemon, 1993; Allensworth, 2004). Other studies, however, show negative impact of grade retention on school dropout rate and educational attainment (Manacorda, 2012; Grissom & Shepard, 1989; Roderick & Nagaoka, 2005). Meta-analyses of grade retention research also suggest that the practice has either negative or no effect on student performance and emotional and behavioral measures (Jimerson, 2001; Holmes, 1989).

A dimension that is missing from the discussion of liberal promotion policy is the magnitude of the practice: What proportion of students who advance to the next level of the education system 2
do so due to liberal promotion policy?<sup>2</sup> This study provides an empirical estimate of the scale of liberal promotion in public schools in Nepal. In order to estimate the size of liberal promotion, I compare the pass rate in internal school exams with the pass rate in the year-end exam. The crucial difference between the two exams is that the scores of the term exams are not reported to the education authorities and grade transition does not depend on these scores. On the other hand, the scores of the end-of-the-year exams are reported to the District Education Office and they determine grade transition. I estimate the scale of liberal promotion as the difference in the pass rate between the term exams and the final exam.

To preview the main findings, the proportion of students passing all five core subjects in the year-end exam is three and a half times higher than that in the trimester exams. The substantially and significantly higher pass rate in the final exams is not explained by increase in student inputs like school attendance and academic reinforcement at home, and it cannot be attributed to the financial incentives that students in a randomly selected subset of schools were eligible for.

The rest of the paper is organized as following. The next section provides an overview of the institutional context of the education sector in Nepal. Section III describes the data and its source and section IV reports the empirical results. Section V explores possible reasons other than lenient grade promotion policy for the observed difference in pass rates. Finally, section V discusses the policy implications of the results and concludes.

# II. **Institutional background**

Until recently, the education system in Nepal was divided into six levels: early childhood development and pre-primary, primary (grades 1 – 5), lower secondary (grade 6 – 8), secondary (grades 9 and 10), higher secondary (11 and 12), and university education. A new education cycle comprising early childhood development, basic education (grades 1 – 8) and secondary education (grades 9 – 12) was initiated under the School Sector Reform Program (SSRP) in 2009.

> The Ministry of Education (MoE) and the Department of Education (DoE) are responsible for the delivery of education in Nepal.  District Education Offices (DEOs) supervise and monitor 2 Eisemon (1997) reports repetition rate for different countries and regions around the world but does not report the magnitude of liberal promotion.  He observes that the culture of repetition or liberal promotion often has its root in the colonial history of the country.  Repetition rate is higher in Francophone and Lusophone countries relative to Anglophone countries in both Africa and Latin America.

3
schools at the district level and act as conduit of funds and grants from the center. The funds flow from the Ministry of Finance (MoF) to schools through the MoE, DoE, and DEOs. Each school has an elected School Management Committee (SMC) that manages day-to-day operation and makes budgetary allocations.<sup>3</sup> Teacher salaries, textbooks, scholarships, and construction expenditure constitute more than 90 percent of the budget on basic education (grades 1 – 8) (Thapa, 2011; The World Bank, 2014).

Education access has expanded and outcomes have improved significantly in the past decades in Nepal. Between 1999 and 2015, the primary net enrollment rate increased from 66 percent to 97 percent and the proportion of out-of-school children of primary school age dropped from 34 percent to 3 percent. Over the same period, the primary completion rate went from 64 percent to 105 percent (World Development Indicators; The World Bank, 2014)<sup>4</sup> . Gender parity has been attained up to the secondary level and disparities in access to schooling across geographical areas, income groups, and ethnicity and caste groups have also been eliminated or substantially reduced compared to a few decades ago.

Despite the progress in schooling access and outcomes, however, quality of education remains low. A national assessment of grade 10 students in 2011 showed poor performance of students in the core subjects, with particularly low scores in Nepali (33 percent) and Mathematics (37 percent) (Educational and Developmental Service Centre, 2011).

Liberal promotion policy was introduced late 1990s in response to the high degree of inefficiency in the system. While the primary net enrollment ratio in 1999 was 66, the gross enrollment ratio was 117 due to high repetition rates. It was estimated that the system had to deliver 11.4 years of schooling to complete five years of primary education (The World Bank, 1999). To improve efficiency at the primary level, the Government of Nepal introduced the twin policies of continuous assessment and liberal promotion starting with the Basic and Primary Education Project (BPEP) in 1999. Under this policy, continuous assessment of student learning achievement would be conducted to provide regular feedback to both students and teachers on student learning levels. At the same time, to reduce school dropout and repetition, students would be granted automatic promotion to the

> 3 Institutional or private schools are privately funded and privately managed.  Community schools, or public schools, are managed by the local community, but the government controls teacher recruitment and management in these schools and they also receive government funding and support.  Public schools are known as community schools after the Education Act 2001 handed over the management of publicly-funded schools to local communities.

> 4 Accessed: March 3, 2016.

4
higher grade. Those who were unable to attain the required learning levels would be provided with opportunities to achieve them in the higher grade (The World Bank, 1999).

# III. **Data**

This study utilizes data collected in the context of a randomized controlled trial of the impact of financial incentives on student performance. The experimental study was implemented in public schools in Nepal to understand if financial incentives are effective in boosting student achievement. The intervention was applied on grade eight students in semi-urban areas of Kathmandu during the 2009 – 2010 school year lasting from April 2009 to March 2010. From a pool of 33 schools, a randomly selected subset of 11 schools comprised the treatment group while the remaining schools constituted the control group.

Grade eight curriculum consists of five mandatory subjects – Nepali, English, Math, Science, and Social Studies – and four peripheral courses. The exam for each core subject is worth 100 points and the pass mark is 32. The exams for the non-core subjects are worth 50 points each with pass mark of 16. Eighth graders take three exams every year: two trimester exams internal to the school and the year-end exam common at the district level. Students in the treatment group were eligible for cash rewards in direct proportion to their scores in each of the three exams. Incentives based on average aggregate score were offered for all three exams taken in the year. In this paper, I focus on the scores in the five core courses for the sake of comparability across all schools.<sup>5</sup> Two student surveys, one at the beginning of the academic year before the announcement of the incentives and one towards the end of the year, collected information on household characteristics and student time use at home. Because the survey was administered in school, the survey information is missing for students who were not present in school on the day of the survey. Since the official information on student attendance is not always reliable, I use the number of students who filled out the baseline and exit surveys as measure of school attendance.<sup>6</sup>

> 5 Schools have some leeway in selecting the elective courses.  Health and Physical Education, and Population and Environment are taught in all but one school, 15 schools teach Moral Education, and 25 teach Vocational Studies.  The results reported in this paper are robust to calculating average aggregate score using all nine subjects.

> 6 In most cases, appointment for surveys was made either the previous evening or the same morning so the visits are unannounced for all practical purposes.  Attendance rate and other information might be nonrandom in the few occasions when the request was not met and the visit was scheduled for a later date.

5
Summary statistics for the sample of schools and students are shown in Table 1. Studentteacher ratio in the sample schools (22:1) is similar to that in public schools in the region in general: in 2011-12, student-teacher ratio at the lower secondary level in public schools in Kathmandu valley was 23 (Department of Education, 2011). Students come predominantly from lower socioeconomic strata with only 14 percent of students’ parents having at least some secondary education. Girls significantly outnumber boys, accounting for 57 percent of the student body.

# IV. **Empirical strategy and results**

In the district where the study was conducted, the questions for the district level exam are set by the District Education Office but the exams are administered by schools and graded by teachers in the same school.<sup>7</sup> Unlike the internal trimester exams, scores from the final exam are reported to the District Education Office and grade transition depends on these scores.<sup>8</sup> My approach to measuring the extent of liberal promotion involves estimating the difference in pass rate between the trimester exams and the year-end exam.

Specifically, in order to test if the differences in pass rates between the trimester exams and the final exam are statistically significant, I regress the dummy variable for pass status on the dummy variables for second trimester and year-end exams, controlling for student fixed-effects as following: where ���� is a dummy variable indicating the pass-fail status of student _i_ in school _j_ in exam _t_ , �� and �� are dummy variables for second trimester and final exams respectively, and �� is a dummy variable for student _i_ . To allow for the outcomes to be correlated for students in the same school, the standard error is clustered at the school level. Average difference in scores or pass rate between first and second trimester exams (pair _P1P2_ ) and first trimester and final exams (pair _P1P3_ ) is captured by the coefficients �� and �� respectively, and the difference between second trimester and final exams (pair _P2P3_ ) is estimated by the difference �� � ��. This specification measures the difference in the probability of passing the second trimester and final exams relative to the first trimester exam when all studentspecific factors like intrinsic ability, family background, and school characteristics that do not change over time are controlled for.

> 7 This is just one of many arrangements; in some districts, answer sheets are swapped with other schools within the same district for grading.

> 8 Students must obtain at least the pass mark in all subjects to progress to grade nine.

6
Before turning to the regression results, I present the simple difference in average score and pass rate between the trimester and year-end exams. Columns III, IV, and V in Table 2 show the pass rates in the first and second trimester exams and the final exam respectively. The difference in pass rate between pairs of exams _P1P2_ , _P1P3_ , and _P2P3_ are listed in columns VI, VII, and VIII respectively. There is no upward trend in pass rate between the two internal exams: the difference is between -2 to 3 percentage points depending on the subject and never statistically different from zero at 5 percent level of significance. There is, however, 63 percentage point difference in the overall pass rate between exam pairs _P1P3_ and _P2P3_ , an increase of 250 percent, and the difference is highly significant ( _p_ -value = 0.000). The subject-specific pass rates are between 13 to 53 percentage points higher in the final exams depending on the pair of exams and subjects compared and all differences are statistically different from 0 at less than 1 percent level of significance.

The regression results are reported in Columns IX, X, and XI of Table 2. The point estimates mirror the simple difference results: the difference in pass rate between first and second trimesters is substantively small and never statistically significant. By contrast, the large and significant difference in pass rate between the trimester exams and the year-end exam persists after controlling for student fixed-effects. Thus time-invariant student characteristics are unable to account for the substantial improvement in scores in the final exam.

It is not known what the true distribution of scores would have been in absence of liberal promotion. For the sake of comparison, I take the distribution in the trimester exams to be the counterfactual. That is, I assume that in absence of liberal promotion, all students who failed in the trimester exams would have failed in the final exams and otherwise.<sup>9</sup> Under this assumption, only a quarter of students would have passed all five core subjects in the year-end exams, compared to the 89 percent that is observed. In other words, three and a half times as many students pass all subjects in the final exam compared to the trimester exams.

# **Alternative explanations**

The substantially and significantly higher pass rate in the final exams is suggestive of lenient promotion, but it is far from conclusive. While the data are not available to test and conclusively rule out all plausible alternative explanations, this section considers a couple of them.

> 9 Although, as seen in Figure 1, there is some spike in density at the cutoff score of 32 in trimester exams as well.  As such, the estimates are likely an underestimate of the true rate.

7
The difference in pass rates could be “genuine” if students, knowing that grade transition depends on the year-end exam, put in more effort for the district exams. To check for this possibility, I compare student inputs to educational production at the beginning and the end of the year, focusing on two key inputs: school attendance and academic reinforcement at home.

Difference in level of inputs is not able to explain the large increase in pass rate between the trimester and final exams. At the beginning of the year, before the announcement of the incentives, 52 percent of students received help with schoolwork at home, either from a household member or from a hired tutor. This figure stays level until the end of the year; 51 percent of students received help with schoolwork at home right before the final exams, a value that is not significantly different from the baseline level (regression not shown). Neither is there a significant difference in school attendance rate; attendance rate was 79 percent at the beginning of the year compared to 82 percent at the end of the year, a statistically and practically insignificant difference of 3 percentage points (regression not shown).

The results are robust to restricting the analysis to the subset of students who transitioned from failing one or more core subjects in the trimester exams to passing all subjects in the final exam. Of these students, 53 percent reported receiving help at home with schoolwork at the beginning of the year and 50 percent said the same at the end of the year, with the difference being statistically insignificant (regression not shown). Likewise, the attendance rate among these students was 79 percent and 82 percent at the time of baseline and endline surveys, a statistically insignificant difference of 3 percentage points (regression not shown). Large increase in average score and pass rate without corresponding increase in two key educational inputs, even among those who demonstrated significant improvement between the two exams, lends further credence to the claim that the improvement was due to lenient grade promotion.

In addition, one could argue that the financial incentives introduced in some schools might have induced behavioral change in students and teachers: students in treatment schools were motivated by the financial incentives to work harder for the final exams or that the final exams in only the treatment schools were graded leniently to award higher rewards to students. The results, however, are robust restricting the sample to control schools; the point estimates are just as high and statistically significant (Table 3). This evidence goes against the argument that financial reward could have caused behavioral change that led to the observed improvement in pass rate.

8
**Implementation of lenient promotion policy**

Next, I analyze the pattern of score distribution in the trimester exams and the year-end district exams to understand how liberal promotion is implemented in practice. There are two ways in which it could be executed. First, scores of all students could be raised by roughly the same magnitude, resulting in rightward shift of the distribution. Alternatively, extra points could be awarded to students only in the lower ends of the distribution, pushing inframarginal students over the passing threshold, but leaving the scores in the top parts of the distribution relatively unchanged.

A casual visual analysis suggests the latter mechanism to be at work. Figure 1 shows the distribution of test scores for the five core subjects in the two trimester exams and the year-end exam. There is significant bunching of scores at or just above the passing threshold and hardly any mass of the distribution lies to the left of the pass mark. To further explore the mechanism of score adjustment, I analyze the differential increase in test scores as a function of baseline academic performance. Table 4 shows the gains in test scores for students in four quartiles of score distribution.<sup>10</sup> Columns I, II, and III show average aggregate score in the first and second trimester exams and the district exam respectively for students in the bottom, second, third, and top quartiles. Simple difference in average aggregate scores between first and second trimester exams are practically small for all quartiles (Table 4, Column IV). In contrast, the difference in scores between the final exam and first and second trimester exams is large and statistically significant for the lower quartiles. Bottom quartile students score 11.8 percentage points higher in the year-end exam compared to the first trimester exam, an increase of 56 percent. The gains decrease monotonically as one moves from the bottom to the top quartile (Table 4, Columns VI and VII). Students in the top quartile do not score any higher in the year-end exam relative to the first trimester exams; the difference is 0.94 and statistically indistinguishable from 0. The null hypothesis that the gains are equal across quartiles can be rejected at conventional levels of significance ( _p_ -value = 0.000). The point estimates from student fixed-effects regression reported in Columns VII, VIII, and IX of are identical to the simple differences. These results demonstrate that scores were adjusted upwards relatively more for students in the bottom parts of the distribution, allowing them to move to grade nine.

> 10 Quartiles are defined within treatment and control groups using the first trimester scores.  Grade seven scores, the intended baseline measure, are missing for more than 20 percent of the students, either because students transferred from schools outside the sample or because they were not enrolled in school in the previous year.  To the extent that incentives increase average scores without causing rank reversals, these quartiles correspond to the quartiles created from “true” baseline test scores.

9
V. **Discussion and conclusion** This paper presents empirical evidence on the magnitude of liberal grade promotion in public schools in Nepal. The pass rate in the year-end exam whose scores determine grade transition is 256 percent higher than the pass rate in internally administered trimester exams. This suggests that as many as seven out of 10 students who transition from grade eight to nine may be doing so due to the policy.

The study has a number of limitations. First, due to the unavailability of data, the estimates are not conclusive. In particular, the study cannot positively rule out the possibility that other inputs like teacher attendance, time spent on classroom instruction, and remedial classes for the end-of-theyear exams were responsible for the observed leap in pass rates. Second, the small number of schools in the study were selected purposively from semi-urban areas of Kathmandu for a randomized controlled trial of student incentives, so these schools are not representative of all the schools in the district or the country. Further research is necessary to establish the external validity of the results.

Although not a causal attribution, liberal promotion policy appears to have been successful in reducing the system’s inefficiency: between 1999 and 2014, the repetition rate at the primary level dropped from 23 percent to 10 percent, the transition rate from primary to lower secondary level climbed from 80 percent to 87 percent, and the survival rate to grade five, the last grade of the primary school cycle, rose from 59 percent to 70 percent (World Development Indicators).<sup>11</sup> There is, however, little evidence of improvement in the quality of education over the same period. The pass rate in School Leaving Certificate (SLC) exams, the terminal exam taken in grade 10, continues to remain low; in 2014, it was 34 percent in community schools compared to 89 percent in institutional schools.<sup>12</sup> According to the national assessment of grade 10 students in 2011, the average achievement level in the five core subjects was only 41 percent (Educational and Developmental Service Centre, 2011). An assessment by Independent Evaluation Group (IEG) of BPEP, the project that introduced the liberal promotion policy, concluded that the policy encouraged children to stay longer in school but “...no means were developed to help the children who were falling behind learn the required material and pass grades or stay in school” (Independent Evaluation Group, 2009).

> 11 Accessed March 15, 2016.

> 12 <u>http://myrepublica.com/society/story/23078/slc-results-2015-declared-with-47-43-pc-pass.html</u> Accessed March 15, 2016.

10
The study points to a number of reforms necessary in the student assessment system in Nepal. First, reporting numeric scores in subject-based examinations and using arbitrary cutoffs as passing thresholds is not the ideal way to assess students and determine who progresses through the education system. Reporting raw scores ignores the measurement error inherent in any measure of student learning. For instance, it unambiguously ranks a student who scores 60 higher than a student who scores 59 when many variables other than true “ability” play a role in determining the score (for instance, the exam questions, the examiner effects, etc.). It also deems a student who scores one point below the passing threshold to have insufficient ability to advance to the higher grade, while a student who obtains the passing threshold is allowed to move on.

In recognition of the limitations of the numeric scoring method, letter grading was introduced in the School Leaving Certificate (SLC) exams starting in 2016.<sup>13</sup> It is recommended that this practice be extended to the lower grades as well. Use of letter grades will facilitate the implementation of the liberal grade promotion policy without the need to artificially report a passing grade in the final examinations. In practice, the passing threshold of 32 is already not strictly enforced due to the liberal grade promotion policy.

On a broader note, the role of formative classroom assessment and system-level assessments should be strengthened in the overall student assessment system. Curriculum-based examinations are the primary form of student assessment in the country. Indeed, an evaluation of Nepal’s student assessment system by the World Bank rated examinations as “established” while classroom assessment and national assessments were rated as “emerging” (The World Bank, 2012). Rather than measuring the underlying ability or aptitude, the marks obtained in the examinations measure students’ level of preparedness for subject-based assessments. The scores are not comparable across subjects and from one year to another, so they cannot be used to monitor the performance of the overall system. At the same time, evidence suggests that formative classroom assessments lead to better long-term learning outcomes because they provide frequent feedback to students about their learning levels (Black & Wiliam, 1998). And large-scale system level assessments provide information on the overall health of the system (Clarke, 2012). Therefore, the liberal promotion policy must be complemented by robust formative classroom assessments and system-level assessments to promote and monitor student learning as they advance through the education system.

> 13 SLC exam is the national exam taken in grade 10 and it is the “gateway” exam for all students who wish to pursue further education.

11