HIAS-E-99

Does remedial activity with math workbook improvelearning? Empirical evidence from scaled-up intervention in
Niger
Takao Maruyama(a) and Takashi Kurosaki(b)(a) Graduate School for Humanities and Social Sciences, Hiroshima University(b) Institute of Economic Research, Hitotsubashi University

June, 2020

Hitotsubashi Institute for Advanced Study, Hitotsubashi University
2-1, Naka, Kunitachi, Tokyo 186-8601, Japan
tel:+81 42 580 8668 http://hias.hit-u.ac.jp/
HIAS discussion papers can be downloaded without charge from:http://hdl.handle.net/10086/27202
https://ideas.repec.org/s/hit/hiasdp.html

All rights reserved.

Ver. June 30, 2020

Does remedial activity with math workbook improve learning?
Empirical evidence from scaled-up intervention in Niger

Takao Maruyama and Takashi Kurosaki12

Abstract
A set of interventions, including distribution of math workbooks and training of activity
facilitators, was scaled up by the government of Niger from 2017 to 2018, targeting
approximately 310,000 students from 1st to 4th grade in around 3,500 public schools. The
scaled-up interventions tried to help the students improve basic math learning throughextra-curricular remedial activity. Because of budget constraint, the distribution of mathworkbooks was limited to students from 1st to 4th grade, not covering 5th and 6th grades.
Focusing on the discontinuity of intervention between 4th and 5th grade students, this study
investigated the impact of the interventions on student math learning using three-round survey
data. The average impact of the interventions is estimated to be 0.36 to 0.38 standard deviationsof test scores. The impact is larger for students with lower baseline scores.

Key words: Educational development; Basic math learning; Community participation;
Scaling-up intervention; Sub-Saharan Africa

Maruyama: Graduate School for Humanities and Social Sciences, Hiroshima University, 1-5-1,
Kagamiyama, Higashi-Hiroshima, Hiroshima 739-8529 Japan (e-mail: tmaru@hiroshima-u.ac.jp);
Kurosaki: Institute of Economic Research, Hitotsubashi University, 2-1 Naka, Kunitachi, Tokyo
186-8601, Japan (e-mail: kurosaki@ier.hit-u.ac.jp).
2 All data in this study are from JICA and were collected by the project “School for All: the Projecton Support to Educational Development through Community Participation.” We are grateful to
Akiko Kageyama for her support for this study. We also appreciate Masahiro Hara and Nobuhiro
Kunieda for their continuous encouragement. Findings, interpretations, and conclusions expressed inthis paper are those of authors and do not represent the views of Japan International Cooperation
Agency (JICA).
11

1. Introduction
Despite the progress in expansion of enrollment since 1990, approximately 617 million primaryand lower secondary school-age children are not reaching minimum proficiency levels inreading and mathematics in the world (UNESCO, 2017). The learning crisis is particularly acutein Sub-Saharan Africa, where most children are not mastering basic reading and mathematics inprimary education (Bashir et al., 2018). One of the reasons of the low-level learning is amismatch between curricula and learning. Pritchett and Beatty (2015) reviewed basic readingassessments in India, Pakistan, Tanzania, Uganda, and Kenya and found that the learning
progress per year of schooling was slow. Curricular expectations are too high, and the mismatch
between curricular and children’s learning levels expands with years of schooling in developing
countries. Even though most children are left behind, teachers are required to completeprescribed syllabus, then their attention tends to be selective, often focusing on a limited
number of children who can keep up (Banerji, 2000; Abadzi and Llambri, 2011).
Several systematic reviews in educational development agree that pedagogicalintervention which tailors teaching to student learning level is effective at improving learning(Evans and Popova, 2016). An example that adapt the approach for improving basic reading andmath is “Teaching at the Right Level (TaRL)” developed by Pratham, an Indian
nongovernmental organization. TaRL divides 3rd to 5th grade students into several groups
according to the assessment result. They are taught basic reading and math with various
materials and activities that match with their proficiency levels. TaRL has helped children to
improve basic reading and math proficiency rapidly (Banerjee et al., 2017). In the process of thescaled-up program by the government, strong leadership of state government and continuous
on-site monitoring and support were essential elements for changing teaching practice
successfully and improving learning (ibid). However, it is still challenging to scale up TaRL
successfully in sub-Saharan Africa, where the capacities of education administration andteachers are often limited.
In the context of limited capacity of education administration and teachers, Niger has

2

developed a package of interventions to improve basic math proficiency: “Paquet Minimum
Axé sur la Qualité” (minimum package for quality learning or PMAQ) developed by the “Ecolepour tous” project (the EPT project) in cooperation with the Japan International Cooperation
Agency (JICA). PMAQ seeks to improve basic math learning through extracurricular remedialactivities by two types of interventions: a one-day training of School Management Committee(SMC) presidents and secretaries (school principals), and a set of distribution of mathworkbooks for each student and a two-day training of activity facilitators. PMAQ was
introduced in pilot schools in two regions (Tillaberi and Niamey) of Niger’s eight regions, andstudents who participated extra-curricular remedial activity improved math learning in the
2013-14 school year (JICA, 2017).
Following the successful pilot, the government of Niger decided to scale up PMAQ to
all public primary schools in Tillaberi region; however, because of budget constraint, thedistribution of math workbooks was limited to 1st to 4th grades. Focusing on the discontinuityof intervention between 4th and 5th grade students, this study investigates the impact of PMAQ
on student math learning. There are mainly three contributions of this paper to the literature.
First, this study investigates the impact of scaled-up intervention on student math learning,
using quasi-experimental data from three rounds of survey. While the number of studies on pilot
interventions increases, rigorous evidence on the scaled-up intervention impact is still scarce ineducation. Although a small pilot succeeds, the scaled-up intervention by the government often
faces implementation challenges (Banerjee et al., 2017; Bold et al., 2018). The ministry and the
EPT project did not plan to conduct rigorous evaluation, but the discontinuity of intervention
between 4th and 5th grades allowed us to investigate the impact of PMAQ. The average impact
on student math learning is estimated to be 0.36 to 0.38 standard deviations. The pre-baselineand baseline data confirm that the identification assumption of the estimation is satisfied.
Second, this study evaluates the heterogeneity of impacts with respect to two aspects
that are important in Niger. One of them is baseline score level. In Niger, 92.4 percent ofstudents in last grade of primary education do not reach sufficient levels in mathematics, and3

68.4 percent cannot solve basic math item such as four operations (PASEC, 2016). Girls’
achievement level of mathematics is lower than boys (ibid). Thus, this study assesses theheterogeneous impact with respect to gender as well. It turns out that while the impact waslarger on students with lower baseline scores, the impact additional for girls was not statisticallysignificant. It demonstrates that the heterogeneous impact by baseline score level was notbrought via the correlation with gender. PMAQ helped children left behind improve basic mathlearning.
Third, the results of this study indicate that PMAQ improved quality of remedialactivity organized by SMC even in a context of limited capacity of educational administrationand teachers. In Niger, district educational offices suffer from the lack of staff and logisticalresources. One district educational office oversaw 249 SMCs on average (Kunieda et al.,forthcoming). Less than one percent of primary school teachers possessed minimum knowledgeof the subjects that they teach (World Bank, 2017). The minimum knowledge indicator reflects
the percentage of teachers who scored above 80 percent in the lower primary portion of
language and mathematics tests (ibid). In most sub-Saharan African countries, the capacity ofteachers is limited like Niger (Bashir et al., 2018). Only 7 percent of teachers met the minimum
content knowledge of language and mathematics in seven sub-Saharan African countries (Boldet al, 2017). PMAQ can help those countries with limited capacity of teachers rapidly overcome
learning crisis.
The remainder of this paper is organized into the following 5 sections: (1) scaled-up
intervention and timeline; (2) evaluation strategy; (3) results; (4) discussions and (5) conclusion.

2. Scaled-up intervention and timeline(1) Scaled-up intervention
PMAQ seeks to improve basic math learning through extracurricular remedial activities

4

organized by SMC3. For this purpose, two types of interventions were designed. The firstintervention is a one-day training of SMC presidents and secretaries (school principals) thatstrengthen the capacity of SMC to develop school action plan integrating organization ofremedial activity using math workbooks. After the training, school conducts basic math test tomeasure students’ proficiency level. SMC organizes a community general assembly to share theresult among teachers, parents and community, and discuss causes of low learning and potential
activities4. Based upon the discussions, SMC prepares a draft of school action plan integrating
remedial activity, and hold a community general assembly again to adopt it.
The second intervention of PMAQ is pedagogical, which consists of distribution ofmath workbooks and a two-day training of activity facilitators (teachers and community
facilitators). The EPT project developed nine workbooks on introduction to number and basic
four operations, which reflect different math proficiency levels5. Math workbooks which matchwith proficiency level are distributed for each student, according to the result of basic mathassessment. Students learn basic math with workbooks by their own paces. Activity facilitators
check their answers and give an instruction for each student. The content is arranged
systematically so students can easily follow and progressively understand more mathematics
3

For further detail of PMAQ, see Hara et al. (forthcoming) and Kunieda et al. (forthcoming).

SMC facilitates the discussion at community general assembly using the analysis framework;
namely (i) inadequate time spent on learning; (ii) lack of suitable learning materials andenvironments; and (iii) ineffective and low-quality teaching and learning practices. For each of thesefactors, the community identifies a corresponding solution (e.g., increasing the amount of time spenton learning, providing suitable learning materials and environments and improving the quality ofteaching and learning practices). These solutions are then integrated into the SMC-organized
extracurricular remedial activities.
5 The math workbooks were issued in nine volumes:
Volume 0 (pre-math): Introduction to handwriting
Volume 1: Introduction to numbers: 0 – 10; Composition and decomposition of numbers
Volume 2: Introduction to numbers: 11-20; Addition/subtraction without carrying and borrowing;
Addition and subtraction with carrying and borrowing
Volume 3: Addition and subtraction of 3-digit numbers; Understanding numbers: 21-79
Volume 4: Addition and subtraction of 2-digit numbers; Number sense 80 – 120; Addition andsubtraction: vertical; Number sense: up to 1,000
Volume 5: Addition and subtraction of 2 to 3-digit numbers; Number sense: up to 10,000; Additionand subtraction of 2 to 3-digit numbers: verticalVolume 6: Multiplication and division: basic; Division with remainders
Volume 7: Number sense: 10,000 – 100,000,000; Multiplication: verticalVolume 8: Division: vertical; Calculation rules and techniques
45principles. For example, students learn what a number is by following workbook pages that
gradually transforms an illustration of a concrete object into a number. The math workbook
includes plenty of well-developed math problems so students can practice their mathematics
skills until they have mastery.
Out of Niger’s eight regions, PMAQ was introduced in pilot schools in Tillaberi and
Niamey regions. Since students who participated extra-curricular remedial activity improved
math learning in the 2013-14 school year (JICA, 2017), the Government of Niger decided toscale up PMAQ in Tillaberi region in collaboration with multi-donor support6. The populationof the region is around 2.6 million which accounts for about 16 percent in the country, and 53.8
percent live in poverty (national average: 59.5%). While the access to primary education in theregion is close to the national average, the quality lags. 97.1 percent of students in the last yearof primary school do not reach sufficient levels in mathematics in the region (PASEC, 2016).
Gender gap is also an issue in educational development. In the region, primary gross enrollment
ratio is 79.3 percent (boys) and 74.2 percent (girls), and primary completion rate is 75.3 percent(boys) and 67.2 percent (girls) (The Ministry of Primary Education in Niger, 2016).
As the scaled-up intervention, remedial activity was organized from March to June2018 for the students in Tillaberi region; however, because of budget constraint, the distributionof math workbooks was limited to 1st to 4th grades of students.

(2) Timeline of scaled-up intervention[Insert Figure 1: Timeline of scaled-up intervention]

The ministry originally scheduled the scale up of PMAQ in the 2016-17 school year7. From
November to December 2016, a training of SMC president and secretary (school principle) on
6

Multi-donor support was provided by the Global Partnership for Education, AFD, Swiss Cooperationand the World Bank.
7 The Niger school year starts in October and ends in June the next year.

6

school action plan development was organized. Math workbooks were to have been deliveredon March 2017; however, as the procurement process was significantly delayed, the ministry
postponed the distribution of math workbooks and the training of activity facilitators to thefollowing school year.
A two-day training of trainers on the use of math workbook was organized in February
2018 for activity facilitators who were selected by each SMC. In March, before the start of theremedial activity at SMCs, the Ministry organized a forum of stakeholders including
departments of the Ministry of Primary Education, the director of local education administration
office, inspectors, pedagogical advisors, executive members of SMC federation 8 , SMC
supervisors, and mayors. At the forum, the target of improvement in basic math learning was
discussed, and the participants committed 30 percent point increase of correct response rate of
1st to 4th grade students through remedial activities. They also pledged to organize remedialactivities for them at least 10 hours per week at each SMC9.
According to the Ministry of Education in Niger (2018), 3,380 targeted schools
planned and conducted remedial activities on basic math. Each SMC implemented an average of
133 hours of extracurricular basic mathematics activities (around 10 hours per week) in 3
months between March and June 2018 supported by 12,735 facilitators, 86 percent of them wereteachers (ibid). Once these remedial activities were completed, correct responses on student
basic math assessment significantly improved from a starting point (baseline) of around 40
percent to 67 percent on average (end-line) (ibid). As evidence for the impact of scaling-up,
these figures are highly inadequate. First, this is a before-after comparison, not controlling forchanges over time that were not related with the project. Furthermore, the data quality is notsatisfactory10. Therefore, we estimate the impact in a more rigorous way, as explained in the
8

The SMC federation is a network of SMCs in a geographical or administrative area (municipality or
district) that supports SMCs in its network.
9 Each SMC federation organized a general assembly to report and discuss the result of forum to member
SMCs, and to pledge to achieve the target. Subsequently, each SMC organized a community general
assembly to discuss the objective, planning and implementation arrangement of remedial activities.
10 SMC conducted basic math assessment and reported the result to local education office, which
complied and forwarded them to the Ministry of Primary Education. There are several potential biases in

7next section.

3. Evaluation strategy(1) Sampling of schools and student test data
Of the 3,655 public primary schools in Tillaberi region, 3,519 schools in 13 departments taught
at least one class of 1st to 4th grades. Niger has two types of public primary school: those thatteach in French (plus local language); and (ii) those that teach Arabic in addition to the othersubjects. The latter type is called “Franco-Arab primary school”, which constitute around 10
percent of public primary schools in the region.
The EPT project selected 31 public primary schools which teaches subjects in French(and local language) and 3 Franco-Arab primary schools. Among the former, 28 schools wereselected through three-stage random sampling11: at the departmental, commune, and schoollevels. The other remaining three were schools where the EPT project piloted upgraded versionof PMAQ which integrates basic reading and writing in addition to basic math component12.
The three Franco-Arab primary schools were randomly chosen from the same departments
selected in the process of above-mentioned random sampling. As a result, the 34 schools,
consisting of 31 public primary schools and 3 Franco-Arab primary schools, are our evaluation
sample.

[Insert Table 1-1: Distribution of sampled schools by department and commune]

the student assessment results reported by each SMC, and compiled by the Ministry of Primary Education.
First, as the math test was voluntarily organized by each SMC, the quality of controlling test organization
varies among SMCs. Secondly, since it was not mandatory for SMCs to submit activity report to the local
education administration office, all the reports were not collected. Third, the numbers in SMC activity
report were summed up at the local education administration offices. In the process, the number might not
be correctly calculated, because of large volume of assessment results. In addition to those potential
biases, the student level individual test result is not available in the data compiled by the Ministry. Given
those limits, this paper uses the data collected by local NGO under the contract with the EPT project.
11 Five departments situated mainly in northern part of the region were excluded from sampling frame
because of security concerns.
12 For the three schools, math workbooks were delivered by the project. On the other hand, other schools
came to local education office to receive workbooks. The transportation expenses were born by themultilateral support.

8

Local NGO conducted three rounds of survey: in March 2017 (pre-baseline); December 2017(baseline); and June 2018 (end-line)13. As explained in the previous section, due to the delay ofmath workbooks procurement, the schedule of scaling up PMAQ was postponed. Then, the firstsurvey on March 2017 which was initially conducted as baseline survey became pre-baseline
survey. As the school calendar in Niger starts from October and finishes in June next year, thepre-baseline survey was conducted in the 2016-17 school year, and baseline and end-line
surveys were organized in the 2017-18 school year.
At each round of surveys, teams of local NGO organized basic math tests to measurelearning outcomes of all grades students at each school14. The test was composed for each grade
as follows:-

Grade 1: (i) single-digit addition (12 questions);-

Grade 2: (i) plus (ii) single-digit subtraction (12 questions);-

Grade 3: (i) and (ii) plus (iii) two-digit addition including problems with carrying andsubtraction including problems with borrowing (11 questions) plus (iv) single-digit
multiplication (9 questions);-

Grade 4: (i), through (iv) plus (v) three-to-four-digit addition including problems withcarrying and subtraction including problems with borrowing (12 questions); and-Grades 5 and 6: (i) through (v) plus (vi) single-digit division including problems withremainders (5 questions).
There were 44 test items for 3rd grade students, 56 for 4th grade students, and 61 for

5th and 6th grade students. We used 56 items for 4th grade students to compare with 4th and 5th
grade students’ test scores.

13

The authors were not involved in the survey design and data collection process. School sampling,
preparation of test items and surveys were conducted by the EPT project and local NGO. JICA provided
all data used in our study.
14 The test took 30 minutes for 1st and 2nd grades, and 45 minutes for the other grades.

9(2) Constructing panel data
Since the three rounds of survey were not designed to track same students, we constructed panel
data of 4th through 6th grade students ex post by manually linking data in the three rounds ofsurvey. As student name was typed at each round of survey, it caused inconsistency of spelling
among the three-rounds. For overcoming such inconsistency of spelling in student name, we
utilized fuzzy-matching method (Robinson, 2019). Specifically, we took four steps to constructpanel data from the three rounds of survey data. First, we removed accent marks above letters instudent names, and transformed student and school name to small letters.
Next, we assigned identification numbers to each student in the end-line data. Personalnames in Niger generally consist of an individual first name, father’s first name andgrandfather’s first name. Some students were registered by all of them, but the others were only
by own first name and father’s name. Thus, we fuzzy-matched end-line data with baseline datausing letters of the student first name, father’s name and school name by grade15. If there werestudents with exactly same name, we used grandfather’s name and matched it again. In case thatstudents still had same name, we excluded it16. As a result, the panel data of two data points(baseline and end-line) include 1,068 students from 4th through 6th grade.
Third, we fuzzy-matched our panel data (baseline and end-line) with pre-baseline data
for the corresponding previous grade or the same grade during the 2016-17 school year. For
example, we fuzzy-matched 4th grade student in the 2017-18 school year with 3rd grade in theprevious school year. Considering that students might have repeated the grade, we
fuzzy-matched the remaining unmatched data with data of 4th grade student in the 2016-17
school year. Our panel data of three data points contains 710 students.
None of the three rounds of survey collected student gender information. We
recovered the student gender information based on the typical correspondence of student firstname and gender. We observed around 800 types of student first name in total in the pooled data
15

The degree of fuzziness allowed one-letter differences in first name and father’s name.
Five pairs of students had same first and family name and belong to the same school. We excluded
them from the panel data sample.
16

10

from baseline and end-line surveys17.
The trends in average math test scores of 4th and 5th grade student group in threerounds of survey is shown in Graph 1. “4th grade student group” is those who were 4th gradestudents in the 2017-18 school year, which includes students who moved up from 3rd grade or
repeated 4th grade. “5th grade student group” is those who were 5th grade in 2017-18, which
includes students who moved up from 4th grade or repeated 5th grade. The trends in averagemath test scores of 4th and 5th grade student groups are parallel from pre-baseline to baseline
survey. On the other hand, the 4th grade average math test scores improved more than 5th gradestudent group from baseline to end-line survey, which we will analyze in the following section.
Table 1-2 presents descriptive statistics of the test results.

[Insert Graph 1: Trends in average math test scores (4th to 6th grade groups),
and Table 1-2: Number of students and test scores by grade group](3) Estimation Strategy
As noted, because of budget constraint, the distribution of math workbooks was limited to
students from 1st to 4th grades. That discontinuity of intervention between 4th and 5th grades
allows us to estimate the impact of PMAQ by equation (1) using the pooled data from baseline
and end-line surveys.

Yijt = α + β1 Grade4i +β2 Postt + λ DIDit+ Xiγ1 + Xjγ2 + εijt , (1)Yijt is the raw test score or its standardized value of student i in school j at the timing of tth
survey; Grade4i is a dummy variable that takes 1 for the student i of 4th grade in the 2017-18
school year, and takes 0 for the 5th grade; Postt is a dummy variable that takes 1 for the end-line

17

The EPT project provided association between student first name and gender. Boys’ and girls’ names
are rarely gender-ambiguous in Niger.

11

data; DIDit is an interaction term of Grade4i (treatment) and Postt, which identifies the impactof PMAQ. Xi is a vector of student characteristics for student i (e.g., the female dummy); Xj is avector of school characteristics for school j (e.g., the dummy for Franco-Arab school or pilot
school of basic reading) and writing activity; εijt is a residual of the regression. Scalers ofparameter α, β1, β2 and λ, and vectors of parameters γ1 and γ2 are to be estimatedeconometrically. The standard errors are clustered at school level.
As the main interest of this study is not on the impact of school characteristics onlearning, we replaced Xjγ2 in the equation (1) with school fixed effects, αj. The equation (1)becomes

Yijt = αj + β1 Grade4i +β2 Postt + λ DIDit+ Xiγ1+ εijt , (1-a).

By using the balanced panel data of two periods instead of the unbalanced pooled data, we
controlled the student characteristics more flexibly, using student fixed effects, αi. Note thateach student attends the same school in both baseline and end-line. In other words, student fixed
effects and school fixed effects are perfectly collinear, rendering the latter redundant. The
estimation model becomes

Yijt = αi +β2 Postt + λ DIDit+ εijt , (2).

By construction, parameters β1, γ1, and γ2 in the equation (1) cannot be identified.
The assumption under which we identify λ is parallel trends in the test scores of 4th
and 5th grade student groups before the scaled-up of PMAQ. If the parallel trend assumption
does not hold, λ cannot correctly identify the impact of PMAQ. The parallel trend assumption ischecked by the equation (3) using the pre-baseline and baseline data. Pre-Treatmentt is adummy variable which takes 1 for the baseline data; DID’it is an interaction term of Grade4i and

Pre-Treatmentt. If the parallel trend assumption holds, λ’ will be close to zero and not be

12statistically significant.

Yijt = αi +β2 Pre-Treatmentt + λ’ DID’it+ εijt , (3)As another test for our parallel trend assumption, we also check the trend of basic math learningof students in different grades, but not using math workbooks in remedial activity. Moreconcretely, we estimate equations (4) and (5) comparing 5th and 6th grade students in 2017-18.
In estimating (4), we use the pooled data of 5th and 6th grade students, while in estimating (5),
we use the panel data of 5th and 6th grade students. Grade5i is a dummy variable which takes 1
for 5th grade students in 2017-18, and takes 0 for the 6th grade; DID’’it is an interaction term of

Grade5i and Postt. If the trends in test scores of those grade students are parallel, λ’’ will beclose to zero and not be statistically significant.

Yijt = αj + β1 Grade5i +β2 Postt + λ’’ DID’’it+ Xiγ1+ εijt , (4)Yijt = αi +β2 Postt + λ’’ DID’’it+ εijt , (5).

4. Results(1) Average impact on basic math learning
The average impact of PMAQ on student basic math learning is estimated at 5.6 score points outof the total of 56 points using the pooled data and equation (1) (Table 2-1 (I)), 5.9 score points
using equation (1-a) which controls school fixed effect (Table 2-1 (II)), and 6.0 score points
when panel data is employed (Table 2-1 (III) and (IV)). All the estimated values are statisticallysignificant at the 1 percent level. Then, we standardized the impact with the mean and thestandard deviation of 4th grade students baseline score. The impact is 0.38 standard deviations
using the pooled data (Table 2-2 (II)) and the panel data (Table 2-2 (III) and (IV)).

[Insert Table 2-1 and 2-2: Average impact of PMAQ on math learning]13

Spider charts of Graph 2-1 to 2-3 display the before/after comparison of correct response rates
for each test item among 4th grade students. Correct response rates of each item improved frombaseline to end-line, but the rates are still low in items No. 31 and 32 (two-digit subtraction withborrowing), and items from No. 52 through 56 (three to four-digit addition with carrying andsubtraction with borrowing). The test items that exhibits improved correct response rate differ
according to the baseline score level. For example, students with baseline score less than the
median correctly answered more single addition and subtraction items at the end-line survey(Items No. 1 through 12 on Graph 2-2). Students with baseline score over the median correctly
answered more test items involving single-digit multiplication (Items No. 36 through 44) andthree-digit addition and subtraction without borrowing (Items No. 45 through 51 on Graph 2-3).

[Insert Graph 2-1 to 2-3: Correct response rate by math item (4th grade students)](2) Testing parallel trends
The trends in average scores of 4th and 5th grades student group from pre-baseline to baseline
survey are nearly parallel (Graph 1). The parallel trends are checked by the equation (3) using
pre-baseline and baseline scores in the panel data. The parameter estimate for λ’ is -0.92 scorepoints out of the total of 44 points which is not statistically significant (Table 3 (I))18. It showsthat the parallel trend assumption holds for identifying the impact of PMAQ through thedifference in differences (DID) of 4th and 5th grades student test scores.

[Insert Table 3: Regression results of parallel trend test (raw test score)]

Next, we checked the trend of basic math learning of students who did not use math workbooks

18

Most 4th grade students were at 3rd grade at the timing of pre-baseline survey. As explained in 4. (2),the maximum scale for 3rd grade students is 44.

14

in remedial activity by comparing 5th and 6th grade student test scores. If the trends of testscore of 5th and 6th grade students are different, the DID of 4th and 5th grade student testscores does not accurately identify the impact of PMAQ. The results from equation (4) and (5)are shown in Table 3 (II) and (III). The parameter estimate for λ’’ is 1.00 score point using thepooled data and 0.47 score point using the panel data. Neither are statistically significant. Those
results indicate that the DID of the test scores of 4th and 5th grades identifies the impact of
PMAQ on student math learning.

(3) Robustness check of estimated impact
To check the robustness of the estimated impact of PMAQ further, we added interaction termsof Xi and Postt, and those of Xj and Postt in equations (1) and (1-a) as below,

Yijt = α + β1 Grade4i +β2 Postt + λ DIDit+ Xiγ1 + Xjγ2 + (Xi×Postt)γ3 + (Xj×Postt)γ4+ εijt , (1-b)
Yijt = αj + β1 Grade4i +β2 Postt + λ DIDit+ Xiγ1+ (Xi×Postt)γ3+ εijt , (1-c).

The estimated values of λ from equations (1-b) and (1-c) remain almost at the same level asreported in the previous sub-section (Table 4-1 (I) to (III)).

[Insert Table 4-1: Regression result of robustness check (1) (raw test score)]

Given the structure of the basic math test, students with higher baseline scores would find it
harder to gain scores at the end-line survey than students with lower baseline scores. Since
average baseline scores of 4th grade students are below those of 5th grade students, thedifference of the baseline score level might engender a bias in estimating the impact of PMAQ.
In order to check such potential bias, we conducted a regression with the sub-samples in thesame range of baseline test scores from 10 to 37 score points, equivalent to 2nd and 3rd quartile15

of the 4th grade baseline scores. The sub-sample of 5th grade students in the bandwidth
represents around half of the panel data sample19. The test scores are standardized with the meanand standard deviation of the whole sample of the 4th grade student data. The estimated valuesof λ with the sub-sample is 0.28 to 0.29, which is slightly smaller than the values with the wholesample (Table 4-2) but still statistically significant.

[Insert Graph 3-1: Cumulative density curve of test scores of 4th grade student group,
Graph 3-2: Cumulative density curve of test scores of 5th grade student group, andTable 4-2: Regression results of robustness check (2)]Although the distribution of math workbooks was limited to 1st to 4th grade students, someschools organized supplementary classes for 5th and 6th grade students. The supplementary
classes for 5th and 6th grade students might have targeted basic math without using mathworkbooks. To check the robustness of the estimated impact of PMAQ, we conducted aregression with 4th grade students, and the sub-samples of 5th grade students of schools where
supplementary classes for the grade were organized. The test scores are standardized with themean and standard deviation of the whole sample of the 4th grade student data. The estimatedvalue of λ with the limited sample is 0.32, which is slightly below the values with the wholesample (Table 4-3) but still statistically significant.

[Insert Table 4-3: Regression results of robustness check (3)](4) Heterogeneous impacts
We analyzed the heterogeneity of impacts of PMAQ on student math learning with respect tobaseline score level and gender. If the impact is larger on students with lower baseline test
19

The percentage of 5th grade students with baseline scores below 9 score points is 9 percent of theoriginal 2 data point panel data sample. The percentage of 5th grade students with baseline below 37
score points is 57 percent of the original two-data-point panel data sample.

16

scores, it indicates that PMAQ helps those students to catch up their basic math understanding
and skills. If the impact is larger on girls than on boys, it shows that PMAQ reduces genderdisparity in basic math learning. As shown in Graph 4, the average baseline score of girls is
lower than boys.

[Insert Graph 4: Boxplot of baseline test score by sex and grade (4th and 5th grade students),
and Table 5: Number of 4th and 5th grade students by test score level and sex]

Because of the limited number of samples, we expanded the equation (1-a) for the heterogeneity
analysis as

Yijt = αj + β1 Grade4i +β2 Postt + (λ0 +λ1 Femalei) DIDit+ Xiγ1+ εijt , (6-a), andYijt = αj + β1 Grade4i +β2 Postt + (λ0 +λ1L Lowest_Scorei +λ1M Median_Scorei) +DIDit+
Xiγ1+εijt , (6-b).

By using balanced panel data, the heterogeneous impact will be accurately estimated. In theequation (6-a), parameter λ1 will identify the treatment effect additional for girls in comparison
with boys. In the equation (6-b), sample students are split into three groups: lowest baseline
score group (the 1st quartile of the score of each grade); median baseline score group (the 2nd
and 3rd quartiles of the score of each grade); and highest baseline score group (4th quartile ofthe score of each grade), and then we define Lowest_Scorei as the dummy variable which takes
1 if the test score of student i falls into the lowest baseline score group and Median_Scorei as
the dummy variable which takes 1 if the baseline test score of student i falls into the medianbaseline score group. Parameter λ1L andλ1M will identify the treatment effect additional forstudents in different groups in comparison with the highest baseline score group.

As the female dummy and lower baseline score dummy are correlated, we included both in a17

same equation. The heterogeneity impact model (6) can be expanded as

Yijt = αj + β1 Grade4i +β2 Postt + (λ0 +λ1L Lowest_Scorei +λ1M Median_Scorei +λ2 Femalei)DIDit+ Xiγ1+εijt , (7), andYijt = αj + β1 Grade4i +β2 Postt + (λ0 +λ1L Lowest_Scorei +λ1M Median_Scorei +λ2 Femalei +λ3L Lowest_Scorei ×Femalei+λ3M Median_Scorei ×Femalei) DIDit+
Xiγ1+ εijt , (8)from which we can test whether there is an interaction effect of two types of disadvantages
combined through examining parameter λ3.
The impact of PMAQ is larger for lower baseline score groups (Table 6-1 (II), (IV)and (V)). The estimated value ofλ1L is 0.58 standard deviations, and that of λ1M is 0.21 standard
deviations, even after controlling interaction term of the DID and gender (Table 6-1 (V)).
PMAQ helped children with lower baseline scores catch up basic math learning with higher
baseline score group. On the other hand, the impact additional for girls is not statisticallysignificant (Table 6-1 (IV) and (V)).

[Insert Table 6-1: Heterogeneous impacts of PMAQ on math learning (standardized test score)]

To examine the robustness of heterogeneous impacts, we added the interaction terms of Xi and

Postt to equations (6) to (8). For instance, when an interaction term of the female dummy andPostt is added, the DID heterogeneity term of girls shows the impact of PMAQ additional on
4th grade girls in comparison to 5th grade girls, independent of the additional impact on boys.
Furthermore, as already discussed, the estimated value of λ might be biased if trends in scoreswithout interventions are different depending on the baseline score level. For these reasons, we
replaced Lowest_Scorei with Lowest_Score’i (taking 1 when the test score of student i is 0 to 9
score points, which is the range for 1st quartile baseline score of 4th grade students), and also18

Median_Scorei with Median_Score’i (taking 1 when the test score of student i is 10 to 37 scorepoints, which is the range of 2nd and 3rd quartile of 4th grade students) in equations (6) to (8).
We also added interactions of Postt and the female dummy, Lowest_Score’i and Median_Score’I
to the equations.
The estimation results are reported in Table 6-2. The impact on the highest score group(λ0) is positive and statistically significant at the 1 percent level (Table 6-2 (II)). 4th gradestudents of the highest score group learned basic math better than 5th grade students of the same
bandwidth of the score. On the other hand, the impact on the lowest score group (λ1L) is positive
but not statistically significant (Table 6-2 (II) and (V)). The lowest score group of 4th grade as
well as 5th grade students caught up with the highest score group of each grade.
The average baseline score of girls is lower than boys, and the time trend in 5th grade
girls’ learning is negative (Table 6-2 (III), (IV) and (V)), which indicates that the genderdisparity in learning outcomes tends to become larger. The estimated value of λ2 is positive andstatistically significant at the 5 percent level (Table 6-2 (III) to (V)), which indicates that 4th
grade girls could learn basic math better than 5th grade girls by using math workbooks in theremedial activity.

[Insert Table 6-2: Robustness check of heterogeneous impacts of PMAQ on math learning(standardized test score)]

5. Discussions
PMAQ improved basic math learning even in a context of limited capacity of educational
administration and teachers. PMAQ helped students with lower baseline score catch up basic
math learning. The remedial activity is organized not by grade or age, but by student proficiency
level. Students could learn math with workbooks which matches with their proficiency levels,
supported by activity facilitators. The learning contents are standardized by math workbooks,
which enabled scaling up remedial activity with quality in a difficult context. The remedial

19activity using math workbooks also helped teachers to reflect their teaching practices in dailyclasses (Hara et al., forthcoming). Teachers tended to have a negative perception on student
learning ability, since most of them could not follow daily classes. In remedial activities of
PMAQ, teachers checked student answers in math workbooks and gave them an instruction
individually. Throughout the process, teachers noticed the progress of student math learning,and changed positively their perception on student learning ability. Students also felt teachers
closer to them, and asked more questions (ibid).
The EPT project has continuously developed a package of interventions for addressing
various educational needs in local communities in Africa. Kozuka et al. (2016) found that apackage of trainings for school principals and SMC representatives increased student enrollment,
decreased student repetition, and lowered teacher absence through establishing community-wide
collaboration in Burkina Faso. The intervention also enhanced social capital in the community(Sawada et al., 2016). A combination of grant provision and training for school principals and
SMC representatives further improved student learning outcomes in French and math in Niger(Kozuka, 2018). This study contributes to the literature on the empirical study on the EPT
approach in Africa through providing rigorous evaluation of scaled up intervention of the novel
package of interventions, PMAQ.
While PMAQ helped students with lower baseline scores improve basic math learning,
it was not sufficient to decrease gender disparity in learning. Girls in Niger socially matureearlier than boys, and hesitate to ask questions or pose answer in daily class, because they worry
what the others think of them. Such behavior hampers girls’ math learning, because they receive
less feed-back from teachers. In the extra-curricular remedial activity of PMAQ, where activity
facilitators check math workbooks individually, girls can learn basic math by their own paces
and ask questions. This study investigated the heterogeneity of impacts with respect to gender.
We found that 4th grade girls who learned with the workbook improved math learning than 5th
grade girls who did not; however, the gender disparity in basic math learning still remains. In
Niger, the percentage of girls who reach to the last grade of primary education in the age group20

is 65 percent, which is lower than boys by 13 percentage points in 2016 (UIS, 2020). Low
learning achievement is a major reason of dropout of girls from primary education (Perlman etal., 2016). PMAQ has room to improve for reducing gender disparity in learning.

6. Conclusion
The scaling-up of PMAQ in Niger’s Tillaberi region tried to help students improve basic mathlearning through extra-curricular remedial activity. PMAQ aims to improve basic math learningby two types of interventions: a one-day training of SMC president and secretary to strengthen
the capacity of SMC to organize basic math remedial activity, and a set of distribution of mathworkbooks and a two-day training of activity facilitators. Because of budget constraint, thedistribution of math workbooks was limited to 1st to 4th grades. Focusing on the discontinuityof intervention between 4th and 5th grades, we investigated the impact of PMAQ on student
math learning using three rounds survey data. The average impact is estimated to be 0.36 to 0.38
standard deviations. The impact is larger for students with lower baseline score, which indicatesthat the remedial activity helped students left behind catch up basic math learning.
In developing countries, a mismatch between curricula and the learning levels ofstudents expands as curricular progresses. While most students are left behind, teachers are
required to finish prescribed syllabus. As a result, most of children finish or drop out fromprimary education without mastering foundational skills for learning: basic reading and math.
The scaling up of PMAQ in Niger shows that once children have a chance of learning whichmatches with their proficiency levels, they can improve math learning. Governments in
sub-Saharan Africa should pursue measures that help children master foundational skills forovercoming learning crisis.

21

Reference

Abadzi, Helen, Stavri Llambiri. (2011). “Selective Teacher Attention in Lower-Income
Countries: A Phenomenon Linked to Dropout and Illiteracy?” Prospects, 41, 491–506.
Banerjee, Abhijit, Rukmini Banerji, James Berry, Esther Duflo, Harini Kannan, Shobini Mukerji,
Marc Scotland, and Michael Walton. (2017). “From Proof of Concept to Scalable
Policies: Challenges and Solutions, with an Application.” Journal of Economic
Perspectives, 31 (4), Fall 2017, 73-102.
Banerji, Rukmini. (2000). “Poverty and Primary Schooling: Field Studies from Mumbai and
Delhi.” Economic and Political Weekly, 35 (10), 795-802.
Bashir, Sajitha, Marlaine Lockheed, Elizabeth Ninan, and Jee-Peng Tan. (2018). Facing
Forward: Schooling for Learning in Africa. Africa Development Forum Series.
Washington DC: World Bank.
Bold, Tessa, Deon Filmer, Gayle Martin, Ezequiel Molina, Brian Stacy, Christophe Rockmore,
Jakob Svensson and Waly Wane. (2017). “Enrollment without Learning: Teacher Effort,
Knowledge and Skill in Primary Schools in Africa.” Journal of Economic Perspectives,
31 (4): 185-204.
Evans, David K. and Anna Popova. (2016). “What Really Works to Improve Learning in
Developing Countries? An Analysis of Divergent Findings in Systematic Reviews.”
World Bank Research Observer, 31 (2), 242-270.
Hara, Masahiro, Takao Maruyama, Akiko Kageyama and Nobuhiro Kunieda. (Forthcoming).
Quality Learning through Community-wide Collaboration: A Methodology to Overcome
the “Learning Crisis” in Niger. In M. Nishimura (Ed.), Community participation withschools in developing countries: Towards equitable and inclusive basic education for all.
Routledge Research in Educational Equality and Diversity Series. New York & Oxon:
Routledge.

22

JICA. (2017). Final Evaluation Report on School for All: The project on support to educational
development through community participation. Tokyo: Japan International Cooperation
Agency.
Kozuka, Eiji, Yasuyuki Sawada and Yasuyuki Todo. (2016). “How Can Community
Participation Improve Educational Outcomes? Experimental Evidence from a
School-Based Management Project in Burkina Faso.” JICA-RI Working Paper No.112.
Tokyo: JICA Research Institute.
Kozuka, Eiji. (2018). “Enlightening Communities and Parents for Improving Student Learning
Evidence from Randomized Experiment in Niger.” JICA RI Working Paper. No.166.
Tokyo: JICA Research Institute.
Kunieda, Nobuhiro, Takao Maruyama, Akiko Kageyama and Masahiro Hara. (Forthcoming).
Educational Development through Community-wide Collaboration: How to Establish a
Sustainable Community-wide Initiative to Improve Education. Nishimura. op.cit.
Ministry of Primary Education in Niger. (2016). STATISTIQUES DE L’ÉDUCATION DE BASEET ALPHABÉTISATION ANNUAIRE: ANNUAIRE 2015-2016. Niamey: The Ministry ofPrimary Education in Niger.
Ministry of Primary Education in Niger. (2018). [Situation de la mise en œuvre du PMAQ dansles écoles (BILAN)]. Unpublished Raw Data.
Ministry of Primary Education in Niger. (2019). STATISTIQUES DE L’ÉDUCATION DE BASEET ALPHABÉTISATION ANNUAIRE. Niamey: The Ministry of Primary Education in
Niger.
Organisation Nigérienne des Educateurs Novateurs (ONEN). (2017). Rapport final de l’étude àmi-parcours dans 45 écoles primaires de la région Tillaberi dans le cadre del’evaluation des résultats des interventions du Projet Ecole Pour Tous Phase II(Mars-Avril 2017). Niamey: JICA.
Organisation Nigérienne des Educateurs Novateurs (ONEN). (2018a). Rapport final de l’étude àmi-parcours dans 45 écoles primaires de la région Tillaberi dans le cadre de23

l’evaluation des résultats des interventions du Projet Ecole Pour Tous Phase II(Décembre 2017-Janvier 2018). Niamey: JICA.
Organisation Nigérienne des Educateurs Novateurs (ONEN). (2018b). Rapport final de l’étude
à mi-parcours dans 45 écoles primaires de la région Tillaberi dans le cadre del’evaluation des résultats des interventions du Projet Ecole Pour Tous Phase II(Juin-Juillet 2018). Niamey: JICA.
PASEC. (2016). PASEC 2014 – Performances du système éducatif nigérien: Compétences etfacteurs de réussite au primaire. PASEC. CONFEMEN. Dakar.
Perlman, Daniel, Fatima Adamu, and Quentin Wood. (2018). “Why Do Adolescent Girls Drop
Out of School in Niger? A Combined Quantitative and Qualitative Analysis.” Marché etorganisations, 2 (32), 179-194.
Pritchett, Lant, Amanda Beatty. (2015). “Slow Down, You Are Going Too Fast: Matching
Curricula to Student Skill level.” International Journal of Educational Development, 40,
276-288.
Robinson, David (2019). fuzzyjoin: Join Tables Together on Inexact Matching. R package
version 0.1.5. https://CRAN.R-project.org/package=fuzzyjoin.
Sawada, Yasuyuki, Takeshi Aida, Andrew S. Griffen, Eiji Kozuka, Haruko Noguchi, andYasuyuki Todo. (2016). “Election, Implementation, and Social Capital in School BasedManagement: Evidence from a Randomized Field Experiment on the COGES Project in
Burkina Faso”. JICA-RI Working Paper No.120. Tokyo: JICA Research Institute.
UIS (UNESCO Institute for Statistics). (2020). Data for the Sustainable Development Goals:
Niger. Retrieved from http://uis.unesco.org/country/NE (Access date: April 23, 2020)
UNESCO. (2017). More than One-Half of Children and Adolescents Are Not LearningWorldwide. Fact Sheet No. 46 (UIS/FS/2017/ED/46).
World Bank. (2017). Republic of Niger: Niger service delivery indicators education 2015.
Washington DC: World Bank.

24

Figure 1: Timeline of scaled-up intervention25Graph 1-1: Trends in average math test scores (4th to 6th grade groups)

26

Graph 2-1: Correct response rates by item (4th grade students)

27

Graph 2-2: Correct response rates by item
(4th grade students below median baseline score)28

Graph 2-3: Correct response rates by item
(4th grade students over median baseline score)29

Graph 3-1: Cumulative density curve of test score of 4th grade student group30

Graph 3-2: Cumulative density curve of test score of 5th grade student group31

Graph 4: Boxplot of baseline test score by sex and grade (4th and 5th grade group)32

Types of public
primary school

Department

Kollo
Public primaryschool which Sayteaches subjects
in French or Gotheye
local language
TorodiFranco-ArabKollo
Torodi
Total

CommuneKollo
Hamdalaye
Say
Gueladio
GotheyeDargol
Makalondi
Torodi
Kollo
N'Dounga
TorodiN. of schools
which haveboth 4th & 5th
grades(t1 and t2) (A)N. of schools
which have
only 4th grade(t1 and t2)(B)

N. of schools
which have
only 5th grade(t1 and t2)(C)

N. of schools of
evaluation sampleof this study(t1 and t2)(A)+(B)+(C)

2
2
3
0
1
2
2
3
1
0
0
16

4
1
1
1
2
1
1
0
0
0
1
12

0
1
0
3
0
0
1
0
0
1
0
6

6
4
4
4
3
3
4
3
1
1
1
34

Note t1: baseline survey; t2: end-line survey
Table 1-1: Distribution of sampled schools by department and commune

33

Pooled Data (2t): Maximum scale (full mark) of test score is 56
Pre-baseline
BaselineEnd-line
N. of students Mean score N. of students Mean score N. of students Mean score(Female)(SD)(Female)(SD)(Female)
(SD)
545
22.0
516
34.3
Grade 4 group(288)(15.4)(290)(14.7)
521
31.0
413
37.7
Grade 5 group(237)(14.7)(199)(14.2)
571
35.1
536
40.6
Grade 6 group(276)(14.6)(272)(14.1)
Panel Data (2t): Maximum scale (full mark) of test score is 56
Pre-baseline
BaselineEnd-line
N. of students Mean score N. of students Mean score N. of students Mean score(Female)(SD)(Female)(SD)(Female)
(SD)
360
23.7
360
36.0
Grade 4 group(197)(15.7)(197)(14.2)
332
32.0
332
38.3
Grade 5 group(152)(14.5)(152)(14.0)
376
35.0
376
40.8
Grade 6 group(196)(14.4)(196)(14.0)
Panel Data (3t): Maximum scale (full mark) of test score is 56
Pre-baseline
BaselineEnd-line
N. of students Mean score N. of students Mean score N. of students Mean score(Female)(SD)(Female)(SD)(Female)
(SD)
240
23.1
240
35.7
Grade 4 group(125)(16.1)(125)(14.6)
220
32.1
220
38.2
Grade 5 group(102)(15.2)(102)(13.9)
250
35.3
250
41.1
Grade 6 group(127)(14.5)(127)(13.6)
Panel Data (3t): Maximum scale (full mark) of test score is 44
Pre-baseline
BaselineEnd-line
N. of students Mean score N. of students Mean score N. of students Mean score(Female)(SD)(Female)(SD)(Female)
(SD)
240
17.2
240
20.7
240
30.8
Grade 4 group(125)(13.1)(125)(13.6)(125)(11.6)
220
23.5
220
28.0
220
32.7
Grade 5 group(102)(13.1)(102)(12.3)(102)(10.8)
250
28.1
250
30.0
250
34.7
Grade 6 group(127)(12.3)(127)(11.2)(127)(10.3)

Table 1-2: Number of students and test scores by grade group
Notes:(1) “Grade 4 group” is defined as a group of students who were in grade 4 in the school year 2017-18.
“Grade 5 group” and “Grade 6 group” are defined similarly.
(2) The first section “Pooled Data” is for students with test scores in 35 schools reported in Table 1-1,
which include students who were not linked across different rounds of survey. Other three sections arefor subsets of the first section. The second section “Panel Data (2t)” is for 1,068 students who weresuccessfully linked between baseline and end-line. The third and fourth sections are for 710 students
who were successfully linked between pre-baseline, baseline and end-line.

34

(Intercept)
Grade 4 (Treatment)×Post(λ)
Grade 4 (Treatment)
Post
FemaleFranco Arab school
Basic reading pilot school
School fixed effect (dummy)
Student fixed effect (dummy)

Adjusted R-Squared
Observations(I)
4th / 5th
PooledModel (1)32.829***(2.445)5.656***(1.349)-8.310***(2.113)
6.697***(1.002)-3.194**(1.175)-3.035(3.339)-5.123(3.707)
No
No
0.151
1,995

(II)
4th / 5th
PooledModel (1-a)(III)
4th / 5th
Panel: 2t
Model (1-a)(IV)
4th / 5th
Panel: 2t
Model (2)5.956***(1.251)-9.627***(1.576)6.493***(1.028)-3.280***(0.896)6.089***(1.183)-7.854***(1.753)
6.247***(0.863)-2.602**(1.170)6.089***(1.653)

Yes
No
0.865
1,995

Yes
No
0.880
1,384

No
Yes
0.958
1,384

6.247***(1.206)***p < 0.01, **p < 0.05, *p < 0.1(1) Robust standard errors are clustered at school level, and in parenthesis. The variance of standard errors is adjusted
by finite-sample correction.

(2) Maximum scale (full mark) of math test score is 56.

Table 2-1: Average impact of PMAQ on math learning(raw test score)35

(Intercept)
Grade 4 (Treatment)× Post(λ)
Grade 4 (Treatment)
Post
FemaleFranco Arab school
Basic reading pilot school
School fixed effect (dummy)
Student fixed effect (dummy)

Adjusted R-Squared
Observations(I)
4th / 5th
PooledModel (1)0.702***(0.159)0.367***(0.088)-0.540***(0.137)0.435***(0.065)-0.207**(0.076)-0.197(0.217)-0.333(0.241)
No
No
0.151
1,995

(II)
4th / 5th
PooledModel (1-a)(III)
4th / 5th
Panel: 2t
Model (1-a)(IV)
4th / 5th
Panel: 2t
Model (2)0.387***(0.081)-0.625***(0.102)0.422***(0.067)-0.213***(0.058)0.387***(0.075)-0.499***(0.111)0.397***(0.055)-0.165**(0.074)0.387***(0.105)

Yes
No
0.508
1,995

Yes
No
0.505
1,384

No
Yes
0.832
1,384

0.397***(0.077)***p < 0.01, **p < 0.05, *p < 0.1(1) Robust standard errors are clustered at school level, and in parenthesis. The variance of standard errors isadjusted by finite-sample correction.

(2) All the test scores at each round of survey are normalized by mean and standard deviation of baseline scores of
4th grade students.

Table 2-2: Average impact of PMAQ on math learning(standardized test score)36

Grade 4 (Treatment) × Pre-Treatment(Baseline) (λ’)
Grade 5 × Post (λ’’)(I)
4th / 5th
Panel: 2t
Model (3)-0.929(1.768)Grade 5
Pre-Treatment (Baseline)

FemaleAdjusted R-Squared
Observations(III)
5th / 6th
Panel: 2t
Model (4)(III)
5th / 6th
Panel: 2t
Model (5)1.000
(1.341)-5.018**(1.991)0.465(1.314)-4.096(2.502)0.465(1.836)5.709***(1.099)-2.295*(1.239)
Yes
No
0.890
2,041

5.782***(1.078)-2.244(1.476)
Yes
No
0.897
1,416

5.782***(1.506)4.500***(1.188)

Post

School fixed effect (dummy)
Student fixed effect (dummy)(II)
5th / 6th
PooledModel (4)

No
Yes
0.939
920

No
Yes
0.972
1,416

***

p < 0.01, **p < 0.05, *p < 0.1(1) Robust standard errors are clustered at school level, and in parenthesis. The variance of standard errors is adjusted
by finite-sample correction.

(2) Maximum scale (full mark) of math test score in column (I) is 44. Maximum scale (full mark) of math test score infrom column (II) to (IV) is 56.

(3) Column (I) of panel data is composed by pre-baseline and baseline data. Column (III) of panel data is composed bybaseline and end-line data.

Table 3: Regression results of parallel trends test (raw test score)37

(Intercept)
Grade 4 (Treatment) × Post(λ)
Grade 4 (Treatment)
Post
FemaleFranco Arab school
Basic reading pilot school
Post×Female
Post×Franco Arab school
Post×Basic reading pilot
schoolSchool fixed effect (dummy)
Student fixed effect (dummy)
Adjusted R-Squared
Observations(I)
4th / 5th
PooledModel (1-b)
32.582***(2.472)5.761***(1.416)-8.359***(2.133)7.242***(1.098)-2.682**(1.247)-2.908(4.911)-4.979(4.474)-1.100(0.955)-0.276(3.813)-0.257(2.185)
No
No
0.150
1,995

(II)
4th / 5th
PooledModel (1-c)(III)
4th / 5th
Panel: 2t
Model (1-c)

5.999***(1.261)-9.650***(1.576)6.773***(1.080)-3.003***(0.964)

6.139***(1.189)-7.879***(1.743)6.500***(0.919)-2.326*(1.195)-0.593(0.639)-0.552(0.655)

Yes
No
0.865
1,995

Yes
No
0.877
1,384

***

p < 0.01, **p < 0.05, *p < 0.1(1) Robust standard errors are clustered at school level, and in parenthesis. The
variance of standard errors is adjusted by finite-sample correction.
(2) Maximum scale of math test score is 56.

Table 4-1: Regression results of robustness check (1) (raw test score)38

Grade 4 (Treatment) × Post (λ)
Grade 4 (Treatment)
Post
FemalePost×Female
School fixed effect (dummy)
Student fixed effect (dummy)

Adjusted R-Squared
Observations(I)
4th / 5th
Panel: 2t(Sub-sample)
Raw Score
Model (1-a)
4.648***(1.236)-2.823**(1.187)9.766***(0.793)-1.547(1.012)-2.045**(1.003)
Yes
No
0.909
672

(II)
4th / 5th
Panel: 2t(Sub-sample)
Standardized
Model (1-a)
0.295***(0.079)-0.179**(0.075)0.621***(0.050)-0.098(0.064)-0.130**(0.064)
Yes
No
0.458
672

(III)
4th / 5th
Panel: 2t(Sub-sample)
Standardized
Model (2)0.288**(0.107)0.560***(0.069)

No
Yes
0.703
672

***

p < 0.01, **p < 0.05, *p < 0.1(1) Robust standard errors are clustered at school level, and in parenthesis. The variance of standard errors is adjusted
by finite-sample correction.

(2) Maximum scale (full mark) of math test score is 56. In column (B) and (C), test scores are standardized with meanand standard deviation of baseline scores of whole sample of 4th grade students.

(3) Panel data (baseline and end-line) is composed by students with baseline score from 10 to 37 score point (2nd and3rd quartile of 4th grade student).

Table 4-2: Regression results of robustness check (2)39Grade 4 (Treatment) × Post (λ)
Grade 4 (Treatment)
Post
FemalePost×Female
School fixed effect (dummy)
Student fixed effect (dummy)

Adjusted R-Squared
Observations(I)
4th / 5th
Panel: 2t(Sub-sample)
Raw Score
Model (1-a)
5.153***(1.380)-10.362***(1.534)7.043***(1.068)-3.741***(1.237)-0.324(0.820)
Yes
No
0.909
672

(II)
4th / 5th
Panel: 2t(Sub-sample)
Standardized
Model (1-a)
0.328***(0.088)-0.660***(0.098)0.449***(0.068)-0.238***(0.079)-0.021(0.052)
Yes
No
0.472
672

(III)
4th / 5th
Panel: 2t(Sub-sample)
Standardized
Model (2)0.327**(0.123)0.438***(0.082)

No
Yes
0.793
672

***

p < 0.01, **p < 0.05, *p < 0.1(1) Robust standard errors are clustered at school level, and in parenthesis. The variance of standard errors is adjusted
by finite-sample correction.

(2) Maximum scale (full mark) of math test score is 56. In column (B) and (C), test scores are standardized with meanand standard deviation of baseline scores of whole samples of 4th grade students.

(3) Panel data (baseline and end-line) is composed by all of 4th grade students, and sub-sample of 5th grade students ofschools which organized additional classes for the students.

Table 4-3: Regression results of robustness check (3)404th grade baseline test score
Higher Med. Lower Total
Sex

Male
Female
Total

43
47
90

86
93
179

34
57
91

163
197
360

5th grade baseline test score
Higher
Med.
Lower
Total
43
30
73

89
84
173

48
38
86

Table 5: Number of 4th and 5th grade students by test score level and sex

41

180
152
332

Grade4 (Treatment)× Post: DID (λ0)
DID × Lowest
score (λ1L )DID × Medianscore (λ1M )DID × Female
(λ2)

(I)
4th/5thPanel: 2t
Model (1-a)
0.387***(0.075)(II)
4th/5thPanel: 2t
Model (6)0.134
(0.093)0.585***(0.123)0.211***(0.070)(III)
4th/5thPanel: 2t
Model (6)0.341***(0.082)0.084(0.050)(IV)
4th/5thPanel: 2t
Model (7)0.106
(0.106)0.580***(0.123)0.210***(0.070)0.054(0.056)-0.165**(0.074)-0.542***(0.051)0.397***(0.055)-2.122***(0.065)-0.910***(0.047)-0.093*(0.050)-0.549***(0.053)0.397***(0.055)-1.975***(0.051)-0.859***(0.033)-0.112**(0.052)-0.541***(0.051)0.397***(0.055)-2.121***(0.064)-0.909***(0.047)-0.107**(0.050)

Yes
No
0.496
1,384

Yes
No
0.785
1,384

Yes
No
0.759
1,384

Yes
No
0.785
1,384

DID ×Lowest score
×Female (λ3L )DID ×Median score
×Female (λ3M )Grade4 (Treatment)

Post-0.499***(0.111)0.397***(0.055)Lowest score
Median score
FemaleLowestscore×Female
Medianscore×Female
School fixed effectStudent fixed effect
Adj. R2
Observations(V)
4th/5thPanel: 2t
Model (8)0.092
(0.127)0.493***(0.166)
0.269***(0.096)0.084(0.089)0.121(0.173)-0.111(0.119)-0.541***(0.050)0.397***(0.055)-2.194***(0.077)-0.904***(0.058)-0.147***(0.053)0.157(0.110)-0.003(0.066)
Yes
No
0.787
1,384

***

p < 0.01, **p < 0.05, *p < 0.1(1) Robust standard errors are clustered at school level, and in parenthesis. The variance of standard errors isadjusted by finite-sample correction.

(2) All the test scores are normalized with mean and standard deviation of baseline scores of 4th grade students.
Maximum scale (full mark) of math test score is 56.

(3) Lowest score dummy takes 1 for students who obtained baseline score below 1st quartile score of 4th gradestudent baseline score. Median score dummy takes 1 for students who obtained baseline score more than 1stquartile score of 4th grade student baseline score, and below 3rd quartile of the score.

Table 6-1: Heterogeneous impacts of PMAQ on math learning (standardized test score)42

Grade4 (Treatment)× Post: DID (λ0)
DID × Lowest
score (λ1L )DID × Medianscore (λ1M )DID × Female
(λ2)

(I)
4th/5thPanel: 2t
Model (1-c)
0.390***(0.076)(II)
4th/5thPanel: 2t
Model (6-a)
0.219***(0.070)0.272(0.165)0.009(0.077)(III)
4th/5thPanel: 2t
Model (6-a)
0.170**(0.068)0.189**(0.093)(IV)
4th/5thPanel: 2t
Model (7-a)
0.128(0.077)0.268(0.168)0.008(0.080)0.187*(0.094)-0.148**(0.076)-0.099**(0.042)0.137***(0.045)-2.419***(0.062)-1.228***(0.030)-0.100***(0.036)-0.104**(0.043)0.163***(0.038)-2.421***(0.062)-1.230***(0.030)-0.100***(0.036)-0.100**(0.042)0.182***(0.047)-2.420***(0.062)-1.229***(0.030)-0.100***(0.036)-0.035(0.042)0.569***(0.125)0.481***(0.061)-0.052(0.045)0.744***(0.097)0.480***(0.042)-0.145*(0.074)0.563***(0.130)0.482***(0.059)-0.149**(0.072)

Yes
No
0.505
1,384

Yes
No
0.782
1,384

Yes
No
0.783
1,384

Yes
No
0.783
1,384

DID ×Lowest score
×Female (λ3L )DID ×Median score
×Female (λ3M )Grade4 (Treatment)

Post-0.501***(0.111)0.413***(0.058)Lowest score
Median score
FemaleLowestscore×Female
Medianscore×Female
Post × Lowest
score
Post × Medianscore
Post×Female
Post×Lowest score

×FemalePost×Median score

×FemaleSchool fixed effectStudent fixed effect
Adj. R2
Observations(V)
4th/5thPanel: 2t
Model (8-a)
0.123(0.090)0.198(0.194)0.057(0.114)0.194**(0.095)0.063(0.396)-0.086(0.184)-0.101**(0.042)0.163***(0.034)-2.475***(0.057)-1.243***(0.046)-0.139***(0.029)0.118*(0.066)0.036(0.064)0.555***(0.153)0.516***(0.071)-0.110(0.075)0.049(0.393)-0.075(0.120)
Yes
No
0.783
1,384

***

p < 0.01, **p < 0.05, *p < 0.1(1) Robust standard errors are clustered at school level, and in parenthesis. The variance of standard errors isadjusted by finite-sample correction.

(2) All the test scores are normalized with mean and standard deviation of baseline scores of 4th grade students.
Maximum scale (full mark) of math test score is 56.

(3) Lowest score dummy takes 1 for students who obtained baseline score below 1st quartile score of 4th gradestudent baseline score. Median score dummy takes 1 for students who obtained baseline score more than 1stquartile score of 4th grade student baseline score, and below 3rd quartile of the score.

Table 6-2: Robustness check of heterogeneous impacts of PMAQ (standardized test score)43