metadata: authors: 0: author: Hasan,Amer
1: author: Jung,Haeil
count: Indonesia
docna: 0: docna: The impact of early childhood education on
            early achievement gaps : evidence from the Indonesia early
            childhood education and development (ECED) project
docty: Policy Research Working Paper
subtopic: Children and Youth,Health Care Services Industry,Social Protections & Assistance,Disability,Reproductive Health,Early Child and Children's Health,Nutrition,Early Childhood Development,Educational Sciences,Services & Transfers to Poor,Economic Assistance,Access of Poor to Social Services
teratopic: Governance,Urban Development,Communities and Human Settlements,Education
abstracts: cdata!: This paper assesses whether the
            Indonesia Early Childhood Education and Development project
            had an impact on early achievement gaps as measured by an
            array of child development outcomes and enrollment. The
            analysis is based on longitudinal data collected in 2009 and
            2010 on approximately 3,000 four-year-old children residing
            in 310 villages located in nine districts across Indonesia.
            The study begins by documenting the intent-to-treat impact
            of the project. It then compares the achievement gaps
            between richer and poorer children living in project
            villages with those of richer and poorer children living in
            non-project villages. There is clear evidence that in
            project villages, the achievement gap between richer and
            poorer children decreased on many dimensions. By contrast,
            in non-project villages, this gap either increased or stayed
            constant. Given Indonesia's interest in increasing
            access to early childhood services for all children, and the
            need to ensure more efficient spending on education, the
            paper discusses how three existing policies and programs
            could be leveraged to ensure that Indonesia's vision
            for holistic, integrated early childhood services becomes a
            reality. The lessons from Indonesia's experience apply
            more broadly to countries seeking to reduce early
            achievement gaps and expand access to pre-primary education.
geo_reg_key: 517198,517191
repnme: repnme: The impact of early childhood education on
            early achievement gaps : evidence from the Indonesia early
            childhood education and development (ECED) project
display_title: The impact of early childhood
            education on early achievement gaps : evidence from the
            Indonesia early childhood education and development (ECED) project
listing_relative_url: /research/2014/02/19154785/impact-early-childhood-education-early-achievement-gaps-evidence-indonesia-early-childhood-education-development-eced-project
topicv3: Child Development m1327025 2151,Education m1327048 470,Early Childhood Education m1327440 1612,Health m1504661 3480,Early Childhood m1582438 4408
publishtoextweb_dt: 2014-02-01T00:00:00Z
historic_topic: Governance,Urban Development,Communities and Human Settlements,Education
no_of_pages: 40
author_keys: 0: author_key: 807987
1: author_key: 1201240
model_extractions: WPS6794 Policy Research Working Paper 6794 The Impact of Early Childhood Education on Early Achievement Gaps Evidence from the Indonesia Early Childhood Education and Development (ECED) Project _Haeil Jung Amer Hasan_ The World Bank East Asia and the Pacific Region Education Sector Unit February 2014
Policy Research Working Paper 6794

# **Abstract**

This paper assesses whether the Indonesia Early Childhood Education and Development project had an impact on early achievement gaps as measured by an array of child development outcomes and enrollment. The analysis is based on longitudinal data collected in 2009 and 2010 on approximately 3,000 four-year-old children residing in 310 villages located in nine districts across Indonesia. The study begins by documenting the intent-to-treat impact of the project. It then compares the achievement gaps between richer and poorer children living in project villages with those of richer and poorer children living in non-project villages. There is clear evidence that in project villages, the achievement gap between richer and poorer children decreased on many dimensions. By contrast, in non-project villages, this gap either increased or stayed constant. Given Indonesia’s interest in increasing access to early childhood services for all children, and the need to ensure more efficient spending on education, the paper discusses how three existing policies and programs could be leveraged to ensure that Indonesia’s vision for holistic, integrated early childhood services becomes a reality. The lessons from Indonesia’s experience apply more broadly to countries seeking to reduce early achievement gaps and expand access to pre-primary education.

This paper is a product of the Education Sector Unit, East Asia and the Pacific Region. It is part of a larger effort by the World Bank to provide open access to its research and make a contribution to development policy discussions around the world. Policy Research Working Papers are also posted on the Web at http://econ.worldbank.org. The authors may be contacted at ahasan1@worldbank.org.

_The Policy Research Working Paper Series disseminates the findings of work in progress to encourage the exchange of ideas about development issues. An objective of the series is to get the findings out quickly, even if the presentations are less than fully polished. The papers carry the names of the authors and should be cited accordingly. The findings, interpretations, and conclusions expressed in this paper are entirely those of the authors. They do not necessarily represent the views of the International Bank for Reconstruction and Development/World Bank and its affiliated organizations, or those of the Executive Directors of the World Bank or the governments they represent._ Produced by the Research Support Team
The Impact of Early Childhood Education on Early Achievement Gaps: Evidence from the Indonesia Early Childhood Education and Development (ECED) Project<sup>∗</sup> Haeil Jung Indiana University Bloomington Amer Hasan World Bank World Bank Sector Board: EDU JEL Classifications: I21; I25; I28

> ∗ This paper builds on joint work with Sally Brinkman, Marilou Hyson, Angela Kinnell, and Menno Pradhan presented in Hasan, Hyson and Chang, (eds.), 2013. The registration number of the trial on which this paper is based is ISRCTN76061874. Preparation of this document received partial funding from the Government of the Kingdom of the Netherlands through the Dutch Basic Education Support Program. We would like to thank Luis Benveniste, Marilou Hyson, Menno Pradhan, Samer Al-Samarrai and Heather Tomlinson for detailed feedback on an earlier version of this paper. All errors are our own. The findings, interpretations, and conclusions expressed in this paper are entirely those of the authors. They do not necessarily represent the views of the International Bank for Reconstruction and Development/World Bank and its affiliated organizations, or those of the Executive Directors of the World Bank or the governments they represent.
# **Introduction**

In 2002, Indonesia made a clear commitment to education – passing a constitutional mandate to allocate at least 20 percent of the total government budget to education. This has led to a large increase in resources, more than doubling education spending in real terms. Despite such substantial financial commitments to education as a whole, a number of challenges persist. In particular, ensuring equitable access to early childhood education and development (ECED) services has historically been a challenge in Indonesia.

To put the nature of the challenge in context, consider that in 2007 the gross enrollment rate in early childhood services for children between the ages of four and six was 23 percent. However, this number masks large differences in the probability of access depending on background. Figure 1 shows differences in the probabilities of access to early childhood services for Indonesian children of various backgrounds using 2007 data from a nationally representative household survey (SUSENAS). On average, a four-year-old child from the richest 25 percent of households in Indonesia had a 33 percent probability of accessing early childhood services while a four-year-old child from the poorest 25 percent of households in Indonesia had an 8 percent probability of accessing such services.

Such inequalities also exist in child development outcomes. **Error! Reference source not found.** shows the average scores of a sample of rural Indonesian children aged 48-60 months on three different measures of child development: the language and cognitive development domain of the Early Development Instrument (EDI), the draw-a-human task, and a test of executive function – the Dimensional Change Card Sort game. In each panel, the performance of children between the ages of 48 and 60 months from richer households (those with a wealth index<sup>1</sup> value above the mean) is compared to that of children from poorer households (those with a wealth index value below the mean). The data shown here suggest that child development among children from poorer households starts to plateau around the age of 54 to 56 months (four and a half years) whereas children from richer households continue to register improvements in their developmental outcomes as they get older.

> 1 The wealth index is derived from a principal components analysis of assets owned by households and is the score of the first principal component normalized to have a mean of zero and a standard deviation of one.

2
In an effort to address such inequalities in access and development and given the government’s appreciation of the importance of the early years in laying the foundation for future human capital development, in 2008 the government launched an Early Childhood Education and Development Project (ECED) in 3,000 poor villages in 50 districts across Indonesia. This paper follows a cohort of four-year-old children that live in 310 villages in nine of these districts. Approximately 200 of these villages were randomly assigned to receive project services according to a phased-in schedule. This paper examines the impact of the project on enrollment as well as on an array of child development outcomes and assesses the degree to which this project was able to reduce gaps in child development between richer and poorer children. Henceforth we refer to this gap as the early achievement gap.

The paper is structured as follows: we first review the growing literature on the impacts of early childhood services on children’s development with a focus on those studies that look at early achievement gaps. Next we describe the Indonesian government’s approach to increasing access to early childhood services through the ECED project. A randomized impact evaluation sought to capture the impacts of this project. We describe how the evaluation was designed as well as how project implementation led to deviations from the design. We present statistical models that, under certain assumptions, allow us to deal with these deviations from design and still causally attribute the impacts we observe to the ECED project. The section on empirical results employs both experimental and non-experimental methods. Given our interest in assessing the degree to which this project was able to reduce early achievement gaps in child development, we compare gaps between children from richer and poor households in villages that received the ECED project to similar gaps among children from villages that did not receive the project. The final section concludes with three specific suggestions for how current policies and programs could be leveraged to ensure that all children throughout Indonesia are able to access and benefit from early childhood services.

# **Literature Review**

Several researchers have found evidence of steep socioeconomic gradients in cognitive development at early ages in countries as diverse as Bangladesh, Brazil, Ecuador, Egypt, India, Mexico, and the Philippines (See Bouguen et al. 2013 and the references cited therein). For example, Paxson and Schady (2007) investigate trends in child development in a sample of 3
disadvantaged Ecuadorian children using the Peabody Picture Vocabulary Test to show that while the differences in age-adjusted scores of three-year-old children are generally small, by age six children in the poorest 25 percent of households have fallen far behind children in the richest 25 percent of households. Berlinski, Galiani, and Manacorda (2008) also note that the impact of preschool attendance was largest for those children from households with less education.

Most of the studies we reviewed have documented the impact of an early intervention on child development but few assessed the degree to which such interventions had differing impacts on children from different socio-economic backgrounds. Burger (2010) is an exception in this regard but even his focus is largely on examples from the US, Canada and Europe.<sup>2</sup> He analyzes the impacts of early interventions on cognitive development – one of several indicators of successful child development – and attempts to categorize programs by whether or not they had differential effects on children from different backgrounds.

While Cunha and Heckman (2007) and Heckman and Masterov (2007) have underscored the fact that early interventions not only promote economic efficiency but reduce lifetime inequality, few papers have systematically documented whether early childhood interventions in developing country settings are able to reduce early achievement gaps.

In this paper, we assess whether a large community-driven program in a middle-income country which aimed to increase access to early childhood education and development services in rural areas was effective in reducing early achievement gaps. In doing so, this paper contributes to the rapidly growing literature that has documented the benefits of intervening early in life to improve health and education outcomes and attempts to add to the literature on inequality in early childhood.

Evidence from around the world has shown that the programs most successful at improving outcomes share several common features which include beginning at birth, involving families, and targeting the poorest children (Naudeau et. al., 2011a; Naudeau et al., 2001b). Intensive interventions which are long-lasting and holistic in that they include health, nutrition, and parenting services in addition to education have been shown to have the largest and most long-lasting impacts.

In the U.S. this evidence draws on the experiences of programs such as Perry Preschool (a half-day preschool program which children attended for two years), the Abecedarian project (a

> 2 The Vietnam case in his study did not allow for sub-group comparisons.

4
full-day, year-round, center-based care from infancy until kindergarten entry) and Head Start. Each of these programs targets poor families and varies in their modes of delivery and intensity. The programs generally show positive effects, but not for all outcomes at all stages. For instance, Currie and Thomas (2000) found that children who participated in Head Start did better in school than their siblings who did not – a finding replicated in Deming (2009) as well as Carneiro and Ginha (2009). However, for longer term outcomes such as employment the evidence is more mixed.<sup>3</sup> In developing countries, a variety of center-based early childhood education and development services have been introduced in a multitude of contexts. Evaluations have found mixed evidence on the benefits of these services. For example, an evaluation from Rio Grande do Sul in Brazil (Young, 2012) compared children before and after entry into the program and showed significant cognitive, social, and motor development. Likewise, a comparison between children in the same communities who participated in the program with those that did not showed strong gains in all of these dimensions (Primeira Infância Melhor, 2011). In Mozambique, Martinez, Naudeau and Pereira (2011) found improvements in primary school enrollment rates, hours per week spent on homework as well as in a number of child development outcomes as a result of preschool attendance over a two-year period. In contrast, a study on the impacts of alternative types of ECD provision in Cambodia (Bouguen et al., 2013) found little difference in development outcomes when comparing outcomes over a three-year period. More comprehensive reviews of evidence from low-income (Engle et al., 2007) and middle-income (Vegas and Santibáñez 2010) settings exist and underscore the variety of results reported here.

# **The Indonesia Early Childhood Education and Development (ECED) Project**

# **_Background_**

ECED services in Indonesia are intended to cater to children from birth to age six. As such, they take up a variety of forms and are overseen by various ministries (Table 1). Kindergartens are regulated either by the Ministry of Education and Culture (MoEC) or by the Ministry of Religious Affairs (MoRA) depending on whether they are regular (TK) or Islamic

> 3 See the summary of various papers presented in Table 4.1 of Alderman, 2011.

5
kindergartens (RA). MoEC regulates Playgroups (KB) while Toddler Family Groups (BKB) are overseen by the National Family Planning Board. Day care centers (TPA) are also overseen by MoEC and are largely urban phenomena. Children up to the age of five generally go to an integrated health service unit called a Posyandu, some of which provide a preschool program (Pos-PAUD). Once children have reached the age of seven, they are expected to begin primary school.

Historically, there has been a distinction drawn between formal and non-formal early childhood services. Since 2010, this distinction has been done away with at MoEC (at least on paper). Now all early childhood services are under the purview of one Directorate General at the Ministry of Education and Culture. In practice, however, the distinction between formal and nonformal services continues with different types of services and teachers eligible for different forms and levels of support from the government.

Each type of early childhood service is intended to cater to children of a specific age ( **Error! Reference source not found.** ). In practice, these age cut-offs are hard to enforce. Some children may continue in playgroups past the intended age of 4 and others may enroll in primary school at the age of six or even five. The incentive to do so is strong since attending kindergarten is not mandatory and most kindergartens charge fees while primary school is compulsory and free.

While there are a variety of early childhood services in Indonesia, the provision of these services has historically been characterized by: 1) low levels of coverage overall and especially for children from birth to age three; 2) largely private provision of services in the face of low levels of public investment; and 3) volunteer teachers with little or no training since very few institutions provided training for early childhood teachers.

# **_The Scope of the Project: Selecting the Districts_**

In an effort to address some of these challenges, the government of Indonesia in partnership with the World Bank and the Dutch government developed the ECED project. The goals of the project were to increase access to early childhood services and increase children’s readiness for school in relatively poor districts with generally low ECED participation. Under this project, districts were selected on the basis of a composite score based on poverty rates, gross enrollment rates, Human Development Index (HDI) rankings, geographical remoteness, 6
whether or not they are border districts<sup>4</sup> and district assurance of being “committed” to early childhood services.

There were 422 districts in Indonesia at the time of the study design. The composite score was used to select 50 districts (12 percent of all districts) for inclusion in the project. Within each of these districts, 60 priority villages were identified on the basis of their poverty rate, a sufficiently large population of children between the ages of 0–6, a sufficiently large overall population and the village’s willingness to contribution financially to the project. Consequently project services are being implemented in 3,000 villages (4 percent of all 69,000 villages in the country).

# **_The Intervention_**

Each district that participated was required to set up a district early childhood services office. Each village that participated in the project received the following:

1. The services of a community facilitator whose job was to raise community awareness on the importance of early childhood services and share information on the benefits available under the project. Community facilitators also provided communities with training on how to prepare proposals for block grants available under the project.

2. Block grants for three years in the amount of US $18,000 per village which were to be spent on establishing two centers. Thus villages received US$3,000 per center per year for three years. These funds came with the requirement that no more than 20 percent could be spent on building new infrastructure. This limit meant that most of the centers established under the project involved rehabilitating existing buildings rather than constructing new ones. The remaining 80 percent could be spent on learning activities, health and nutrition and management and administration of the center.

3. Teacher training of 200 hours duration for two teachers per center. Teacher training was delivered via a cascade training model. Teachers were predominantly women from the village who often had children of their own. Some had prior work experience in health and education. Others had no such prior experience.

While the original intent of the project had been to offer services to all children ages 0–6, in practice, the most common form of service provision selected by communities was the

> 4 The government of Indonesia designates some of its poorest performing districts as 3T districts: _Terpencil_ (isolated/remote), _Terluar_ (border), _Tertinggal_ (lagging).

7
establishment of a playgroup for four to six-year-olds, typically offered two hours a day, three times a week. This package (community facilitation, block grants, teacher training and playgroups) is effectively the intervention evaluated in this paper.

# **_The Evaluation Design_**

Given the scope of the project, the government originally decided to roll out implementation in three batches roughly nine months apart. Thus during the design of the evaluation the decision was made to randomly assign villages to planned phases of the project. Analysts worked with the government to hold a public lottery and assign some villages randomly to the first batch - batch 1 (100 villages) and some villages to the last batch - batch 3 (120 villages). Villages in the second batch (batch 2) were not included in the evaluation since it was felt that nine months was too little time to gauge the impact of such an intervention. Thus according to this design some villages were randomly assigned to control group status but only for 18 months. At the end of this period, batch 3 villages also received the intervention. Given that after this period, there would be no villages without the intervention, analysts also collected information on a group of villages that would never receive the project (90 villages). These villages were not randomly selected. Instead they were identified by government officials as being similar to the villages that had been randomly assigned to batches 1 and 3. Together these three groups of villages (batch 1, batch 3 and villages that never received the project) constitute the source of the data used in the impact evaluation.

# **_The Evaluation in Practice_**

Due to issues with timing of budget disbursements, in reality the design above was not adhered to. In reality the government rolled out the program in two phases. The first roll-out comprised what was originally referred to as batch 1 and roughly half of batch 2. The second roll-out comprised the remaining half of batch 2 and batch 3. Given that project implementation deviated from its original design, for the purposes of this evaluation, we use actual date of funds disbursal to confirm when villages actually received the project. As a result, we end up with 105 villages in batch 1 and 112 villages in batch 3. The remaining 93 villages never received the project. Thus the evaluation in practice only has small deviations from the evaluation in design.<sup>5</sup>

> 5 See tests of balance between original and actual batches as reported in Hasan, Hyson and Chang (eds.), 2013.

8
**Error! Reference source not found.** depicts the evaluation as it was designed and as it was implemented.

# **The Data: The Surveys, the Measures Collected, and the Children**

# **_The Timing of Survey Data Collection_**

The analysis in this paper is based on two rounds of data – a baseline survey which was conducted from March to June 2009 and a follow-up survey which was conducted approximately 14 months later from July to August 2010. There were discrepancies between the timing of the surveys and project implementation – these are depicted in Figure 4. As a result, villages in batch 1 had already implemented the ECED project for about 6 months by the time the baseline survey was fielded. Likewise, villages in batch 3 (which had not received the ECED project at baseline) started implementation about nine months before the follow-up survey was fielded. Villages without the project had no services provided through the project either at baseline or at follow-up. In each village, data were collected on households, caregivers and the development of two cohorts of children.

# **_The Measures_**

Child development was assessed using an extensive array of internationally-validated and locally-adapted<sup>6</sup> child development measures including:

1. The short version of the Early Development Instrument (EDI)

2. The Strengths and Difficulties Questionnaire (SDQ) 3. Height and weight measurements 4. Demonstrations of child skills (and mother reports of these skills), drawing on a study conducted by the University of San Carlos Office of Population Studies 5. Tests of executive function using the Dimensional Change Card Sorting (DCCS) game 6. Drawing tasks (based on the Draw-a-Man test) 7. Expressive and receptive language tasks Several of the measures above assess multiple domains of child development. A higher score is better for all measures except for the Strengths and Difficulties Questionnaire (SDQ) where lower scores are better and represent fewer difficulties.<sup>7</sup>

> 6 For details on how instruments were adapted and tested, see Pradhan et. al. (2013).

9
# **_The Children_**

The original raw data contain information on two cohorts of children in these 310 villages: children who were one year old and those who were four years old in 2009 (when baseline data were collected). These children were followed and re-observed in 2010 when they were two and five years old, respectively. Given the project’s focus on children between birth and the age of six, analysts considered that following these children would allow us to understand the trajectory of child development in the age range of interest: 0–6.

This paper focuses on whether the package of interventions (community facilitation, block grants, teacher training and playgroups) introduced by the government had any impact on children’s development and whether it helped reduce early achievement gaps between children from richer and poorer households. Consequently, we exclude from our analysis the one-year-old children as they were too young to have attended the playgroups. Thus, we form our analysis sample consisting of children who were four years old in 2009 (baseline year) and likely to have attended playgroups. In the sample of four-year-old children in the baseline survey, we have 1,109 children in batch 1 villages, 1,184 children in batch 3 villages, and 940 children in villages that never received the project. 49.7 percent of these children are girls and have an average age of 53 months when they are first observed. Their caregivers are on average 32 years of age and on average report being the primary caregiver for four children.

Table 2 shows the descriptive statistics for enrollment and child development outcomes. We organize table into columns showing the means and standard deviations for each group of villages: batch 1, batch 3 and villages without the project (henceforth the comparison group) and by baseline and follow-up for each group to examine their similarities and differences. Our ECED enrollment variable is set to 1 if children had enrolled in any ECED service at baseline or follow-up. Otherwise, it is set to 0. We standardize most child development outcomes with mean 0 and standard deviation 1 among all children across baseline and follow-up. This allows us to interpret the progress in different child outcomes as a result of the ECED project in the same metric. The only exception is the test of executive function which required children to play a three-stage Dimensional Change Card Sorting game. This card sorting game is scored as 1 if the

> 7 This is true of all SDQ domains. Typically one domain is reverse-scored but in this paper, we have aligned all SDQ domains in the same direction for ease of interpretation. Appendix Table 1 provides details on each of these measures of development.

10
children pass at least one stage and 0 otherwise. Thus ECED enrollment and the test of executive function are measured in percentage points while all other outcomes are measured in standard deviation units. The visible patterns are as follows.

First, at baseline most of the outcomes for children in batch 1 villages are better than those for children in batch 3 villages and for children in villages without the project. This is to be expected because batch 1 villages had been receiving the ECED project services for about 6 months at baseline. Baseline ECED enrollment for batch 1 children is 61% - much higher than for batch 3 children (34%) and children in villages without the project (36%).

Second, at baseline most outcomes for children in batch 3 villages are similar to those for children in villages without the project. As we discussed earlier, a group of villages was identified by government officials to make a comparison group for batches 1 and 3. Because of the early implementation of the program in batch 1, we cannot compare batch 1 villages with this comparison group of villages. However, we can check whether batch 3 villages are comparable to these villages without the project in terms of enrollment and child outcomes. We find that they are very similar to each other at baseline. None of the outcomes are statistically different between two groups. The ECED enrollment at baseline is 34% and 36% for batch 3 villages and those without the project, respectively and other child outcomes are also very similar. The close comparability of these two groups of villages is highly encouraging for us to use a difference-indifferences (DiD) approach to examine the impact of the ECED project on enrollment and child outcomes.

Third, for children in all batches most outcomes show improvements during the time that elapsed between baseline and follow-up. In other words, children grew up and measures of their development improved even in the absence of the project. ECED enrollment also increases over time, even for villages that did not receive the ECED project. This reminds us that we need to control for improvements in outcomes driven only by aging. Therefore, we have a time dummy in the DiD model to control for the effect of age. Our DiD estimators yield the exclusive impact of the ECED project after accounting for the effect of age on child development.

In Table 3, we compare key characteristics of children who were not enrolled in ECED services at baseline depending on whether they live in project villages or non-project villages and examine their families’ enrollment decisions at the time of the follow-up. We note that at baseline, children who did not enroll in ECED services have very similar levels of low parental 11
education and household wealth irrespective of which village they live in (last row). Two-thirds of these children have mothers and fathers with less than an elementary education and their household wealth is below average. Next, we examine the enrollment status of these children at follow-up. In non-project villages, parents only have two choices – continue to keep their children un-enrolled or enroll them in an existing ECED service. In project villages there are three choices– continue to keep children un-enrolled, enroll them in an existing ECED service or enroll them in a project-provided ECED service. Looking at the average characteristics of children who were not enrolled in baseline (last row), those who were never enrolled in either baseline or follow-up (row 1) and those that chose to enroll at follow-up in a project-provided ECD service (row 3) suggests that those who attended project services are poorer (their wealth z- score is -0.19) than the average child who was not attending at baseline (their wealth z-score is - 0.13), but not as poor as the child who never enrolls (their wealth z-score is -0.43). Similarly, children who decide to enroll in existing ECED services in non-project villages are much better off (their wealth z-score is 0.11). Taken together, the evidence in Table 3 seems to suggest that project-provided services do a better job of reaching the moderate poor than non-project provided services. However, there is little indication that the extreme poor are being adequately reached.

# **Statistical Model**

We use two main analyses in evaluating the impact of the ECED project on child outcomes. First, using batch 3 children as the treatment group and children in non-project villages as the comparison group, we estimate the impact of the ECED project using the Difference-in-Differences (DiD) method. The basic idea of the DiD method is to follow two groups that are similar at baseline and to estimate the difference in outcomes at follow-up after an intervention on one group and no intervention on the other. We take advantage of the fact that batch 3 children had no intervention at baseline, but had nine months of intervention at follow-up, while villages without the project had no intervention either at baseline or at follow-up.

The regression model is as follows.

(1) yit = β0 + Diδ1+Ttδ2 + DiTtδ3 + xiβ+ uit.

yit is enrollment (1 if ever-enrolled and 0 if never-enrolled) or child development outcomes at baseline ( _t_ =1) and follow-up ( _t_ =2) for a child _i_ . Di is the dummy variable indicating 1 for batch 3 12
and 0 for the comparison group while Tt is the dummy variable indicating 1 for baseline and 0 for follow-up. Thus, δ1 captures the difference between batch 3 and villages without the project at the baseline. If villages in the comparison group are comparable with those from batch 3 in terms of child development outcomes and ECED enrollment, this estimate should be close to 0. On the other hand, δ2captures the age effect on child outcomes, which includes the advancement of children’s outcomes and ECED enrollment as they get older regardless of the ECED project. The time difference is about 14 months. DiTt is the interaction term between the intervention group dummy and the time dummy. Thus, δ3 indicates the impact of the ECED project. xi is a vector of explanatory variables that include caregiver, household, and child characteristics.<sup>8</sup> The estimated impact of the project is not on children who are necessarily enrolled in the projectprovided services but rather it is on children who were offered the chance to enroll in the projectprovided services. This estimated impact is more relevant for policy makers since most social programs are based on voluntary participation of eligible individuals. We use robust standard errors clustered at the village level. The key identification assumption is that at the time of follow-up differences in outcomes between villages that received the project in batch 3 and the comparison group should be the same as those at baseline if there is no ECED intervention or if the intervention has no effect. We also run a fixed effects model on regression (2), which controls for all observed and unobserved time-invariant child characteristics.

Second, we use the random assignment of villages into the treatment (batch 1) and control (batch 3) groups to estimate the impact of different intervention durations on child outcomes. At the time of the baseline survey, children in batch 1 villages had already received the treatment for about 6 months and children in batch 3 villages had not yet received treatment. However, at the time of the follow-up survey, batch 1 villages had been running the new ECED services for about 20 months and batch 3 villages had been running them for about nine months. Taking advantage of the different timing of program implementation and survey, we estimate the impact after six months of project implementation using batches 1 and 3 at baseline and estimate the impact of nine months versus 20 months of the project implementation using batches 1 and 3 at follow-up.

> 8 We control for caregiver, household, and child characteristics as follows; sex, age, education, health, literacy, and number of children of a caregiver; sex, age, education, literacy, and health of a household head and wealth, neighborhood, marital status, religion, and social integration of a household; sex, health, and age in months of a child.

13
The OLS regression model in these cases is as follows:

- (2)     yi = β0 + Diδ+ xiβ+ ui.

yi is enrollment (1 if ever-enrolled and 0 if never-enrolled) or child development outcomes at baseline or follow-up for a child _i_ . Di is the dummy variable indicating 1 for batch 1 and 0 for batch 3. We run this model for baseline and follow-up separately. Assuming that the randomization is valid, the estimated δ yields the unbiased impact after six months of project implementation at baseline and the unbiased impact of nine months versus 20 months of the project implementation at follow-up. xi is a vector of explanatory variables that include caregiver, household, and child characteristics.<sup>9</sup> Again, the estimated impact of the ECED project is not on children who enrolled in project-provided services but on children who were offered the chance to do so. We use robust standard errors clustered at the village level.

Using these two analytical methods, we examine changes in enrollment and child outcomes for all children who were four years old at baseline (age five at follow-up). We then examine two subgroups of interest, (a) children from poor households whose wealth is less than the average at baseline and (b) children who never enrolled in any existing ECED services at baseline.

In the difference-in-differences approach we examine how the development of children who live in project villages (specifically batch 3 villages) differs from children in non-project villages as they go from age four to age five. In the simple difference approach relying on the randomized assignment of villages to batch 1 or batch 3, at baseline we are comparing the differences in development of four-year-olds when some villages have received an ECED intervention for six months while others have not. Likewise at follow-up we are comparing the differences in development of five-year-olds when some villages have received an ECED intervention for 20 months while others have received it for nine months.

> 9 We control for caregiver, household, and child characteristics as follows; sex, age, education, health, literacy, and number of children of a caregiver; sex, age, education, literacy, and health of a household head and wealth, neighborhood, marital status, religion, and social integration of a household; sex, health, and age in months of a child.

14
# **Empirical Results**

# **_DiD Estimation Results Using Batch 3 Villages and Those without the Project_**

Figure 5 and 6 summarize our significant findings from the DiD estimators using a fixed effects model.<sup>10</sup> We present the impact of the ECED project on enrollment in any early childhood services as well as on eight other child development outcomes (social competence, language and cognitive development, communication and general knowledge, language tasks, draw-a-house task, test of executive function using the dimensional change card sort game, conduct problems, and pro-social behavior). We only include noticeable and statistically significant findings.<sup>11</sup> We present results for (a) all children (regardless of whether they were enrolled in any ECED service at baseline), (b) children who were not enrolled in pre-existing ECED services at baseline, and (c) children from poorer households – those households that report below average wealth.<sup>12</sup>

# **_Enrollment_**

Figure 5 reports the estimated impact of the ECED project on enrollment. Each bar represents our estimates of a different group of children: all children (solid bar) and poor children (shaded bar). We further disaggregate these children on the basis of their enrollment status at baseline. The left-hand set of bars shows the estimated impact regardless of enrollment status at baseline and the right-hand set of bars shows the estimated impact for the subset of these children who never enrolled at baseline. As shown in this figure, children from poor households made more progress in ECED enrollment at follow-up than all children. Also, children who never enrolled at baseline are more likely to enroll at follow-up. As a result, the impact of the ECED project on enrollment is about 6 percentage points higher for poorer children who were never enrolled at baseline (15.6 percent) than for all four-year-old children (9.7 percent).

> 10 All other regression estimates are available upon request.

> 11 In total we analyzed 22 outcomes including ECED enrollment, child development outcomes, nutrition outcomes and parenting practices.

> 12 Household wealth is measured using principal components analysis. A wealth measure is created using the score of the first principal component derived from a series of questions on asset ownership. The resulting score is standardized to have mean 0 and standard deviation 1. Poor households are those with a standardized wealth below 0.

15
# **_Child Development Outcomes_**

In Figure 6, we present the estimated impact of the ECED project on six child outcomes. Each child outcome has estimates for four groups as in Figure 5 based on ECED enrollment at baseline and household wealth. For the domains of social competence, language and cognitive development, and executive function (card sorting), we see a greater impact of the ECED project on children who were not enrolled at baseline or those who are from poor households when compared to all four-year-olds. For example, the impact of the ECED project on social competence is about 0.2 standard deviations larger for children who never enrolled and are from poor households than the impact for all four-year-old children. Children who never enrolled or are from poor households also have significant and considerable improvements as a result of the ECED project in communication and general knowledge, language tasks, and the draw-a-house task. Figure 7 presents two domains of the SDQ: conduct problems and pro-social behavior. As we mentioned earlier, negative scores in these domains indicate fewer difficulties and therefore improvements in child development. The ECED project has little impact on pro-social behavior for all four-year-old children but it greatly reduces pro-social behavior problems of children from poor households (-0.215 standard deviations). The reduction in pro-social behavior problems of poor children who were not enrolled at baseline is even larger (-0.287 standard deviations).

# **_Corroborating Estimation Results Using Randomized Assignment of Batches 1 and 3_**

In Figure 8, 9 and 10 we present the estimated impact of the ECED project on enrollment and child development outcomes using the randomization used to assign villages to batch 1 and batch 3. These estimates corroborate our difference-in-difference findings. We only include noticeable and statistically significant findings.<sup>13</sup> As we mentioned earlier, we estimate the difference between batches 1 and 3 controlling for household, caregiver and child characteristics at baseline and follow-up. At baseline, batch 1 had implemented the ECED project for six months and batch 3 had not started their program yet. At follow-up, batch 1 had implemented the ECED project for about 20 months and batch 3 had implemented the ECED project for about nine months. As a result, the estimated difference between the two batches at baseline indicates the estimated impact of six months of the ECED project and the estimate difference between the two batches at follow-up indicates the estimated impact of nine months versus 20 months of

> 13 All other regression estimates are available upon request.

16
project implementation. The estimated differences between the two batches at baseline and follow-up are reported side by side for all children and poor children. It is not possible to present results for never-enrolled children because the intervention had already happened in batch 1 villages for six months at time of the baseline.

Consequently, as shown in Figure 8, at baseline, the differences in enrollment between the two batches are about 25 to 26 percentage points for all children and poor children, respectively. This represents the impact of 6 months of implementation of the ECED project. It is larger than the estimated impact using the DiD estimators, which was about 10 to 20 percentage points and represented the difference between villages that had received the project for nine months when compared to villages that had not received the project. The estimated impact after the first six months is slightly larger for poorer subgroups of children than all children in general. At follow-up, the difference between batches 1 and 3 is much smaller - about 6 to 10 percentage points. This appears to be driven by the different durations of program implementation, nine months versus 20 months.

Figure 9 reports the impact of the ECED project on standardized child outcomes in a manner similar to Figure 6. The only statistically significant estimates of impact are in the social competence domain. At baseline, the estimated impact on social competence is about 0.13 and 0.20 standard deviations for all children and poor children, respectively. This is the impact of six months of implementation in batch 1 villages. This is compatible with the estimated impacts on social competence reported in Figure 6. However, the estimated difference is negative or close to 0 at follow-up for both groups - though they are relatively small and statistically not significant. Similarly, we have a small negative impact of the ECED project on communication and general knowledge at follow-up. This implies that at follow-up children in batch 3 villages are doing better in terms of communications and general knowledge than children in batch 1 villages. Figure 10 presents the impact on two domains under the SDQ, conduct problems and pro-social behavior, as in Figure 7. There are no statistically significant differences between batch 1 and batch 3 villages either at baseline or at follow-up.

Overall the results obtained using experimental methods (i.e. by comparing children in batch 1 and batch 3 villages at baseline and then again at follow-up) corroborate the findings of the non-experimental methods (comparing the progress of children in batch 3 villages between baseline and follow-up with the progress of children in comparison group villages).

17
Having established that this intervention was able to register an impact on enrollment and children’s development, we turn next to the question of whether this impact was sufficiently large to help narrow early achievement gaps between richer and poorer children.

# **The ECED Project and Early Achievement Gaps between Richer and Poorer Children**

Our findings in the previous sections suggest that children from poorer households make more progress in a number of child outcomes compared to all children. Thus, in order to examine the extent to which the ECED project is able to reduce early achievement gaps, we investigate whether the project narrows the gap in child outcomes between children from richer households (those with a wealth Z-score ≥ 0) and poorer households (those with a wealth Z-score <0).<sup>14</sup> We modify our DiD model to estimate the progress made by poor children in batch 3 villages compared to those in the comparison group without the ECED project. We interact the DiD variables with an indicator variable for poorer children as follows: (3) yit = β0 + Diδ1+Ttδ2 + Piδ3 + DiTtδ4 + PiTtδ5 + DiPtδ6 + DiTtPiδ7 + xiβ+ uit.

As in equation (1), yit is enrollment (1 if ever-enrolled and 0 if never-enrolled) or child development outcomes at baseline ( _t_ =1) and follow-up ( _t_ =2) for a child _i_ . Di is the dummy variable indicating 1 for batch 3 and 0 for the comparison group while Tt is the dummy variable indicating 1 for baseline and 0 for follow-up. Pi is the dummy variable indicating 1 for poorer children and 0 for richer children. Thus, δ1 represents the difference between batch 3 and the comparison group, δ2 represents the progress due to age, δ3 represents the difference between poor and rich children at the baseline in the comparison group, and their interactions are captured in δ4 to δ6. In this regression model, δ7 captures the progress made by poor children in batch 3 because they were exposed to the ECED project.

Figure 11 depicts several observations from such an analysis. First, it shows (A) the change in the rich-poor gap in child outcomes between baseline and follow-up for children in batch 3 villages. This gap decreases for enrollment and most child outcome measures. More specifically, for children in batch 3 villages, the rich-poor gap in social competence, communications and general knowledge and pro-social behavior decreases between baseline and

> 14 Throughout this paper – rich or poor should be read as relatively richer and poorer. All households in this sample are rural Indonesian households and are broadly representative of the typical rural Indonesian household which is not rich by most definitions. However, within these rural households there are those that are relatively richer and those that are relatively poorer.

18
follow-up. Second, it shows (B) the change in the rich-poor gap between baseline and follow-up for children in comparison villages. Here we observe that with one exception (card-sorting) on all dimensions the rich-poor gap increased between baseline and follow-up. Third, it shows (A) – (B) the difference between the change in the rich-poor gap for children in batch 3 villages and the change in the rich-poor gap for children in comparison villages. (A) is represented by the estimated δ 5<sup>+δ</sup> 7<sup>, (B) is represented by the estimatedδ</sup> 5<sup>, and (A) – (B) is represented by the</sup> estimated δ 7<sup>. Thus, assuming that the change in the rich-poor gap between baseline and follow-</sup> up in comparison villages is a valid counterfactual for the change that could have happened in batch 3 if there had been no ECED project, the reduction in the gap between rich and poor children as a result of the ECED project is (A) – (B).

In the absence of the ECED project, the rich-poor gap in child outcomes widens substantially in comparison villages as shown in (B) of Figure 11. For example, the rich-poor gap in the language and cognitive development domain of the EDI and in a child’s ability to draw a house increases by about 0.24 and 0.30 standard deviations, respectively. On the other hand, most of reported rich-poor gaps narrow in batch 3 as shown in (A). Specifically, we find decreases in the rich-poor gap in batch 3 in social competence by 0.246 standard deviations, in the communication and general knowledge domain of the EDI by 0.228 standard deviations, in pro-social behavior by 0.216 standard deviations, and in the test of executive function (i.e. the Dimensional Change Card Sort game) by 8 percentage points.

The causal reduction in the gap between rich and poor children as a result of the ECED project is presented in (A) – (B) of Figure 11. In the case of social competence, the gap closes by 0.3 standard deviations, in the language and cognitive development domain of the EDI the gap closes by 0.185 standard deviations, in the communication and general knowledge domain of the EDI, the gaps closes by 0.298 standard deviations, and in the case of pro-social behavior it closes by 0.377 standard deviations. Thus, our findings imply that the greater progress for poor children as a result of the ECED project in batch 3 leads to a reduction in the gap between rich and poor children.

19
# **Policy Implications**

As this paper has shown, it is possible for well-designed early childhood services in rural areas to narrow (and in some cases close) early achievement gaps. The results reported here are consistent across a number of domains. For most outcomes studied in this paper the early achievement gap decreased for children from project villages. In particular, in project villages, the early achievement gap in social competence and in communication and general knowledge decreased significantly as did the early achievement gap in pro-social behavior problems. In contrast, there was no change in the early achievement gap on these dimensions among children from non-project villages. For other outcomes, such as the draw-a-house task – the early achievement gap increased in both project and non-project villages. However, the increase in the gap was twice as large in non-project villages as in project villages suggesting that the ECED project may have prevented even more adverse outcomes from materializing.

As Indonesia pursues its vision to ensure that all children have access to holistic, integrated early childhood education and development services, a number of programs and policies could be leveraged to ensure that this vision becomes a reality. For the specific case of Indonesia, we discuss three programs where the potential synergies between government priorities for HI-ECED and program objectives seem greatest. However, similar programs exist in many low- and middle-income countries. Thus the opportunities described here could also be explored elsewhere as well.

# **_Link Conditional Cash Transfers (CCTs) to ECED Attendance_**

Like many other conditional cash transfer (CCT) programs, the Indonesian CCT program, _Program Keluarga Harapan_ , (PKH) seeks to reduce poverty and improve people’s welfare by providing cash transfers to families if they comply with a number of conditions relating to health and education, such as going to well-baby check-ups or ensuring children attend school. In its present form, PKH provides families with a cash incentive to ensure that children attend primary school. There is no condition or transfer related explicitly to ECED attendance though a number of conditions are focused on improving early health outcomes.<sup>15</sup>

> 15 A key motivation behind the ECED project was the fact that children’s school readiness needs to be improved and that ECED services are key in doing so.

20
To better align the government’s flagship poverty reduction program with its own vision to ensure that all children have access to ECED services, the government could add a condition whereby families with younger children (less than primary school age) would receive conditional cash transfers if they attended an ECED service.

# **_Leverage Existing Community-Driven Development Platforms_**

The country’s community-driven development program – _Program Nasional Pemberdayaan Masyarakat_ (PNPM) – provides incentives not directly to families, but to communities – if they meet or exceed thresholds of primary and junior secondary school enrollment. For many years now, most PNPM communities have exceeded those enrollment thresholds. One way to align government priorities with community incentives would be to include ECED enrollment or access thresholds in the PNPM program. Thus communities which meet or exceed certain pre-specified ECED enrollment thresholds would be eligible to receive incentives as well.

# **_Ensure Sustainability of Existing Services_**

A third option would be to reassess the _Bantuan Operasional PAUD_ (BOP) program, in which the government provides a cash transfer to some ECED services on a per-child basis. In 2012, this program provided certain ECED services with Rp. 240,000 per student per year (approximately US$25 / student / year) for up to 25 students. In its 2013 manifestation BOP has been transformed into a flat transfer of Rp. 7,200,000 per institution per year (approximately US$720 / institution / year). This is the amount if there are more than 26 students per institution. For an institution with up to 25 students the transfer is Rp. 5 million and for an institution with up to 15 students the size of the BOP transfer is 3.6 million. In rural communities where institutions are much more likely to have more than 26 students than in urban areas this means that BOP is a smaller per student transfer. As suggested in our findings, children in rural areas are more likely to benefit from having access to ECED services. Thus, there is scope to improve the benefit-incidence of BOP and make it a more progressive transfer.

21
# **Conclusion**

Our present findings underscore the importance of increasing access to ECED services particularly to children from poorer backgrounds. It remains to be seen whether the kinds of gains found in our study persist beyond the short term analyzed here. It will also be important to gauge whether the government is able to ensure the sustainability of these services and expand them to reach other children in similar districts.

22