metadata: authors: 0: author: Alam,Andaleeb Jahan
1: author: Baez Ramirez,Javier Eduardo
2: author: V. Del Carpio, Ximena
count: Pakistan
docna: 0: docna: Does cash for school influence young
            women's behavior in the longer term ? evidence from Pakistan
docty: Policy Research Working Paper
subtopic: Labor Markets,Educational Sciences,Gender and Development,Rural Labor Markets
teratopic: Gender,Social Protections and Labor,Education
abstracts: cdata!: The Punjab Female School Stipend
            Program, a female-targeted conditional cash transfer program
            in Pakistan, was implemented in response to gender gaps in
            education. An early evaluation of the program shows that the
            enrollment of eligible girls in middle school increased in
            the short term by nearly 9 percentage points. This paper
            uses regression discontinuity and difference-in-difference
            analyses to show that five years into the program
            implementation positive impacts do persist. Beneficiary
            adolescent girls are more likely to progress through and
            complete middle school and work less. There is suggestive
            evidence that participating girls delay their marriage and
            have fewer births by the time they are 19 years old. Girls
            who are exposed to the program later, and who are eligible
            for the benefits given in high school, increase their rates
            of matriculating into and completing high school. The
            persistence of impacts can potentially translate into gains
            in future productivity, consumption, inter-generational
            human capital accumulation and desired fertility. Lastly,
            there is no evidence that the program has negative spillover
            effects on educational outcomes of male siblings.
geo_reg_key: 517198,517212,517191
repnme: repnme: Does cash for school influence young
            women's behavior in the longer term ? evidence from Pakistan
display_title: Does cash for school influence young
            women's behavior in the longer term evidence from Pakistan
listing_relative_url: /research/2011/05/14242914/cash-school-influence-young-womens-behavior-longer-term-evidence-pakistan
topicv3: Labor Markets m1326488 855,Education m1327048 470,Gender m1327621 615,Labor Force Participation m1504050 2746,Stipend m1583603 4402
publishtoextweb_dt: 2010-07-01T00:00:00Z
historic_topic: Gender,Social Protections and Labor,Education
no_of_pages: 57
author_keys: 0: author_key: 1068366
1: author_key: 945620
model_extractions: WPS5669 Policy Research Working Paper 5669 Does Cash for School Influence Young Women’s Behavior in the Longer Term?

Evidence from Pakistan _Andaleeb Alam Javier E. Baez Ximena V. Del Carpio_ The World Bank Independent Evaluation Group Public Sector Unit May 2011
Policy Research Working Paper 5669

# **Abstract**

The Punjab Female School Stipend Program, a femaletargeted conditional cash transfer program in Pakistan, was implemented in response to gender gaps in education. An early evaluation of the program shows that the enrollment of eligible girls in middle school increased in the short term by nearly 9 percentage points. This paper uses regression discontinuity and differencein-difference analyses to show that five years into the program implementation positive impacts do persist. Beneficiary adolescent girls are more likely to progress through and complete middle school and work less.

There is suggestive evidence that participating girls delay their marriage and have fewer births by the time they are 19 years old. Girls who are exposed to the program later, and who are eligible for the benefits given in high school, increase their rates of matriculating into and completing high school. The persistence of impacts can potentially translate into gains in future productivity, consumption, inter-generational human capital accumulation and desired fertility. Lastly, there is no evidence that the program has negative spillover effects on educational outcomes of male siblings.

This paper is a product of the Public Sector Unit of the Independent Evaluation Group. It is part of a larger effort by the World Bank to provide open access to its research and make a contribution to development policy discussions around the world. Policy Research Working Papers are also posted on the Web at http://econ.worldbank.org. The author may be contacted at jebaez@worldbank.org.

_The Policy Research Working Paper Series disseminates the findings of work in progress to encourage the exchange of ideas about development issues. An objective of the series is to get the findings out quickly, even if the presentations are less than fully polished. The papers carry the names of the authors and should be cited accordingly. The findings, interpretations, and conclusions expressed in this paper are entirely those of the authors. They do not necessarily represent the views of the International Bank for Reconstruction and Development/World Bank and its affiliated organizations, or those of the Executive Directors of the World Bank or the governments they represent._ Produced by the Research Support Team
# **Does Cash for School Influence Young Women’s Behavior in the Longer Term? Evidence from Pakistan**

Andaleeb Alam Javier E. Baez Ximena V. Del Carpio<sup>1</sup>

## JEL Classification: I25, J13, J21, O15

Keywords: conditional cash transfers, female education, female labor participation, fertility, Pakistan

> 1 Javier Baez is an Economist with the Independent Evaluation Group of the World Bank. Ximena V, del Carpio is an Economist with the Social Protection Unit, East Asia and Pacific Region. Andaleeb Alam is a Consultant with the Independent Evaluation Group. Special thanks go to Mark Sundberg, Emmanuel Skoufias, John Hoddinott, Andrew Warner, Martha Ainsworth and Kathleen Beegle for insightful comments. Tu Chi Nguyen provided superb research assistance. Special recognition goes to operational staff in the World Bank, in particular Dhushyanth Raju, Sofia Shakil and Mariam Qaiser. We also thank various government agencies and research centers in Islamabad and Lahore, Pakistan, especially Muhammad Asif and Sohail Malik (PMIU), Shamim Rafique (Bureau of Statistics), Minhaj ul Haque (Population Council), and Feyza Bhatti (Mahbub Ul Haq Human Development Center). Many thanks go to many others inside and outside the World Bank who provided helpful comments. We acknowledge financial support from the Norwegian Agency for Development Cooperation. The findings, interpretations, and conclusions expressed in this paper are entirely those of the authors. They do not necessarily represent the views of the International Bank for Reconstruction and Development/World Bank and its affiliated organizations, or those of the Executive Directors of the World Bank or the governments they represent.
# **1. Introduction**

Many developing and transition countries have implemented some modality of conditional cash transfer (CCT) program to incentivize poor households to make investments in the human capital of their children. These programs are a popular tool for alleviating short-term poverty and reducing the inter-generational transmission of poverty. Though CCTs have some basic elements, they vary in terms of scale, transfer size, conditionalities, eligibility, and many implementation features. In spite of these differences, evidence arising from impact evaluations indicates that most of these programs fulfill their short-term objectives through increases in immediate consumption and use of educational and health services. Indeed, a recent literature review shows that CCTs increased the consumption of participant households and 87 percent raised the school enrollment and attendance of children (World Bank, 2010).

However, little is known about the longer-term impacts of these programs on human capital accumulation and measures of human welfare. Most evaluations to date only assess immediate changes in behaviors related to school attendance, enrollment and visits to health centers. While these investments are critical for the accumulation of human capital of children, such changes say little about whether the accumulation is taking place and whether immediate impacts also influence the future prospects of children in their progress in school, productivity and employability (Filmer and Schady, 2009).<sup>2</sup> Likewise, the evidence is thin regarding indirect effects on outcomes such as marriage and fertility decisions, which have plausible implications for the long term on the welfare of the beneficiaries.

This paper contributes to the literature by focusing on the impacts of a gender targeted CCT program in the longer-term. The program investigated is the _Female School Stipend Program_ (FSSP) which was implemented in the province of Punjab, in northeast Pakistan. The evaluation identification strategy relies on a regression discontinuity design. The analysis focuses on examining impacts on school progression, middle and high school completion, early labor market outcomes, marriage and fertility decisions of adolescent girls who have been participating in the program for up to four years. The paper also measures distributional impacts

> 2 Evidence from CCTs in Mexico and Cambodia, for instance, suggests that the programs increased grade progression but did not have impacts on scores in mathematics and language tests.

2
across various participant groups and explores spillover effects of the program on the enrollment of boys.

Results shows that after four years of implementation, participant girls are 3 to 6 percentage points more likely to complete middle school; and younger cohorts of girls who are exposed to the program later-on have a greater likelihood of transitioning to high school (by 4 to 6 percentage points) and completing at least one high school grade (by 5 percentage points). In terms of labor market outcomes, the positive effects of the program on progression and completion seem to be accompanied by a reduction in the labor force participation of beneficiary adolescent girls of 4 to 5 percentage points. In addition, the younger cohorts of girls also appear to work at least two to three hour less per day relative to non-participants. Lastly, there is some evidence (marginally significant in statistical terms) that participant girls (between 15 to 19 years of age) tend to delay their marriage by 1.4 years and have 0.3 fewer children.

# **2. Related Literature**

The last decade has seen a substantial increase in the prevalence of CCTs and, with it, a parallel increase in impact evaluations that assess their effectiveness. The wave of CCTs and evaluations started with the Mexican _Oportunidades_ program, which has been operating since 1997. Evaluations of the program based on a randomized design and multiple rounds of panel data show significant positive impacts on school enrollment and attainment, child labor, immediate consumption, and some health outcomes (Gertler, 2000; Hoddinott et. al., 2000; Behrman et al., 2001; Schultz, 2004). These assessments have been corroborated by subsequent evaluations of similar interventions in Brazil, Honduras, Colombia, Cambodia, Nicaragua, Turkey, Pakistan, Malawi, and El Salvador. For the most part, their findings are consistent with the results of the _Oportunidades_ program regarding positive impacts on the use of educational and health services and short-term consumption (Olinto et al., 2003; Glewwe and Olinto, 2004; Attanasio and Mesnard, 2005; Attanasio et al., 2005a and 2005b; Filmer and Schady, 2006; Maluccio and Flores, 2005; Ahmed et al., 2007; Chaudhury and Parajuli, 2008; Baird et al., 2009).

In addition to positive effects on school attendance and enrollment, some evaluations also suggest that grade progression improves with CCTs. For instance, de Janvry et al. (2006) show that the _Bolsa Escola_ program in Brazil increases grade advancement by 6 percentage points; an 3
experimental analysis of the Honduras _PRAF II_ program finds that the probability of matriculating to subsequent grades increases by 2 to 4 percentage points among beneficiary children. Comparable effects have also been documented for CCT programs in Mexico, Nicaragua, and Paraguay (Behrman et al., 2001; Maluccio and Flores, 2005; Veras Soares et al., 2008).

The literature also gives evidence on the indirect effects of conditional transfers on the labor market outcomes and marriage and fertility decisions of adolescents, which are thought to operate mostly through the conditionalities that require children to attend school regularly. In fact, rigorous evaluations of programs in Brazil, Cambodia, Colombia, Mexico, and Nicaragua confirm that treated children not only spend more time in school but also have relatively lower participation in income generating activities and domestic work (Behrman et al., 2005; Maluccio and Flores, 2005; de Janvry et al., 2006; IFS – Econometria - SEI, 2006; Ferreira et al., 2009). This effect is also observed among adolescent females. For example, the _Oportunidades_ program in Mexico lowers the participation of young girls in labor markets and child care by around 10 percent and 40 percent, respectively (Behrman et al., 2005; Dubois and Rubio-Codina, 2009). Furthermore, increased time spent in school and improved knowledge could also result in changes in marriage, sexual and fertility decisions. It is shown that CCTs have caused delays in marriage and onset of sexual activity in Malawi and Mexico, reductions in the number of sexual partners in Malawi, and reductions in pregnancy rates in Malawi and Turkey (Baird et al., 2009; Behrman et. al., 2005; Gulemetova-Swan, 2009; Ahmed et. al., 2007).

Evidence regarding the impacts of CCTs designed to explicitly address gender disparities in investments in human capital, such as the program evaluated in this paper, shows that they have succeeded in increasing school enrollment and attendance of girls. At least four interventions that were set up to address these issues achieve positive impacts in the short term. The _Bangladesh Female Secondary Stipend_ Program is shown to raise the enrollment of girls in sixth through eighth grade, by 8 to 12 percent (Khandker et. al., 2003). Similarly, the _Zomba Cash Transfer_ experiment in Malawi more than doubles the re-enrollment of girls between 13 and 22 years old who had dropped out of school before (Baird et. al., 2009). Another example is the _Japan Fund for Poverty Reduction Scholarship Program_ in Cambodia, which has a large positive effect (around 30 percent) on the school enrollment and attendance of girls (Filmer and 4
Schady, 2006). A most directly related to this analysis, the short-term impact evaluation of the FSSP in Pakistan –the program analyzed in this paper –shows an increase in the enrollment of girls of about 9 percentage points between 2003 and 2005 (Chaudhury and Parajuli, 2008). Although the impacts of these studies differ by magnitude, and the empirircal approaches vary, results appear to be consistent across programs and implementation contexts.

Nonetheless, the evidence of the connection between the provision of incentives for school participation and the well-being of adolescent girls in the longer-term remains very thin. Therefore, this paper aims to begin filling this evidence gap by assessing whether girls who benefited from a school stipend program show improved outcomes in the longer-term, or up to four years after their first exposure to the program.

# **3. Schooling Context and the Project**

## **_Schooling in Punjab_**

The literacy rate in Punjab in 2003 was approximately 54 percent (45 percent in Pakistan as a whole) compared to 56 and 63 percent for South Asia and all low income countries, respectively (Lloyd, 2004).<sup>3</sup> Divided into 36 districts (34 districts at the time the program began), Punjab is home to over 55 percent of the total Pakistani population.

School enrollment in Punjab, particularly in secondary education, is largely constrained by the scarcity of schools. Primary education is supplied by both public and private schools. On the contrary, the secondary level is dominated by the public sector, and not every village has access to middle schools (grades 6 to 8) and high schools (grades 9 and 10).<sup>4</sup> For instance, whereas 84 percent of the households in Punjab report that their children are within walking distance (15 minutes) of a primary school, only 55 percent of the households have access to a middle or high school within the same travel time (Pakistan Social & Living Standards Measurement Survey - PSLM, 2004). Differences in access are even larger in rural areas, making it even far less likely that children in rural villages enroll in schools at these levels (Andrabi et al., 2006; Sathar et al., 2003). In 2003, before the program started, while enrollment in primary

> 3 Estimates for Pakistan and region were obtained from published documents ―Pakistan at a Glance‖ 2004 available online; Punjab estimates were derived from the MICS 2003-04 Report, table titled Major Summary Results.

> 4 Around 90 percent of private schools offer only primary classes (Andrabi et al, 2006).

5
school for children aged 6-10 was 58 percent, enrollment at the middle school level for children aged 11-14 was 29 percentage points lower (Multiple Indicator Cluster Survey – MICS, 2003).

Education levels of females are low, both in absolute and compared to boys, especially in rural areas. At baseline, girls in program areas were less likely than boys to be enrolled in school. While enrollments in primary and middle school were very similar for boys and girls in nonstipend districts, large differences are observed in stipend districts – up to 8 percentage points (MICS, 2003). Such gaps in enrollment by gender are even wider when the samples are restricted to rural areas of Punjab. In terms of school attainment, numbers for 2001 (before the program started) show that women in Pakistan achieved on average 4.9 and 1.3 years of schooling in urban and rural areas, respectively – about 1.9 and 2.8 fewer years of education than men. These disparities by gender are also evident for Punjab, where the levels of enrollment and educational attainment are systematically lower for women in rural and urban areas (Table 1).

Finally, there are also marked differences in school enrollment and access between districts in Punjab covered and not covered by the GSP program. Enrollment levels in program districts were well below those of non-stipend districts. In 2003, for instance, enrollment in primary school for children aged 6 to 10 was 20 percentage points lower in stipend districts; similarly, only 20 percent of children 11-14 years old were in middle school compared to 34 percent in non-stipend districts (MICS, 2003). As shown in Table 2, access to school was also more difficult in stipend districts before the program began, where higher fractions of households live in villages without a school and, thus, their children had to travel longer distances to go to school (MICS, 2003; PSLM, 2004).

# **_The Female School Stipend Program_**

The FSSP was designed as part of a broader educational reform of the Provincial Government of Punjab (GoP) to improve the educational attainment among girls and decrease gender inequities, particularly at the middle school level.<sup>5</sup> Prior to the start of the program, as noted before, female enrollment in primary and secondary schools was low, both in absolute terms and relative to that

> 5 The Punjab Education Sector Reform (PERSP) seeks to address flaws identified in the educational system, including: lack of infrastructure or substandard quality of infrastructure, which limited access to schools; bad quality of service delivery and financial management; limited capacity of staff at all levels of the system; weak community and parental involvement; and a rise in the private sector provision of schooling with weak oversight for standards and quality.

6
of boys.<sup>6</sup> Additionally, low enrollment for girls is further compounded by low retention and completion rates (Sawada and Lokshin, 2009). A large body of analytical work has identified various community characteristics (access to roads, transport, share of teachers that reside in the village where the school is located, scarcity and poor quality of schools for girls) and household characteristics (poverty status of the family, school costs, parental education and occupation) that are associated with gaps in school entry for girls (Khan, 1993; Sathar and Lloyd, 1994; Lloyd et al. 2005; Das et al. 2006; Lloyd at al. 2007). In response to this, the GoP mobilized resources in 2003 to finance the FSSP seeking to target a clearly disadvantaged group (girls) in districts with lagging literacy rates. None of the ineligible districts were covered by the FSSP but were still eligible for other components of the wider educational reform, such as the distribution of free textbooks, school rehabilitation, and new teachers, among other things.<sup>7</sup> The main goal of the FSSP was to promote participation in public education for girls in middle school (grades 6 through 8).<sup>8</sup> Program benefits comprise a quarterly stipend of approximately PKR 600 (equivalent to US$10) per female student.<sup>9</sup> Beneficiary girls were targeted based on their district of residence (districts with the lowest literacy rates in the province – below 40 percent), and enrollment in eligible grades (grades 6 through 8) in public schools; eligibility was conditional on a minimum school attendance rate of 80 percent, as reported regularly by the school.

The FSSP started in late 2003 and covered 15 eligible districts. Although planning for the program began in early 2003, stipends were not distributed until the first quarter of 2004, with compensation for attendance in the fourth quarter of 2003, as reported and accounted by the monitoring unit. During the first quarter of the program, about 156,000 girls received the stipend. In 2006, the FSSP was extended to include high school (grades 9 and 10); however, the impact

> 6 The gender disparity in Pakistan is well established in the literature; for example, authors such as Khan (1993), Behrman and Schneider (1993), Alderman et al. (1995), Alderman et al. (2001), Holmes (2003), Lloyd et al. (2005), and a gender assessment by the World Bank (2005) make this point clearly.

> 7 Parts of these activities were either phased in throughout the province over the years based on need or were implemented only in places where demand was most salient. In this respect, the evaluation accounts for three components that could potentially affect the outcome of the _GSP_ : school improvements<sup>7</sup> , the recruitment and allocation of new (probably better qualified) teachers, and the phased distribution of free textbooks to all primary school children.

> 8 Since 2006 a public subsidy to low cost private schools was put in place; this program known as stipend to Foundation Assisted Schools covers girls in approximately 50 percent of the GSP program. Additionally, an education voucher scheme was also implemented throughout Punjab, but targeting and modality are orthogonal to the program.

> 9 It is estimated that the costs of schooling per quarter are approximately $9, leaving some funds left over for the family to use for other needs

7
of this extension is not the main focus of this paper. The FSSP is still active in Punjab, and as of 2007, it covered 245,000 beneficiaries in middle school, which correspond to a take-up rate of 51 percent.<sup>10</sup> According to actual quarterly stipend data, the program spent around US$7.3 million in 2007 on stipends for grades 6 through 8.

A previous impact evaluation of the FSSP investigated the short-term impacts of the program on female enrollment. The evaluation uses difference-in-differences (DD) and Regression Discontinuity Design (RDD) models on provincial school censuses to estimate net growth in female enrollments in grades 6-8 (middle school) from 2003 (before the program started) to 2005 (two years into the program) and in stipend public schools relative to nonstipend public schools. Other models estimated compare the differences in female growth enrollment relative to male growth enrollment in a triple-difference framework. Even though the magnitude of program impact estimates varies across different models, this evaluation found that the FSSP increased the number of females enrolled in grades 6-8, on average, by 9-23 percentage points (Chaudhury and Parajuli, 2008).

# **4. Data**

This paper estimates the impact of the program with information at the school and household levels. The main data source for the first approach is the Punjab public school census collected annually in October from 2003 to 2009 by the provincial education department, including data from before and during the program implementation. It covers all public schools in Punjab and focuses on school information (including village name and date of establishment), infrastructure, teaching staff, enrollment, and some organizational indicators.<sup>11</sup> The enrollment data are reported by gender of the school and separated into different grade levels. Schools are also disaggregated by urban and rural areas. The total number of schools is 60,000-62,000 from 2003-04 to 200910, however the sub-sample for this analysis is a pool of cross-sectional data that includes

> 10 The take-up rate is difficult to calculate because eligibility to the program is determined by the grade, and not the age, of girls in stipend districts. For simplicity, this analysis estimates the take-up rate as the ratio of participants (245,000 girls covered by the program in 2007) relative to the sum of the actual and potential participants (those enrolled in grade 6-8 in private schools and those whose highest grade is 5-7 but are not enrolled at the time of the survey).

> 11 This survey is part of the institutional strengthening capacity effort within the PERSP and is used for monitoring resource allocation and performance in the sector. The survey does not cover private and religious schools in Punjab; a private school census was undertaken in 1999-2000, on the basis of which it was estimated that over one-quarter of all students were enrolled in private education.  A National Education Census, also conducted in 2006, enumerated all public and private schools.

8
approximately 4,000 middle schools for girls in stipend and non-stipend districts. These are the schools that could be eligible for the program at both baseline and follow-up.<sup>12</sup> In order to extract contextual information and construct covariates, this analysis uses several household surveys. The household surveys include the Pakistan Social and Living Standard Measurement survey (PSLM) collected in 2004-05, the Punjab Multiple Cluster Survey (MICS) collected in 2003, and the Pakistan Integrated Household Survey (PIHS) collected in 2001. They use multi-topic questionnaires focusing on a comprehensive set of social and economic indicators as well as information on infrastructure supply and quality, thus allowing for construction of some covariates at the district level. These surveys are representative at the district level and disaggregated by urban and rural areas.<sup>13</sup> Other sources include educational censuses and administrative data that provide information on the supply of educational services and other programs. This evaluation uses the National Education Census collected in 2005 on both public and private schools to construct additional covariates. It also relies on the administrative data of other elements of the educational reform implemented in Punjab at the same time as the FSSP, such as distribution of free textbooks, recruitment of new teachers, construction of school facilities, and improvement of school councils.

The main data source for the estimation of impacts at the individual level is two waves of the MICS collected in 2003 (pre-program) and 2007-08 (post-program). This household survey covers a wide range of social and economic indicators, all repeated in both rounds for comparability. The sampling design follows the sampling frame of the previous population census (1998) and provides estimates for the province, for urban and rural areas, and for each of the districts in Punjab and the Cantonment of Lahore. Sampling for 2003 was stratified by rural, urban, and large city areas. In the follow-up survey in 2007-08 the sample size was vastly

> 12 The sample of eligible schools includes only girls’ schools that offered grades 6-8 at both baseline and follow-up (less than 1 percent of middle schools are mixed). Schools that were nonfunctional upgraded to middle level or above or downgraded to primary or lower level during the program implementation period were excluded, as these would not have been eligible for the stipend at both baseline and follow-up. Of the remaining schools, only those schools that had non-zero enrollment in grades 6-8 at both baseline and follow-up were retained. Finally, although in theory all public schools in Punjab are single-gender, the data shows that a small number of mixed schools exist. These schools were also excluded from the final sample.

> 13 It is important to note that, although lower-level data provide more precise estimates, the questionnaires are significantly shorter; thus, some key economic welfare indicators are excluded.

9
enlarged to increase the precision and make it representative at the Tehsil level (a level below the district). The total number of surveyed households is 30,932 in 2003 and 91,280 in 2007-08. For the empirical models of this paper, the analysis uses pooled cross-sectional data comprised of girls aged 12-19 selected based on the criteria described above.

The surveys used to construct covariates for the school-level analysis are again used for the household-level analysis. This includes the PSLM, which relies on a similar sampling strategy as the MICS. Other data come from the PIHS, the National Education Census, administrative data of other programs under the PESRP, and the annual Punjab public school census collected in 2003.

Given the targeting of the program to low-literacy districts, stipend and non-stipend districts are likely to differ in aspects other than the treatment and a baseline comparison confirms this. Since literacy rates are likely correlated with other underlying socio-economic characteristics, such factors may, in turn, be associated with different education attainment and divergent trajectories in human capital accumulation between households in stipend and nonstipend districts. Indeed, as shown in Table 3, at the community level, households in stipend districts are more likely to be below the poverty line, to be located in a rural village, and to spend a little less on education. The stipend districts also have less access to public transport and to schools of different levels (defined as a distance of two kilometers or travel time of 15 minutes) as well as fewer private middle schools relative to public schools. At the school level, schools in stipend districts tend to have worse school facilities, but the differences are very small, except for electricity. On the other hand, stipend schools exhibit more favorable student-teacher ratios. Finally, at the household level, households in stipend districts appear to have more dependents and be headed by individuals with approximately 1.6 fewer years of education. They are also less likely to have dwellings connected to gas and electrical services, and to have access to family planning and health centers.

In order to deal with potential biases that may rise due to these differences, we run econometric specifications which control for factors at the community, school, and household levels that could contribute to the outcomes, as well as district and cohort-fixed effects. Control variables that may be endogenous to the stipend program are added to the models using their 10
mean values at baseline. Furthermore, econometric models in the analysis also add binary indicators to control for the plausible effects of other programs related to improving the education system within the PERSP.<sup>14</sup>

# **5. Empirical Strategy**

As the FSSP was not randomly allocated, a simple comparison of girls in stipend and nonstipend districts is unlikely to identify the causal effect of the program on the outcomes of interest. Due mainly to budgetary constraints, the FSSP was rolled out only in the neediest districts of Punjab (those with literacy rates below 40 percent). In total, 15 out of 34 districts, most of which are located in southern Punjab, were eligible for the program (Annex A). This targeting may pose an obvious selection problem since stipend districts were chosen based on characteristics (possibly both observed and unobserved) that may be correlated with low educational attainment. Moreover, there are two further issues that could make it difficult to identify the causal effect of the program in the absence of a reliable counterfactual. One is participation bias, namely families which decide to enroll girls in public middle school and therefore participate in the program may be different from other families with girls in relevant ages but not in public school. The other issue is the focus on outcomes that are conditioned on the enrollment of girls. In this case, the samples for analysis may suffer from negative selection bias if they include lower-ability girls in stipend areas that would not have attended school in the absence of the program.<sup>15</sup> Since the program design and implementation did not plan for an impact evaluation, no specific data were collected for the purpose of evaluation based on a counterfactual framework.

> 14 The covariates used in the models include: (1) Community-level variables: access to schools (distance to primary, middle, and high schools, and access to public transport), school supply across education levels (ratio of primary to middle schools and middle to high schools), share of the private sector in total enrollment (private/public enrollment ratio), dropout rates at different education levels, poverty (poverty headcount ratio), and share of education in household expenditures; (2) School-level variables: proxy of school quality (student/teacher ratio), school facilities (electricity, drinking water, toilets, and boundary walls); (3) Household-level variables: access to basic services (health care, water, electricity and gas), location (rural or urban), household structure (household size and dependency ratio), birth order, and parental education (education of household head); (4) other programs: distribution of free textbooks, recruitment of contract teachers, construction of school facilities, and strengthening of local school councils.

> 15 Another potential concern for identification strategy is that school enrollment/attendance and labor force participation are joint decisions. However, this is problematic for analyses that focus on the relationship between the use of school inputs (enrollment and attendance) and labor market participation. In contrast, this paper exploits the extra incentive provided by the stipend and the conditionality to send girls to school which this analysis argues to be exogenous after controlling for some observables at their baseline values, time-invariant unobserved factors between stipend and non-stipend districts (through DD method), and the exogenous variation in program eligibility around the literacy cut off (through RDD).

11
We attempt to overcome some of the issues discussed above with the use of cross-sectional data at the school and household levels from 2003 through 2009 and quasi-experimental methods. More specifically, the analysis contrasts cohorts of girls in treatment and control districts in double-difference and RDD frameworks.<sup>16</sup> While double-differencing compares the changes in the outcomes of analysis between stipend and non-stipend districts overtime, RDD exploits the literacy rate cut-off for program eligibility across districts. The RDD models are also implemented within a DD framework as done by Chaudhury and Parajuli (2008). Therefore, both models compare the trends in stipend and non-stipend districts, which helps address possible selection bias introduced by time-fixed characteristics related to nonrandom program placement. In order to check the sensitivity of the findings to the issue of participation bias, additional models were estimated with samples that do not condition on enrollment in middle school.<sup>17</sup> The validity of the identifying assumptions of the DD models and the implications on the expected selection on returns to schooling are discussed in the robustness section.

We construct treatment and comparison groups of girls at both the school and household levels. At the school-level, the analysis draws upon administrative data from school censuses to estimate the annual average changes in total female enrollment at public schools in stipend districts (treatment schools) for the period between 2004 and 2010 relative to the pre-program level (2003-04 academic year). These changes are then compared with the corresponding annual changes in female enrollment observed for public schools in non-stipend districts (control schools) during the same period. This approach is limited, however, by the fact that schools are the units of analysis, and it is not possible to tease out other behavioral responses regarding the investments in the human capital of girls.

To overcome that, we also use household surveys to estimate changes at the individual level. The research design for this approach follows for the most part the same identification strategies and assumptions as the school-level analysis. However, the richness of information of the household-level data, as compared to school censuses, allows the analysis to take advantage

> 16  It was not possible to also use matching techniques with the pool of cross-sectional data available. The only common level of aggregation between baseline data —that would be used to predict program participation and construct the propensity score for each household— and follow up data was the district. This implies that matching is only possible at the district level and the province has only 34 districts.

> 17 These samples are restricted to girls who finished primary school (having at least five years of schooling). Additional econometric exercises also show that the GSP program did not have indirect effects on either enrollment in grades 4 and 5 or completion of primary school.

12
of the extra variation at the individual level and the better accuracy in measuring girls’ exposure to the program. The outcomes of interest at this level include relative changes between baseline (2003) and follow-up (2007-08) in partial measures of school attainment (middle school completion, and high school enrollment and completion), labor market participation, marriage, and fertility. The comparison is made between cohorts of girls with various levels of exposure to the program in stipend districts (treatment cohorts) and similar cohorts in non-stipend districts (control cohorts). Program impact estimates at the school and household levels should be interpreted as impacts on all eligible girls, whether or not they participated in the program (intent-to-treat impacts).<sup>18</sup> In what follows, both approaches are explained in more detail.

# **_School-level Analysis_**

The treatment and comparison cohorts of girls for this analysis are constructed synthetically based on their enrollment at different grades. The treatment cohorts were enrolled in public middle school in stipend districts; thus, they should have been exposed to the program for at least one year between baseline (2003-04) and subsequent academic years up to 2009-10. Based on these criteria, seven distinct cohorts can be identified (Table 4). The counterfactuals consist of the corresponding cohorts enrolled in public schools in non-stipend districts. For example, a cohort that consists of girls in grade 6 in stipend districts in 2004-05 is expected to progress to grade 7 through 10 in the succeeding years between 2005-06 and 2008-09. If their progress through school is consistent, they should have been exposed to the program for all three years of middle school (grades 6 to 8) between 2004-05 and 2006-07, and to the expansion of the stipends to girls in high school (grades 9 and 10) which started in 2006-07. Girls in the corresponding cohort but in non-stipend districts should not have been exposed.

The outcome of interest for this analysis is the percentage change in grade level enrollment (grades 6-10 each), which is estimated using the following specification:

> 18 In the context of the GSP, the intent-to-treat analysis measures changes in female enrollment of public schools in stipend districts relative to public schools in non-stipend districts even though not all girls in the stipend districts participated.

13
Where _i_ indexes schools, _X_ includes baseline socioeconomic characteristics, _FSSP_ is a binary variable that takes the value 1 for a stipend-eligible school and 0 otherwise. measures the net impact of the stipend program.

Therefore, the school-level analysis measures whether the increase in enrollment experienced by a cohort when they first joined the program (the short-term impact) is continued through successive grades relative to the trend observed for the comparison cohort. A sustained increase in net enrollment could be due to increased transition across grades (the push effect), reduction in drop-out rates, and/or influx of new entrants (including both rejoiners and those who switch schools). Yet, the school-level approach cannot distinguish these channels of impacts since the school censuses do not provide information to measure them separately. However, preprogram and post-program data from household surveys indicate that female rejoining (new entrants) and grade repetition at the middle school level is very low.<sup>19</sup> This suggests that net changes in female enrollment (if they happen) are largely driven by the combined effects of the program on both transition and dropout rates.

# **_Household-level Analysis_**

The units of observation used for this analysis consist of girls expected to be exposed to the program for at least one year between 2003-04 and 2007-08. Although the household survey data used for this analysis provide rich information on a wide variety of indicators, they do not contain explicit information on the treatment status and length of exposure of girls to the FSSP.

Therefore, some steps were taken to select the girls who fall within an age range and are likely to have been exposed to the program for at least one year. The motivation for constructing synthetic cohorts based on age and grade criteria, as opposed to only an age based criterion, was to arrive at a cohort that is more reflective of actual participants<sup>20</sup> Annex B explains more fully

> 19 Both rejoiners and repeaters constituted around 0.5 percent of total enrollment in middle school at baseline. Not only were these rates similar between stipend and non-stipend districts in 2003-04, there has been no differential growth in these rates between 2003-04 and 2007-08. School switching is another source of compositional change within the cohort. At baseline, around 2 percent of all children enrolled in middle schools switched from private to public schools while a similar proportion switched from public to private (the difference is statistically insignificant). So at baseline, compositional shifts stemming from re-entry, repetition and school switching accounted for some 5 percent of total enrollment in middle school level. There are no comparable data on school switching in the 2007-08 survey, but the change in repetition and rejoining rate at the middle school level remained comparable between stipend and non-stipend districts between 2003-04 and 2007-08.

> 20 The previous impact evaluation of the FSSP (Chaudhury and Parajuli 2008) used cohorts of children 10-14 years old from cross-sectional household surveys to triangulate the findings derived from school data. However, an examination of the age-grade

14
the construction of the cohorts for analysis, and addresses the assumptions and concerns underpinning the construction. Girls are included in the treatment cohorts if they live in the stipend districts and in the comparison cohorts if they live in non-stipend districts. In addition, both groups have to meet one of the following criteria to ensure that the analysis is restricted to the relevant girls, namely those who could enroll in middle school for at least one year during the period of study: (a) girls who were enrolled in grades 7-12 in 2007-08 and/or grades 6-11 in the previous year (2006-07) are assumed to have been exposed for at least one year to the program; (b) for the rest of the girls who were enrolled in neither 2007-08 nor 2006-07, only those whose highest grade completed is at least grade 6 and at most grade 10, and who are likely to have been enrolled in a middle school grade between 2003-04 and 2005-06 (based on the age-grade distribution of the reference cohort at baseline) are assumed to have been exposed for at least one year to the program.

We use the household data to estimate the influence of the program on such longer-term outcomes as educational attainment and labor market, marriage, and fertility decisions. The general specification of the base regression equations for the DD analysis is as follows: where _i_ indexes individuals, _t_ is the time subscript, _T_ is a dummy variable taking a value of 1 for post-program year and a value of 0 for the pre-program year, _FSSP_ identifies the location of the girl between stipend and non-stipend districts, and X is a vector of baseline household demographics, community-level characteristics, and measures of education market place and school quality. _P_ is a vector of dummy variables to capture other programs that were introduced by the GoP along with the FSSP and sought to build school facilities, provide free textbooks, recruit contract teachers, and revitalize school councils.<sup>21</sup> The outcomes of interest ( _Y_ ) measure educational outcomes (middle school completion, middle to high school transition, high school

> distribution of girls in middle school in Punjab indicates that this cohort has errors of inclusion and exclusion. More specifically, the youngest girls of this cohort (10-11 years old) have a high probability to remain in primary school whereas a substantial fraction of girls 15-17 who attend school are enrolled in middle school.

> 21 These programs were _per se_ not targeted at stipend districts but could have been implemented differentially: for instance, girls’ schools in stipend districts were worse off and it is possible that any program on rehabilitation of school infrastructure would focus on schools with many missing facilities, many of which happen to be in stipend districts.

15
completion), labor force participation (extensive and intensive margins) marriage and fertility.<sup>22</sup> The coefficient , the interaction of time and treatment status, gives the average net impact of the program. Additional specifications of this model also include district and cohort fixed effects. Because observations within a district could be serially correlated, standard errors are clustered at the district level.

We also estimate program impacts in a RDD framework, exploiting the fact that assignment to the stipend district (beneficiaries) is determined by a threshold base on the average literacy rate of the district (Chaudhury and Parajuli, 2008). Thus, girls in stipend districts with literacy rates of 40 percent or less are eligible for FSSP, with all girls in districts above the 40 percent literacy threshold comprising the control group. The RDD specification includes a control function L for distance of district literacy rate (subscript D) from the cutoff. It is assumed that the control function is linear, making this a parametric specification. The identifying assumption here is that making treatment status conditional on literacy would imply treatment to be independent of the error term given the literacy rate. The general specification of the RDD is as follows: The parametric RDD specification extrapolates from above and below the cut-off point to all observations in the data, and allows for use of covariates. A batch of RDD specifications is run adding covariates and new programs to equation (3). To avoid misspecification issues arising from the functional form, we also estimated non-parametric RDD models with different bandwidths and weights on either side of the discontinuity.

# **6. Empirical Results**

## **_Educational Outcomes_**

Theory provides ambiguous results on the effects of this sort of program on grade progression, school completion, and educational attainment. The cash transfer has an income effect. This is

> 22 A possible caveat to this analysis is that the girls studied were still young (12 to 19 years old) at the time of the follow-up survey. This means that many had probably not reached their full educational attainment and were not yet married or have children. However, this age range is still relevant for the analysis of program effects on early marriage and childbearing, both of which are prevalent issues for young women in Pakistan.

16
the only effect relevant for families that would have sent their daughters to middle school with or without the program. At the same time, for families with daughters out of school, the program’s condition on school attendance also induces a substitution effect since it reduces the price of education.

Both effects point toward an increase in the demand for education, particularly among credit-constrained households in environments with imperfect capital markets. On the one hand, if this demand is sustained over time, the higher utilization of education services can lead to higher school progression and completion. On the other hand, there might be perverse incentives or constraints that prevent program participants from accumulating more human capital. For example, families may try to keep their daughters in those grades that make them eligible for the transfers, in effect preventing them from progressing and graduating. Moreover, assuming no changes in supply, schools in program areas may become overcrowded due to the additional demand, which can hinder progression and completion. We seek to shed some light on the net effects of these conflicting forces in the context of the FSSP.

The analysis based on school-level data provides systematic evidence that the program helps girls progress through middle school. The point estimates from DD and RDD models indicate that the program raises enrollment not only in grade 6 but also in grades 7 and 8 for the same cohorts (Table 5).<sup>23</sup> For instance, cohort C sees higher percentage changes in enrollment in stipend than in non-stipend districts in both grade 7 in 2005-06 and grade 8 in 2006-07 (the second and third year of exposure to the program). The increases relative to enrollment in stipend versus non-stipend are 18 and 32 percent, respectively. Although the relevant cohorts (cohorts B to G) participate in the program at different times and for different periods, estimates of program effects on enrollment yield consistent results for all of these cohorts and grades over time (positive and significant impacts). The insignificant impacts in 2004-05 are expected since

> 23 The school-level approach measures changes in absolute enrollment between stipend and non-stipend districts. These changes, however, may be driven by a plausible mechanical relationship between higher fertility, population growth, and the number of girls enrolled given that stipend districts are poorer. Unfortunately, there are no school-level data available to do the estimation for a pre-program period and check if changes in enrollment in stipend district occurred even before the program began. Yet, the empirical models include a number of socio-economic covariates at their baseline values which are also likely to explain differences in fertility between stipend and non-stipend districts. Furthermore, the analysis at the household level looks at changes in progression and completion rates which, by definition, are not affected by the size of the cohorts.

17
awareness of the program was not widespread during its first year.<sup>24</sup> For subsequent cohorts, program impacts are still large and statistically significant, ranging from 11 to 32 percent.<sup>25</sup> Impacts of comparable magnitude have been found in similar female-targeted CCTs in Bangladesh, Cambodia, and Malawi (Khandker and others 2003; Baird and others 2009; Filmer and Schady 2006).

An interesting result is that program impacts on female enrollment increase with school grades for the same cohort. Several factors may be behind this. First, the program might have incentivized girls who had left the education system to re-enter school. However, this does not seem to explain the changes for middle school grades since re-entry is not only very low (less than 0.5 percent) but also similar across stipend and non-stipend districts. Second, and more plausible, the program may have differential effects in retaining girls in schools, perhaps more in higher grades. Dropout rates in Punjab (as in most developing settings) increase with grades and age. In fact, a simple DD analysis between 2003-04 and 2007-08 using household survey data shows a decrease of 2-3 percentage points in drop-out rate in middle and high school grades in stipend districts, relative to non-stipend districts, while there are not changes in dropout rates for noneligible grades. Finally, the expansion of the FSSP in 2006-07 to cover grades 9 and 10 may provide additional incentives for girls to finish middle school and transition to high school.

The findings also show that the increases in enrollment attributed to the program remain as long as girls are enrolled in middle school. However, such effects are not continued to grades 9 and 10. Point estimates of the effects on enrollment in grade 9 only become positive and statistically significant for some econometric specifications in the academic year 2007-08 and 2009-10 when the expansion of the FSSP to high school was fully implemented. This may suggest that this second phase of the program has positive impacts on enrollment in high school.

The evidence on school progression based on household-level data seems to validate the findings obtained with school data as it also suggests that the program increases the chances that

> 24 Qualitative evidence based on interviews with officials in charge of implementing the program indicates that campaigns to raise the awareness of the existence of the program began only at the second year of the program.

> 25 The parameter of interest for the models discussed here is changes in the number of girls relative to number of girls enrolled at baseline. For instance, if the numbers of girls enrolled increased by five (from 16 to 21), this corresponds to a 32 percent change in enrollment. This, however, should not be confused with a change in enrollment rate in percentage points. Suppose that the universe of girls that should be enrolled in middle school is 100, the increase in enrollment rate in percentage points for the above scenario would be around five, from 16 percent to 21 percent.

18
girls complete middle school. Overall, results summarized in Table 6 indicate that 12-17 year-old girls who were exposed to the program (for different periods) between 2003-04 and 2007-08 are on average around 3.3 percentage points more likely to have completed middle school (grade 8) relative to the control cohorts. This is equivalent to an increase of 4.5 percent relative to the baseline value of the treatment group.<sup>26</sup> The impacts on middle school completion are larger for girls aged 15-16 (6 percentage points with 5 percent significance) but none for girls older than 16.

We also find some evidence that the program helps girls transition from middle to high school and complete grade 9, especially those younger than 17.<sup>27</sup> The first outcome analyzed is the probability that a girl enrolled in grade 8 during the program period progressed to grade 9 or beyond.<sup>28</sup> The average impact for the entire cohort of 12-19 years old girls is not statistically significant (Table 6). However, for girls aged 15-16, the impacts are positive and significant (5.5 percentage points relative to a baseline value of 58.5 percent). This cohort is also 5 percentage points more likely to complete grade 9 than the control group (Table 7). One reason for this is that these younger girls benefit more from the FSSP as they are not only exposed longer but they joined the program when there was more awareness of its existence. Second, and perhaps more important, only the younger cohorts have the additional incentives induced by the expansion of the FSSP, which began in 2006, to also provide conditional transfers to enrollees in high school.

# **_Labor Market Outcomes_**

Theoretically, the direction of the effects of the FSSP on working behaviors of the beneficiaries in the longer-term is unclear. While the program introduces incentives for girls to spend more time in school and work less, if girls are an important source of labor for the household, they may give up leisure time rather than working less. Moreover, girls are expected to obtain better jobs and higher wages as they progress through school and accumulate more education. At the

> 26 Evidence of similar impacts has been also found for Bolsa Escola and PRAF II, two CCT programs in Brazil and Honduras (de Janvry and others 2006; Glewee and Olinto 2004).

> 27 2007-08, the year of the follow-up survey, may be too early to capture the impacts on this younger cohort (that is, age 15-16) of the program on high school (that is, grade 10) completion, as the majority of girls enrolled in grade 10 at the time of the follow-up survey are aged 15-16

> 28 The transition from middle to high school is a critical juncture in schooling attainment in Punjab as well as the rest of Pakistan, particularly in poor rural communities. In fact, rates of transition from middle to high school were low at baseline. The distribution of school attainment (measured as the number of years of schooling) for adolescents and adults exhibits a significant drop between grades 8 and 9 in both stipend and non-stipend districts.

19
same time, schooling might have diminishing marginal returns and increasing marginal costs. Thus, girls have more incentives to join the labor market as their ages and years of accumulated schooling increase. Additionally, since the FSSP seems to facilitate school progression, girls may begin working at earlier ages if they are able to complete their desired schooling sooner. In sum, while it is more possible that the program would decrease working in the short-term, the combination of these other factors may have an ambiguous impact on working over the long term. In order to shed light on this, this section presents results of program impacts on two outcomes related to work for girls aged 12-17: labor force participation and time devoted to working.

Participation in labor markets (the extensive margin of labor supply) is proxied by the probability of looking for a job and participation in work for pay or unpaid work (unpaid family helper and unpaid work outside the home). Results of DD and RDD models indicate that there is a statistically significant reduction in the labor force participation of around 4-5 percentage points among girls in stipend districts, which is largely driven by a reduction in girls’ participation in unpaid family work. Point estimates of all empirical models are not sensitive to different specifications, sub-samples, or approaches. Furthermore, the decrease in labor force participation is notable. Four years after the program started, the labor force participation rates of girls 12-19 are almost equal between stipend and non-stipend districts even though they were roughly 4-5 percentage points higher for the former group at baseline. Unfortunately, there are no available data (regarding time use, for example) to further investigate whether this reduction in labor participation is fully or partly explained by the extra time spent by beneficiaries in school-related activities (Table 8).

We also examine the effects of the program on the intensive margin of the labor supply for those girls who are working for pay at the time of the survey. Samples are restricted to not only girls that work, but also those that are 15-19 years old since no comparable data exist at baseline for younger cohorts. Although the average impact on girls 15-19 is statistically insignificant, the results indicate that cohorts aged 15-16 worked 7 days less per month (around 2.8 hours per day), relative to their counterfactuals in non-stipend districts (significant at the 10 20
percent level). This is also consistent with the hypotheses described earlier about higher impacts for the younger cohorts (Table 8).

# **_Marriage and Fertility Outcomes_**

There are multiple channels through which the program could influence marriage and fertility outcomes. Most of these channels are transmitted through the effects of the program on the decisions about education (and perhaps also work) — as girls have more incentives to stay in school longer, they could delay their marriages and sexual activity and alter their decisions about fertility (such as timing of childbearing, birth spacing, and total fertility rates). On the other hand, if the program increases grade progression and help girls finish school earlier, they might move into marriage and childbearing sooner.<sup>29</sup> Furthermore, social norms and cultural tradition in Pakistan may figure prominently in marriage and fertility decisions, such as making it difficult for girls to resist the pressure of marriage and child bearing. Besides, the local socioeconomic determinants of marriages may mean that the additional schooling brought about by the program could increase the economic status of women and with it the chances of getting married. When combined, these different factors could manifest themselves as opposite effects. Therefore, establishing the net effect of the FSSP is, to a large extent, a matter of empirical analysis.

We first investigate the possibility that the program changes the probability of marriage and age at marriage for girls aged 15 to 19. Overall, estimates from DD and RDD models do not show any impacts of the program on the probability of being legally married. However, when looking at age at marriage, the analysis identifies that after the program started girls in stipend districts are likely to marry about 1.2-1.5 years later. Results are significant in statistical terms and consistent across model specifications and subsamples with different age cohorts.

The analysis then turns to the impacts of the program on fertility outcomes. The sample for this part of the analysis consists of girls aged 17 to 19, since these are the ages when girls are most likely to have been married and possibly given birth. This group of girls was likely to receive the stipend during the early years of the program (2004-05) but not likely to be enrolled

> 29 Women in Pakistan typically move into childbearing shortly after marriage.

21
in middle school at the time of the survey (2007-08). The first outcome indicator measures the probability that girls have given birth. This analysis reveals no differential effects on the probability of girls 17-19 having given birth. A second exercise looks at the number of births given for the sub-sample of girls who became mothers during the period under study. The empirical analysis based on this sample indicates that, relative to the control group, girls exposed to the FSSP have on average 0.3 fewer children. However, this result is marginally significant around the 10 percent level.<sup>30</sup> Even though both are partial measures of fertility given the young age of the girls and do not reflect fertility targets, the findings may still signal some changes in fertility that may be sustained over the long-term. In the event that the total fertility rates of both treatment and control groups converge over time, the reduction in the number of children by early adulthood still implies that girls in stipend districts tend to delay their subsequent births (Table 9).

# **_Heterogeneity of Program Impacts_**

This section presents evidence on the heterogeneity of the FSSP effects across some socioeconomic characteristics of program beneficiaries such as socioeconomic status, parental education, location (rural or urban), age, and length of exposure of the beneficiaries.<sup>31</sup> We first find that girls who live in rural villages and have parents with no education benefit much less from the program in terms of the probability of completing middle school, but their labor participation also decreases more. The evidence indicates that most educational benefits (as measured by middle school completion) attributed to the program accrue to girls who belong to households in urban centers and have parents with at least primary education. On the other hand, these girls also reduce their participation in the labor market less than girls living in rural areas and with parents that have no education. Unfortunately, with the data available, it is not possible to tease out the underlying reasons that may help reconcile the lower school attainment and lower labor force participation identified for girls in these rural, less-educated households (Table 10).

> 30 A caveat of this analysis is that the follow-up survey has a significant number of missing values for the number of births. Further calculations show that this issue is equally prevalent in both stipend and non-stipend districts. Moreover, the problem of missing values does not appear to be systematically correlated with some determinants of fertility.

> 31 Estimates from the DD and RDD models shown in tables 4 to 8 were obtained from a DD framework for the whole sample with a control function of district literacy rates that determine program eligibility. The fact that the results are very similar may suggest that there is low impact heterogeneity across districts. Yet, this does not imply that there is not impact heterogeneity for different types of subgroups defined based on some socioeconomic characteristics.

22
Program impacts on labor force participation also appear to be larger for the poorest households. Reductions in labor force participation associated with the FSSP among girls from poorer households (those below the median consumption level of the district) are more than twice those found for the rest of the beneficiaries. There is also some suggestive evidence that participant girls in the poorest households are more likely to delay marriage.

There is evidence in the data that the effects of the FSSP on educational outcomes vary with the year that the girls join the program and the length of exposure. As noted in the previous section, the results show that the younger cohorts of girls who joined the program after two years of implementation are more likely to complete middle and high school. As noted above, the two most plausible reasons of this are an increase in the awareness of the program (due to informational campaigns launched in the second year of operation of the program) and the expansion of the FSSP introduced in 2006-07 to cover grades 9 and 10. Finally, the analysis also shows that girls with more than one year of exposure to the program are more likely than other beneficiaries to complete one grade of high school and less likely to be married at the time of the survey. This may provide suggestive evidence of positive marginal effects of treatment length.

# **_Impact on Male Siblings_**

In theory, educational stipends that are targeted to girls could have indirect effects on households’ investments in the education of ineligible siblings. Models of schooling decision show that, in addition to income and substitution effects, school-focused conditional transfers also have a displacement effect. While for eligible children the three effects tend to produce positive impacts on enrollment, for ineligible ones the displacement effect often runs in the opposite (negative) direction. For instance, due to the income effect of the transfers, budgetconstrained households may be more able to keep all children in school or enroll those out of school. In contrast, the displacement effect may incentivize parents to reallocate child work (inside and outside the home) away from eligible girls toward boys and other ineligible girls. As a result, the net indirect effects of a program depend on the relative size of these opposing forces.

The existing literature gives mixed evidence on the indirect effects of CCTs on ineligible children. In Colombia, for example, the negative displacement effect was found to offset the 23
positive income effect in the context of a CCT program (Barrera-Osorio et al. 2008). While the program increased the enrollment of recipients, ineligible siblings were more likely to drop out of school and enter the labor market. In a contrary example, evidence for a scholarship program in Cambodia that targeted poor children making the transition from primary to lower secondary school shows that the school enrollment of ineligible siblings was not affected by the program (Ferreira et al. 2009).<sup>32</sup> We exploit three different identification strategies to examine the spillover effects of the FSSP effects on current enrollment, school completion (primary and secondary), and school choice (private or public) for boys in school age (6 to 17 years old). The first strategy implements a DD analysis that compares boys in households with at least one exposed girl in stipend districts to similar boys in non-stipend districts at baseline and follow-up. The second strategy also looks at relative changes over the longer term but uses boys in households without exposed girls in stipend districts as the comparison group. Finally, the third strategy integrates both of the designs above in a triple difference framework (DDD). The analyses draw on household data from MICS (2003 and 2007-08), and the subsamples used for each of the outcomes are selected based on gender-specific age and grade restrictions.

None of the empirical models estimated for each of the research designs reveal any spillover effects of the program on the enrollment and school completion of boys. The results, summarized in Table 11, indicate that the enrollment and school completion rates of boys with female siblings who have been exposed to the FSSP exhibit statistically similar trends as boys with no eligible siblings in both stipend and non-stipend districts. The only exception is a decrease in enrollment of 3 percentage points in one of the models of the DD analysis that is based on the comparison with boys living with ineligible girls in stipend districts. Yet, this result does not hold when the analysis is extended to DDD models.<sup>33</sup>

> 32 The CESSP Scholarship Program in Cambodia provides modest transfers to eligible girls that are equivalent to approximately 2-3 percent of the total expenditures of the average recipient households. The relative size of the GSP in Pakistan is similar. The monthly stipend represents 3.4 percent of median household expenditures of the recipient households in 2004 (Ferreira and others 2009; Chaudhury and Parajuli 2006).

> 33 Results that look at the average years of schooling completed do not reveal any effects either. Comparisons for school-age boys at baseline and follow-up indicate that this result is not driven by systematic difference in of age-grade distributions.

24
Consistent with what was observed through qualitative work conducted in the field as part of this impact evaluation, there is, however, indication of strong indirect effects of the program on school choice among boys. We find that families with daughters eligible to the FSSP tend to respond to the program by enrolling their sons in private schools. In fact, several econometric models show a statistically significant average increase in enrollment of boys in private schools of nearly 4 percentage points compared to a baseline level of 30 percent. Further analysis indicates that this increase in the share of male enrollment in private schools is largely driven by boys 6 to 12 years old who were sent to primary schools.<sup>34</sup> This behavioral response may be explained by reasons related to the supply, cost, and quality of private schools that offer primary-level education. First, there has been a significant increase in the number of private primary schools in rural and urban Punjab in the 2000s. Second, fees in private schools are often low and below the stipend provided by the FSSP (200 Rs. per month). In fact, a national census of private schools in 2000 found that the median rural private school charged 60 Rs. per month.<sup>35</sup> Third, there are also differences in quality; children who study in private schools score substantially higher in tests in all subjects (Andrabi and others 2006). Therefore, a potential indirect effect of the FSSP may introduce gender disparities in learning as more boys in eligible households are enrolled in better performing schools. More work, however, is needed to better understand this possible indirect effect.

# **7. Robustness Checks**

As eligibility to the FSSP was not assigned randomly, the analysis conducted in this paper has to rely on quasi-experimental techniques and their corresponding identifying assumptions. To assess the internal validity of this evaluation, we discuss the robustness of the results to a few issues that could contaminate the design of the evaluation. We start with the underlying assumption of the double-difference analysis, namely that the outcomes investigated were progressing along similar trends across stipend and non-stipend districts before the program began. Program impact estimates based on double-difference models may be biased if either of the outcomes for girls in stipend districts (who started from a lower base) was converging to the value in control districts even before the program was in place.

> 34 Aslam (2009) also finds that boys are more likely than girls to enroll in private schools in Punjab. 35 School fees in the private sector may increase by more than three-fold at the secondary level.

25
Yet, there is no evidence that longer-term impacts of the program identified in this paper are driven by differences in pre-program trends. Two main checks were conducted to directly investigate this. The first uses pre-baseline and baseline data from PIHS (2001) and MICS (2003) to run double-difference models on schooling, labor market, marriage, and fertility outcomes for the same cohorts of girls.<sup>36</sup> In principle, one should not expect to see impacts of the FSSP since the program began after the period covered by this _placebo test_ . In fact, results of this analysis show that all outcomes examined in the paper were progressing in similar trends for stipend and non-stipend districts at the preprogram time. The second check consists of doubledifference models using data from MICS (2007-08) to examine the trends of school attainment for two slightly older cohorts of girls (ages 20-22 and 23-25) that, by definition, finished school before the program started and, thus, did not benefit directly from it. Again, results of these exercises do not reveal any differential pre-program trends that may contaminate the main findings of the paper (Table 12).

Another important concern stems from the fact that the empirical analysis is based on a proxy measure of the level of exposure of girls to the program. Consequently, program duration for the synthetic cohorts used in the analysis is likely measured with some error. This measurement error in the indicator of treatment could produce imprecise or biased estimates of program impacts. If this is the case, the parameters that measure the effects of the FSSP on the longer-term outcomes would be biased toward zero (also known as ―attenuation bias‖). Although there is no sensible way to correct for this type of bias in the context of this evaluation, the results of the analysis are still very informative as they can be interpreted as lower bounds of the true impacts.

There are various problematic compositional changes in the sample that could take place. An example is that the FSSP likely induces the self-selection of lower–ability female students into middle schools. The marginal girl who is most likely to be brought into school by the

> 36 A limitation of this analysis is that PIHS 2001, the household survey used for the prebaseline period, is representative only at the province level. The survey was structured around three main domains: rural, major cities, and other urban areas. The first two domains (rural and major cities) were stratified by district and, thus, can be used to construct the treatment status of girls. The ―other urban‖ domain was stratified by division, a higher administrative level that may include both stipend and non-stipend districts. Therefore, the DD analysis is restricted to the ―rural‖ and ―major cities‖ domains and assumes that the sample statistics based on PIHS 2001 are representative for the two groups of analysis as a whole (that is, stipend and non-stipend districts).

26
stipend program may be, on average, relatively poorer, less motivated, and spend less time on school work than the girls who are already enrolled in school.<sup>37</sup> Thus, there is a possibility that at least some of these new entrants are drawn from the left-hand side of the ability distribution. As a result, there may be a selection bias on the expected returns since the new entrants are expected to benefit less from schooling compared to the girls who were already attending school regularly before the program started. This type of selection bias may be important for evaluations like this one that investigate the impacts on educational indicators such a progression and completion, which condition on school enrollment (itself an outcome of the program). Although the findings of this paper are not adjusted for the self-selection of lower-ability students, the sign of the bias suggests that they still provide lower bounds of the actual impacts.<sup>38</sup> Similarly, there is no sign the program causes changes to the age-grade distribution in stipend districts. In addition to bringing new entrants with relatively lower expected returns into school, the program could have motivated overage girls to go back to school. This may add extra noise to the construction of the synthetic cohorts and the measures of treatment status. However, comparisons of the age-grade distributions at baseline and follow-up within stipend districts, as well as between stipend and non-stipend districts, do not indicate that over-age girls are more likely to enroll in schools in stipend districts after the FSSP began. Furthermore, various model specifications allow for cohorts that are wide enough to capture over-age girls. We do not find evidence that migration could affect the composition of the samples of analysis.

An additional recurrent concern for evaluations of this type is that the program could either attract migrants from control regions to stipend regions or stimulate permanent migration among program participants. The decision to migrate permanently may be influenced by factors that also determine participation in the program and the outcomes of interest (that is, endogenous migration). Both issues may alter the composition of the samples in a way that compromises the internal validity of the analysis. Yet, there are several reasons why this may be less of an issue in this paper. First, the modest size of the transfer (nearly $2.5 per month) is not expected to encourage households to move from richer non-stipend districts to stipend districts. Second, out-

> 37 Filmer and Schady (2009) find evidence of this type of selection to explain why a scholarship program targeted to poor students in Cambodia does not appear to have positive effects on academic tests.

> 38 An alternative to correct for this type of selection is to construct either parametric or nonparametric bounds on quantile-specific program impacts based on symmetric truncation of the distribution of the treatment and control groups. See Lee (2002) and Angrist and others (2004).

27
of-province family migration in both stipend and non-stipend districts is rather uncommon, particularly in rural areas. Instead, most migration is in the form of temporary out-migration of a member of the household — often of working men. It is very unlikely that this migration was affected by the stipend program. Finally, some simple econometric models give no evidence that the FSSP affect either type of migration after controlling for most of the covariates used in the analysis.

A final compositional issue could arise if the conditional stipend motivates households located in non-stipend districts but adjacent to stipend districts to send their girls to schools there. This would increase the demand for education in stipend districts but reduce it in non-stipend districts, leading to no changes in net enrollment across the province. In order to check the possibility of crossover effects between contiguous stipend and non-stipend districts, econometric models of educational outcomes were estimated on a subsample of schools and households located in treatment and comparison districts that do not have common borders.<sup>39</sup> The direction, magnitude, and significance of the findings using these samples are for the most part consistent with the results obtained with the full sample, with the exception of some models that employ school level data for which the findings are qualitatively similar but not statistically significant.

A natural issue for a program like the FSSP focused exclusively to public schools is families could have moved their daughters who were previously enrolled in private schools to public schools in order to become eligible for the stipend. As a result, the observed increase in female enrollment in the public system may be offset (partially or completely) by a fall in enrollment in private schools, hence reducing the overall effectiveness of the program. Nevertheless, the main set of results of this paper uses household-level data and defines progression and completion irrespective of the type of school in which the girl is enrolled. Therefore, the DD and RDD models using these data provide estimates of the net impacts of the FSSP that already account for the possible crowding out effect induced by the program.<sup>40</sup>

> 39 The sample size of these models is reduced in some cases by more than 50 percent.

> 40 At baseline, there were no statistically significant differences between stipend and non stipend districts in the proportion of girls switching from public to private vs. private to public at the middle school level

28
In order to check for potential issues of selection in participation, we also run empirical models for all the outcomes on the sample of girls eligible to the program irrespective of their likely participation in the program The sample for this includes girls ages 12-19 that were eligible for the program even though they were never enrolled in grades 6-8 during the duration of the program. However, although impacts estimates based on this larger cohort are closer to the intent-to-treat parameter, a problem of this approach is that the results of this analysis are likely to get diluted as the units of observations may include a significant number of girls that did not participate; they could have left school before the program started and never rejoined.<sup>41</sup> Yet, some of the results from the base models remain.<sup>42</sup> A final concern for internal validity is that certain impacts on educational outcomes could be mistakenly attributed solely to the program when they are actually the combined effects of the stipend (a demand-side intervention) and changes on the supply side. This may be the case if more schools — particularly public schools — were built at the same time as the program in stipend than non-stipend districts. There is, however, no evidence of such systematic change in school supply. In fact, data from the annual Punjab school census (EMIS) for the period 2003-10 shows that the number of public schools was more or less unchanged. For instance, during 200407, the number of schools in stipend and non-stipend districts increased by only 0.34 and 0.15 percent, respectively, compared to the number of schools at baseline. Furthermore, the trends in the school supply of stipend and non-stipend districts are very similar in the period before and during the program, 2000-07 (Figure 1).

# **8. Conclusions**

This paper aims to fill the evidence gap on longer-term impacts of CCTs by evaluating a femaletargeted CCT program in Pakistan, the Female School Stipend Program. The evidence indicates that impacts are sustained beyond the short term and could help reduce gender gaps in schooling.

> 41 This bigger cohort could be problematic as it may include girls who completed grades 5, 6 and 7 but stopped going to school well before the program started

> 42 For middle school/grade 9/grade 10 completion, and middle to high school transition outcomes, average impacts on cohort age 12-17 are not statistically significant but have a positive sign across alternative definitions. However, for the cohort age 15-16, schooling outcomes are positive and statistically significant for some definitions. Impacts on labor outcomes are robust to the inclusion of eligible but not exposed girls, but with lower significance levels (5%-10%). Results for marriage and fertility outcomes (with the exception of age at marriage) have a negative sign across alternative definitions, but are not statistically significant.

29
Indeed, after four years of implementation, the FSSP is found to help girls in stipend districts progress through and complete middle school. Furthermore, the younger cohorts of girls in stipend districts in particular are also more likely to matriculate into and complete at least one high school grade. In terms of labor force participation and marriage and fertility decisions, there is some evidence (although in some cases marginally significant in statistical sense) that the program encourages girls to work less, delay their marriage, and have fewer children.

We also find that, as a result of the stipend, boys residing in the same household as girls in participating districts are more likely to be enrolled in private primary schools. These schools are growing in number, more affordable — school fees are lower than the stipend — and perform better in terms of students’ learning, which means the FSSP may indirectly increase the gender gap in learning, at least at the primary level. However, further research is needed to investigate whether learning disparities may arise as a result of this change in the school choice of boys. If this is the case, the evidence should perhaps motivate actions to either address the discrepancy in quality between private and public schools or expand the FSSP benefits to include private schools so that girls have similar school choice as boys.

The impacts on educational attainment may have important implications for future productivity and welfare of beneficiaries. As in many other developing countries, the gradient between education and poverty is evident in Pakistan. Women who complete middle and high school, for instance, live in households that enjoy nearly 14 and 29 percent (equivalent to $5.3 and $10.8) higher monthly consumption per capita, respectively, relative to women with less than middle school education. A big part of the reason is that education provides women with more marriage opportunities. Another reason is that higher schooling also enables women to attain higher earnings. Simple back-of-the-envelope calculations estimate that the female wage premiums for middle and high school education (relative to lower than middle school education) are 24 and 157 percent, respectively. These improvements represent an annual increase in earnings of $16 and $100, respectively, which might more than compensate the annual stipend cost of the program ($34).

In addition, the impacts of the FSSP may have further dynamic effects on other dimensions. First, women who have higher education and/or marry at later age tend to have 30
fewer children. In fact, women in Punjab with middle and high school education have around 1.8 fewer children than those with lower than middle school education by the end of their reproductive life. Simple extrapolations also indicate that the 1.4 year delay in marriage of beneficiaries associated with the program could lead to 0.4 fewer births by the end of their childbearing years. Furthermore, more educated women are expected to not only have fewer children but also invest more in their human capital, which may lead to positive intergenerational effects.

The results of this paper could provide critical information for policy makers in assessing the effectiveness and efficiency of programs like the FSSP; however, more needs to be done in future research to more fully explore the dynamics of program impacts and the effects on learning. First of all, many of the beneficiaries observed in this paper are still young (12-19 years old), so they might not have reached their full potential in terms of human capital. In addition to continue investigating the impacts on final school attainment and fertility, more needs to be done in subsequent works to identify the effects of the program on future productivity and wages. Second, and related to this, it would be useful to understand whether the impacts on schooling are also translated into improvements in learning. Knowledge on these areas is critical to establish whether CCTs and comparable programs also contribute to poverty reduction in the long-term.

31