rafmacalaba/data-use-mentions-extended huggingface dataset:
Policy Research Working Paper 5465

# **Abstract**

Low student learning is a common finding in much of the developing world. This paper uses a relatively unique dataset of five semiannual rounds of standardized test data to characterize and explain the short-term changes in student learning. The data are collected as part of the quality assurance system for a public-private partnership program that offers public subsidies conditional on minimum learning levels to low-cost private schools in Pakistan. Apart from a large positive distributional shift in learning between the first two test rounds, the learning distributions over test rounds show little progress. Schools are ejected from the program if they fail to achieve a minimum pass rate in the test in two consecutive attempts, making the test high stakes. Sharp regression discontinuity estimates show that the threat of program exit on schools that barely failed the test for the first time induces large learning gains. The large change in learning between the first two test rounds is likely attributable to this accountability pressure given that a large share of new program entrants failed in the first test round. Schools also qualify for substantial annual teacher bonuses if they achieve a minimum score in a composite measure of student test participation and mean test score. Sharp regression discontinuity estimates do not show that the prospect of future teacher bonus rewards induces learning gains for schools that barely did not qualify for the bonus.

This paper—a product of the Education Unit, South Asia Region; and the Education Unit, Human Development Network—is part of a larger effort of the departments to rigorously evaluate innovative government programs supported by World Bank lending operations. Policy Research Working Papers are also posted on the Web at http://econ.worldbank. org. The authors may be contacted at fbarrera@worldbank.org and draju2@worldbank.org.

_The Policy Research Working Paper Series disseminates the findings of work in progress to encourage the exchange of ideas about development issues. An objective of the series is to get the findings out quickly, even if the presentations are less than fully polished
receipt discreetly at a different time that verifies how much was deposited into their savings account. Consequently, our protocols ensure the privacy of savings deposits—while leaving the process of obtaining payment of the cash component of earnings on paydays unchanged. These payment protocols were the same for both the Private and Non-private treatments.

_Enrollment and roll-out._ In Phase 1, workers were enrolled in the experiment in three staggered waves, with data collection for Phase 1 ending in March 2019. Phase 2 was then conducted from April 2019-July 2019.

# **6 Empirical strategy**

## **6.1 Data sources**

Our primary data source is Olam’s detailed daily administrative data at the worker level. This includes individual daily attendance, output (the quantity of nuts processed), and the internal validity of our estimates, since treatment assignment is stratified within a given phase.

21
These numbers fit relatively well the average probabilities shown above and show that the model supports a scenario in which government-led renegotiations following a previous renegotiation are only a small subset of all renegotiations at the initiative of the government.

### Stationarity of the βs.

Another important point to consider is the question of the stationarity of the βs in a multiperiod model. When there are successive renegotiations, it could be questioned whether such a stationarity assumption is realistic. Given the complexity of the model, however, we leave this issue for further analysis (see for example Laffont and Martimort, 2002) and content ourselves with observing that this problem is made less relevant by the fact that overall successive renegotiations are a rather small subset of the total.

Moreover, note that empirically this provides justification for the way projects are handled in our data base. They leave the sample when they are renegotiated, and enter as a new, different project if the concession carries on with significant modifications, thus avoiding the problems of confusing projects which distinct underlying technological characteristics.

# 3 Empirical Evidence

## 3.1 The Data

We use an original data set, developed by the World Bank, which describes the characteristics of nearly 1,000 concessions awarded in Latin American and Caribbean countries from 1982 to 2000<sup>19</sup> . As in GLS, and to keep results strictly comparable, we again focus on water and transport concessions in five countries (Argentina, Brazil, Chile, Colombia and Mexico), between 1989 and 2000. We have information on the general details of the projects (sector, activity, year of award, award criteria, size and duration of the concession), on the institutional and regulatory context, the type of regulatory framework put in place (price cap or rate of return), as well as the evolution of the main economic variables (growth, exchange rate, inflation) and the timing of national and local elections. Table 1 presents the full list and definitions of variables used in
DCFTA that are about 2.5 times larger than the gains estimated in the present study. We explain in appendix I that the larger estimated gains of the CASE study are due to due to a combination of two effects: (i) larger assumed distortions in the CASE study; and (ii) different modeling assumptions. The larger the distortions are, the more gains there are from their removal. As we explain in appendix I, the larger distortions in the CASE study partly reflect the fact that our study was based on estimates of distortions in 2010, while the CASE study uses estimates of distortions from 2006 or 2007. Since Armenia has implemented substantial reforms in the interim, the initial distortions in the CASE study are significantly higher. This is the case with border costs. But it is also due, in some cases, to the fact that we had greater data available to us that allowed a more accurate estimate, for example in the services survey and estimates that we conducted. Further, we assume that the benefits of services commitments in a DCFTA are limited to EU investors, while the CASE study assumes those commitments will be extended multilaterally.

The paper is organized as follows. In section II, we provide an overview of the estimation of the ad valorem equivalents of barriers in Armenian services sectors. We provide an overview of the model in section III and a discussion of the data in section IV. The central results are presented in section V and sensitivity results are presented in section VI. Conclusions are presented in section VII. In appendix A, we discuss the trade and tariff data in some detail. We document the calculation of ownership shares by sector and region in appendix B. How we obtained estimates of the Dixit-Stiglitz elasticities in goods is described in appendix C. Our estimation of the reduction in trade or border costs as a result of a DCFTA is presented in appendix D and our estimate of the reduction in standards costs is presented the section E. The estimates and methodology of the ad valorem
# **<mark>1. Introduction</mark>**

Women earn less income and are less likely to participate in the labor market, especially in low- and middle-income countries (World Bank, 2012). There are several ways to increase women’s involvement in economic activities, such as micro-credit program <mark>s, self-help groups, or programs that are specially designed to help</mark> improve their access to infrastructure and technology.<sup>1</sup> In this paper, we study a simple but important factor—childcare—that can release women from domestic work and encourage them to participate in the labor market. Specifically, we examine the impacts on women’s labor market outcomes of pre-school (age 1-5) childcare in Vietnam.

Analyzing the most recent household surveys for Vietnam in the past decade, we find that childcare (attendance) has a strong effect on women’s participation in the labor market, as well as the probability of their working in a formal wage-earning job. <mark>Childcare also helps increase women’s total annual wages and household income per capita and reduces poverty. The effect of childcare is larger for younger children than older ones. For younger children, moreover, a medium-term effect exists after 2 years, increasing the probability that women will have a wage-earning job by 38 percentage points and reducing self-employed farm work. We also find heterogeneous effects of childcare on maternal employment, where its impact on the probability of having a wage job is larger for younger and highly-educated women. Interestingly, childcare positively impacts the probability of men having a wage-earning job as well.</mark> A key challenge in measuring the effects of childcare is the endogeneity issue. Women who send children to childcare services differ from those who do not (e.g., they may be richer or simply have less time for childcare). We address this problem by using the threshold in the birth months of children—which is exogenously given
# **Table 1. Oil Subsidy and CO2 production**

|**Dependent**<br>**Variable:**<br>|**(1) All samp**|**le countries**|**CO2 Em**<br>**(2) MENA**|**issions**<br>**subsample**|**(3) MENA oilgr**|**oup subsample**|
|---|---|---|---|---|---|---|
|**populaton**|0.00000120|0.00000415<sup>***</sup>|0.00000278|0.00000179|0.00000986<sup>***</sup>|0.0000103<sup>**</sup>|
||(1.43)|(2.86)|(0.72)|(0.38)|(4.10)|(3.00)|
|**GDP**|1.99e-10<sup>***</sup>|7.27e-11<sup>**</sup>|1.72e-10<sup>*</sup>|2.16e-10|1.94e-10<sup>**</sup>|3.03e-10<sup>*</sup>|
||(14.52)|(2.60)|(1.89)|(1.21)|(3.48)|(2.25)|
|**fossil fuel**<br>**subsidies as a**|-660695926.3|-484726044.4|-121424410.6|25300623.5|376621923.3<sup>**<
countries than in other countries at similar income levels.<sup>6</sup> Part of this is due to high rents and urban transport costs (respectively 55% and 42% higher than in comparable places elsewhere) although it extends to other commodities. Prices of food and other goods are also relatively high. Henderson and Nigmatulina (2016) show that, across developing country cities, high prices are negatively associated with a measure of connectivity between people in the city.

Our model captures these features, and is developed in a series of stages. Section 2 lays out the ingredients, characterizes equilibrium and undertakes comparative statics to establish the determinants of city specialization, as well as of city size, density and rent levels. While Section 2 maintains the assumption of constant returns to scale in tradable production, Section 3 moves to environments with increasing returns, establishing the possibility of multiple equilibria and the roles of history and expectations in shaping outcomes. Section 4 concludes and offers policy implications.

> 5 These findings on labor costs and price levels are consistent with earlier work by Gelb et al. (2013, 2015), although this work does not have an urban focus.

> 6 Nakamura et al. also report findings from the Economist Intelligence Unit’s Worldwide Cost of Living Survey.  While this is produced principally for expatriates its findings are consistent, indicating African cities are 30% more costly for households than cities in low- and middle-income countries elsewhere.

5
6

- Related to this point, the fact that IDAAN enjoys such a high degree of discretion in determining the identity of the beneficiaries raises the risk that subsidies may be misdirected for political ends.

- The tendency to use non-payment of bills as an eligibility criterion for the subsidy scheme introduces a perverse incentive for households not to pay their bills.

- The subsidization of the full value of bills for the 'social cases' undermines any incentives for these households to make efficient use of water resources, and, moreover, means that they are losing the habit of paying for these services.

This analysis suggests that the existing subsidy apparatus could not continue in its present form.

# **The Incidence of Poverty**

A fundamental starting point in assessing the need for a new subsidy programme is to characterize the incidence of poverty among the customers of IDAAN. In its official definitions, the government of Panama distinguishes between two levels of poverty.

- _The poverty_ line-defined as the level of expenditure needed to pay for a basket of basic services comprising nutrition, housing, transport, health and education. In 1997, it was set at US$726 per capita per year.

- _The extreme poverty_ line-defined as the level of expenditure needed to pay for a basket of basic nutrition alone. In 1997, it was set at US$470 per capita per year.

On this basis, Table 3.1 summarizes the geographical distribution of poverty in Panama using data from the national Living Standards Measurement Survey conducted in 1997. The data clearly indicate that poverty in Panama is predominantly a rural phenomenon. This is an important finding, given that IDAAN only services the urban areas of the country. Indeed, only 11% of the urban population live in conditions of poverty, while less than _5%_ live in conditions of extreme poverty.

**Table 3.1: Distribution of poverty across rural and urban areas in Panama**

||**Extre**|**me poor**|**Po
**Table 4:** Effects of Turnovers on Nepotism

||**Village head survey**|**Bureau**|**crat survey**|
|---|---|---|---|
||Employs relative|Parent was leader|Parent served ingovt|
||(1)|(2)|(3)|
|New village head|-0.385<sup>_∗∗∗_</sup>|-0.066|-0.168<sup>_∗∗_</sup>|
||(0.155)|(0.043)|(0.069)|
||[0.178]|[0.051]|[0.082]|
|Observations|442|1067|1067|
|Control mean|0.36|0.054|0.27|
|Robust p-value|0.008|0.109|0.0340|
|<br>Bandwidth size (%)|12.7|17.5|22.5|
|Effective obs.|149|466|550|

_Notes_ : This table reports RD estimates of _γ_ in equation (1) obtained via the non-parametric method from Calonico et al. (2014). Units of observation are village heads in column 1 and bureaucrats in columns 2-3. The dependent variable is: in column 1, a dummy equal to 1 if relatives of the village head are employed in the village government; in column 2, a dummy equal to 1 if the bureaucrat reports having a parent who served as village head; in column 3, a dummy equal to 1 if the bureaucrat reports having a parent who served in the village government. See Section 4 for details. * p _<_ 0.1, ** p _<_ 0.05, *** p _<_ 0
# **4 Discussion and Conclusion**

Teacher incentives have the potential to encourage teachers to exert more effort and ultimately lead to increase in student learning, but they could also lead to distortionary behaviors which could be detrimental for learning. The details of program design have been shown to matter for impacts, especially if they interact with teachers’ intrinsic motivations differently. In the context of primary schools in Guinea, this impact evaluation shows that teacher incentives led to improvements in student learning outcomes after one year of program implementation —with in-kind rewards having impacts that are roughly twice as large as those of recognition rewards. After two years of the program, the impacts are substantially muted —they roughly halved in size— and no longer generally statistically significantly different from zero. This reduction in effect sizes was likely driven by the Ebola epidemic that was emerging close to the two-year mark of the program. Analysis of enrollment patterns across years and across cohorts does not suggest that there were any perverse enrollment impacts of the program. Data from the school inspections suggest an improvement in teaching preparation and practice: teachers made better use of resources in the classroom, were better prepared for the class, and implemented better teaching practices. Moreover, the results from lesson observations mirror the results on student learning.

Male and female teachers respond differently to the different types of incentives. While there are many potential explanations for this result, we have suggestive evidence that the different types of incentives may be activating different underlying sources of motivation. In particular, the degree to which a teacher identifies themselves with the profession—potentially signaling a greater degree of intrinsic motivation—might be related to their responsiveness to the recognition incentive.

These results have potential implications for policy. They suggest that teachers do indeed respond to performance incentives, and can do so in ways that improve student learning outcomes, even in a low-capacity environment. However, non-pecuniary rewards—which are often preferred by policy makers—might only be effective for some types of teachers.<sup>16
languages were introduced in 1993, this group comprised of Somaligna (6%) and three languages within the diverse Southern Nations, Nationalities, and Peoples’ Region (SNNPR) totaling an additional 4 percent of the population (Boothe and Walker, 1997; Zenebe Gebre, 2014).<sup>22</sup> At this point every language spoken by at least two percent of the population had been introduced, and as time passed political calculations are likely to have a role in the selection process for less prominent languages. Therefore, this paper focuses on the implementation of the initial phase of the MTI project.

The final key aspect of the introduction of the MTI program was the need to translate the primary school material from Amarigna into the new languages. Up to this point in time, any written translations of these languages had used the Ge’ez script; however, due to the historical connotations of the use of Amarigna, each region was given the option to use a different script in translation, and every region outside of Tigray selected to translate their schooling material into the Roman script. For Tigray, this allowed them to translate all of the material locally and introduce the MTI program on schedule in 1991 (Boothe and Walker, 1997;

> 21For consistency, each language will be referenced using the name from the 2007 Ethiopian Census.

> 22The languages were Hadiyigna, Gedeogna, and Kembatigna.

A.11
Fourth, an examination of the firm-level dynamics in Eastern Africa does not support the view that preferential access led to durable benefits. For this analysis, we draw upon customs transaction-level data for four representative African countries. The sustained dynamism of Kenya and the late growth of Ethiopia in their exports of apparel to the US were driven largely by new exporters that entered the market after 2010, rather than by incumbent exporters that had benefitted from large preference margins during the early AGOA period. This pattern rules out dynamic benefits internal to the firms but we cannot rule out dynamic externalities across firms within the apparel industry. In Madagascar and Mauritius, a large-scale exit of firms accompanied the contraction of exports after the end of the MFA and the withdrawal of AGOA benefits to Madagascar in 2009. Exports recovered only after Mauritius was granted liberal rules of origin (ROO) in 2009, and hence a wider preference margin, and Madagascar regained AGOA preferences in 2015. This pattern suggests that the ups and downs in firms’ export performance were driven by fluctuations in the preference margin rather than by dynamic benefits internal to the firm. Furthermore, growth was concentrated in surviving firms with almost no role played by new firms which does not support the existence of dynamic externalities. Thus, in none of the four countries is there evidence of firms learning-by-doing and becoming internationally competitive due to the preferential access they enjoyed in the early AGOA period.

Finally, we explored the reasons for the heterogenous responses to AGOA. While a thorough analysis was not possible, two reasons may explain why the impact of AGOA differed across African countries. Low tariffs on own imports may help explain the initial success of Southern African countries, because such regimes allowed easy access to imported inputs even compared to that in other countries where duty-drawback and other schemes involved higher transactions costs. The establishment of effective special economic zones (SEZs), which combined liberal trade 6
> of these regressions is also reported. ***, **, and * indicate significance at the 1%, 5%, and 10% levels, respectively.

of the policy, the amount of NBE bills (in black) went from zero to roughly 500 million birr in April 2011 and kept growing steadily afterwards. At the same time, safe assets (which include NBE bills) increased one-to-one with this variable. This shows that the policy was successful in leading Ethiopian banks to become safer with the introduction of the policy, and banks did not substitute away from one type of liquidity (bank-to-bank deposits, cash, etc.) to NBE bills, but rather increased the overall aggregate.

Last but not least, we have referred to this policy as a macroprudential regulation as well as a “financial regulation tax”. Indeed, we find that bank profitability suffers in the aftermath of this. For this purpose, we collect data from bank annual reports and run a similar regression to equation (2): The difference is that while balance-sheet variables (NBE bills, safe assets) are measured monthly, the profits are only measured yearly. Analogously to the previous subsection, we report the coefficients _βy_ of equation (3) in Table 3 and plot the yearly coefficients in Figure 2. By inspecting both elements, 12
- 19 -

agriculture to nonfarm activities (Freeman and Norcliffe, 1985; Okelo, 1973; and Williams and McClintock, 1981). Evidence from Kenya an4 Sierra Leone suggests that agricultural surpluses account for between 15 and 40 percent of nonfarm investment funds (Child, 1977; Liedholm and Chuta 1976).

Surpluses generated in nonfarm activities likewise furnish funds for productivity enhancing investments in agriculture. Kitching (1977), in a review of sixteen farm management surveys from East Africa, finds repeated evidence of nonfarm earnings playing a crucial role in farmer acquisition of productive agricultural assets, especially land. The positive impact of off-farm earnings on agricultural investment has also been observed in Kenya (Collier and Lal, 1986; Haugerud, 1984), Nalawi (Cleave, 1974), Nigeria (Berry, 1975) Tanzania (Cleave, 1971) and Uganda (Nassel and Parnes, 1969).

B. LaboL FlwEl Monfarm labor usage moves contracyclically to demands of the agricultural calendar resulting in substantial seasonal labor flows between the rural farm and nonfarm sectors. We estimate that 20 to 40 percent of the rural labor force works in both farm and nonfarm activities, this range representing the sizeable magnitude of the labor flows moving back and forth between the rural farm and nonfarm sectors. 10/ C. Backward Linkages from Agriculture to Rural Input Suppliers Rural enterprises can supply some of the agricultural inputs required by farmers. For example, seasonal data from Sierra Leone indicate that blacksmithing activity reaches its peak during the height of the agricultural season because of the demand for repair services (Liedholm and Chuta, 1976). Of course, the type and magnitude of backward linkages varies depending on agricultural technology, size of holding, type of crop, and whether production is irrigated or rainfed.