rafmacalaba/datause-encoder-data huggingface dataset:
journal articles in SSCI- and SCOPUS-indexed journals, we also present data on citations in Google Scholar to Bank publications other than journal articles.<sup>17</sup> Table 5 reports citation data from Google Scholar for different types of publications. Nearly 30% of Bank publications have never been cited. The “failure rate”—and the mean and median citations per publication—vary by publication type. The types with the lowest failure rates and the highest median citations per publication are journal articles and working papers: only 15% of working papers fail to get a single citation; the figure for journal articles is 22%. Failure rates are highest, and median citations lowest, for book chapters and conference volumes; book chapters, for example, have a failure rate of nearly 50%. Books lie in the middle in terms of failure rates and median citations, with a median citation of 39 and a failure rate of 30%.

So we find that, at least for World Bank publications, volume and scholarly influence go hand-in-hand: the more common types of Bank publication (journal articles and working papers) are also more influential in terms of citations. This is reflected in the h-indices. World Bank staff have authored 260 journal articles that have been cited at least 260 times, and have authored 155 working papers that have been cited at least 155 times (there is some overlap between these categories). By contrast, they have published only 135 books that have been cited at least 135 times and only 125 book chapters that have been published 125 times.

Figure 4 shows the citation curves for journal articles and books, while Figure 5 shows the normalized citations curve. We see that the journal article citation curve lies everywhere above that of books; if we assume more citations imply more influence (whatever the precise relationship between the two) we can conclude that articles have had greater influence. We can also conclude from the normalized citation curves that articles also have higher average influence (again, assuming only that influence increases with citations).

Which Bank books are cited and which are not? Around 70 percent of Bank books are published in a “flagship series,” the WDR being the best-known of these series. Others include _Directions in Development_ , and _Latin America and Caribbean Studies_ . Table 6 reports citation data (from Google Scholar) for book series containing at least six publications.

We focus first on the Bank’s flagship report, the WDR. At the time of writing, our count is that the 32 WDRs, from 1978 to 2009, had a total of 1473 citations in Google Scholar, an

> 17 We were not actually able to obtain citation data for all 20,000 records in our database.

16
1.8

the vaccine impotent and useless. Yet, in oplte of the above-mentioned shortcomings WHO statistics appear to be the best available. Caution should still be used, however, as not all countries collect new coverage data every year. In such cases, it appears that WHO reports the previous year coverage unchanged. This is why some 17 countries (mostly African) report the exact same coverage for each disease in 1987 and 1988, and four in 1986 and 1987.

In 1974, at a time when fewer than 5 per cent of children in developing countries were immunized, the World Health Organization (WHO) established the Expanded Program on Immunization (EPI) whose goal was to reduce infant morbidity and mortality by making immunization available for all children of the world by 1990. The emphasis was put on diphtheria, pertussis, tetanus, measles, poliomyelitis, and tuberculosis.13 Although the target was not achieved, immunization coverage increased dramatically in the 1970s and into the 1980s. A comparison of immunization coverage in 1981 and 1986-87 by UNICEF, points towards substantial progress, in the 1980s, in the rate of coverage. In countries with very high child mortality,14 tuberculosis coverage rose from 26 to 46 per cent, diphtheri **a** nd tetanus from 14 to 27 per cent, poliomyelitis from 7 to 28 per cent, and measles from 19 to 33 per cent. In countries with high child mortality, tuberculosis coverage improved from 55 to 72 per cent, diphtheria and tetanus from 36 to 66 per cent, poliomyelitis from 30 to 62 per cent, and measles from 24 to 55 per cent. Countries with 13R.H. Henderson, A. Galazka, and J. Keja, "Reducing vaccine-Preventable Morbidity and Mortality: the Expanded Program on Immunization,' in John Last (ed.) (1986).

14 UNICEF uses a country classification by level of child mortality, as opposed to the Bank which uses levels of income. The end result is, however, roughly equivalent.
**Timeline and experimental design** We present the timeline of key activities associated with the IAPP project and the experiment in Figure 1. As the pulses are particularly suited for cultivation in the dry season, which starts in roughly November in Barisal, initial meetings with farmer groups were implemented a few months prior in July 2012. These initial farmer group meetings were used to collect our sampling frame for our baseline survey, which we describe in additional detail, along with other aspects of data collection, in Section 3.3. The baseline survey covered, among other things, agricultural production during the November 2011—March 2012 dry season, which we refer to as the “Pre-demo year”.

Figure 1: Timeline <!-- Start of picture text --> One year Two-years<br>Pre-demo Demo post-demo post-demo<br>Baseline survey First midline survey Second midline survey Endline survey<br>2011 2012 2013 2014 2015<br>Q4 Q1 Q2 Q3 Q4 Q1 Q2 Q3 Q4 Q1 Q2 Q3 Q4 Q1 Q2 Q3<br>Farmer Crop Adoption season<br>selection demos<br><!-- End of picture text --> _Notes:_ Dark gray solid lines indicate the boro season each year, denoted _Pre-demo_ , _Demo_ , _One year post-demo_ and _Two years post-demo_ , respectively. Survey periods are highlighted in black. Light gray brackets denote project interventions.

After initial farmer group meetings, we randomly assigned one fourth of the 110 villages in our study sample into one of three treatment arms plus a control group. Figure 2 summarizes the experimental design and sample sizes under each arm. We describe the activities implemented under each of the four experimental arms below.

**Control** In contrast to our three treatment arms, no demonstration resources were provided through farmer groups in control villages. However, as noted in Section 2, a number of activities associated with the promotion of the new seeds were implemented in control villages. Extension agents participate in regular meetings with the farmers groups to discuss and promote the new seeds, as well as other agricultural practices, regardless of treatment status. In addition, the new seeds were locally multiplied at the Union-level and available on all markets in the study area.

analysis.

11
domestic child care is frequently combined with another activity.<sup>45</sup> The EMNV98 also collected information on simultaneous activities but as separate variables. The respondents were asked specifically about the amount of time they allocated to child care as a simultaneous activity. The difficulty with this approach is that the primary activity with which the second is associated cannot be identified.<sup>46</sup> For these reasons, the SSA TUS2000 data are superior in terms of accuracy and the information on time use that they provide.

However, the SSA2000 TUS gives rise to fundamental difficulties for policy analysis due to missing information on all economically active members of the household. There are also problems with the data on individual and household incomes. These are discussed below. In contrast, the EMNV98 provides data on all active members and, furthermore, the information on time use is combined with data on a wide range of income and economic activity variables that are needed for computing wage rates.

The importance of designing a time use survey to collect complete information on economically active members is emphasized by Klevmarken (1999).<sup>47</sup> Surprisingly, Harvey and Taylor (2000) in their survey on time use do not clarify this point. To the contrary, they suggest that different units may be equally important, that the analysis “may focus on the whole population, the subset of people who participate in an activity, an individual, a household, an activity, or an episode”. The kinds of descriptive statistics that can be generated for several of these units are illustrated in Statistics South Africa (2001). While the results are interesting, they cannot in general provide information on the behavioral parameters required for deriving the welfare effects of policy changes, nor

> 45 There would seem, however, little to be gained from collecting information on a third simultaneous activity. This has been the experience with time use survey data for developed countries.  The ABS, for example, collected data on three simultaneous activities in its 1992 TUS but dropped the third activity in its 1997 survey.

> 46 To satisfy the time constraint it is necessary to weight primary and secondary activities.  This requires information on the primary activity with which the secondary activity is associated.

> 47 In an oral discussion at the IZA 2002 Time Use Conference Klevmarken was particularly critical of the planned US time use survey for failing in this respect.  His concerns are captured by the end line of his 1999 article: “Time use data are too valuable to be left only to statistical agencies and national accounting people”.

45
||1<br>New<br>product<br>line|2<br>Upgraded<br>existing<br>product<br>line|3<br>New<br>Techno<br>logy|4<br>Discontinued<br>at least 1<br>product|5<br>Opened<br>a new<br>plant|6<br>Closed<br>at least<br>one<br>existing<br>plant|7<br>New JV<br>with<br>foreign<br>partner|8<br>New<br>Licensing<br>Agreement|9<br>Outsourced<br>a major<br>activity|10<br>Brought<br>in-house a<br>previously<br>out-<br>sourced<br>activity|11<br>CORE|12<br>INDEX8|13<br>INDEX2|
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
|Capacity||||||||||||||
|Utilization>=80%|0.331<sup>a</sup>|0.295<sup>a</sup>|0.447<sup>a</sup>|-0.274<sup>a</sup>|-0.025|-0.565<sup>a</sup>|-0.221<sup>c</sup>|0.087|-0.03|0.032|0.417<sup>a</sup>|0.229<sup>a</sup>|-0.383<sup>a</sup>|
||[0.072]|[0.077]|[0.075]|[0.079]|[0.115]|[0.115]|[0.134]|[0.117]|[0.106]|[0.113]|[0.066]|[0.063]|[0.076]|
|Constant|1.265|-0.141|-1.104|-3.858<sup>a</sup>|-10.951<sup>a</sup>|-7.376<sup>a</sup>|-7.301<sup>a</sup>|-8.278<sup>a</sup>|-6.963<sup>a</sup>|-2.839<sup>b</sup>||||
||[0.808]|[0.781]|[0.814]|[1.098]|[2.341]|[1.731]|[1.548]|[1.511]|[1.618]|[1.146]||||
|Observations|13823|13818|13889|13817|13476|13454|13763|13791|13472|13483|13886|13541|13541|
|Pseudo R-square|0.0942|0.1722|0.1174|0.0912|0.1024|0.0775|0.1252|0.111|0.1012|0.1118|0.0964|0.0734|0.0713|
|F-Test||||||||||||||
|Medium= Large|0|0|0|0|0|0|0|0|0.0065|0|0|0|0|
|Joint Significance||||||||||||||
|Test of Industry||||||||||||||
|Dummies|0|0|0|0|0|0|0|0|0|0.033|0|0|0|

a b c , , and represent significance at 1%, 5% and 10% respectively 37
# **1 Introduction**

Can judicial reforms improve judicial efficiency and increase firms’ economic performance? Cross-country and within-country evidence show that legal efficiency, in the form of higher speed and lower procedural formalism, is a strong correlate of economic development and market performance (Alencar & Ponticelli, 2016; Djankov et al., 2008; Chemin, 2018). As legal origins account for much of the cross-country variation in legal efficiency (La Porta et al., 2008), what is the potential for _de jure_ reforms to increase _de facto_ efficiency? And, can improvements in legal efficiency produce firm-level economic gains?

In this study, we show that judicial reforms can improve judicial efficiency at a margin that can produce meaningful economic impacts for firms engaged in court cases. We do so in the context of a Senegalese judicial reform that aimed to increase the speed of civil and commercial adjudications and, in doing so, improve the investment climate. The reform gave first-instance judges the power to desk reject cases and the responsibility to complete pre-trials in four months—which, on average, accounted for over two-thirds of total case duration. As such, the reform we study explicitly aimed to curb high levels of procedural formalism, characteristic of civil law systems (Djankov et al., 2003).

We proceed in two steps. First, we exploit hearing-level court case data to document the causal impact of the judicial reform on procedural efficiency and the quality of legal decisions. Second, we combine monthly firm tax records and primary firm survey data to measure the firm-level economic effects of the improvement in pre-trial speed legislated by the reform.

We start by showing that a reduction in _de jure_ procedural formalism increased _de facto_ legal efficiency in the context of the civil and commercial court of Dakar, Senegal. In two ways, we contribute novel evidence to a literature that has linked legal reforms with judicial efficiency. First, we construct a flexible event study design, combining the multiple reform application cutoffs across the six chambers of the court between October 2013 and March 2014 with 1
Table 12: Pupil-teacher ratios

||(1)|(2)|(3)|(4)|
|---|---|---|---|---|
|Schools|All|All|Treatment|Comparison|
|Treatment|7.86<br>(3.04)**|7.99<br>(2.71)***|||
|Pre-program PT-ratio||0.31|0.37|0.25|
|||(0.11)**|(0.18)**|(0.14)*|
|Dummy for treatment school within 4km||-6.26|-3.68|-8.34|
|||(2.86)**|(5.18)|(3.00)**|
|Number of schools within 4km||-0.57|-0.7|-0.65|
|||(0.58)|(1.23)|(0.54)|
|Experience(PC1)||-0.44<br>(2.73)|||
|Experience(PC1)*Treatment||-1.9<br>(4.13)|||
|Constant|26.48<br>(2.15)***|25.26<br>(5.72)***|31.18<br>(10.72)***|28.25<br>(6.16)***|
|Obs|50|50|25|25|
|R-squared|0.12|0.4|0.26|0.44|
|Adj. R-squared|0.1|0.32|0.15|0.36|
|Maximum number of visits|10|10|10|10|

Notes: The dependent variable is the observed pupil-teacher ratio, averaged over all visits in terms 2 and 3 of 2000 and terms 1 through 3 of 2001. The pre-program pupil-teacher ratio is the average pupil-teacher ratio over terms 1 through 3 of 1999 and term 1 of 2000 (one observation for each term). Standard errors are in parentheses. * signi...cant at 10 percent; ** signi...cant at 5 percent; *** signi...cant at 1percent percent of the cases, the porridge was served at 9.30 am or later. This is likely to be an optimistic estimate since the ...gures are based on the timing reported by the teacher rather than on actual observation. In addition, the schools reported that the feeding took half an hour on average, but casual observations suggest that this ...gure might be too low, especially in large schools. One half hour easily accounts for 15 to 20 percent of teaching time in preschools.

Second, the school meals program increased crowding in the classrooms. Even though several treatment schools hired additional preschool teachers, the observed number of pupils per teacher in treatment schools was 34, compared to 27 in comparison schools (Table 12). There were no reports that the increase in pupil-teacher ratio lead to disciplinary problems, though one can imagine that the larger class sizes did not foster learning. In fact, crowding in the classrooms 34
**_The scale transformation Command_**

2

# **Test Score Gap Robustness to Scaling: The** **_scale_** **_~~t~~ ransformation_ Command**

Andres Yi Chang World Bank Group Washington, D.C. ayichang@worldbank.org

**Keywords:** STATA, scale ~~t~~ ransformation, test scores, measurement, achievement gaps, robustness to scaling, psychometrics

**JEL Classification:** C15, C49, C87, I29

# **1 Introduction**

With enrollments at a historical high for low- and middle-income countries, the focus in education has shifted away from enrollment and toward improving the quality of education (World Bank World Development Report, 2018). Countries around the world are focusing on both increasing learning and closing substantial gaps across sub-populations, particularly in favor of disadvantaged groups. These gaps have been extensively documented in the United States, where a large literature has identified stubborn test score gaps by race (Bertrand and Pan 2013; Fryer and Levitt 2010), socio-economic status (Reardon 2011, 2013) and sex (Hanushek and Rivkin 2006; Clotfelter et al. 2009; Fryer and Levitt 2004, 2006, 2013).

Gap measurement is essential to make inferences about equitable learning. What is surprising though, despite the seemingly large gaps across sub-groups, is that different studies often come to different conclusions. For instance, Fryer and Levitt (2004) first noted that a substantial black-white test score gap that had received extensive attention in the literature (see Coleman et al. 1981; Kaufman and Kaufman 1983; Krohn and Lamp 1989; Phillips et al. 1998; Phillips 2000) disappeared after controlling for covariates. Additionally, they found that over the first 4 years of school, blacks lose about -.10 standard deviations per year compared to other races. Bond and Lang (2013) then demonstrated that even this seemingly rock-solid racial gap in the United States is in fact, not robust to arbitrary scale transformations. That is, transformations that minimize or maximize learning by weighting test-items differently, while still preserving order, create lower and upper bounds that are too wide to be informative or definitive.

These findings contribute to an ongoing debate around test score measurement and, in particular, on whether the assumptions that justify cardinality are satisfied in most tests (Ballou 2009; Ho 2009; Bond and Lang 2013; Jacob and Rothstein 2016). Three alternatives have been proposed. The first is to argue that the test at hand has been constructed in a manner that allows for cardinal measures of ability; this is the approach of Rasch measurement and Domingue (2014) provides statistical methods to verify the assumptions required for cardinality. The second is to give up on cardinality entirely and focus only on ordinal comparisons. Reardon et al.’s (2017) _HETOP_ , Williams’s
Turning to the implications of survey design for the FAO’s FBS-CV method, we use the 7 modules from our experiment to calculate a CV of calorie availability, following FAO (1996, Appendix 3).<sup>11</sup> Specifically, we collapse the data to 10 deciles of daily per capita kilocalories available and then calculate the CV across the medians of these 10 groups, subtracting 0.05 from that CV to account for other errors. The FAO motivates these manipulations, which serve to lower the CV, by the desire to purge the CV of random variation, seasonal variation, and measurement error. Furthermore the FAO forces any CV to lie between 0.20 and 0.35, setting any outliers to their nearest acceptable value.

In addition to the CV, we need an estimate of the mean of calorie availability per capita in order to fully parameterize the log-normal distribution. We use two approaches for the mean estimate. First, we take the mean from the personal diary ( _μ_ *= 2677 kcal per person per day) as the estimate which is likely to be closest to the truth. Second, we set _μ_ * to the module-specific mean of calorie availability. The motivation for this second approach, which is purely illustrative, is that the FAO derives this mean from the FBS, which also differ in methodology and implementation across countries, so we would like to reflect this sensitivity in estimates of the average food balance.

The third and final component needed to replicate the FAO calculations is to use the sample average of daily energy requirement (2068 kcal per person per day). Using these estimates, the FAO approach produces a mean ( _μ_ ), a standard deviation ( _σ_ ) and a _z_ -score ( _z_ ): From the _z_ -score, _p_ -values are derived in the cumulative standard normal distribution to indicate the area to the left of the average requirement of 2068 kcal/day. This number is the percentage of people who fall to the left of 2068 kcal/day assuming a log-normal distributional form parameterized by _μ_ * and CV. When hunger rates estimated in this same manner are aggregated across countries they yield the widely reported estimates of 870 million hungry people that is noted in the introduction of this paper.

> 11 This is the last FAO report on which we have a detailed description of the exact mechanisms used. The methodology used has been modified slightly since then, as explained Annex 2 of FAO (2012) and footnote 4.

16
# 1. **Introduction**

Several industrialized countries have a long tradition of monitoring socioeconomic inequalities in health and of formulating policies to try to combat them. As early as the first half of the nineteenth century, occupation was added to the death certificate in Britain, and tabulations of mortality rates by occupational group, along with commentaries, became a regular feature of the government's annual mortality reports (Whitehead 2000). The evidence showed large-and persisting-gaps in mortality at all ages between the poorer and more affluent sections of British society (Drever and Whitehead 1997). These gaps have frequently been claimed to be unjust, and in recent years new policies have been developed in the United Kingdom to reduce them (Acheson 1998). Several other European countries (Fox 1989)-the Netherlands (Kunst, Geurts, and van den Berg 1992; Kunst, Geurts, and van den Berg 1995; Mackenbach and others 1997), Sweden (Vagero and Erikson 1997) and others-have seen similar levels of monitoring, research, and policy initiatives.

Only recently, has the issue of socioeconomic inequalities in health started to receive attention in the developing world. Until then, the focus had, for the most part, been firmly on population averages. In some circles-influential ones at that-this continues to I