rafmacalaba/datause-encoder-data huggingface dataset:
characteristics. Students' parents and teachers answered a survey at the same time. This information was needed to control for teacher and socioeconomic characteristics.

The C.E.E sample consists of students from 198 schools randomly chosen from four different types of schools: Urban, RURAL, NATIVE and CONAFE from five different states.' The participation of each school type, relative to the total is shown in Table 2.2 Table **2. Number of schools by type and state, 1992**

|State|**Urban**|**Rural**|**Native**|**CONAFE**|**Total**|
|---|---|---|---|---|---|
|Chiapas|6|13|14|5|38|
|Guerrero|4|14|12|4|34|
|Hidalgo|3|11|12|8|34|
|Oaxaca|7|17|15|12|51|
|Michoacan|7|15|10|9|41|
|Total|27|71|64|44|198|

Source: PARE's database.

# **3.** Impact of the PARE **program on learning and achievement**

# 3.1 Control and the experimental groups

The literature generated by the PARE points toward a mixed conclusion about the impact of the program.<sup>3</sup> This was partly due to incomplete and faulty implementation, especially in urban areas. By design, the program intended to provide a number of simultaneous actions (components), which together would impact on educational outcomes. For pedagogical reasons the total was to be greater than the sum of the parts. The actions were to affect the behavior of students, parents, teachers, principals and supervisors; they were to provide the target schools with supplies, didactic materials and 1 CONAFE stands for Consejo Nacional de Fomento Educativo.

2 The Native school refers to schools offering services to populations which mother tongue is not the Spanish.

3 The PARE program has generated a voluminous literature produced mainly by the _Direcccion General de Evaluacion_ of the _Secretaria de Educacion Publica_ and by the _Centro de Estudios Educativos A. C._ (CEE). The CEE was chosen by the executing agency of the PARE program (the _Consejo Nacional de Fomento Educativo,_ CONAFE) to monitor and evaluate the program. Its conclusions were summarised in the document "Determinacion del Impacto del PARE en el Aprovechamiento y la Retencion Escolares," Tercer Infonne, Tomo IV, Mexico, D.F., March 1996. After an extensive analysis of the data the report concludes (my translation) on page 21: "... the variable PARE [a dichotomous variable identifying schools which had access to the program] had a significant impact in only two of the estimated equations. They are, first, the equation referring to performance in mathematics in urban schools of the states' capitals; second, the equation for performance in Spanish in rural schools closer to the states' capitals. ... only for schools in these two sub-samples did students achieve performance levels greater than those in comparable schools which remained outside the PARE program." _5_
||Depend<br>|ent Variable<br>|
|---|---|---|
||(1)|(2)|
||<br>Ln Height<br>at baseline|<br>Years of schooling<br>at baseline|
||(0.079)|(1.409)|
|Dwelling has good flooring|0.016*|0.261|
||(0.009)|(0.163)|
|Height of mother|0.002***|0.010|
||(0.001)|(0.010)|
|Height of mother missing|0.261***|1.279|
||(0.089)|(1.580)|
|Mother resides in household|0.007|0.127|
||(0.015)|(0.260)|
|Father resides in household|-0.011|0.106|
||(0.009)|(0.157)|
|Mother deceased before child is 15 years old|-0.010|0.238|
||(0.011)|(0.189)|
|Father deceased before child is 15 years old|-0.006|-0.096|
||(0.008)|(0.146)|
|Constant|4.368***|-6.227***|
||(0.118)|(2.101)|
|Number of observations|718|718|
|Test of Excluded Instruments_F_-Value|5.58|4.95|
|Test of Excluded Instruments_p_-value|0.019|0.026|

25
# **4.3 Processes and health treatment practices**

The findings in section 4.2 show that the intervention, which resulted in large improvements in health outcomes and utilization in the short run (Björkman and Svensson, 2009), also resulted in long-run improvements in health. Below we try to identify possible channels through which these changes came about. The intervention was intended to improve health outcomes by strengthening the community’s ability to monitor and hold providers to account which, in turn, would influence health workers’ incentives to provide both more and better health care. We next turn to assessing the evidence for these two links.

## **Processes**

In table 7, we use data on a set of process measures that proxy for the community’s ability and extent of monitoring and engagement. The data in panel A were collected through visual checks by the enumerators during the post-intervention health facility survey. Four years after the initial intervention, the average standardized effect of having a suggestion box, numbered waiting cards, a staff duty roaster, and posters informing patients about their rights and that services are free is significantly positive. For two of the variables, having a suggestion box and a staff duty roaster, the individual effects are also significantly positive and the point estimates, relative to the control group, are fairly large.

Panel B uses information collected at the household level. There are positive long-run effect on all individual indicators; i.e. whether the household has received information about the HUMC’s role and responsibilities, whether the performance of health workers was discussed in Local Council meetings, whether a household member told the staff about the household’s (dis)satisfaction with the health services received, whether they think that the staff at the facility works closely with the community, and whether they know somebody who monitored the health facility and staff performance.

## **Health treatment practices**

Tables 8-11 focus on health workers’ behavior; i.e. treatment practices and management.

Table 8 looks at a set of generic measures of behavior. The intervention had no significant long-run impact on the likelihood that any equipment was used during the patient’s last examination, column (i), or waiting time before being examined, column (ii). Note, however, that as utilization increased in the treatment group (see table 6), the total number of patients examined with equipment increased. Column (iii) reports the impact on absenteeism based on data from three unannounced visits. The point estimate is close to zero and insignificant. There is suggestive evidence, however, that the absence rate for staff residing in the community fell in the treatment clinics (the results are available upon request).

The general condition of the clinic – a measure of the conditions of the floor, walls and furniture 13
sample of respondents into a treatment group surveyed after the intervention and a control group surveyed immediately before the intervention.<sup>3</sup> The terms “natural” and “quasi” experiments are often used in an imprecise and - interchangeable sense. However, we advertently refer to the Botswana devaluation as a _quasi_ experiment and distinguish it from natural experiments. While a common feature of natural and quasi experiments is that an intervention generated by some force outside the control of the researcher assigns subjects into treatment and control groups (Meyer 1995; Robinson et al. 2009), the defining characteristic of natural experiments is that treatment assignment occurs in a random or ‘as-if’ random way (Dunning 2008, 2012). However, as emphasized by Cook and Campbell (1979) and Achen (1986), what distinguishes quasi-experimental designs from natural and controlled experiments is that assignment to treatment is nonrandom, which means that the treatment and control groups are imbalanced—or nonequivalent—at the outset. This means that even a macroeconomic shock, for example, a surprise devaluation, may not be strictly exogenous because nonrandom treatment assignment may make treatment status correlated with other factors that affect the outcome. In a regression framework, nonrandom assignment to treatment may therefore imply that treatment status is not statistically independent of the error term—at least not unconditionally— and that confounding is a potential challenge to a causal interpretation of the estimated treatment effect.

While the survey data we use are a random and representative sample of 1200 adult citizens in Botswana, the key source of nonrandom assignment to treatment and control is that the sampling of respondents before and after the devaluation is geographically imbalanced. Overall, 216

> 3 The survey started on May 28 and ended on June 12, 2005. Since the devaluation was announced late in the afternoon (17:00 hours) on May 29, no interviews started after the announcement of the devaluation (the final interview began at 16:57 hours).

9
<!-- Start of picture text --> wo a<br>2 Le<br>£8 De _eeeetecces te<br>& ae _ eee”<br>@ aa ae<br>oY Le » wa 4<br>= LE ee peeaeee-— ae<br>2012m2 2013m6 2014m6 2015m6 2016m8<br>Month<br>* (0,7] + (7,14]<br>° (14,21] - (21,28]<br>. (28,35] - (35,42]<br>* (42,49] + (49,56]<br>—-— (56,63]<br><!-- End of picture text -->
**35**

# Table **5.** Selected Economic IndLcators in Four Counties, 1985 (yuan per capita of agricultural population)

|Indicator|Wuxi|Nanhai|Jieshou|Shangrao|China|
|---|---|---|---|---|---|
|Average per caplta income of<br>rural population|754|1,029|285|322|398|
|Rural gross value of<br>industrial output|4,656|2.313|244|49|207|
|Gross value of agricultural<br>output|604|610|210|266|429|
|TVCEgross profits|512|208|9|4|24|
|TVCEwage bill|364|276|13|23|36|

- Note: For China as a whole, the figures are per member of the rural population. The agricultural population is an administratively defined category used at local levels, which includes people engaged in nonagricultural activities but not receiving grain rations or other subsidies.

- Sources: Information from fieldwork of World Bank/Institute of Economics of the Chinese Academy of Social Sciences research project. State Statistical Bureau of China (1986).
- 29 -

user's account is low and a red light would indicate that a defective or fraudulen **t** ag is used, at which time the video enforcement system automatically akws a picture of the offending vehicle. (In principle, even though a traffic cop could be dispatched immediately to apprehend violators, in practice they are merely sent tickets in the mail as with speeding violators.) The - reader then transmits the data to a central computer system composed of a system of commonly used microcomputers and a minicomputer - which records infonnation such as the identification code, the location, date, time or the vehicle class code and performs off-vehicle charging either by prepayment or postpayment. A motorist can choose to pay in advance or in arrears. For prepayment, a user's account is linked to a bank account and a pre-specified sum is deducted periodically (alongside the toll rate itslk) via electronic funds transfer (giro payments) or credit card payment. As is cufrently done at toll faciities in Europe and the United Stat, prepayment could also be performed without a bank account by placing a prepaid cash deposit in advance, as with a numbered account arrangement (thus protecting the individual's privacy). For postpayment, conventional monthly or quarterly billing statements can be automatically sent out and straightforwardly processed. Even with automatic _vehicle Wentificadon,_ the optional nature and wide variety of payment options would therefore not pose a threat to citizen's civil liberties.

38. Smart card-based systems form the latest tof-the-art technology in AVI (Thompson, 1990).19 Smart card technology is based on a two-way communication link between a 'smart' on-board unit and a roadside antenna. An on-board unit is composed of a smart card, a smart card reader and a transponder.20 In addition to the standard integrated circuits found in radio 9 As a precusor of th chip card, the plastic mqpageic osipe card (as exemplified by tbe credit card and bank card) is ubiquios and vey chea to manufactr - less tha a dollar ea Th magneic strpe card suffers from: 1) the defect of being asly copied; 2) low mwwry capcity; and 3) the requiremnt for it to be used together with a mechaical, motor driven reader tha is unrelable.

I The Teepsos system _is the_ only smUt ard sydsm in opeton s May 1990 an Italy's Autostrad. in the Italin sysdm, h tbrnpde - locte in the oa-bard muit itseffeponb for onu an d reepto of roadside to vehicl i t ad operatsat S.72H. Hence it is over s times faster than an _RF_ technology which oprates at 91S NHz, alowing for added measures of encryptio and scurity. In additiao to beu a supplier to Toleps, AT&T h **_ndy_** teamed up with Vapor Cda, Inc. to itgrate its _'ec doat'_ smrt cad used in baning applicais (which has thie cqpacity to stor fur pages
# **_Box 1:  Structure of STPD classification of measures_**

# **_A Conditions on market entry_**

- _A1 Legal forms of entry (including foreign equity limits) A2 Quantitative measures (for firms and natural persons)_

- _A3 Conditions on licensing/investment screening/qualifications relating to market entry (firms /natural persons)_

- _A4 Other conditions on market entry_

# **_B Conditions on operations_**

- _B1 Conditions on supply of services_

- _B2 Conditions on service supplier_

- _B3 Conditions on government procurement_

- _B4 Other conditions on operations_

# **_C Measures affecting competition_**

- _C1 Conditions on conduct by firms_

- _C2 Governmental rights/prerogatives (including public ownership) C3 Other measures affecting competition_

# **_D Administrative procedures and regulatory transparency_**

- _D1 Administrative procedures_

- _D2 Regulatory transparency (including licensing)_

- _D3 Nature of regulatory authority (measures related to nature of regulator) D4 International standards_

- _D5 Other regulatory environment and administrative procedures_

# **_E Miscellaneous_**

Every individual policy measure is mapped to one category in the above taxonomy and is generally defined in a binary way (yes or no response). As an exception to this rule, by their very nature a few measures require a quantitative reply (e.g. level of foreign equity allowed, number of days required to process a licence application). Each policy category includes measures from across all modes of supply as applicable; for example, category A (conditions on market entry) will thus include the measures affecting entry through each of the modes of supply covered in the database, i.e. cross-border, commercial presence and presence of natural persons.

The meta-classification of measures described above is intended to facilitate convergence towards a single classification and quantification methodology. Therefore, the classification remains, at the measure level, fully compatible with the classification utilized in the OECD Services Trade Restrictiveness Regulatory Database and the previous classification of measures developed by the World Bank more than a decade ago.

6
A substantial lack of consistency occurs for which classes have no materials, to potentially target for additional materials or grants. On the class level, 20.5% of those with no materials per the QAS tool are also so identified in the diagnostic research tool, and 33.3% on the school level. There are no school districts with no materials.

One quality enhancement action that would have relatively more consistency would be targeting remediation to weaker students, i.e., following up with the bottom 10% or 25% of classes in terms of letter identification, potentially with reading tutors, literacy coaching, or other literacy enhancement actions. For the bottom 10% in terms of letter identification, there is 61.1% consistency on the class level, 72.7% on the school level, and 66.7% on the district level. For the bottom 25% in terms of letter identification, 79.5% of the classes and 81.5% schools targeted by the QAS tool would also be targeted by the diagnostic research tool, and 100.0% of the same districts.

# **Table 3. Simulations of quality enhancement policy targeting (percentage of QAS classes, schools, or districts identified as targets for quality enhancement action that are also identified in the diagnostic research tool)**

||**Top**<br>**10%**<br>**overall**|**Top**<br>**25%**<br>**overall**|**Bottom**<br>**10%**<br>**overall**|**Bottom**<br>**25%**<br>**overall**|**Bottom**<br>**level of**<br>**pedagogy**|**Bottom**<br>**10%**<br>**hazards**|**Bottom**<br>**25%**<br>**hazards**|**No**<br>**materials**|**Bottom**<br>**10%**<br>**letters**|**Bottom**<br>**25%**<br>**letters**|
|---|---|---|---|---|---|---|---|---|---|---|
|**Class**|0.306|0.516|0.435|<br>0.474|0.530|0.588|0.779|0.205|0.611|0.795|
|**School**|0.250|0.988|0.409|<br>0.966|0.417|0.250|0.711|0.333|0.727|0.815|
|**District**|0.250|1.000|0.833|0.833|0.167|0.333|0.667|.|0.667|1.000|

# _Source:_ Authors’ calculations

# **6 Discussion and conclusions**

LMICs have struggled with the quality of education systems, with a learning crisis: learning basic skills is declining even as access has expanded (World Bank 2018a; Le Nestour, Moscoviz, and Sandefur 2022; UNICEF 2022). Regularly assessing learning and acting on this evidence are cornerstones of improving quality (World Bank 2018a). QAS are a critical mechanism to achieve these goals and ultimately to improve quality, particularly at the pre-primary level, when there are no national assessments to draw on (World Bank 2013; Raikes, Neuman, and Burton 2019). There has, however, been limited evidence on how QAS work in practice in LMICs. This paper investigated how consistently QAS measure quality, and lessons learned on how QAS should be used in LMICs, in light of their measurement challenges.

# **_6.1 Summary_**

Leveraging parallel data collection efforts in schools, we investigated the consistency of QAS tool data with more detailed diagnostic research tool data. Consistency varied substantially across items, with almost perfect consistency only for two out of 35 items and substantial consistency for only five items. Measures of foundational skills were some of the items with 21
Yemenis’ lives, and the country that Yemenis would like to see emerge from the conflict (e.g., World Bank 2022a; World Bank 2022b).<sup>8</sup> Importantly, truthful answers to many of these questions are inhibited both by fears about safety and also by social desirability bias.

# **Section 4a. Data**

In order to assess the potential of anonymous internet-based surveys to elicit more sensitive information than more commonly used survey modalities, we fielded both an internet-based survey and a mobile phone survey. For the internet-based survey, we performed a Random Domain Intercept Technology (RDIT) survey, which delivers anonymous opt-in surveys to internet users. Specifically, the company that performs RDIT surveys has purchased a number of internet domains names that are close spellings of commonly used websites or are expired websites that are no longer in operation. Internet users that accidentally reach one of these misspelled or expired websites are offered the chance to participate in an anonymous survey through a series of proprietary algorithms.<sup>9</sup> We fielded an RDIT survey for the month between January 11 and February 10, 2019. The survey was offered to 198,049 internet users, of which 5,198 completed the entire survey.<sup>10</sup> Respondents are first informed that they have randomly been selected to participate in a survey that is voluntary and completely anonymous. In the empirical analysis, we only use completed surveys when comparing responses in the internet survey to the mobile phone survey.<sup>11</sup><sup>_,_12</sup>

> 8There continue to be some anonymous key informant interviews that shed some light on these issues. For example, see ICG (2021) and ACAPS (2020) for descriptions of how the private sector has survived following the de-banking of Yemeni firms in international financial systems and how food supply chains are currently working in the country, respectively.

> 9RDIT cannot be blocked by state surveillance or internet control and evades firewalls by operating on hundreds of thousands of rotating domains simultaneously, as opposed to surveys offered through social media that operate on a single website (e.g., RIWI 2016).

> 10Some of the questions are only asked to subsets of respondents based on a response to a previous question; and the internet survey also has a module that is randomized across respondents where onethird of respondents are posed one of three modules. Thus, the sample sizes of questions that are compared between the surveys varies. In all cases the number of observations is listed in the analysis.

> 11The results are qualitatively identical if we use all partial answers.

> 12The survey also keeps track of limited amounts of metadata of respondents who are offered to participate in the survey. This information includes the browser type (e.g., Chrome, Android, etc.), the version of the browser type, the operating system, the version of the operating system, the type of device (e.g., smart phone, desktop, etc.), and the region associated with the IP address. However, the region can depend on the internet infrastructure in each country, where, for example, all IP addresses are

10
analysis is conducted using five Crisis Response Surveys from Armenia, Bulgaria, Montenegro, Romania, and Turkey.

Findings from the 5 countries reveal that households reduce some human capital investments to cope with income shocks. In fact, sampled households are more likely to adopt health-related coping strategies as more often than education-related coping strategies. The results from Armenia, Bulgaria, Montenegro, and Turkey show that households affected by income shocks (i) reduce visits to doctors and (ii) reduce spending on medicine and medical care significantly more than the matched comparison group. Households affected by income shocks reduce education investments, but do not adopt harmful education-related coping strategies, such as withdrawing children from schools or moving children from costly private to cheaper public schools.

We interpret the evidence that households adopt health related coping strategies over education related coping strategies as evidence that parents tend to protect education investments because out-of-pocket expenses for education are low in all five countries studied here and because disrupting education even temporarily can be difficult to reverse. King (2009) argues that school enrollments are generally sticky in the short run because parents are reluctant to pull their children out of school in the middle of an academic year. Thomas et al. (2004) find that households affected by income shocks in Indonesia protected education investments of older children, sometimes at the expense of the younger children in the household. In addition, households may perceive a shock to be temporary and hence, may be reluctant to impact their child‘s long term human capital accumulation by withdrawing the child from school.

On the other hand, utilization of health services, even in countries that provide ―free‖ services, usually has some out-of-pocket expenses associated with the utilization. Frankenberg et al. (1999) and Cutler et al. (2002) also find that households affected by income shocks reduced health care utilization in Indonesia and Mexico during crises because of the out-of-pocket expenses. Furthermore, missing a health care appointment or missing a few doses of medicines may not always lead to bad health outcomes.

Implicit in our analysis in this paper is that households maximize long term welfare prior to the income shock and therefore, any reductions in human capital investments after the shock are 18