rafmacalaba/data-use-mentions-extended huggingface dataset:
# **4. Robustness checks**

# **A. Design-based simulations**

The results in the previous section use just a single sample to compare results from two estimators: a direct estimator, using just survey results, and a small area estimator, which uses auxiliary information to improve the accuracy and precision of the survey results. However, it is difficult to come to a firm conclusion when using just a single sample. Therefore, we simulate the small area estimates 100 times, using 100 different samples and calculating 100 different point estimates for direct and small area estimates. We do not calculate MSE on each simulation due to computing limitations.

Table 8 - Simulation results for municipal estimates

||(1)<br>Male LFP|(2)<br>Female LFP|(3)<br>Male<br>Unemp|(4)<br>Female<br>Unemp|
|---|---|---|---|---|
||mean<br>(median)|mean<br>(median)|mean<br>(median)|mean<br>(median)|
|**Simulated relative bias**|||||
|Direct|0.186|-0.222|0.128|1.451|
||(0.534)|(-0.302)|(-58.406)|(-100.000)|
|SAE|-0.274|1.755|35.627|71.286|
||(-0.656)|(-0.561)|(11.734)|(24.950)|
|**Simulated mean absolute**|||||
|**error**|||||
|Direct|0.060|0.070|0.023|0.015
encompasses a large coverage (i.e. general government); (ii) whether the FC make forecasts; (iii) whether the FC makes forecasts on the preparation of the fiscal policy; (iv) whether the FC makes forecasts assessments on the fiscal policy; (v) whether the FC makes recommendations to the government; (vi) whether the FC makes long-term sustainability analyses; (vii) whether the FC establish consistency with the objectives; (viii) whether the FC evaluates the costs of fiscal measures; (ix) whether the FC makes ex-post analysis; (x) whether the FC publish public reports; and (xi) whether the FC has a high media impact. We focus on the period 2009-2016 to evaluate the efficiency of postcrisis policies. The comparison of the procyclical bias across groups of countries with and without these features (Table C.2 and Figure C.2) shows that countries with fiscal councils have lower procyclicality bias; but the largest impact comes from attributes of the council, especially those that examine sustainability and evaluate the costs of different measures. The features are associated with countercyclical spending. Transparency through public reports and the media exposure also matters, in line with predictions by Alesina et al. (2008).

# **[insert Table C.2; Figure C.2]**

# **_Bottom-up solution: Fiscal decentralization_**

Though fiscal decentralization may be an effective mechanism to enhance the provision of public service delivery in developing countries, such as the access to primary education, the access of drinking water, the access of refuse and sewage disposal facilities. The discussion still exists around the optimal level of fiscal decentralization for both spending and expenditures. So, the decentralization might be also effective in the struggle against fiscal procyclicality of the general government because of the application of subsidiarity principle. Indicators of fiscal decentralization generally refer to the fiscal composition ratio established by the IMF’s Government Finance Statistics Yearbooks (see Blume and Voigt (2008) and Blume
allocation method, because the targeting strategy was being refined up until the program was stopped in 2021.

The final PMT was chosen based on the above evaluation criteria and program considerations, namely a tradeoff between model accuracy and complexity. We find that, in general, more complex models (i.e., including more variables) only marginally improve targeting accuracy. However, the inclusion of more information raises the cost of obtaining information and administering the program. Therefore, we propose a parsimonious model (i.e., one that uses less variables) that provides high targeting accuracy.

# 4. Results

To arrive at a final model for the Sudan PMT, we split the data into a training and testing set using a 75% training and 25% testing split. In an effort avoid overfitting on in-sample characteristics of the sample data, all modeling decisions, including any variable transformations or groupings, were based on the training data before arriving at a small number of potential models that were applied to the testing data to explore model performance on unseen, out-of-sample observations. This is done in an effort mimic how the PMT would be applied in practice and to understand out-of-sample predictive performance of the PMT. After exploring model performance on out-of-sample data, the top performing models were selected and estimated using the complete sample of data.

In an effort to improve predictive accuracy of the Sudan PMT, we analyzed the raw household survey data and selected and transformed variables to better capture the relationship with per capital household consumption and the country context.<sup>17</sup> For all categorical variables, we explored each category for intuitive groupings that were related to household income using cross-tabulations and boxplots. For continuous variables, we explored non-linear relationships and, if non-linear relationships were present, we applied transformations, so that the relationship between continuous predictors and the outcome variable of log per capita household consumption was approximately linear. Examples of these analyses are found in Annex 4.

Table 3 below presents the main results of the Sudan national
These control variables are needed to make sure that comparisons are made across similar workers. The standard practice in this respect is to estimate Mincerian equations, where the explained variable is the log of labor earnings and the explanatory variables are individual characteristics like those listed above. The analysis is completed by splitting the sample between the individuals who do work in the formal sector, and are therefore covered by labor market policies and institutions, and those who do not.<sup>2</sup> The criteria used to identify the formnal sector vary across studies. In some cases, the sample is split according to the size of the establishments these individuals work in. But more elaborate criteria are applied too. Consider, for instance, the study on C6te d'Ivoire by Vijverberg and van der Gaag (1990). This study applies factor analysis, involving 15 job attributes, to data from the 1985 Living Standards Survey (CILSS). The first component of these data is then used as an indicator of the formality of jobs. This component turns out to be highly correlated with six of the job attributes. It is higher when the wage is subject to minimum wage legislation, when there is a signed contract, when the worker receives paid holidays, when he or she is eligible for paid sick leave, when there is a retirement plan, and when the worker receives social security benefits.

A different approach is adopted by Miller and Vall&e (1995) in the case of Cameroon. Using a plant-level based survey of workers produced by the Regional Program on Enterprise Development (RPED), these authors consider three sectors: formal regulated, formal unregulated and informal. Firms in the formal regulated sector are characterized by multiple ownership, or by some form of public ownership, or by having complained about being constrained by labor regulations. At the other end, firms in the informal sector are characterized by individual ownership, and by either having apprentices or carrying no

- 2 Some of the studies also correct for the
|-0.02|
|50 or more persons|Public|0.01|0.00|
|Unknown but less than 11 persons|Public|0.10|0.01|
|Unknown but more than 10 persons|Public|0.01|-0.02|
|Total<br>_Note: Kosovo LFS 2018 is used. Wages are imputed a_|Public<br>_s described in Section 2._|0.14<br>_Labor demand elasticity_|0.01<br>_value is set at -0.5 (-_|

_Note: Kosovo LFS 2018 is used. Wages are imputed as described in Section 2. Labor demand elasticity value is set at -0.5 (- 0.8) for skilled labor and -0.7 (-1) for unskilled labor. Only employees with the wage between the old and the proposed minimum wage are assumed to be affected. Aged 15-64._

**Increases in the wage bill are expected to be small.** Table 4 shows the simulated increase in a wage bill due to the policy for the public and private sectors. For the public sector, simulation results suggest no increase in the government wage bill. The fiscal rule that came into effect in 2018 establishes that the overall wage bill of the government is capped with the increase in the last available year’s nominal GDP growth rate. An increase

25
thus a substantial share of the papers adopt Heckman-type methods to assess the effects of adjustment lending. A smaller subset uses control group matching techniques and/or before and after analysis. We employ all three types of analysis here.

There have also been recent attempts to assess the effects of developing country policy reform on trade using similar selection techniques (see for example, Lederman and Ozden, 2004 and Jinjarik, Salinas, and Tsikata, 2004). As in trade, where specific indicators exist to assess outcomes such as the ratios of imports and exports to GDP, outcomes in the financial sector can be assessed via standard indicators such as the ratio of M2 or private credit to GDP. However, in the trade literature it has been easier to pinpoint specific reform episodes tied to trade agreements and legislative actions.<sup>2</sup> In the financial sector, where reform efforts often were carried out under broader multi-sector adjustment projects, and thus are harder to identify, little empirical work has been done.<sup>3</sup> We therefore rely on a comprehensive database of financial sector reforms, including both financial sector adjustment loans and multi-sector adjustment loans with financial sector components, that was recently put together by the World Bank’s Operations Evaluation Department (OED).

The remainder of the paper is organized as follows. Section 2 describes our data, including the indicators that we use to assess outcomes and the variables that summarize World Bank lending to support financial sector reform from 1992 to 2003. Section 3 describes the basic regression models and section 4 summarizes the base results. Section 5 presents results from two methods that address self-selection problems that could be affecting the base results. Section 6 offers additional robustness checks, including regional regressions and models that test whether improvements in financial indicators came before or after World Bank loans were received. Section 7 concludes with a brief summary of the results from the various methods.

> (1997), Summers and Pritchett (1993), Przeworski and Vreeland (2000
Saharan Africa, lag because of a combination of low starting points and difficult circumstances (Easterly 2009; Clemens and others 2007; World Bank 2010).

**Figure 1. Current global distance to the MDGs is wide ranging**

<!-- Start of picture text --> a. Distance of latest indicators to the 2015 goals<br>100%<br>96%<br>90% 91%<br>90%<br>88%<br>80%<br>80%<br>77%<br>70%<br>69%<br>67%<br>60%<br>57%<br>50%<br>40%<br>30%<br>20%<br>10%<br>0%<br>b. Distance to the trajectory to be on track to achieve the goals by 2015<br>100%<br>100% 100% 100% 100%<br>90% 90% 91%<br>80%<br>79% 79%<br>70%<br>68%<br>60%<br>50%<br>40%<br>30%<br>20%<br>10%<br>0%<br>MDG 1.a MDG 1.c MDG 2.a MDG 3.a MDG 3.a MDG 4.a child MDG 5.a MDG 7.c MDG 7.c<br>extreme hunger primary gender parity gender parity mortality maternal access to safe access to<br>poverty completion (primary) (secondary) under five mortality water sanitation<br>rate<br><!-- End of picture text --> _Source:_ Authors’ calculations based on data from the World Development Indicators database. _Note:_ Distance to goal achieved in this graph is a weighted
An important question is whether economic policy uncertainty affects global value chain (GVC) related trade more than other trade. There are contrasting effects which play out over time because GVC trade requires larger upfront relation-specific sunk investments and is concentrated in capital intensive sectors. These investments make GVC trade flows stickier and less responsive to any change in the environment, including economic policy uncertainty. However, since economic policy uncertainty affects investment choices, it should eventually have larger effects on GVC trade compared to other trade. These effects are confounded by the fact that consumer and producer investment goods are typically produced through longer GVCs. Trade in these investment goods is likely to be more sensitive to demand for investment which is affected by policy uncertainty. The combination of these forces determines whether economic policy uncertainty affects GVC trade differently from other trade.

We conduct a panel estimation covering 18 countries over 24 years. Our results, which are robust across a wide range of specifications, suggest a statistically significant negative impact of policy uncertainty on trade growth. A 1 percent increase in uncertainty is associated with a 0.02– percentage point reduction in goods and services trade volume growth. For example, based on these estimates, the increase in economic policy uncertainty since mid-2018 may have caused a 1 percentage point decrease in world trade growth. The finding is robust to focusing on subcategories of trade and employing different specifications. GVC trade appears to be as sensitive to uncertainty as non GVC trade.

Looking at the impact on components of trade may help illuminate the economic channels through which policy uncertainty affects trade. We find that economic policy uncertainty negatively affects imports of capital goods, imports of non-durable consumer goods as well as imports used to produce exports. These findings are consistent with the view that economic policy uncertainty reduces trade growth by reducing foreign firms’ incentives to invest in serving the market, by inducing consumers’ precautionary saving and by affecting firms that are part of global value chains.

The implication of this result is that economic policy uncertainty has contributed to the sluggishness in
- Amin, M. and Y. C. Soh (2021). Does greater regulatory burden lead to more corruption? evidence using firm-level survey data for developing countries. _The World Bank Economic Review 35_ (3), 812–828.

- Avenyo, E. K., M. Konte, and P. Mohnen (2021). Product innovation and informal market competition in sub-saharan africa. _Journal of Evolutionary Economics, 84_ (C), 605–637.

- Ayyagari, M., P. Juarros, M. S. Martinez Peria, and S. Singh (2021). Access to finance and job growth: firm-level evidence across developing countries. _Review of Finance, forthcoming_ .

- Banerjee, A. and E. Duflo (2014). Do firms want to borrow more? testing credit constraints using a directed lending program. _Review of Economic Studies, 81_ , 572–607.

- Banerjee, A., E. Duflo, R. Glennerster, and C. Kinnan (2015). The miracle of microfinance? evidence from a randomized evaluation. _American Economic Journal: Applied Economics 7_ (1), 22–53.

- Banerji, A. and S. Jain (2007). Quality dualism. _Journal of Development Economics 84_ (1), 234–250.

- Beck, T., A. Demirg ̈u ̧c-Kunt, and V. Maksimovic (2005). Financial and legal constraints to growth: Does firm size matter? _The Journal of Finance 60_ (1), 137–177.

- Beck, T., A. Demirg ̈u ̧c-Kunt, and V. Maksimovic (2008). Financing patterns around the world: Are small firms different? _Journal of Financial Economics 89_ (3), 467–487.

- Betz, F. and F. Ravasan (2016). Collateral regimes and missing jobcreation in the mena region.

- Bigsten, A., P. Collier, S. Dercon, M.
.193, so one standard deviation increase in cumulative sales would be associated with a 0.7 percentage point (=0.193*0.038) increase in ROE. Finally, regressions 7-9 in Table 3 suggest that there are no statistically significant short-term differences in cost-to-income ratios a few years

> 20 We control for lagged ROA in the specifications except where the dependent variable was ROA, ROE or NIM. 21 All bank-level variables come from the Bankscope (1995-2014) and Bank Focus Orbis (2015-2017) databases and all macro variables come from the Global Financial Development Database.

11
**_Core Indicators for Determinants and Performance of Electricity Sector Reform in Developing Countries_**

and that vulnerable income groups are protected. This type of question can be addressed through case studies involving cost-benefit analysis and distributional impacts. The main core indicators with reference to the components of our reform model are price and consumption data for income groups (Table IX), changes in access and quality of service (Table V), regulatory framework (Table IV), and where possible environmental impacts of reforms (Table X).

# **5.3 Electricity sectors indicators**

# **I. Sector endowments and characteristics**

|**Focus area**|**Indicator**|**Data Source**<br>**(UK as example)**|**Reference**|
|---|---|---|---|
|_? Resource mix_|**? Electricity generation mix:**<br>**(i) net generation capacity (MW)**<br>**(ii) energy supplied (MWh)**<br>? Primary energy supply mix (MTOE)|IEA and OECD||
|_? Energy security and_<br>_resource independence_<br>_? Reserve generation_<br>_capacity_|**? Electricity consumption, domestic**<br>**production, import, export (MWh)**<br>? Energy consumption, domestic production,<br>import, exports (MTOE).<br>**? Self-sufficiency in electricity – domestic**<br>**production GWh / domestic production + net**<br>**imports (%)**<br>? Energy self-sufficiency – domestic/total (%)<br>? Reserve capacity – at maximum demand<br>(MW, and as % of total installed capacity)||
Sekhon, J. S. (2013). Genetic matching for estimating causal effects: A general multivariate matching method for achieving balance in observational studies.</mark> _<mark>Review of Economics and Statistics</mark>_ <mark>,</mark> _<mark>95</mark>_ <mark>(3), 932-945.</mark> <mark>Duffy, B., Smith, K., Terhanian, G., & Bremer, J. (2005). Comparing data from online and face-to-face surveys.</mark> _<mark>International Journal of Market Research</mark>_ <mark>,</mark> _<mark>47</mark>_ <mark>(6), 615-639.</mark> 24
Policy Research Working Paper 7889

# **Abstract**

This paper estimates the welfare impact of rural electrification in India using nationally representative household panel survey data for 2005 and 2012. Analysis based on a propensity-score-weighted fixed-effects model finds that while electrification is associated with a broad range of social and economic benefits, the size of the effects depends importantly on the reliability of electricity service. Gaining access to electricity combined with a reliable power supply is associated with a 17 percent increase in income during the sample period, but gaining access to electricity alone is associated with only a 9.6 percent increase in income. The net gain from both increasing the access rate and reducing power outages in rural India is estimated to be US$11 billion a year. Moreover, India’s rural electrification policy appears to be progressive because lower-income households benefit more from access to electricity than higher-income households during the sample period.

This paper is a product of the Office of the Chief Economist, South Asia Region. It is part of a larger effort by the World Bank to provide open access to its research and make a contribution to development policy discussions around the world. Policy Research Working Papers are also posted on the Web at http://econ.worldbank.org. The authors may be contacted at fzhang1@worldbank.org.

_The Policy Research Working Paper Series disseminates the findings of work in progress to encourage the exchange of ideas about development issues. An objective of the series is to get the findings out quickly, even if the presentations are less than fully polished. The papers carry the names of the authors and should be cited accordingly. The findings, interpretations, and conclusions expressed in this paper are entirely those of the authors. They do not necessarily represent the views of the International Bank for Reconstruction and Development/World Bank and its affiliated organizations, or those of the Executive Directors of the World Bank or the governments they represent._ Produced by the Research Support Team
arms is larger for men than for women, although the resulting gender gaps in SDG indicators are not statistically different.

The findings of this paper suggest that SDG indicators 1.4.2 (a) and 5.a.1 can be reasonably monitored through the use of proxy respondents and/or aggregated land data, in the case of Armenia and potentially in other cases with relatively high rates of documented land rights. However, in conducting analyses of land rights beyond SDG monitoring, biases are still observed with the use of these design decisions for rights other than legal documentation, including the right to sell, right to bequeath, and perception of tenure security. In future work, the behavioral mechanisms behind the biases suggested by the proxy respondent and aggregation approaches will be unpacked. Given the uniqueness of the Armenian land context, the study will also be replicated in an alternative context to assess the generalizability of the results to areas with lower rates of documented land rights.

19
the above Methods</mark>**

<mark>The selection of covariates for constructing propensity scores has been widely studied.</mark><sup>11</sup> <mark>Covariate selection is usually done in a variety of ways, such as stepwise regression that drops variables that are not significant (Berk and Newton 1985, Lieberman et al. 1996), or conducting one-step covariate selection based on theoretical and logical significance. Some studies argue that there are no serious errors if the model for propensity score adjustment is mis-specified (Drake, 1993), other studies note that the choice of covariates is critical for propensity score weighting (Isaksson and Forsman, 2003). Covariates may be country specific (Stenbjerre, 2002). In countries with homogenous populations and high Internet penetration, there could be fewer covariates that discriminate web survey participation, unlike in countries that have low Internet penetration. There is also a general consensus in the literature that webographic questions, in addition to regular demographic variables, are needed to calculate the propensity scores more accurately.</mark> Several studies examine non-economic averages and how adjusting can potentially reduce bias between volunteer Internet surveys and non-Internet probability-based surveys. Most studies indicate that various weighting schemes do not adequately reduce the difference between weighted Internet survey data and the reference survey. Varedian and Forsman (2003) investigate

> 10 Many Internet survey companies, such as, Harris Interactive, increasingly use webographic questions that are thought to best capture the differences between the general population and those willing to answer a web survey.

> 11 There is a tendency to use only those covariates that are statistically significantly different between treatment and comparison groups. Rosenbaum (2002) offered three cautions against this approach because the relationship between the outcome variable and one covariate is not important, statistical significance is not relevant always especially because it depends on the sample size.

7