rafmacalaba/data-use-mentions-extended huggingface dataset:
recent studies using survey data of US city managers finds higher satisfaction of employees when performance pay is used (Stazyk 2010). The research does however highlight the complexity of the issue as it suggests that there is a crowding-out effect on extrinsic motivation for staff with a distinctively high level of intrinsic motivation — but ultimately this does no harm to effort or to job satisfaction.

(Dowling and Richardson 1997) evaluate the effect of performance pay on UK National Health Service managers, a study, since it focuses on management rather than physicians, that is more closely related to core public administration jobs than healthcare jobs. Using self-reported data from a survey, they find a modest positive effect of pay incentives on manager motivation and effort.

Whether PRP should be used is one thing — when and where it can be used is another. (Dahlstrom and Lapuente 2009) theorizes, using transaction cost economics, that when senior officials share a career path with elected politicians and thus are unlikely to act impartially, then PRP is less often used as those officials are unable to provide credible commitments to their staff. Why would one work hard to achieve a goal if it transpires that the goalposts might be arbitrarily moved for political reasons at the last moment? They find empirical support for this prediction.

# **_Public sector craft jobs_**

Observational studies about performance-related pay for public sector craft jobs are more optimistic concerning their effectiveness. There is more evidence to draw on from developing countries, particularly in relation to teaching and health care.

# **Tax administration, job placement**

Revenue authorities are examples of public sector agencies that can be classified as craft organizations where outputs — number of audits conducted and tax fines collected — are more easily measurable and there is a clearer link between the efforts of, for example, individual tax auditors and revenue collection.

14
.00|
|Urban cluster|167,357|0.19|0.00|0.40|0.00|1.00|
|Rural|167,357|0.45|0.00|0.50|0.00|1.00|
|**DB urban classifcation:**|||||||
|Core|167,357|0.31|0.00|0.46|0.00|1.00|
|Suburb|167,357|0.09|0.00|0.29|0.00|1.00|
|Town|167,357|0.21|0.00|0.41|0.00|1.00|
|Other rural|167,357|0.60|1.00|0.49|0.00|1.00|

Source: International Urban Poverty Database.

Note: The statistics above are based on the household-level data pooled for 16 SSA countries. Real per capita consumption expenditures are deflated using a spatial deflator calculated using the GHSPOP 1km dataset and the DOU method.

60
# **3. Impact evaluation design**

Our starting sample consists of all firms that participated in NaN in 2018, in three out of six SEBRAE-PR offices. We limited the number of offices participating in the intervention to simplify the logistics. From the starting sample, we dropped firms that (i) had participated in a previous round of NaN (before 2018), (ii) adopted all 29 business practices, as measured by the questionnaire applied during the first NaN visit, or (iii) whose firm identification number (CNPJ) was not found in the 2015 RAIS database, which we use to measure employment, as described below. RAIS data is available with a time lag, and 2015 was the latest year we were able to access at the time of randomization. This selection criterion thus restricts our sample to firms that were at least three years old.

The final sample includes 866 firms. We randomly assigned these firms to a control and four treatment groups. The control group received the standard six-page report described in section 2. The treatment groups received the standard six-page report, plus one of the four versions of the information sheet. Appendix figure A1 provides an overview of the five experimental groups.

Randomization was done by computer, in weekly batches, to enable returning the report one to two weeks after the first NaN visit. Randomization was also stratified by SEBRAE-PR office and by whether the firm had previously used any SEBRAE service.

# **4. Data, timeline, and baseline summary statistics**

Our data comes from four different sources. Appendix figure A2 shows the timeline of the intervention and data sources. First, we have administrative data from SEBRAE-PR on use of SEBRAE services between January 2017 and March 2020, corresponding to about 12 months before and 18 months after the intervention. As outcome variables, we code dummies indicating whether a firm used a SEBRAE service within three, six, twelve, and eighteen months relative to month zero, which we define to be the month when the
28 Similarly, Table 7 shows that nontariff barriers were also structured along lines that would reinforce the trade distorting effects of the agreements preferential tariffs. According to the UNCTAD data, nontariff restrictions were applied to about 21 percent of all tariff line items, which is almost onehalf the corresponding ratio (41.9 percent) for Mercosur's first decile products. Similarly, the trade coverage ratio for products falling in the first three deciles averaged almost 60 percent (about 3 times the ratio for all imports) and reached 84 percent for the first decile group.

# V. Policy Implications

This paper identified dramatic changes in the Mercosur countries' trade patterns over the period 1988-94 and particularly over its latter half. It also argued that these are substantially due to the changes in trade policy and probably mainly those introduced under Mercosur transitional arrangements. Statements of this kind depend on comparing actual data with a counterfactural. Two natural candidates present themselves. The counterfactual implicit in this paper is that, but for these policy developments, Mercosur's trade patterns would not have changed much and that the shift towards apparently uncompetitive capital-intensive intra-block trade would not have occurred. The implication is that if the Mercosur countries had achieved an equivalent degree of liberalization on a non-discriminatory basis they would have maintained a more efficient import structure, paying less and/or obtaining better goods, and they would have purchased more from their trading partners outside the block.<sup>21</sup> Given the size of the trade effects identified, the evidence that preferences and the application of industrial policies within RTAs can be distortionary is both compelling and disturbing. It is true that one could imagine other explanations

> **21The** reader should specifically note that trade patterns in 1994 are being compared with those in 1988 and not with some "free-trade" counterfactual. The assumption is that 1994 would have been no more distorted than 1988. We believe that the latter is less distorted geographically because, in the earlier period, trade barriers were applied on a most-favored-nation
were delicensed. Later in 1991 India went through a balance of payment crisis which resulted in a Stand-By arrangement with IMF in August 1991. In this period IMF requested macroeconomic stabilization policies and structural reforms. As a part of the structural reforms licenses were removed for another group of industries. A number of industries were retained from the licensing reform due to security and strategic concerns, social reasons, hazardous nature of the products, and to elitist consumption.<sup>6</sup> Aghion et al. (2008) and Chari (2009) provide a detailed discussion of this reform and list the industries that were delicensed and when they were delicensed. Delicensing reforms were mostly unanticipated.<sup>7</sup> I use the data on licensing reform compiled by Aghion et al. (2008). They use various issues of Handbook of Industrial Policy and Statistics, press notes, and notifications issued by the federal government to code when different industries were exempted from industrial licensing. In Table 3, I present the percentage of firms that belong to a 4-digit NIC industry that has been delicensed. In 1989 around 40% of the firms belonged to an industry that was delicensed on or before 1988. In 1992, the share of delicensed firms more than doubles.

India had several other major market reforms. One of these reforms was on international trade. Average tariff rates and non-tariff barriers were among the most restrictive in Asia towards the end of 1980s. Topalova and Khandelwal (2011) show that there has been radical changes in economic policies during 1990s and this has led to a continuous decline in average tariff rates from 97% in 1989 to 46% in

> 6 There were also some industries that were delicensed in different years than 1985 and 1991 such as manufacture of domestic appliances, n.e.c which was delicensed in 1993.

> 7 Aghion et al. (2008) provide a discussion of this conclusion.

7
# **_Comparison to previous studies_**

Diaz and Handa (2006) use propensity score matching techniques to construct alternative control samples from the Mexican national household survey. They compute average treatment effects by comparing the immediate treatment sample after eight months of receiving program benefits with the delayed treatment sample (who had not yet received benefits), on the one hand, and their newly constructed control samples, on the other. They conclude, “The PSM [propensity score matching] technique requires an extremely rich set of covariates, detailed knowledge of the beneficiary selection process, and the outcomes of interest need to be measured as comparably as possible in order to produce viable estimates of impact” (p.341). In our case, the outcomes are measured in identical ways in the delayed treatment and control samples, and the control sample is constructed following the beneficiary selection process as closely as possible. Our selection of covariates for the propensity score matching closely follows Behrman et al. (2009b), who use almost identical covariates in comparing the effects on schooling outcomes of the short-run differential exposure (between the immediate and delayed treatment samples) with the long-run differential exposure (between the immediate treatment and control samples). They find that longer exposure produces larger effects, and the differences between the order of magnitude of the short- and long-run effects are reasonable. This finding suggests that the propensity score matching technique we use can produce reliable estimates of average treatment effects.

The interpretation of the difference between the distributions of the weighted treatment and control samples as a treatment effect depends on the extent to which the weighting procedure manages to correct for possibly unobserved heterogeneity caused by the imperfect randomness of the assignment to treatment and control groups. Of course, it is not possible to test this directly, but we can compare our results to the findings in the literature that consider differences in 23
14 This decline in real spending should not be misinterpreted. It is estimated that Russian GDP dropped by 40% over 1991-1995. Thus, when measured as a percent of GDP, regional budget expenditures in 1995 were still above their 1991 level. And given the much higher rate of decline in federal budget expenditures, it is worth noting that in 1995 Russian regions controlled a higher share of the consolidated budget than in 1992, which reflects a dramatic fiscal decentralization in Russia that took place over 1991-1994 (World Bank, 1995a).'<sup>9</sup> The individual components of the national economy expenditures show widely varying trends over time. Food subsidies (never a large percentage of the total) were given by fewer and fewer regions from one year to the next, and for 1995 no food subsidy data were reported in the budget. The single largest category of expenditure, housing, dropped by 24%, that is, less than overall national economy expenditures in the period 1993-95 (thus, from 1993 to 1995 housing expenditures as a percentage of national economy expenditures increased from 56% to 62%.) The 34% decline in agricultural subsidies from 1993-95 matched the decline in overall national economy expenditures in that period, while transportation rose slightly from 1993 to 1995 (albeit with a dip in 1994), and the category 'bther' representing mainly subsidies to industry, energy and construction, declined by 59% from 1993 to 1995 (recall, however, the qualification to the comparability of 1993 and 1994 data with those of 1995.) Per capita investment in the Russian Federation dropped on average by 33% in real terms over the period 1992-1995, while loans to enterprises remained basically unchanged, while constituting a relatively small part of overall expenditures.

The relative declines in per capita total budget revenue were greater than for total budget expenditures. As a result, the regional budget surplus, which was quite large in 1992, almost disappeared. From 1992-95, pre-transfer revenue dropped by about 27% on average
## **Figure 22.  Returns to Education in Latvian Labor Market by Source and Group of Employees, 2002**

(upper panel: general education ladder, lower panel: vocational education ladder) <!-- Start of picture text --> 0.80<br>0.70 higher vs secondary,<br>"reward"<br>0.60<br>0.50 higher vs secondary,<br>"access"<br>0.40<br>secondary vs basic,<br>0.30<br>"reward"<br>0.20<br>secondary vs basic,<br>0.10 "access"<br>0.00<br>0.35<br>0.3 postsecondary<br>vocational vs technical<br>0.25 secondary, "reward"<br>0.2 postsecondary<br>vocational vs technical<br>0.15<br>secondary, "access"<br>0.1<br>technical secondary,vs<br>0.05 basic, "reward"<br>0<br>technical secondary, vs<br>basic, "access"<br>all malefemaleLatviannon-Latvianpublicprivateurban rural<br>allmalefemaleLatviannon-Latvianpublicprivateurban rural<br>log points<br>log points<br><!-- End of picture text --> _Notes:_ ”Reward” component is returns to education when major group of occupations and firms size are controlled. ”Access” component is difference between returns without and with occupation and firm size controls. Returns are numerically smaller than in Appendix Tables 24 and 25 because they are measured in logarithmic points rather than percent to allow decomposition. Professional higher education is not separated from academic. Presented results refer to full-time employees. _Source:_ Calculation based on LFS data.

# **Experience and Earnings**

**Age–earnings profiles.*
in PNG that precipitated a humanitarian disaster and a forced displacement crisis. The analysis also demonstrates that the change in reported prevalence of IPV was not as substantial and was less precisely estimated, and illustrates that one likely explanation is the underreporting of IPV incidents in modules that directly ask women about the incidents that they experience. And lastly, the analysis explores potential mechanisms that explain the decline in attitudes about IPV for the entire population of men and the decline in reported prevalence among a subset of the population of women.

However, there are many avenues that the analysis is unable to explore given data limitations. First, as discussed above, we are not fully able to identify how the roles and responsibilities of women change in the household, and thus we cannot fully identify all the ways in which the disaster might have changed the intrahousehold dynamics that might result in changes to IPV. More detailed survey instruments and combining quantitative household data with some more qualitative discussions could better identify how disasters might change household dynamics. Although more subjective in nature, more detailed questionnaires that inquire why women think their husbands might commit acts of intimate partner violence could help better explain these potential pathways and understand IPV more generally.

Second, these results are relatively short-term results and the data coverage following the disaster limits our ability to more fully investigate the full impacts of the disaster. Whether these changes persist more than seven months following the disaster is unable to be explored here; and we are unable to explore whether data collected even more closely in time to the disaster or whether data collected from regions even more strongly impacted than those analyzed here might have resulted in similar or even stronger findings. But the continuation of a survey in the midst of such disasters are relatively uncommon and these results add additional information relative to other settings where data might be collected long after the disaster ends.

And lastly, these results only investigate how disasters might change IPV. Although some of the results might have implications for what might improve the IPV situation in the country more generally,
is more than one respondent in a given country-policy-cluster pair and these respondents did not hold their respective public sector positions during the same period of time.<sup>19</sup> **_α_ 2** is a vector of coefficients for our respondent-level controls contained in **X2** _i_ . Here _δn_ denotes unmeasured time-invariant determinants of the World Bank’s policy influence in countrypolicy-cluster _n_ and _rn_ is a country-policy-cluster indicator dummy, with _N_ representing the total number of countries and _k_ representing the total number of policy-clusters in the sample. The term _εi_ is an error term that denotes all unmeasured idiosyncratic factors that affect the World Bank’s reported level of policy influence at the individual respondent level.

In order to measure exposure to the World Bank’s lending and non-lending services, we extract information on the timing, policy focus, and cross-country distribution of ESW products, DPLs, PAs, and investment loans from the organization’s Business Intelligence Database and Development Policy Actions Database. The number of World Bank ESW products received by a country in a given policy cluster and year is our primary variable of interest ( _n_ _~~e~~ sw_ _~~m 1~~ p_ ) and test of _H1_ . The World Bank assigns each of its ESW products to one of four policy clusters: Macroeconomic and Structural Policy (MSP), Human Development (HD), Sustainable Development (SD), and Governance (GOV). Therefore, we cross-walk these policy cluster-categorized ESW data to the 24 policy domains to which _2014 Reform Efforts Survey_ respondents anchored their evaluations of the World Bank (see Table A1 in the Appendix). This approach allows to define exposure to the World Bank ESW “treatment” at the country-policy-cluster-year level, so if a respondent worked in a specific country and policy cluster that received ESW from the World Bank during a year in which he or she evaluated the policy influence of the World Bank, we identify the respondent as having experienced exposure to the World Bank ESW
- Jacob, B. A. and L. Lefgren (2003). Are idle hands the devil’s workshop? Incapacitation, concentration, and juvenile crime. _The American Economic Review 93_ (5), 1560–1577.

- Jaitman, L., D. Caprirolo, R. Granguillhome Ochoa, P. Keefer, T. Leggett, J. A. Lewis, J. A. Mej ́ıa-Guerra, M. Mello, H. Sutton, and I. Torre (2017). The costs of crime and violence: New evidence and insights in latin america and the caribbean. _Inter-American Development Bank_ .

- John, O. and S. Srivastava (1999). The Big-Five Trait Taxonomy: History, Measurement, and Theoretical Perspectives. In L. Pervin and O. John (Eds.), _Handbook of Personality: Theory and Research_ , pp. 102–138. New York: Guilford Press.

- Jones, D., D. Molitor, and J. Reif (2018). What do workplace wellness programs do? evidence from the illinois workplace wellness study. Technical report, National Bureau of Economic Research.

- Kahneman, D. (2011). _Thinking, fast and slow_ . Macmillan. New York and London.

- Kassam, K. S., A. R. Markey, V. L. Cherkassky, G. Loewenstein, and M. A. Just (2013, June). Identifying Emotions on the Basis of Neural Activation. _PLoS One 8_ (6), e66032.

- Korjus, K., A. Uusberg, H. Uusberg, N. Kuldkepp, K. Kreegipuu, J. Allik, R. Vicente, and J. Aru (2015, February). Personality Cannot Be Predicted from the Power of Resting State EEG. _Front Hum Neurosci 9_ (63), 1–7.

- Krueger, A. B. (2003). Economic considerations and class size. _The Economic Journal 113_ (485
on governance. First, governance can be measured. While many measures suffer drawbacks, advances in how measurements are made and how they are used (e.g. addressing bias, reporting errors) show that this is still a fruitful endeavor. Second, governance measures can be useful both for advocacy and analysis. Organizations like Transparency International show how creating an index can keep an otherwise vague issue like corruption visible as a public policy issue. At the same time, an extensive research literature is using governance indicators to further our understanding of how different aspects of governance affect a country‟s performance. Finally, misuse and misinterpretation of governance measures are common. By recognizing particular errors related to misattribution of causality and considering particular mechanisms to be separable from their context, it should be possible to use these measures more effectively.

The term “governance” has entered the health sector literature in at least three different ways, paralleling and influenced by these other bodies of work. As in the corporate and public sector literature, part of the health sector literature has looked at governance from the perspective of relationships among actors as they influence the behavior of specific organizations such as hospitals (Harding and Preker 2003) or mandatory health insurance institutions (Savedoff and Gottret 2008). Much as the corporate and public sector literature was extended to include broader social goals, another stream of work in the

> 3 <u>http://info.worldbank.org/governance/wgi/index.asp</u>

______________________________________________________________________________________ -5-
the standard deviation of the leader’s contribution to growth is around 1.5% (for all leaders) and is statistically nonzero. As leader quality is normally distributed with mean zero in their paper, this is equivalent to Jones and Olken (2005) estimating the distribution of leader quality across all leaders, shown in the solid red line in Figure 1. Their finding is pioneering in showing that leaders matter for growth in general. But it does not tell us much about the contribution of any particular leader, which is the problem faced by constituents and international actors (and is the focus of dayto-day political and academic debates).

<!-- Start of picture text --> Figure 1: Comparing this Paper and Jones and Olken (2005)<br>45<br>40<br>Observe growth data ( ), Seretse Khama (BWA)<br>35 solve signal extraction problem leader contribution<br>ii −ii<br>gg̅ , gg̅ probability density<br>30 (this paper) [Posterior]<br>2)<br>25 ALL leaders growth ii ii −ii ii ii<br>contribribution. μμ |gg̅ , gg̅ ~NN(μμ̂ , RRD<br>20 Probability density from ii ii ii ii −ii<br>Jones and Olken (2005) μμ̂ = ψψ gg̅ −γγ gg̅<br>2 ii μμ ii<br> ) [Prior] RRD = σσ 1 −ψψ<br>15<br>ii μμ<br>μμ ~NN(0, σσ<br>10<br>5<br>95% CI=<br>0 ii ii<br>μμ̂ ± 2RRD<br>-5% -4% -3% -2% -1% 0% 1% 2% 3% 4% 5% 6% 7% 8%<br>Leader contribution to per