rafmacalaba/data-use-mentions-extended huggingface dataset:
# **3. Challenges in Measuring Impact**

Impact evaluations which measure the effects of business training programs on business performance rely primarily on survey data to measure outcomes. In order to obtain credible and useful estimates, studies must have sufficient power, measure impacts over an appropriate time horizon, deal with survey attrition and selective survival and start-up of firms, and address the possibility that training changes how firms report business outcomes even if it doesn’t change those outcomes. We discuss each of these challenges and assess how well existing studies have met them in order to provide context for the results discussed in the next section.

# _Power_

The power of a statistical test is the probability that it will reject a null hypothesis given that the null hypothesis is false. A starting point for most business training evaluations is to test the null hypothesis that the intervention had no effect, so the power of the experiment is a measure of the ability to detect an effect of training if such an effect does exist. The key determinants of the power of a study are the size of the sample, how much heterogeneity there is in the sample (the more diverse the set of firms the harder it is to measure a change for them), whether the intervention takes place at an individual or group level (power is lower for a given sample size when treatments are allocated at the group level), and the size of the treatment effect. Low take-up rates dilute the treatment effect, reducing power.

Table 5 compares different studies in terms of these components of power. Most studies conduct individual level randomization, with the exception of several studies working with microfinance groups, where different repayment groups are allocated to treatment or control, and Calderon et al. (2012) who first randomize at the village level, and then at the individual level within the treatment villages. A typical study involves approximately 200 to 400 individuals or groups in each of the treatment and control, although sample sizes have been smaller for the studies based on specific
” _Quarterly Journal of Economics_ 113 (3): 903–47.

- Corden, W., and P. Neary. 1982. “Booming Sector and De-Industrialization in a Small Open Economy.” _Economic Journal_ 92 (December): 825–48.

- Davis, S., and J. Haltiwanger. 1992. “Gross Job Creation, Gross Job Destruction and Employment Reallocation.” _Quarterly Journal of Economics_ 107(3): 819–63.

- Gaulier, G., D. Taglioni, and S. Zignago. 2012. “Export Performance in the Wake of the Global Crisis: Evidence from a New Database.” Mimeo.

- Hausman, R., J. Hwang, and D. Rodrik. 2007. “What You Export Matters.” _Journal of Economic Growth_ 12: 1–25.

- Helpman, E., M. Melitz, and Y. Rubinstein. 2008. “Estimating Trade Flows: Trading Partners and Trading Volumes.” _Quarterly Journal of Economics_ CXXIII: 2.

- Krugman, P. 1996. “Making Sense of the Competitiveness Debate.” _Oxford Review of Economic Policy_ 12: 17–25.

- Malouche, M., J. Reis, and S. Waglé, 2012. “The Character of Export Growth in Developing Countries 1998-2008”. Mimeo.

- Melitz, M. J. 2003. “The Impact of Trade on Intra-Industry Reallocations and Aggregate Industry Productivity.” _Econometrica_ 71 (6): 1695–1725.

28
|<br>No|<br>Yes|<br>Yes|<br>No|
|<br>No of countries|149|149|149|100|100|100|92|
|No of observations<br>Note: * p<0.10, ** p<0.<br>Development Indicators|1020<br>05, *** p<0.<br>(WDI), Open|1020<br>01. Standard er<br>Data Watch (|1020<br>rors are cluster<br>ODIN), Global|361<br>ed at the count<br>Data Baromete|361<br>ry level. Data<br>r (GDB), and|361<br>from the Worl<br>Open Data Ba|92<br>d Bank's World<br>rometer (ODB).|

Note: * p<0.10, ** p<0.05, *** p<0.01. Standard errors are clustered at the country level. Data from the World Bank's World Development Indicators (WDI), Open Data Watch (ODIN), Global Data Barometer (GDB), and Open Data Barometer (ODB). In cases where data are missing for a particular covariate, the data are imputed forward using the nearest available value. Estimates with country fixed effects not available for the Global Data Barometer, because the indicator contains only one time period.

69
people can be trusted and that people are helpful, but less likely to agree that people are fair. They are also less worried about crime. These differences, however, are small, and they are almost zero with respect to caution towards foreigners and donations. In Appendix A.3, we report the descriptive statistics for the estimation sample.

The goal of the econometric analysis is to see whether the _change_ in attitudes after the inflow versus before was different among people in high-inflow vs. low-inflow areas. The descriptive analysis in Figure 4 suggests that this was not the case: for most outcomes, the gap between people’s attitudes in high- and low-inflow areas is similar before and after the inflow.

**Data on Anti-Immigrant Violence** To investigate whether the inflows of refugees and asylum seekers affect natives’ behavior towards foreigners — and not only their attitudes — we use information on anti-immigrant violence in Germany. We use a geo-referenced event dataset reporting all instances of anti-refugee violent actions that have been documented by the project _Mut gegen rechte Gewalt_ by the Antonio Amadeu Foundation. The foundation collected information on anti-immigrant incidents such as assaults, attacks against refugee housing and arson, as well as anti-immigrant demonstrations. For each incident, the database records information on the time, location, number of victims and perpetrators as well as a description of the incident and a link to the original source. The information was mostly taken from newspaper articles, press releases by the German police, parliamentary interpellations, as well as publicly accessible reports by local and regional organizations offering advice to victims of right-wing violence. For our analysis, we use a dataset provided by Bencek andˇ Strasheim (2016), who scraped the database from the Foundation from 2013 to 2018. Based on their code, we additionally scraped all entries up until 2020.

Figure 5 shows the overall number of anti-immigrant incidents in Germany between 2014 and 2017 as well as the number of asylum applications per month. The number of incidents was low until mid-2015 and increased in the second
basis, to effectively compete in services ―trade,‖ it still is likely that it requires more of a domestic presence than trade in goods.<sup>4</sup> Given the key role of FDI in the effective delivery of services, commitments to foreign investors are key aspects of trade negotiations in services. With the growing importance of services trade and FDI in services, it is important to have a framework with which we can analyze the impacts of liberalization of barriers to FDI in services.

The way my co-authors and I have modeled services builds crucially on the growing body of empirical evidence that shows that increased access to services, increased foreign direct investment in services or services liberalization increases the productivity of the manufacturing sectors and of the economy as a whole. (See Rutherford and Tarr, 2008, 2010; Markusen, Rutherford and Tarr, 2005; Jensen, Rutherford and Tarr, 2006, 2007, 2010; Balistreri, Rutherford and Tarr, 2009; Jensen and Tarr, 2008, 2011; Balistreri and Tarr, 2011.) Further there is evidence that increased access to imported intermediate goods increases economy-wide productivity. I summarize the computable general equilibrium modeling literature on services below, where I argue that our approach is the first to incorporate the liberalization of barriers against foreign direct investment in services where endogenous productivity impacts play an important role in the results. In our full applied models, we also incorporate the endogenous productivity impacts from increased access to intermediate goods. My co-authors and I have shown that in applications where liberalization of barriers against foreign investors in services is important, such as in accession to the World Trade Organization, the estimated welfare gains depend crucially on our modeling framework—our approach yields estimated welfare gains that are several multiples of the estimated gains of models without foreign direct investment in

> 4 Data on the sales of foreign affiliates of U.S. firms suggests that sales through FDI is the most important channel for U.S. firms to sell services to foreigners (Francois and Hoekman,
**Appendix 9.  Change in Reduced Coping Strategy Index Following the 2017 IPC Announcement**

||(1)|(2)|(3)|(4)|(5)|
|---|---|---|---|---|---|
|VARIABLES|ln (rCSI)|ln (rCSI)|ln (rCSI)|ln (rCSI)|ln (rCSI)|
|Post Announcement Indicator|-0.014|-0.009|-|0.014|0.014|
||[0.019]|[0.021]||[0.030]|[0.030]|
|Quarter Indicators||||||
|____________________<br>Q2 2017 Indicator|-||0.057***|-|-|
||||[0.020]|||
|Q3 2017 Indicator|-||-0.003|-|-|
||||[0.015]|||
|Q4 2017 Indicator|-||-0.012|-|-|
||||[0.021]|||
|Q1 2018 Indicator|-||-0.012|-|-|
||||[0.033]|||
|Q2 2018 Indicator|-||-0.071**|-|-|
||||[0.030]|||
|Q3 2018 Indicator|-||-0.078*|-|-|
||||[0.038]|||
|Interactions Between IPC Status and the Post<br>Announcement Indicator||||||
|____________________<br>Post Announcement Indicator*IPC 4 Indicator|-||-|-0.056|-|
|||
was unable to present us with any current DHIS data on their performance against the six indicators. We also noted that the new PS is fairly unaware of SOML and, unlike his Abia and Anambra counterparts, did not seem to know the various indicators.

# 8.2 Factors that contribute to divergence in outcome indicators between states

|**Existing levels of state**<br>**funding**|The 36 Nigerian states have very different levels of existing revenue available to them and<br>therefore different levels of additional resource to draw on in order to supplement SOML<br>funds. It is apparent that whilst the delay in SOML funds arriving in some states meant<br>those states were unable to begin any substantial interventions, there are other, more<br>financially viable states who were able to continue complementary activities even in the<br>absence of fresh resources. A state like Rivers which has a higher Internally Generated<br>Revenue (IGR) and receives a greater federal allocation than any other state, has drawn on<br>a large bank of resources to supplement SOML efforts, building health infrastructure,<br>improving health worker remuneration and heavily investing in equipment at health<br>facilities. The simple fact of state indebtedness is likely to have a major impact on health<br>interventions since it directly impacts health worker salaries and therefore motivation.|
|---|---|
|**Presence and capability of**<br>**the private sector**|As indicated, the ADG states were selected from a group of 16 states that began with a<br>higher baseline on the six coverage indicators; apparently this was considered a rough<br>proxy for existing levels of “capability”. These states were judged to be higher in existing<br>capability and lower in existing donor activity. But this is a crude measure of capability. It<br>has since been
# **E Additional treatment effects analysis**

## **E.1 Robustness of treatment**

Our robustness tests focus on the five main summary outcomes. First, in Table E.1, we show robustness to alternative ways of constructing the indexes and pooling or averaging of endlines. Columns 2–4 report results from the main paper for comparison. Recall that in this main specification we averaged endline surveys (at 2 and 5 weeks, and 11 and 13 months), took an index of composite measures rather than individual survey questions and used equal weights. In columns 5–7, we do the same except use randomization inference to assess statistical significance. In columns 8–10, we pool our composite measures from both endline surveys and cluster our standard errors by individual. In columns 11–13, we do the same except weight each survey question equally. In columns 14–16, we use covariance-weighted indexes from Anderson (2008) and average both endlines.<sup>16</sup> The conclusions from these three specifications are quantitatively similar to those from the main specification. Exceptions are as follows:

- The impact of cash and therapy on the covariance-weighted antisocial behaviors index is not significant in the long term at conventional levels. This is because half of this index’s weights come from domestic violence and number of arrests, two components that were unaffected by treatment. If we exclude domestic violence from the index and recalculate covariance weights, cash and therapy lead to a .26 standard deviation decline in antisocial behaviors in the long run (columns 19, significant at 99% level).

- Cash increases antisocial behaviors in the long term in some specifications. In Column 15 we see that after a year the men who report cash only increased their antisocial behaviors by 0.17 standard deviations. In the other specifications, the coefficients are positive as well but smaller and not statistically significant. One possibility is that receiving a cash grant and failing, or having the money
that includes motor skills, problem solving, socio-emotional skills, language and cognitive capacity. Given that the Malagasy use of the term ‘intelligence’ encompasses similar constructs as the ASQ-I, our objectives were to 1) compare caregiver-perceived child intelligence in relation to other children in the community with how children scored on the ASQ-I, and 2) explore what child, caregiver, and household factors were associated with discordance between the two measures.

# 2. DATA

# 2.1. Study Setting and Design

Given that specific developmental domains of cognition, language, vision, and hearing largely occur in the early years of life (Thompson & Nelson, 2001), we focus on children under four years of age. Cross-sectional data of rural women and children aged 16-42 months were collected as part of a cluster-randomized controlled trial in Madagascar (Fernald, et al., 2016). The sample represents five regions of the country, which were selected because they have some of the highest rates of stunting and food insecurity in Madagascar: Amoron’ i Mania, Androy, Atsimo Atsinanana, Haute Matsiatra, and Vatovavy-Fitovinany. The study tested the value added of intensive nutrition counseling, lipid-based supplementation, or a home visit parenting program on child growth and development. All pregnant women and women with age-eligible children for the intervention were eligible to participate in the trial. A total of 3,560 caregivers and children were interviewed and assessed at endline. The measure of ECD, ASQ-I, was complete for 3,533 children. Missing values for caregiver age, caregiver education, and child birth order were imputed using data from other rounds. After imputation, complete data on child-, caregiver-, and household-level characteristics were available for 3,361 children.

5
However, to the best of our knowledge, this is the first such work using tax data from developing countries. Also, unlike most previous studies,<sup>2</sup> the tax authorities have not imposed any extra penalty on the visited properties. Together, these two characteristics represent a setting where auditing programs were less likely to work and make a stronger case for the adoption of such programs in a developing country setting.

Our results, also open a discussion on the importance of considering enforcement strategies while designing tax law. The distributional impacts of a tax system are frequently addressed through tax policy design separately from tax enforcement strategies. The presence of heterogeneous effects in enforcement strategies illustrates the case for which a progressively designed tax system where the mainly poorest are induced to be compliant may yield a regressive outcome. We are investigating these issues more deeply in another working paper, "Policy Learning in Taxation".

This paper is structured to provide a comprehensive exploration of the evaluation of a tax audit program in Gorontalo, Indonesia, from theoretical models to empirical findings. Section 2 is a literature review, focusing on two pivotal areas: the economics of tax compliance, where we examine the theoretical underpinnings and empirical evidence on audit-like strategies and their effectiveness, and Machine Learning for causal inference with heterogeneous treatment effects. Section 3 offers a background description of revenue collection in Indonesia and motivates the importance of fostering tax compliance, followed by a description of the context within which the tax audit experiment was conducted in Gorontalo. Section 4 presents the empirical results of the study. Section 5 presents the conclusions.

2The exception is Hebous et al., 2023 for Norway.

4
<!-- Start of picture text --> Standard deviation<br>10 of residuals<br>10<br>120<br>8 100<br>80<br>° 60<br>—Primary 40<br>expenditure 20<br>4 —Energy index (RHS) 0<br>1992 1996 2000 2004 2008 2012 2016 2020<br><!-- End of picture text --> <!-- Start of picture text --> Standard deviation Index<br>42 of residuals 140<br>40 120<br>100<br>8 80<br>6 60<br>4 —Revenues 40<br>—Energy index (RHS) 29<br>Q Sse?—<br>1992 1996 2000 2004 2008 2012 2016 2020<br><!-- End of picture text --> <!-- Start of picture text --> Standard deviation Index<br>1g Of residuals 140<br>16 120<br>14<br>12 400<br>10 80<br>: 60<br>4 40<br>2 —Government consumption 209<br>0 —Energy index (RHS) 0<br>1992 1996 2000 2004 2008 2012 2016 2020<br><!-- End of picture text --> <!-- Start of picture text --> Standard deviation Index<br>5 Of residuals 140<br>4 120<br>100<br>3 80<br>2 60<br>—Primary balance 40<br>' —Energy index (RHS) 20<br>0 ——_____—— 0<br>1992 1996 2000 2004 2008 2012 2016 2020<br><!-- End of picture text -->
However, the program was clearly subject to the same constraints in the political economy that influenced the incidence of past fiscal contractions in Argentina. The program expanded into poor areas when the budget increased, but it retreated from poor areas when the program was cut. It was the program's disbursements to non-poor areas that were protected. Given the low wage rate offered, the direct benefits from the work are still very likely to have favored the poor, even after the cuts. So the design features of the program undoubtedly helped protect the poor from cuts.

In conclusion, the time series data for Argentina suggest that action to support propoor sending at times of aggregate fiscal contraction is warranted. Social spending in general, and targeted social spending in particular, took a heavy hit at times of fiscal austerity. The add-on program studied here was able to achieve far more pro-poor targeting than pre-existing social spending. The new program was clearly not immune to the same underlying forces in the political economy that help protect spending on the non-poor from aggregate fiscal contractions. But the program helped the families of poor unemployed workers at a time of need; given the pattern of past public spending, it appears unlikely they would have received such help otherwise.

25
25 referred to as a "frequency" index, shows the share of all tariff line products covered by one or more nontariff restrictions.19 Although these measures have some limitations (see Laird and Yeats, 1990 for a discussion), they show the extent to which nontariff restrictions are available to reinforce the effects of the block preferences. The data refer to 1988 and have been subject to significant change since then. However, again appealing to the normal persistence in patterns of protection and noting that the Treaty of Asuncion provides for the removal of NTBs on intra-bloc trade, the data very likely identify the elements of discrimination. We recognize that this is all a bit tenuous, but we believe there is content in these statistics. Given the tariff data, however, our thesis that discrimination exists does not depend on these NTB measures.

Table 7 summarizes statistics relating to these barriers. The data strongly suggest that tariff preferences, and the protection they provide for intra-trade, have been a major factor behind the recent pattern of trade changes. For example, Mercosur tariffs on the most dynamic products -- the first decile 19The frequency index (F) for importing country j shows the percentage of tariff lines covered by some preselected group of nontariff measures and is defined by, _(5)_ Fj = (ED1N + 1 _Nt)_ 100 where Ni is tariff line item i, Di is a dummy variable that takes a value of unity if one or more NTBs is applied to the item or zero otherwise, and N, is the total number of lines in the product group. The above summation is made over all countries exporting to importing country j. Similarly, the trade coverage index (C1) is defined as,

- (6) _Cj_ = ((ZDi,t m **V;,t n)/TVi,t** .). 100

where Vi,tn is the value of imports in tariff line item i in year (t-n) and Di,t is a
of economic uncertainty (Altig et al., 2020, Baker et al., 2021), utility from weather (Baylis, 2020), political polling data (Beauchamp, 2017), Arab spring protests (Acemoglu, Hassan and Tahoun, 2018), and which politicians are influential (Mankad and Michailidis, 2015).<sup>2</sup> Accordingly, social scientists frequently use social media to assemble data more quickly and at a higher frequency than is typically available in standard datasets.

Moreover, Twitter does not only reflect offline attitudes, there is also evidence that exposure to certain material on Twitter can affect offline behavior. Alatas et al. (2019) worked with celebrities in Indonesia willing to endorse vaccines; they find that a Twitter endorsement affects vaccine takeup and knowledge of study participants’ social networks. Donald Trump’s tweets about the federal reserve affected the interest rates and other macroeconomic variables (Bianchi et al., 2023). This influence is not limited to material directly attributable to politicians and celebrities; quasi-random exposure to Twitter (driven by the home counties of South by Southwest festival attendees in 2007) led to anti-Muslim hate crimes during Donald Trump’s campaign (M ̈uller and Schwarz, 2020).

This potential to influence offline behavior underscores the importance of understanding the determinants of participation in political and social discussions on Twitter. Indeed, there is evidence that women use social media differently than men. Men are followed and retweeted by both men and women in greater numbers in medicine (Zhu et al., 2019) and journalism (Usher, Holcomb and Littman, 2018). Bamman, Eisenstein and Schnoebelen (2014) find that men and women use different linguistic styles on Twitter; men were more likely to use numbers, quantifiers and technology words while women were more likely to use pronouns, emotional terms and family terms. Gender

> 2Though this finding is not universal; Zeitzoff, Kelly and Lotan (2015) study discussion on the Iran-Israel conflict on Twitter. They