rafmacalaba/datause-encoder-data huggingface dataset:
17 Model Validation The aim of validatlon Is to demonstrate that the model can acceptably reproduce historically observed outcomes. Since the objective of model construction was to deveiop a tool for simulating policy Interaction effects, the period for validation should be the periods over which these effects are to be studled. This requirement effectively limits the validation period to the relatively recent past given that the data base used In model estimatlon contains some time series that go back only to 1972. For this reason, and because avallable Information Is as yet Incomplete for the most recent years, the 1974-85 period was selected for model validatlon.

Given that the ultimate objective Is counterfactual simulations of pollcy Interactions, the model should be capable of simulation over a number of years with only Initial historical values for endogenous varlables as Input data to the model. Of course, the behavioral relatlonships defined by the parameterized model equations will embody the expectatlons of economic agents as conditloned by historical experience and rational expectations based on that experience. Hence, the validation test selected was sinultaneous solution of model equations over the twelve years 197485, with only Initial historical values for 1972-73 used to provide data for predetermined lagged ondogenous variables. Hlstorical values were used for pollcy and exogenous varlables In the validatlon test.

The simulated values of endogenous variables and their actual historical values are pioted together over 1974-85 In Figures 1 to 33 of Appendix B. In general. simulated values track historical values quite well. Of course, the parameterized equations do make use of dummy variables (that temporarily shift Intercepts) to explaln
# **Figure 3:** Average life satisfaction and objective welfare

<!-- Start of picture text --> Satisfaction with life and HH expenditures Satisfaction with life and HH expenditures<br>SWE<br>SWE<br>GBR<br>SVNITAFRADEU TJK GBRFRADEU<br>CZE UZB<br>POL SVKHRV SVN<br>TJK UZB MNG BLRALB MNEEST KGZ MNG KSVBLRKAZ TUR POL SVKESTCZE ITA<br>MDA KSVAZEKAZMKDBIHTURSRB LTUHUNRUS LVA AZEALB MNERUSLVA HRV<br>KGZ ARM UKR ROUBGR GEO MDAUKR MKDBGRBIHSRB LTU<br>ARM<br>GEO ROU HUN<br>3 4 5 6 7 3 4 5 6 7<br>Log(HH consumption) Log(HH consumption)<br>Satisfaction with life and the asset index Satisfaction with life and the asset index<br>SWE<br>SWE<br>GBR<br>ITA FRASVNDEU TJK FRADEUGBR<br>CZE UZB<br>POL HRV SVK SVN<br>TJK UZB MNG ALB MNE BLR EST KGZ MNGKAZ TURKSVBLRPOL EST SVKCZEITA<br>MDA AZE KAZ BIH RUSTURHUNSRBKSVLTUMKD LVA AZE ALB RUSMNE LVA HRV<br>KGZ ARMUKRROU BGR GEOMDA UKR BIH BGR SRBLTUMKD<br>ARM<br>GEO ROU HUN<br>-2 -1 0 1 2 3 -2 -1 0 1 2 3<br>Asset index Asset index<br>Satisfaction with life and GDP per capita (PPP) Satisfaction with life and GDP per capita (PPP)<br>SW E<br>SWE<br>GBR<br>SVN ITA FRA DEU TJK FRA GBRDEU<br>CZE UZB<br>HRVPOL SVK SVN<br>TJKUZBMNG ALB MNEBLR EST KGZMNG KAZBLRTUR POLEST SVKCZE ITA<br>MDA BIHAZEMKDSRBKAZ TUR LVA LTURUSHUN ALBAZE MNE LVA HRVRUS<br>KGZ ARMUKR BGRROU MDAGEOUKRBIH MKDSRB BGR LTU<br>ARM<br>GEO ROU HUN<br>0 10000 20000 30000 40000 0 10000 20000 30000 40000<br>GDP per capita (PPP USD) GDP per capita (PPP USD)<br>8 1<br>7 .8<br>6 .6<br>5 .4<br>Life satisfaction (1-10 scale)<br>4 .2<br>Satisfaction (disagree - agree): agree or strongly agree<br>8 1<br>7 .8<br>6 .6<br>5 .4<br>Life satisfaction (1-10 scale)<br>4 .2<br>Satisfaction (disagree - agree): agree or strongly agree<br>8 1<br>7 .8<br>6 .6<br>5 .4<br>Life satisfaction (1-10 scale)<br>4 .2<br>Satisfaction (disagree - agree): agree or strongly agree<br><!-- End of picture text --> _Source: LiTS II and WDI._ Consistent with the literature, average life satisfaction measures are positively correlated with the means of objective welfare measures (household expenditures, asset index or per capita GDP in PPP terms). However, average satisfaction with life is not always aligned with countries’ relative levels of economic development. In particular, Mongolia and the countries of Central Asia appear to have higher average life satisfaction levels than their incomes would suggest. Of the 10
# Table 3: Descriptive Statistics

||Mean|Std. Dev.|
|---|---|---|
|_Work status (of all population age_≥_15)_|||
|By employment|||
|Work during the survey week or have a job to return to|0.69|0.46|
|By formality|||
|Formal|0.18|0.38|
|Informal|0.50|0.50|
|By industry|||
|Agriculture|0.29|0.45|
|Manufacturing|0.11|0.31|
|Construction|0.04|0.19|
|Commerce|0.11|0.31|
|Service|0.11|0.31|
|All other|0.04|0.21|
|_Determinants of work status_|||
|Province-level minimum wage (THB/day)|140.53|13.77|
|Non-municipal area|0.67|0.47|
|Female|1.50|0.50|
|Education - primary|0.62|0.49|
|Education - some secondary|0.17|0.38|
|Education - secondary|0.10|0.30|
|Education - vocational|0.05|0.21|
|Education - university|0.06|0.23|
|No. of children (age ≤ 5)|0.95|1.04|
|No. of children (age 6-14)|0.58|0.80|
|No. of elderly (age > 64)|0.31|0.60|
|Age (15-29)|0.36|0.48|
|Age (30-49)|0.40|0.49|
|Age (50-64)|0.16|0.36|
|Age (>64)|0.08|0.28|
|N (unweighted sample size)<br>|4,770,735||

Source: Thai Labor Force Survey (1997 - 2005) Notes: N = unweighted sample size (individual respondents) for 32 quarters from 1997-2005. Mean and Std. Dev. are calculated using population weight given by the survey.
countries” (Ianchovichina, 2017). This paper explores one potential explanation for this puzzle: the role of non-humanitarian and non-neutral foreign military interventions.<sup>1</sup>

|**Table 1** **Av**|**erages of som**<br>|**e major ind**<br>|**icators (per c**<br>|**ountry per pe**<br>|**riod)**<br>||||
|---|---|---|---|---|---|---|---|---|
||_External_<br>_Intervention_<br>_Int_nh_|_Conflict_<br>_Incidence_<br>_PRIOCW_|_Religious_<br>_Polarization_<br>_RELPOL_|_Ethnic_<br>_Polarization_<br>_ETHPOL_|(1)/(3)<sup>1</sup>|(1)/(4)<sup>2</sup>|(2)/(3)|(2)/(4)|
||(1)|(2)|(3)|(4)|(5)|(6)|(7)|(8)|
|MENA|.370|.267|.470|.525|0.79|0.70|0.57|0.51|
|SAFRICA|.166|.179|.701|.537|0.24|0.31|0.26|0.33|
|ASIAE|.095|.136|.507|.458|0.19|0.21|0.27|0.30|
|LAAM|.084|.086|.404|.646|0.21|0.13|0.21|0.13|

Data sources: IMI data (Pearson and Baumann, 1993) for external military interventions of non-neutral and nonhumanitarian type, _Int_nh,_ in (1); PRIO for conflict incidence, _PRIOCW,_ in (2); L’Etat des religions dans le monde and The Statesman’s Yearbook for religious polarization, _RELPOL,_ in (3); WCE for ethnic polarization, _ETHPOL,_ in (4). Note: MENA stands for Middle East and North Africa; SAFRICA is Sub-Saharan Africa; ASIAE is East Asia; and LAAM is Latin America. Columns (5) and (6) display numbers for the incidence of external intervention per unit of religious and ethnic polarization, respectively. Columns (7) and (8) display numbers for the incidence of civil conflicts per unit of religious and ethnic polarization, respectively.

Previous studies of civil war incidence have emphasized different explanatory factors, but virtually all have related civil war to domestic factors and processes. Theoretical studies of internal conflict have focused on grievance-motivated rebellions (Gurr, 1970), the factors creating opportunities for collective action in mobilization (Tilly, 1978), and the role of rents from conflict in promoting support for violence (Collier & Hoeffler, 2004). Many studies have explored the hotly contested link between ethnic and religious diversity and social conflict. Fearon and Laitin (2003) do not find a link between ethnic heterogeneity and conflict, but others insist that ethnic cleavages may increase the risk of conflict (Ellingsen, 2000; Cederman & Girardin, 2007; Montalvo and Reynal-Querol, 2005) and the duration of civil wars (Collier, Hoeffler, and Soderbom, 2004).<sup>2</sup> Arguing that there is less violence in highly homogeneous and highly heterogeneous societies, and more conflict in societies where a large ethnic minority lives side by

> 1 Other explanations for this puzzle, referring specifically to the period after the Arab Spring, are discussed in detail in Ianchovichina (2017).

> 2 Collier et al. (1999) argue that the duration of civil wars is positively, though non-monotonically related to the level of ethnic fractionalization of the warring society. The implication is that polarized societies would generate longer civil wars because the cost of coordinating a rebellion for a long enough period could be prohibitively high in very diverse societies.

3
# ANNEX B Univariate filters

Univariate statistical filters decompose a series yt into trend, cyclical, and noise components. The trend component is used as a proxy for potential output. Although they are all essentially weighted moving averages of the series yt , they differ in their weights.

Five univariate filters are applied to estimate potential output: filters based on Hodrick and Prescott (1997), three band-pass filters (Baxter and King 1999; Butterworth 1930 and Gomez 2001; Christiano and Fitzgerald 2003), and a filter based on an Unobserved Components Model. The measures are estimated for 37 advanced economies and 52 EMDEs for 1980Q1-2022Q1 (table 10). Forecasts from the Global Economic Prospects report provide data to 2024. A smaller sample is used in comparisons with other approaches, to ensure consistency of samples (tables 11 and 12).

## Hodrick-Prescott filter

The Hodrick-Prescott (HP) filter minimizes deviations of a series yt from its trend τ t , assuming a degree of smoothness λ of the trend. The HP filter chooses the trend τ t that minimizes: where T is the sample size. A larger λ indicates a smoother trend. For λ =0, the trend is equal to the actual series and for λ >+ ∞ the trend is a linear time trend with a constant growth rate. Typically, the value of λ is set at 1600 for quarterly data. The trend is estimated based on past values as well as projected values of the series yt.

## Band-pass filters

The three band-pass filters aim to isolate fluctuations in a time series which lie in a specific band of frequencies. They eliminate slow-moving components (trend) and very high frequency components (noise) and define the intermediate components as the business cycle. Specifically, the three band-pass filters differ in their approximations of the optimal linear filter (also known as the “ideal” band-pass filter) to deal with finite time series.

The Baxter and King (BK) filter is a moving average of the data with symmetric weights on lags and leads. Therefore, it loses observations in the beginning and towards the end of the sample. It is particularly well-suited when the raw series follows a near-independent and identically distributed process (Christiano and Fitzgerald 2003). Specifically, the BK filter is given by: ˆ _ct_ = _b_ ( _L_ ) _yt_ , where b(L) is the lag polynomial given by: 36
needs to be made by the losers and the amount of assistance that they would require to ease the transition.

|**TABLE 3 Net Gainers from**|**Formula 3 (Per Stude**|**nt)**|
|---|---|---|
||**Average Net Gain**|**Per Capita State**|
|**Category**|**(1993 Pesos)**|**GDP (1993 Pesos)**|
|High Net Gainers|227|967|
|Medium Net Gainers/Losers|-59|1,135|
|High Net Losers|-375|2,390|

|**TABLE 4 Loss**|**es of Losers**<br>||||
|---|---|---|---|---|
||**All losers**||**Big Losers**|**Actual**<br>**spending in**<br>**1997**|
||**Loss (1998**<br>**billion**<br>**pesos)**|**States**|**Loss**<br>**(1998**<br>**billion**<br>**pesos)**<br>**States**|**(1998 billion**<br>**pesos)**|
|**Lower**<br>**secondary**|||||
|**Formula I**|**2.23**|**19**|**1.43**<br>**6**|**24**|
|**Formula II**|**2.25**|**17**|**1.86**<br>**6**|**24**|
|**Primary**|||||
|**Formula III**|**3.61**|**21**|**2.53**<br>**8**|**44**|

9
cover the full cost of service. Nevertheless, even when cost is low such as in Egypt and Vietnam, the high public sensitivity toward electricity pricing prevents the government from removing subsidies completely. In Vietnam, while end user tariffs follow cost, tariff increases above 7 or 10 percent require the approval of the Ministry of Industry and Trade or the Prime Minister, respectively. In Egypt, bold tariff adjustments of up to 40 percent were enacted in 2016 and 2017, but the low tariff base means that more substantial increases are needed to cover costs. Political pressure to provide subsidized or free power to certain groups of consumers (e.g. agriculture sector) also undermine the cost recovery effort.

**Figure 4: Tariffs, Collection and Costs of 17 Case Studies (average for 2010-17, in 2017 US$/kWh)**

<!-- Start of picture text --> 0.40<br>0.35<br>0.30<br>0.25<br>0.20<br>0.15<br>0.10<br>0.05<br>0.00<br>Operating Cost Capital Cost Cash Collected Billed tariff revenue<br><!-- End of picture text --> _Notes: *_ Split of operating and capital cost for Morocco is not available. _Source:_ World Bank staff.

# **#2: Few countries adequately manage volatile costs and maintain cost recovery levels over time**

27. None of the cases maintained full cost recovery throughout the entire period of observation. Six of the cases reached full-cost recovery in at least one of the observed years, but even when tariffs attain cost recovery levels for certain periods of time, the sustainability of this outcome is challenged due to cost volatility faced by the sector. This is true for countries at both ends of the cost recovery spectrum (e.g., for both Colombia and Tanzania), as well as countries that are completely relying on fossil-fuels (e.g., Dominican Republic) and countries relying entirely on renewable energy (e.g., Tajikistan). The full data are presented in Table 8 in Annex 2. Year-on-year volatility of cost recovery is particularly high in Rajasthan (coefficient of variation of 16 percent), Pakistan (17 percent), Senegal (16 percent), Tajikistan (19 percent) and Tanzania (23 percent). The only two exceptions with low volatility in cost recovery levels are Andhra Pradesh and Odisha.

28. The sources of volatility most often observed are exchange rate, fuel cost, fuel mix and debt service costs (see Table 3). Colombia, for example was above full capital cost recovery in 2011-2016, but below that level in 2010 because of the impact of drought on hydropower costs. Kenya’s KPLC experienced worse financial performance in 2009-2012 because of a drought that reduced hydro capacity. Tanzania’s 14
Table 3: Census summary statistics

||Year|Areas|Subareas|Households|Land area<br>(’000s km<sup>2</sup>)|Outcome<br>Assets|s calculated<br>Poverty|
|---|---|---|---|---|---|---|---|
|Burkina Faso|2019|346|8,763|748,961|274|X||
|Madagascar|2017|1,515|14,412|5,007,602|597|X||
|Malawi|2018|420|18,700|796,925|118|X|X|
|Mozambique|2017|1,282|67,239|6,133,769|802|X||
|Sri Lanka|2012|331|13,984|4,842,300|66|X||
|Tanzania|2012|169|13,857|2,612,518|947|X|X|
|Vietnam|2019|711|11,159|2,322,464|332|X||

Note: The table shows the number of areas and subareas, the number of households, the total land area in thousands of square kilometers, and the outcomes calculated for different countries. Assets are available in all countries, while we impute poverty into the census using concurrent household surveys in Malawi and Tanzania.

# **2.2 Geospatial features**

We pull geospatial features from Google Earth Engine using Python and the earth engine api library. Table A1 in the appendix lists the geospatial features used. Importantly, we often derive several additional statistics from different indicators. For example, data on temperature is used to construct average temperature, maximum temperature, and minimum temperature during the year, while data on pollution is used to generate many distinct indicators.<sup>8</sup> In addition, we also create features by aggregating to higher levels by taking means. For example, we calculate mean urbanity at the admin3 level for all countries, and create predictors with other variables in similar ways. By combining these features in different ways and across different levels of aggregation, we end up with several hundred different predictive features. While we include all of these features in the machine learning methods, we use lasso to select features for the EBP model, following others (Engstrom et al., 2022; Masaki et al., 2022; Newhouse et al., 2025).<sup>9</sup> We pull the most recent data available for all countries. Different geospatial features have different time coverage, meaning we sometimes pull data from after the census. For example, our preferred land classification data only goes back to 2019, so we extract 2019 data for the 2012 Tanzania census. We recognize that this is not ideal and can result in additional stress-testing for the proposed methods (e.g., by introducing noise into the data).

> 8More concretely, we construct weather variables separately by month throughout the year of the census, maximum and minimum values throughout the same year, as well as climate-related variables that are long-run means and standard deviations from the 10 years prior to the census. We construct similar variables (except the long-run averages) for pollution.

> 9All three machine learning methods have their own regularization methods which make lasso redundant.

10
profound effect on the control and thus the performance of the enterprise. Again, the presence of foreign services providers appears to have the most robust effect on the performance of Czech firms in downstream manufacturing.

As the next step, we estimate our model in first differences as specified in equation 4. In this more demanding specification, the differencing removes all influences that are fixed at the level of the firm, industry or region, as did the firm fixed effects before. In addition, regional dummy variables in the differenced equation now purge all differences in trends that are specific to the region of operation. The estimates, based on the OLS productivity and presented in table 4, confirm our earlier conclusions. We find a positive and statistically significant correlation between the presence of foreign providers of services and the productivity of Czech firms relying on services inputs. As before, statistically significant effects are also found for the index of policy changes and the extent of privatization. We reach similar conclusions when we employ the TFP measures based on the Olley-Pakes approach (table 5).

15
underlying securities, both of which were grossly underestimated by the rating agencies (Coval et al., 2009).

With a high proportion of structured finance products that had initially been rated AAA having been downgraded to junk status, investors lost confidence in this market. By late 2008 the structured finance market had virtually closed down, with almost no new issues, and specialists did ot expect it to reopen for years. Evidently, then, no tax could have a further corrective effect in discouraging issues.

Nor was there ever much revenue potential in these securities. Quarterly issuance of them peaked in 2006-7 at around USD 100 billion per quarter. As primarily buy-andhold securities, the transactions tax revenue from the primary issue would be a high fraction<sup>16</sup> of the total lifetime tax revenue from that issue – a mere USD 10 million for the peak quarter (assuming a tax rate of 0.01 percent).

# _CDS_

The relatively sudden emergence of the credit default swap market starting in the late 1990s has been identified as a significant contributor to the growing distortions of the credit market during the following decade (Tett, 2009). By 2008, the gross amount of debt insured through CDS was thought to exceed ÙSD 60 trillion,<sup>17</sup> though many of the contracts were back-to-back and resulted in negligible net risk. The net amount of CDSinsured debt may not have exceeded USD 15 trillion. These amounts have subsequently declined. Even on this net amount, the flow of premia was only a fraction of the sums insured (especially considering that most of the debt insured was highly rated. Indeed, the first CDS contracts entailed annual premia of just 0.02 percent of the nominal amount insured. Riskier debt of course carries a much higher premium. Even on the sovereign debt of some European Union countries, CDS premia have approached 400 basis points (4 percent) at times during the recent crisis.

> 16 Transactions data on CDOs is not collected by the BIS.

> 17 The BIS half-yearly estimate of the nominal value of outstanding Credit derivatives (most of them CDS) peaked at USD 58 trillion at end-December 2007.  At that date, the gross market value of the contracts was USD 2 trillion, a figure which jumped to USD 5 trillion by the end of 2008 because of the movements in premia and hence in the replacement values of each of the outstanding positions.

15
**Roadmap** The paper is organized as follows. Section 2 describes the data, before Section 3 presents the method for identifying the unexpected component of demand shocks at the firmlevel. Section 4 describes the econometric model, while 5 reports the corresponding empirical results. Section 6 concludes the paper.

# **2 Data**

The empirical analysis in this paper combines and examines several sources of panel data from Portugal spanning the period 2006-2018. We provide a brief description of each data source in this section and give further details in Appendix A.3.

**Employer-employee data** : _Quadros de Pessoal (QP)_ [Personnel Records] is a high-quality compulsory census run by the Ministry of Employment covering the population of firms with wage earners in manufacturing and services. Each firm is required by law to provide information on an annual basis about its characteristics and those of each individual that comprises its workforce. Firm-level information includes annual sales, number of employees, industry code, geographical location, date of constitution and share of capital that is foreign-owned. The set of worker characteristics includes wages (monthly base wage, overtime pay, and other components of pay), gender, age, schooling, date of starting, detailed occupation and hours worked. A worker may also be matched to the firm in which he is employed. Extensive checks have been performed to guarantee the accuracy of worker and firm data. After these checks, we kept for analysis full-time wage earners working at least 100 hours a month, and aged between 20 and 60 years old.

**Firm census** : Using common unique firm identifiers, we supplement the firm-year data from QP with information from _Sistema de Contas Integradas das Empresas (SCIE)_ [Enterprise Integrated Accounts System], a yearly census of firms run by National Statistics Institute (INE). The main objective of SCIE is to characterize the economic and financial behaviour of firms. This data set includes information on total sales, investment, employment, value added, wage bill, industry, location, among several other variables.

**International trade statistics** : We merge the above data sets with yearly data on firms’ export transactions from _Estat ́ısticas do Com ́ercio Internacional (ECI)_ [Foreign Trade Statistics] from INE. This is the country’s official information source on imports and exports. It comprises

> common shocks associated with a single global recession (e.g. financial factors), which may plausibly have heterogeneous impacts across firms. The focus of our analysis is also distinct in that we separately identify the effects of these unexpected shocks on the different components of pay, show that wage effects apply mainly to top earners, and to firms initially led by highly skilled top executives. This enable us to make further progress in identifying specific mechanisms whereby firm-specific unexpected demand shocks impact wage setting.

5