rafmacalaba/datause-encoder-data huggingface dataset:
between survey incomes and national accounts incomes in 27 OECD countries, finding discrepancies in annual growth rates of 0.32 percentage points in the United States and 0.55 percentage points in Germany.

A number of studies have systematically reviewed the discrepancies between survey and national accounts data. The most complete reviews were conducted by Ravallion (2003), Karshenas (2001, 2003) and Deaton (2005), who all assess the discrepancies globally with a sample of household surveys and national accounts data from 1980s and 1990s. They document significant HHS - NAS gaps, but their estimates varied substantially from each other and they were limited by relatively small samples of matched NAS and HHS data. Since these global assessments, the availability of household survey data in poorer countries have expanded considerably and many countries have revised both their survey and national accounts data and methods. We now also have better metadata on the types of household surveys and their comparability, which can help us better understand discrepancies.

The main contributions of this paper are threefold. We compile a data set of 2,095 household survey means from 166 countries matched to corresponding national accounts aggregates. Using this much larger compilation of matched NAS and HHS data, with broader geographic coverage and with temporal coverage stretching from 1965 to 2019, we revisit the HHS - NAS gaps and update the findings of Ravallion (2003), Karshenas (2003) and Deaton (2005). Second, we overcome the heterogeneity often seen in household surveys by extending the analysis to a subsample of the data containing only comparable spells, which is particularly important for understanding the discrepancies in estimates of growth. Third, we illustrate how the discrepancies in levels can lead to divergent representations of how living standards, poverty, and inequality differ across countries and over time.

Our findings suggest that disparities in income and consumption measures between surveys and national accounts are much larger than found by both Ravallion (2003) and Deaton (2005). On average, across all countries, we find that per capita consumption means are about 22 percent lower in surveys compared with national accounts (i.e., HFCE), and per capita income means from surveys are about 52 percent lower than per capita GDP. This indicates considerably larger discrepancies than what was found in Deaton’s and Ravallion’s assessments. Ravallion 3
for 36% of Kenya’s GDP on average between 2015 and 2019 (Appendix Table A4). Excluding VAT-exempt sectors such as agriculture, non-market services and finance, the share increases to 67% of residual economic activity, implying an informal sector share of 33%. The share of informal firms and people employed in the informal sector is even higher. Of the 7.4 million businesses identified in the 2016 MSME report, only one fifth were licensed, and only 2.5% appeared in VAT data (KNBS, 2016). Similarly, VAT-registered private sector firms employed 5% of Kenya’s workforce in 2019.

With data on the formal sector only capturing a proportion of overall economic activity, we now turn to the question on whether the inability to observe informal firms in the VAT data might result in distinct trade patterns that deviate from overall firm-to-firm trade.

# **3 Representativeness of the Formal Firm Network**

In this section, we establish two key stylized facts about the extent to which trade among formal firms might be representative of overall economic activity. First, we show that trade among formal firms is highly concentrated around urban centers and places greater emphasis on inter-county rather than within-county trade. This contrasts with other measures of economic activity that include less formalised activities and which we find to be more geographically dispersed. Second, we document that the incidence of informality varies systematically across geography, sectors, and positions in the supply chain, suggesting that informality is indeed not evenly distributed across the economy. As a result, we expect overall trade patterns to diverge from the ones we document for the formal sector.

## **Fact 1: Formal sector trade is more spatially concentrated than overall economic activity and distinctly centered around urban hubs.**

### **Formal sector data reveal a high degree of spatial concentration compared to overall economic activity.**

Our first observation is that formal sector trade flows are strongly concentrated around Kenya’s largest metropolitan areas Nairobi and Mombasa. In 2019, as much as 68% of the total sales within the network of formal firms was generated by Nairobi-headquartered firms (see Table 2). However, in the same year, as little as 9% of Kenya’s population lived in Nairobi County and the city contributed only 33% of Kenya’s GDP outside the agricultural sector (also reported in Table 2).<sup>15</sup> These comparisons suggest that the role of urban centers in the Kenyan firm network is disproportionate relative to their population and their contribution to aggregate GDP.

> 15While Nairobi’s metropolitan area extends into neighboring counties, for simplicity, references to Nairobi in this paper exclude these areas.

10
1987/88, at the field level they went largely unnoticed because the pest pressure was low and despite poor control, damage remained below tolerance limits.

The resistance levels were at a more moderate level in the 1990/91 season. For example the RF was 41/3 on pigeonpea at ICRISAT and 80/6 on cotton. In fact the highest level recorded there was 830/60 on pigeonpea in January 1991.

(c) The changes in resistance are clearly influenced by the seasonal migration patterns as exhibited by a high degree of correlation between the two. Thus, two sorts of changes in resistance are observed both of which are entirely consistent with the migration patterns described in an earlier chapter. First, the post rainy season dilution of resistance is probably due to the fact of influx of predominantly susceptible moths emerging from TABLE 2 Cypermethrin RFs wrt the Reading strain recorded during November.

|YEAR|ICRISAT|GUNTUR|
|---|---|---|
|1987|125|325|
|1988|2|30|
|1989|214|2100|
|1990|41|80|

Source: Presentation by N.J. Armes, ICAR/USDA/IOPRM Joint Meeting, Hyderabad October 1991.

large areas of unsprayed sorghum in Maharashtra. Second, the rise in resistance from November to February, observed around ICRISAT mirrors the development of resistance in Guntur. This arises as a 51
R&D is the type of innovation characterized by the smaller variation between the two surveys. Although this may be in part driven by the smaller number of firms reporting R&D, the lower differences can also be an indication of a better familiarity of respondents with the concept of R&D investments and therefore smaller measurement errors due to cognitive or framing problems. Finally, innovation rates for marketing methods are higher in the long questionnaire for almost all countries in the sample for the same reason discussed above.

Despite the fact that innovation rates are lower in the long than in the short questionnaire, they are still very high, particularly when the distinction between innovation at the technological frontier and adoption (and adaptation) of existing technologies is not considered. Almost half of the countries in the sample exhibits product and process innovation rates of 40 percent or higher. Reported innovation in marketing is even higher with the innovation rate lower than 50 percent only in two countries. Lower, but still high rates are reported for R&D, ranging from 7 percent in Nepal and Pakistan to 26 percent and 28 percent in Namibia and Malawi and to an even higher 58 percent in India (innovation rates by country and type of innovation are presented in table A3).

Given the existence of considerable differences in innovation rates across countries and the high innovation rates reported in both surveys it is important to verify whether measured innovation rates are accurate. As described above, the long questionnaire provides a description of the new or significantly improved product and processes introduced for all firms. This allows us to verify innovation information and to build a “clean” innovation variable.<sup>12</sup> The cleaning and verification process was possible for around 48% of firms for product innovations and 53% of firms for process innovations (a description of how the cleaning exercise was implemented is provided in the Appendix). The remaining firms reported no innovations in their products and processes. The implication of this, as we will explore below, is that the correctness of the reply for non‐innovators cannot be verified. This is a problem in the analysis of the determinants of discrepancies in innovation rates since some of the elements that may bias the responses

> 12 The following analysis is based on the sample of 11,150 firms for which this information is available. Follow‐up questions to be used for the verification and cleaning process were not asked for marketing innovation or R&D. As such the innovation rates based on these two measures cannot be verified. An additional verification was conducted by using the year in which the innovation was introduced to make sure that all reported innovations were introduced in the correct reference period. The question asked whether new or significantly improved process or product were introduced during the past three years but in a few cases respondents reported innovation introduced before that period.

13
11 It is clear from this discussion, and from Meller's case study of Chile (presented in this issue), that adjustment is likely to have involved secondary adjustment costs beyond the income loss and poverty increase resulting from the need to service a higher external debt. It is precisely in this context that there is a role for government policy to attempt to minimize the secondary costs of adjustment by undertaking appropriate structural adjustment policies to minimize the social costs of adjustment by appropriate complementary measures.

We have not yet discussed how shifting resources from nontradables to tradables is likely to affect the distribution of income. Suppose that the poor get all their income from the supply of labor services while the rich get all their income from the return to capital. For the time being, we shall assume that both groups have identical preferences. We assume further that tradables are labor intensive and that capital is sector specific in the short run. Then, we know that the real incomes of the owners of the tradable-sector capital stock will improve, that of the owners of the capital stock in the nontradable sector will decline in absolute terms, and that owners of labor services will fall in between.'<sup>2</sup> In the long-run, we know from the Stolper-Samuelson theorem that, under the usual competitive assumptions, an adjustment that involves an increase in the relative price of tradables will shift the distribution of income in favor of labor because of the rise in the wage-rental ratio that accompanies the resource shift toward tradables.

But the extent of the income redistribution depends on the economy's structure. Figure 3 contrasts the same two economies that have to adjust to a reduction of capital inflows (which is equivalent to an increase in transfers abroad). Consider again the long-run case, where all factors are mobile across sectors. In the right-hand panel, the curves labelled NN show the locus of points where there is internal balance, that is equilibrium in the nontradable goods market. Starting from a position of equilibrium, where income Y is equal to absorption A (both shown on the horizontal axis), an increase in expenditures will require an increase in the relative price of nontradable goods - a fall
of those countries with 50 years of data extending all the way back to 1965. These provide important examples of the potential effects of successful versus failed policy reforms, with Thailand experience fluctuations in performance and an overall drop, Israel experiencing fluctuations and an overall increase, and Finland experiencing a relatively steady increase in performance.

Figures 9.0-9.6 present results for a few select countries with error bars capturing standard errors of the original tests. This enables us to capture the measure of uncertainty around our estimates. We observe relatively tight confidence intervals for Finland relative to Germany for example, revealing that Finland’s education progress is robust, while Germany’s recent gains in the last fifteen to twenty years might in fact be closer to flat progress, although there has been significant progress since the mid-1990s.

In Figures 11.0-11.4 we map out HLO coverage and learning trends for each country from 1965-2015. Notably, only 26 countries have test score data extending continuously back 20 years, and only 8 countries have test scores extending continuously back 50 years. Despite this, this dataset presents the richest panel dataset on globally comparable education quality to date.

Figure 10 demonstrates a statistically significant and positive relationship between educational achievement and economic growth. This association is consistent with results from Hanushek and Woessmann (2012).

26
52

|**Category**|**< -20%**|**-20–0%**|**0–10%**|**10–25%**|**>25%**|**Median**<sup>**a**</sup>|
|---|---|---|---|---|---|---|
|**Region (excluding high-income**|**countries)**||||||
|East Asia & Pacific|25|19|13|25|19|6|
|Europe & Central Asia|10|5|10|20|55|28|
|Latin America & Caribbean|14|21|25|29|11|6|
|Middle East & North Africa|8|17|17|33|25|17|
|South Asia|14|0|14|43|29|20|
|Sub-Saharan Africa|21|21|19|21|19|6|

_Sources:_ Table A1.8 and author’s calculations. a. The median is in percentage of the price effect for consumption.

Figure A1.1: Impact of oil intensity of the economy and exchange rate on the change in vulnerability in the developing world by region, 1999–2009 <!-- Start of picture text --> East Asia and Pacific<br>90 85 86 Europe and Central Asia<br>Latin America and the Caribbean<br>80 75<br>Middle East and North Africa<br>70 64 South Asia<br>58<br>60 56 55 Sub-Saharan Africa<br>50 44 45<br>37<br>40<br>33<br>29<br>30 25<br>19 19 18<br>20<br>11<br>10<br>0<br>0<br>Declining oil intensity of GDP Offset exceeding 25% Appreciating exchange rate<br>% of countries in each region<br><!-- End of picture text --> _Sources:_ Table A1.8 and author’s calculations.
|**Table 1.5. Medium-Term and Lo**|**ng-Term Income Mobility, R**<br>**All hou**|**LMS 1994-2015 (per**<br>**seholds**|**centage)**<br>**Households with 50**|**% or more members**|
|---|---|---|---|---|
||**Unconditional**|**Conditional**|**Unconditional**|**Conditional**|
|**Panel A: 1994-2004**|||||
|Upward mobility|27.7|38.1|24.7|36.2|
|Immobility|48.6|48.6|51.1|51.1|
|Downwardmobility|23.8|38.2|24.1|38.4|
|**Panel B: 2004-2015**|||||
|Upward mobility|26.1|37.5|25.2|36.5|
|Immobility|47.4|47.4|47.4|47.4|
|Downwardmobility|26.5|38.8|27.3|40.3|
|**Panel C: 1994-2015**|||||
|Upward mobility|34.7|46.6|29.0|38.0|
|Immobility|44.0|44.0|51.9|51.9|
|Downwardmobility|21.3|35.9|19.1|28.4|

**Note** : Estimation results are obtained based on total household income per capita. The terciles are defined using the cross-sectional sample for each year. All numbers are deflated with December to December regional CPIs and weighted with household weights, where the first survey round in each period is used as the base year. We use panel of household heads and define heads according to RLMS recommendations as: (1) the oldest working-age male in the household, (2) if there is no working-age male, then the oldest working-age female, (3) if there is no working-age female, then the youngest retirement-age male, (4) if there is no retirement-age male, then the youngest retirement-age female, (5) if there is no retirement-age female, then the oldest child. We use two types of panel: with all households and with households that have 50% or more members in other wave. Estimation sample size is 1926 and 1055 panel households from the 5th and 13th rounds of the RLMS, 1410 and 569 panel households from the 13th and 24th rounds of the RLMS and 753 and 173 panel households from the 5th and 24th round of the RLMS respectively.

53
levels (e.g., WFP 2018a).

Second, and related to the tightening of the ports, there have been substantial fuel shortages since the beginning of the conflict (e.g., OCHA 2018). The lack of fuel affects households’ ability to attain a minimum level of welfare in a number of ways. For example, households without sufficient access to fuel cannot rely on generators and must rather rely on solar power, which is less reliable<sup>6</sup> ; there is limited ability to ship food across the country; and the collection of trash is interrupted, which further contributes to the growing cholera epidemic and other related health issues (e.g., Al Jazeera 2019).

Third, beginning in November 2016, the government began paying public salaries infrequently, if at all. Nearly 30 percent of households relied at least in part on government salaries prior to the conflict, and in the capital, this figure was nearly 50 percent (e.g., World Bank 2017b). Thus, many households lost income that was helping to support them at precisely the time supply chains were disrupted and prices of basic commodities and food items began to rise (e.g., WFP 2018a). In addition, anecdotal evidence suggests that the lack of payment of employees has led to an exodus of public sector employees. This, in turn, has significantly reduced the number and quality of public services provided to the Yemeni people and likely has hampered institutional capacity (e.g., OCHA 2019a).

And lastly, there have been a series of macroeconomic shocks related to the conflict that also caused food and fuel prices to escalate rapidly (e.g., OCHA 2018). In particular, a currency crisis in 2018 rapidly increased both food and fuel prices by nearly 50 percent in three months and significantly worsened the food security situation in the country (e.g., Favari et al. 2021). Aside from rapid increases in prices, there also has been a gradual increase in both food and fuel prices that continues to erode household purchasing power at a time when a large share of households struggles to find stable employment (e.g., ACAPS 2020; Favari et al. 2021).

Combined, these price shocks and the challenge of finding stable employment have resulted in a persistent food emergency. In 2015, only three months after the conflict 6These statistics on household satisfaction with electricity are based on authors’ calculations using questions added to a monthly WFP survey. Also, from the same survey, access to electricity grids is rare in the country.

7
level evidence by using different proxies for corruption<sup>8</sup> , as well as the Freedom House index of political freedom as a proxy for overall institutional quality. They find that the relationship between corruption and growth is non-monotonic with corruption having negative effects only at high levels of incidence.

# **3 Data and Methodology**

To assess the effects of corruption on firm performance this paper uses the 2009 EBRD/World Bank Business Environment and Enterprise Performance Survey (BEEPS) of over 11,000 firms in 28 transition and developed countries.<sup>9</sup> The BEEPS was specifically conceived to assess the extent to which government policies and practices facilitate or impede business activity. It therefore provides a vast array of information on the behaviour and performance of firms, which allows to explicitly model the possible influence of various firm characteristics on the occurrence and impact of corruption at the firm level.

Table 1 lists the countries included in the sample. It shows that there is substantial variation in terms of income group (based on the World Bank classification for 2008)<sup>10</sup> and EU membership. Such high dispersion in income per capita provides a particularly rich sample, that allows controlling for specific country characteristics linked to the level of development and, in particular, to the quality of the institutional environment. The business environment is examined by asking firms to assess how various factors affect business operations, including infrastructure, financial services, government regulation, tax administration, judiciary functions. Corruption is also examined, allowing us to model its occurrence and impact on the operation and performance

> 8 The authors use the International Country Risk Guide (ICRG), the IMD index of corruption is published by the Institute for Management Development (IMD) and the corruption perceptions index (CPI) compiled by Transparency International.

> 9 Previous rounds of the BEEPS surveys were carried out in 1999, 2002 and 2005.  Unfortunately, given the changes in survey design, a meaningful link between the 2009 and earlier versions is not feasible.  Also, the survey nature of the data leads to the loss of many observations in multivariate regressions, owing to non-response rates. 10 Economies are divided according to 2008 GNI per capita. The groups are: low income, $975 or less; lower middle income, $976 - $3,855; upper middle income, $3,856 - $11,905; and high income, $11,906 or more.  See http://web.worldbank.org/WBSITE/EXTERNAL/DATASTATISTICS/0,,contentMDK:20420458~menuPK:641331 56~pagePK:64133150~piPK:64133175~theSitePK:239419,00.html (accessed 15 October 2009).

12