rafmacalaba/data-use-mentions-extended huggingface dataset:
In this study, we bridge the gap by proposing a comprehensive country- and product-level Discrepancy Index database that captures both the frequency of misreporting and its impact on overall recorded trade value. The database was created by leveraging the UN Comtrade database, capturing all available Harmonized System (HS) data from 1988 to 2022. Central to this approach is use of a bilateral Discrepancy Index (DI), which calculates the level of divergence between reported data and its corresponding mirror (data reported by trading partner).

This study makes several contributions towards measuring discrepancies and assessing reporter reliability in the UNSD Comtrade database. First, we provide a comprehensive coverage of discrepancy measurements across all periods of observation, reporting countries, products, and product detail for the HS nomenclature. Hence, resources provided here can assist researchers aiming to trace discrepancies to specific causes, whether logistic or otherwise. Second, we show that the functional form of DI presented here avoids some of the pitfalls of commonly used trade asymmetry measures and is therefore more suited for penalizing discrepancies that are least likely to be explained by some logistic factor. Third, we demonstrate how the international merchandise trade reporting discrepancy database can aid discrepancy analysis and resolve data reliability issues in the UN Comtrade database. By measuring discrepancy in reported values across time, reporting nomenclatures, trading partners and products, we identify an encouraging trend of an increasing quality of reported trade over time. We identify persistent serious quality problems linked to some reporters and note potential effects of reporting discrepancies on important trade indicators such as trade balances. Thus, this study also contributes to ongoing discourse regarding quality of data in international merchandise trade by providing measures and tools for its assessment and outlining important considerations in practical applications of trade data. More importantly, our results can equip trade analysts with the necessary tools to make informed decisions regarding the use of a country’s reported data or its mirror.

The rest of this paper is organized into five sections. Section 2 discusses the numerous factors contributing to discrepancies in international merchandise trade reporting
crimination of any kind on grounds of race, sex, religion, language, political opinion, membership in a union, national origin, social origin, disability, or any other situation.<sup>39</sup> Other results of the surveys related to access to institutions show that hidden forms of institutionalized discrimination can cause certain groups to have difficulty exercising their political rights and, as a consequence, equal access to public services. For example, in Bulgaria, regulations on the issuance of identification cards require a registered address. Those regulations adversely affect mostly Roma minorities, who often have problems obtaining housing and do not have a registered address. Also, the Bulgarian Act on Social Benefits for Children, introduced in 2015, suspends child benefits for children of underage girls, affecting mostly Roma women, who tend to have children at a younger age than do women in the rest of the population. Those laws, coupled with the widely reported economic underperformance of Roma minorities in Bulgaria (Gatti and others 2016), give rise to interesting research questions on the correlation between seemingly neutral laws and the disproportionate effect that such laws can have on the continual economic marginalization of minorities in the country.

Similar findings come from Morocco, where the Nationality Law of 1958, as amended in 2007, provides that Moroccan nationality is acquired by birth except when the minister of justice opposes it. The Amazigh people from the south, however, usually lack birth certificates. As a result, the law setting out the requirements for foreigners to obtain Moroccan nationality—including knowledge of the Arabic language, which those ethnic minorities often do not speak—also applies to them. That apparently neutral provision in the nationality law makes obtaining nationality harder for the Amazigh people than for the Moroccan Arabic majority and, as a consequence, hinders access to social services and the labor market (UNCERD 2010). In the area of cultural rights, however, Morocco has made some progress in the new 2011 Constitution by recognizing Tamazight, the language of the Amazigh
<!-- Start of picture text --> ARMED CONFLICT BY REGION, 1946-2021<br>» AFRICA = AMERICAS * ASIA * EUROPE * MIDDLE EAST<br>60<br>55<br>50<br>45<br>40<br>B 35<br>g<br>8 30<br>¢<br>Fa 25<br>3<br>20<br>Is<br>10<br>5<br>0<br>FELELER ERR ERLE TAAL ARARETE LER EERERERRAREREREE EERE RERER GH RARARRARRARRERRERE RE<br>Based on UCDP 22.1 data<br>So<br>s<br>8<br>s<br>a<br>82 Es<br>8<br>as 3<br>8g<br>:<br>So<br>° Saat Sg<br>2000 2005 2010 2015 2020<br>year S<br>—— Battles ——— Explosion\Remote Violence Violence 2000 2005 2010 2015 2020<br>———- Violence against citizens against citizens citizens _——— Protests Year<br>——— Riots ——— FCSILICUS Countries |. ———— Non FCS/LICUS Countries<br><!-- End of picture text -->

<!-- Start of picture text --> So<br>s<br>8<br>Es<br>3<br>g<br>So<br>gSg<br>S<br>2000 2005 2010 2015 2020<br>Year<br>——— FCSILICUS Countries |. ———— Non FCS/LICUS Countries<br><!-- End of picture text -->

<!-- Start of picture text --> s<br>a<br>82<br>8<br>as<br>:
Weixing Wu. 2020. “Social Ties and the Selection of China’s Political Elite.” _American Economic Review_ 110 (6): 1752–1781.

- Fukuyama, Francis. 2014. _Political Order and Political Decay: From the Industrial Revolution to the Globalization of Democracy_ . New York: Farrar, Straus and Giroux.

- Gehlbach, Scott, and Philip Keefer. 2011. “Investment without Democracy: Ruling-party Institutionalization and Credible Commitment in Autocracies.” _Journal of Comparative Economics_ 39 (2): 123-139.

- Ghanem, Dalia, Shu Shen, and Junjie Zhang. 2020. “A Censored Maximum Likelihood Approach to Quantifying Manipulation in China’s Air Pollution Data.” _Journal of the Association of Environmental and Resource Economists_ 7 (5): 965–1003.

- Gibson, John, Susan Olivia, Geua Boe-Gibson, and Chao Li. 2021. “Which Night Lights Data Should We Use in Economics, and Where?” _Journal of Development Economics_ 149: 102602.

- Gulzar, Saad, and Benjamin J. Pasquale. 2017. "Politicians, Bureaucrats, and Development: Evidence from

32
American sample). In particular, the Global Findex database includes two sets of questions to each surveyed individual which are essential for our purposes of measuring � and �:

- Question FIN24 asks: “Now, imagine that you have an emergency and you need to pay [1/20 of GNI per capita in local currency]. Is it possible or not possible that you could come up with [1/20 of GNI per capita in local currency] within the next month?”The possible answers were “yes”or “no”.<sup>24</sup> We use this question, like in the second group of studies which rely on self-assessed measures of capacity to deal with ...nancial shocks, to calculate the share of HtM individuals, �.

- Questions FIN37 and FIN38 ask if “[i]n the past 12 months, have you, personally, received any ...nancial support from the government? This money could include payments for educational or medical expenses, unemployment bene...ts, subsidy payments, or any kind of social bene...ts. Please do not include wages or any payments related to work”and “[i]n the past 12 months, have you, personally, received a pension from the government, military, or public sector?”, respectively. The possible answers were “yes”or “no”. We use these questions to identify the share of social transfers reaching HtM individuals, �.

# 4.2 Share of HtM individuals is more than twice as large in emerging markets

Figure 2 shows the share of HtM individuals, �, in each country. Red bars depict emerging markets while blue bars indicate developed countries. For ease of reading, countries with labels correspond to those used in our sample of six Latin American and 17 developed ones. The visual impression is striking: a majority of blue bars lie to the left of the ...gure and the majority of
prioritize nutritional issues is crucial for initiating efforts to address the problem, especially when resources are limited.

# 3. Literature Review

Prior studies have utilized SAE techniques to map food insecurity and nutritional trends at the subnational level. Within this context, we identify two noteworthy trends in employing SAE to better understand nutritional outcomes in developing countries. One prevalent approach combines census data with household expenditure information, utilizing caloric intake thresholds as indicators of malnutrition. For instance, Hossain et al. (2020) applied the SAE methodology to estimate food insecurity at the district level in Bangladesh. They integrated data from the Household Income and Expenditure Survey 2010 with the Bangladesh Population and Housing Census 2011. Their approach gauged food insecurity prevalence based on a per capita calorie intake threshold of 2,122 kcal per day, derived from the survey.

Guha and Chandra (2021) also combined household expenditure data with census data to obtain estimate of food insecurity indicators at the district level for the rural area of the state of Uttar Pradesh in India. Their empirical evidence indicate that the estimates generated by SAE approach are reliable and representative. Spatial maps showing district level inequality in distribution of food insecurity in Uttar Pradesh is also produced. The disaggregate level estimates and spatial maps of food insecurity are found to be directly relevant to help the monitor the sustainable development goal indicator 2.1.2 - severity of food insecurity.

Another notable approach combines census data with standardized health surveys like the Demographic and Health Surveys (DHS) to estimate stunting rates at the subregional level. For instance, Sohnesen et al. (2017) employed this method in Ethiopia to estimate undernutrition within different woreda (sub-regions). Their findings revealed substantial variations in the prevalence of undernourished children within each woreda, highlighting location-specific challenges. Notably, they found limited correlation between the percentages of underweight and stunted children across woredas.

Other studies employed SAE to estimate dynamic changes in nutritional outcomes. In fact, to estimate the trends in age-specific chronic childhood undernutrition in all of Bangladesh's 64 districts
Future expectations can also inform economic self-interest. Benabou and Ok (2001) show that expectations of socioeconomic mobility can affect preferences for redistribution. Individuals with below-average incomes who would benefit from redistributive policies financed by higher taxes may oppose such policies if they expect to be upwardly mobile. This prediction, known as the Prospects of Upward Mobility hypothesis, has been borne out by the data in experimental settings (Checchi and Filippin 2004; Agranov and Palfrey 2020) as well as in observational data (Alesina and La Ferrara 2005; Alesina and Fuchs-Schuendeln 2005; Cojocaru 2014; Rueda and Stegmueller 2019).

The literature on preferences for redistribution points to the importance of self-interest, cultural factors, and social mobility, both past and future (Corneo and Gruner 2002; Alesina and Giuliano 2009; Alesina, Stancheva, and Teso 2018; Cavaille 2023; see also Mengel and Weidenholzer 2022 for a detailed survey of this literature). Less is known about the role these factors play in shaping preferences for policy choices unrelated to redistribution. This paper aims to contribute to closing this knowledge gap. Several earlier studies consider the drivers of support for market reforms at the beginning of the post-Soviet transition (Duch 1993; Denisova et al. 2021; Rovelli and Zaiceva 2013) or in Latin America (Kaufman and Zuckermann 1998). Denisova et al. (2012) find that economic hardships during the post-Soviet transition—such as the experience of downward mobility, the lack of human capital suitable for a market economy, and the lack of private assets— significantly increased the desire for revising the results of earlier privatization.

Several earlier studies have examined the individual determinants of preferences for investments in education for a small subset of European countries (Busemeyer and Garritzmann 2017; Garritzmann, Busemeyer, and Neimanns 2018). Wiese and Eriksen (2024) investigate linkages between expectations of mobility and willingness to pay for improvements in public education and health care using data from the 2016 round of the LiTS, framing it as a test of the POUM hypothesis with respect to indirect
and responsiveness of teachers toward the child, use of teaching strategies that encourage child participation, capacity for group and one-on-one interaction between teachers and children, supervision and disciplinary techniques) qualities of early learning centers.

Items were translated, back translated, reviewed, and adapted by Malawian early child development experts. Some items were also drawn from previous evaluations of CBCCs (Chiuye & Chimombo, 2011; Munthali et al., 2014). Our decision to use these measures, as opposed to other standardized Western tools such as the Early Childhood Environment Rating Scale (ECERS; (Clifford et al., 2010; Harms & Clifford, 1982)) and the Classroom Assessment Scoring System (CLASS; (Pianta et al., 2007)) was based on several factors. The CBCCs are unlike preschools and kindergartens for which the ECERS and CLASS were developed in that they are initiated, run and staffed by community volunteers with minimal education, training and oversight. Thus, the centers are not universal in form or function: they may operate erratically based on the availability of structures, teachers and materials; they tend to serve a wide range of ages, and the emphasis of preparing children for primary school may be secondary to other goals, such as providing a safe place for children to play, and get a meal. Our measures, informed by those developed in low- and middle-income countries, with added information from local experts, strove to measure quality in a contextually relevant manner. While these measures may not be externally valid (e.g., for comparing results across preschool interventions), they are useful for providing tools that can be used within Malawi for evaluating quality of centers. Our items might also be relevant to efforts at creating a universal measure of early learning environment quality.

We used the data collected under the observational tool to develop a _Classroom Observation Index_ with the purpose of measuring classroom quality. At each round, the index computes a weighted average of a set of underlying observational variables (31, 34
In the United Kingdom, a test commissioned by the Department for Work and Pensions found that people with minority ethnic or foreign-sounding names are one-third less likely to be shortlisted for jobs than people with “white”-sounding names (Wood and others 2009). Also in the United Kingdom, among those age 16 years old and older, almost all ethnic minority groups have a higher proportion of their male population unemployed than do their white counterparts (Sims 2012, 11). In the United States, the General Society Survey, a national probability survey representative of the U.S. population, reports that, between 2003 and 2008, 38 percent of the lesbian, gay, and bisexual people who were open about their sexual orientation in the workplace and 78 percent of transgender people experienced at least one form of discrimination, be it in hiring, promotion, or job retention (Sears and Mallory 2011). In Belgium in 2009, 45 percent of all employment discrimination complaints received by the Centre for Equal Opportunities and Opposition to Racism were race related; of those, 36.5 percent concerned access to employment and 56.1 percent concerned conditions of work (ILO 2011a, 28).

The access to the labor market indicator identifies restrictions to accessing the job market, such as limitations to certain forms of discriminatory employment or hiring practices, and the existence of workplace protections for people belonging to ethnic, religious, and sexual minorities. The surveys revealed that all six pilot economies have constitutional provisions or national laws that prohibit discrimination in the workplace on ethnic and religious grounds.<sup>60</sup> In Bulgaria, Mexico, and the Netherlands, that prohibition also applies to sexual orientation,<sup>61</sup> and in Bulgaria, it also applies to gender identity. All of the pilot economies prescribe equal remuneration for work of equal value for individuals belonging to religious and ethnic minorities. The same applies to LGBTI communities in Bulgaria, Mexico, and the Netherlands.<sup>62
# Supplementary information

Supplementary Table 1. Directory of identifiers (present in all datasets)

|**Variable name**|**Label**|**Data processing notes**|**Additional notes**|
|---|---|---|---|
|_country_|Country name|||
|_wave_|Wave number|||
|_household_id_obs_|Household ID<br>(panel<br>identificator)|Variable created by assigning<br>a numeric code to each<br>unique household. The first<br>number of the code<br>determines the country.||
|_household_id _merge_|Household ID<br>(to merge with<br>raw data)|ID to merge with raw LSMS<br>data|Can be merged with the following<br>variables in the raw data<br>(concatenated variables are separated<br>by hyphens):<br>ETH:_household_id_(wave 1)_,_<br>_household_id2_(wave 2)_,_<br>_household_id2_(wave 3)_,_<br>_household_id_(wave 4)_, household_id_<br>(wave 5)<br>MWI:_case_id_(wave 1),_y2_hhid_<br>(wave 2)_, y3_hhid_(wave 3)_, y4_hhid_<br>(wave 4)<br>MLI: concatenation of_grappe_and<br>_menage_(wave 1), concatenation of<br>_grappe_and_exploitation_(wave 2)<br>NER:_hid_(wave 1)_,_concatenation of<br>_GRAPPE, MENAGE_and<br>_EXTENSION_(wave 2)<br>NGA:_hhid_(wave 1)_, hhid_(wave 2)_,_<br>_hhid_(wave 3)_, hhid_(wave 4)<br>TZA:_hhid_(wave 1)_, y2_hhid_(wave<
near and far from terrestrial network of internet cables. They find that the probability that an individual (in one of the 12 selected countries) is employed increases between 6.9 and 13.2 percent when fast internet becomes accessible. Importantly, such probability is significant for skilled occupations, while unskilled jobs are unaffected by access to fast internet.

In parallel, another branch of research examines the effects of ICT adoption in rural areas, focusing mainly on those that have just received mobile coverage, and finds positive effects on the labor market. For instance, Klonner and Nolen (2008) study the effect of mobile network coverage in the rural labor market outcomes in South Africa. To identify the causal effect of network coverage on market outcomes the authors use terrain characteristics as instrumental variables. The authors find that employment increases by 15 percentage points when a region receives mobile coverage, and that this is even reflected in poverty reduction. Atasoy (2013) studies the effects of broadband internet adoption from 1999 to 2007 across counties in the United States. The author finds that broadband adoption is correlated with a 1.8 percentage points increase in the employment rate, but the impact is higher in rural counties. Similarly, Lu and Song (2020) rely on survey data from college students in China to study the effects of ICT on college students’ labor market outcomes and find that students who have received education on ICT-related subjects are more likely to obtain a job and a higher salary than their peers who did not receive such training.

Another stream of emerging studies focuses on the labor enhancing effects of the growth of new businesses based on online-platforms and AI. The common agreement is that digital platforms are transforming markets, making them more efficient, increasing product demand and ultimately, creating new jobs. For instance, in the taxicab market, the “Uber technology” has been linked to increased number of rides and the creation of over 30,000 jobs just from one startup (Vergara-Cobos, 2020). Our survey of the literature
|Repair|Other only|Sanitation &<br>other|Total|
|---|---|---|---|---|---|---|
|SL|146<br>(73%)|7<br>(4%)|2<br>(1%)|31<br>(16%)|14<br>(7%)|200<br>(100%)|
|Control|14<br>(70%)|0<br>(0%)|0<br>(0%)|5<br>(25%)|1<br>(5%)|20<br>(100%)|
||160|7|2|36|15|220|
|Total|(73%)|(3%)|(1%)|(16%)|(7%)|(100%)|

_Notes_ : Data source: Client survey and administrative data. Sanitation loan usage was reported for those clients who took a sanitation loan according to administrative data from the MFI and confirmed it during the interview. Hence, information is missing for clients who did not confirm to have taken a sanitation loan in the survey.

The household survey collected information on toilet ownership, whether it was not in use and the reasons for this, and a number of dimensions of toilet quality, which allow us to directly test whether households made a variety of sanitation investments. We focus on three types of sanitation investments: (i) construction of new toilets; (ii) repair of existing toilets; and (iii) upgrade of toilets. We measure the construction of new toilets through (i) toilet ownership, regardless of whether it was functioning,

> 40Even though the evidence presented above suggests that the loan label might have played a role in affecting the demand
Finally, section 6 offers concluding remarks, suggests directions for future research, and acknowledges the limitations of our approach.

# 2. Sources of discrepancies in international trade reporting

A growing body of literature has studied the relative contributions of several factors to the observed discrepancies in international merchandise trade reporting (Ferrantino & Zhi Wang 2008; Carrère & Grigoriou, 2015; Demir & Javorcik, 2020; Kee & Nicita, 2022). These studies have identified both structural and non-logistic reasons for the lack of dual recording and observed inconsistencies in successfully mirrored records. In this section, we outline the causes of discrepancies and explore their effect on the quality of the obtained data.

Discrepancies in international trade reporting are structural, i.e., they are driven by logistic reasons related to the valuation methods and reporting guidelines for international merchandise trade. These guidelines are detailed in the International Merchandise Trade Statistics: Concepts and Definitions 2010 (IMTS2010), published by the United Nations Statistical Division (UNSD). The UN Comtrade databases, the primary source of official merchandise trade data collected and disseminated by UNSD https://comtradeplus.un.org/, strive to align with the IMTS2010 guidelines in the empirical data that they provide. As a result, data obtained through UN Comtrade (or any source that adheres to these recommendations) inherently possess several structural discrepancies. An example of this structural discrepancy occurs in valuation methods that typically consider freight and insurance costs for imports but not for exports. Therefore, structural discrepancies arise because the reported import value exceeds that of the corresponding mirror. Many have noted that extreme (and even negative) empirical measurements of difference between reported imports and mirror records suggest discrepancies that cannot entirely be explained by differences in valuation methods (Carrère & Grigoriou, 2015).

Moreover, IMTS2010 recommends listing the country of origin for imports, but the last known destination for exports (United Nations, International Merchandise Trade Statistics). This will structurally create transactions without a corresponding mirror in cases where exports pass through intermediate countries before