rafmacalaba/datause-encoder-data huggingface dataset:
Policy Research Working Paper 5653

# **Abstract**

Evidence is crucial in strategic planning for HIV response as well as the monitoring of results. This paper presents a simple tool that can be used to analyze the methodological quality of HIV data, its scope and limitations, and whether it can be accurately compared with other evidence. The tool, utilized as a questionnaire or checklist, first identifies the type of document containing the data; the status and means of its publication; the epidemiological design used; and the priority population. Next, the document is analyzed against a verification list based on eight methodological criteria: 1) objective; 2) inclusion and exclusion criteria; 3) sample size; 4) sampling technique; 5) error-reduction strategies; 6) data analysis technique; 7) limitations; and 8) confidence intervals and/or statistical significance. The paper also presents the results of implementing the tool in three Latin American countries.

This paper is a product of the Global HIV and AIDS Program, Health, Nutrition and Population; Human Development Network. It is part of a larger effort by the World Bank to provide open access to its research and make a contribution to development policy discussions around the world. Policy Research Working Papers are also posted on the Web at http:// econ.worldbank.org. The author may be contacted at rrodriguezgarcia@worldbank.org.

_The Policy Research Working Paper Series disseminates the findings of work in progress to encourage the exchange of ideas about development issues. An objective of the series is to get the findings out quickly, even if the presentations are less than fully polished. The papers carry the names of the authors and should be cited accordingly. The findings, interpretations, and conclusions expressed in this paper are entirely those of the authors. They do not necessarily represent the views of the International Bank for Reconstruction and Development/World Bank and its affiliated organizations, or those of the Executive Directors of the World Bank or the governments they represent._ Produced by the Research Support Team
# **References**

Akhmedjonov, Alisher. 2011. “Do Higher Levels of Education Raise Earnings in Post-Reform Russia?” _Eastern European Economics_ 49(4): 47-60.

- Angrist, Joshua D., and Alan B. Krueger. 1991. “Does Compulsory School Attendance Affect Schooling and Earnings?” _The Quarterly Journal of Economics_ 106(4):979–1014.

- Arabsheibani, Reza G. and Anita Staneva. 2012. "Returns to Education in Russia: Where There Is Risky Sexual Behaviour There Is Also an Instrument." (No. 6726). Institute of Labor Economics (IZA).

- Becker, Gary S. 2009. _Human Capital: A Theoretical and Empirical Analysis, with Special Reference to Education_ . University of Chicago Press.

- Belskaya, Volha, Klara Sabirianova Peter and Christian M. Posso. 2020. "Heterogeneity in the Effect of College Expansion Policy on Wages: Evidence from the Russian Labor Market." _Journal of Human Capital_ 14(1): 84-121.

- Borisov, Gleb. 2007. “The Vintage Effect on the Russian Labor Market.” _Eastern European Economics_ 45(2):23–51.

- Brainerd, Elizabeth. 1998. “Winners and Losers in Russia’s Economic Transition.” _American Economic Review_ 88(5): 1094-1116.

- Broecke, Stijn. 2015. “Experience and the Returns to Education and Skill in OECD Countries.” _OECD Journal: Economic Studies_ 2015(1):123–147.

- Calvo, Paula Andrea, Luis Felipe López-Calva, and Josefina Posadas. 2015. _A Decade of Declining Earnings Inequality in the Russian Federation_ . The World Bank.

- Card, David. 1999. “The Causal Effect of Education on Earnings.” Pp. 1801–1863 in _Handbook of labor economics_ . Vol. 3. Elsevier.

- Carnoy, Martin, Prashant Kumar Loyalka, Greg V. Androushchak, and Anna Proudnikova. 2012. “The Economic Returns to Higher Education in the BRIC Countries and Their Implications for Higher Education Expansion.” _Higher School of Economics Research Paper No. WP BRP_ 2.

- Cheidvasser, Sofia, and Hugo Benítez-Silva. 2007. “The Educated Russian’s Curse: Returns to Education in the Russian Federation during the 1990s.” _Labour_ 21(1): 1-41.

- Clark, Andrew. 2003. “Returns to Human Capital Investment in a Transition Economy: The Case of Russia, 1994‐1998.” _International Journal of Manpower_ 24(1): 11-30.

- Dearden, Lorraine. 1998. "Ability, Families, Education and Earnings in Britain." Institute for Fiscal Studies Working Paper no. W98/14.

- Fleisher, Belton M., Klara Sabirianova, and Xiaojun Wang. 2005. “Returns to Skills and the Speed of Reforms: Evidence from Central and Eastern Europe, China, and Russia.” _Journal of Comparative Economics_ 33(2): 351-70.

- Ganzeboom, Harry B.G. and Donald J. Treiman. 2019. "International Stratification and Mobility File: Conversion Tools." Amsterdam: Department of Social Research Methodology, <http://www.harryganzeboom.nl/ismf/index.htm>. [Date of last revision: 2019/10/05]

- Gimpelson, Vladimir. 2019. “The Labor Market in Russia, 2000-2017.” _IZA World of Labor_ 2019:466.

- Graeser, Paul. 1988. “Human Capital in a Centrally Planned Economy: Evidence.” _Kyklos_ 41(1):75–98.

- Harmon, Colm and Ian Walker. 2000. "Returns to the Quantity and Quality of Education: Evidence for Men in England and Wales." _Economica_ 67: 19-35.

19
51 <!-- Start of picture text --> 60 High-income countries<br>High-income, incl. decoupled payments<br>50 Developing countries<br>40<br>30<br>20<br>10<br>0<br>1955-59 1960-64 1965-69 1970-74 1975-79 1980-84 1985-89 1990-94 1995-99 2000-04 2005-10<br>-10<br>-20<br>-30<br>percent<br><!-- End of picture text --> Figure 1. NRAs to agriculture in high-income (HIC), transition (ECA),<sup>a</sup> and developing countries, 1955–2010 (%). Five-year weighted averages, with decoupled payments included in the dashed line.

a ECA is a term used by the World Bank to denote the transitional economies of Central and Eastern Europe and Central Asia.

Source: Anderson 2009, Chap. 1, updated from estimates in Anderson and Nelgen 2012b.
- [12] Roy, Prithvijit, Tarun Kabiraj, and Arijit Mukherjee, 1999. "Technology transfer, merger, and joint venture: A comparative welfare analysis," Journal of Economic Integration 14(3): 442-466.

- [13] Saggi, Kamal, 1999. "Foreign direct investment, licensing, and incentives for innovation," Review of International Economics 7: 699-714

- [14] Saggi, Kamal, 1996. "Entry into a foreign market: Foreign direct investment versus licensing," Review of International Economics 4: 99-104.

- [15] Salant S.W., S. Switzer, and RJ. Reynolds, 1983. "Losses from horizontal merger: the effects of an exogenous change in industry structure on Cournot-Nash equilibrium," Quarterly Journal of Economics 98: 185199.

- [16] Smarzynska, Beata, 2001, 'Technological leadership and the choice of entry mode by foreign investors', mimeo, The World Bank.

- [17] Teece, David J., 1977. "Technology transfer by multinational firms: The resource cost of transferring technological know-how," Economic Journal 87: 242-261.

- [18] Svejnar, Jan and Stephen Smith, 1984. "The economics of joint ventures in less developed countries," The Quarterly Journal of Economics 99(1): 149-168.

- [19] UNCTAD, World Investment Report, 1995, Geneva, United Nations.

- [20] UNCTAD, World Investment Report, 2000, Geneva, United Nations.

- [21] Yu, Chwo-Ming and Ming-Je Tang, 1992. "International joint ventures: Theoretical considerations," Managerial and Decision Economics 13: 331-342.

25
Figure 2: Emerging Exports in Morocco by Technological Content <!-- Start of picture text --> 1980 1985 1990 1995 2000 2005 2010<br>Primary Resource Based Low-Tech<br>Med-Tech High-tech<br>Source: Authors' estimates using UN-Comtrade Data<br>40<br>30<br>20<br>Exports, % of total<br>10<br>0<br><!-- End of picture text --> Figure 3 shows the composition of Morocco’s merchandise exports in 2010 – its percent figures add up to 100. As shown, using rounded data, the main four items (accounting for 71 percent) were as follows: garments (25 percent); inorganic salts and acids (21 percent); vegetables, fruits, and fish (15 percent); and electric wire (SITC 7731; 10 percent). Among electronics, “Electronic microcircuits” (SITC 7764) and “Diodes and transistors” (SITC 7763) are the two most important items (a total of 5 percent). Figure 4 shows the evolution of export values 1990‐2010 for the world (the global total) and for Morocco for a set of products that are among Morocco’s emerging exports (with the scales for world and Moroccan values on the left and right vertical axes, respectively). In general, it suggests that Morocco’s export growth for these products is not dramatically different from the growth in total world exports (or imports). It also suggests that, inside the Moroccan economy, different sectors are subject to common policy changes and constraints, as a result competing in markets for labor and other factors and responding to changes in exchange rates and other changes in incentives.

8
There are a few points to note in the regional breakdown as well. First, the few Sub Saharan African countries that have bound tariffs, have done so at levels on average much higher than in **the** other regions. Also, the simple average bound tariffs on agricultural products (where all countries have bound 100% of tariff lines) in Africa and the Europe and Middle East region, tend to be much higher than in Asia and Latin America. In terms of applied tariffs, Latin America and the Caribbean countries **liave** the lowest average tariffs both in manufactures and agriculture, but the differences among the other three regions are not large.

**Table 6**

**Tariff** Averages

# (in %)

|||**BOUND**||**A**|**PPLIE**|**D**||**MARGI**|**N**|
|---|---|---|---|---|---|---|---|---|---|
||**M**|**A**|**T**|**M**|**A**|**T**|**M**|**A**|**I**|
|42 developing countries|42|59|49|17|21|19|23|34|31)|
|High Income (4)|17|29|20|_5_|20|8|12|10|12|
|Middle Income (23)|34|48|39|16|18|17|18|30|2:2|
|Low Income_(15)_|64|86|75|24|27|_25_|38|_52_|**_5S)_**|
|Latin America and the Caribbean (13)|33|40|38|13|14|13|21|26|25|
|Asiaand Pacific_(12)_|34|49|37|19|26|21|14|18|I .|
|Sub-Sahara Africa (12)|63|84|74|20|19|20|39|59|5 4|
|Europe, Middle-East and North Africa (5)|37|70|46|21|29|23|16|41|2|

' .planations M: Manutactures A Agriculture T Total _Source: WTO. IDB, 'TO, TPR, Finger et. al. 1996, import weighted._ For most countries, the TPRs also contain a systematic estimation of escalation in the tariff schedule. Escalation is measured by calculating the average tariff rates applied to three groups of products, raw materials, intermediate products and final goods, which are consistently defined at the HS 6 digit level. Table 7 summarizes the information and reflects the judgements contained in the TPRs: thus a rating of "1" in the table is given to cotuntries where "substantial" escalation has been found, involving rising average applied tariffs for all three product groups in ascending order. Negative escalation **"2"** is 16
# **6. Minimum standards and impoverishment**

There is still a difficulty with the "catastrophic" payment approach, namely that it is blind as to how far "catastrophic" payments cause hardship. It seems likely most societies will be more concerned about someone exceeding the threshold by, say, five percentage points if their income is $0.75 a day than if it is $30 a day. An alternative perspective is that of impoverishment, the core idea being that no one ought to be pushed into poverty-or further into poverty-because of health care _expenses. This_ position is evident in the discussions in the World Bank's 2000/2001 WDR (World Bank 2000) and in its _Voices of the Poor_ consultative exercise (Narayan et al. 2000). In a sense, this approach gets to the heart of the concerns over health care payments-that health care utilization is a response to an unforeseen and unsolicited "shock" and can be sufficiently costly to represent a threat to a household's ability to purchase other goods and services that may, like health care, make a difference to its members' ability to survive flourish as a human beings.

## **_6.1. Measuring the impoverishing effects of health care costs_**

Figure 4 provides a simple framework for examining the impact of out-of-pocket payments on the two basic measures of poverty-the headcount and the poverty gap. It also allows us to relate progressivity and redistributive effect to poverty impact. The figure is a variant on Pen's parade. The two parades plot income (before and after out-ofpocket payments) along the y-axis against the cumulative percentage of individuals ranked by pre-payment income along the x-axis. Reading off each parade at the poverty line gives the fraction of people living below poverty, while the area below the poverty line above each parade gives the poverty gap. It is assumed in Figure 4 that the poverty line is the same for post-payment income as for pre-payment income-this is an issue we return to in a moment.

Formally, the relevant concepts and measures can therefore be defined as follows. Let _zPe_ be the pre-payment poverty line (which may be different from the post-payment poverty line for reasons discussed below) and xi be individual i's pre-payment income. Then define _Pi'e =1_ if _x, <_ z". Then _the pre-payment poverty headcount_ is equal to: _25_
3.9%. Using country averages across all firms, Azerbaijan has the lowest amount of crime losses at 0.20% of sales, while El Salvador has the highest with 1.73%.

We also estimate a model where the dependent variable is a dummy variable indicating whether or not a firm has experienced crime in the last fiscal year. The survey question is phrased as follows: “In fiscal year [insert last complete fiscal year], has this establishment experienced losses as a result of theft, robbery, vandalism or arson?” This estimation elucidates the relationship between economic growth and the incidence of crime as opposed to the burden of crime.

# _2.2 Explanatory Variable_

The main variable of interest is annual real GDP per capita growth rate, which is available for the years the firm surveys were undertaken. The sample mean for the rate of economic growth is 2.66% and the standard deviation is 4.2%. Belarus and Romania are the fastest growing economies in the sample with growth rates of 10.2% and 9.6% respectively. The country level average of crime losses and economic growth rates are presented in table A4 for each country in the appendix. We also use 5 year averages (2002-2007) of real GDP per capita growth and find that the results are retained.

# _2.3 Other Explanatory Variables_

We control for several firm level and country level variables. The degree of crime a firm faces may depend on its size, the sector it belongs to, and its locale (Amin, 2009; Glaeser and Sacerdote, 1999). The costs and benefits of criminal activity targeting firms may vary depending 9
to 8.3 percent, indicating that poor households were becoming poorer. Simulations also indicated that income inequality  the Gini coefficient  would rise from 39.3 to 41.3.

When compared with recently released post-crisis data, our ex-ante simulation model performed reasonably well in predicting the direction and magnitude of changes in poverty. Although, postcrisis EU-SILC data are not available, the household budget surveys (HBS) indicate that poverty rates increased from 10.12 percent in 2008 to 18.05 percent in 2009: an increase of about 8.0 percentage points.<sup>13,14</sup> and our simulation model predicted a poverty increase of about 6.0 percentage points. Similarly, between 2008 and 2009, the poverty gap increased by 3.6 percentage points compared to our predicted increase of 2.4 percentage points. Some differences may have arisen because our income data sources differ from those of the HBS. However, poverty numbers from post-crisis HBS data show the usefulness of our model in predicting exante effects of a crisis, despite many data limitations.

Much of the social impact of the crisis can be traced through crisis effects on the labor market. In 2009, GDP contracted 18 percent and the sectors most severely affected were trade hotels and  restaurants, construction, and manufacturing a loss of more than 126,000 formal jobs, comprising about 11.2 percent of the workforce. The large contraction in overall employment reflects employment contractions of 19 percent in trade, hotels, and restaurants, 16 percent in construction, and 14 percent in manufacturing. Finally, substantial differences exist in the impact of the financial crisis across regions and specific population groups in Latvia.

The Government of Latvia implemented several programs to mitigate crisis impacts on households, including increasing the threshold rate of the Guaranteed Minimum Income program, the main national poverty-targeted program; and a self-targeted public works program for unemployed people. Our simulations indicate that these measures may have helped cushion crisis impacts for some of the hardest hit households. However, the scale of the crisis exceeded the ability of programs launched under the Emergency Social Safety Net to offset negative impacts. As a result, poor and near-poor people continued to suffer considerable hardship.

> 13 We have SILC data collected in 2009 but since SILC income data refer to the previous year’s earnings, the 2009 SILC income data cannot be used as post-crisis income. 14 Poverty numbers are based on HBS income data. A household is considered to live in poverty if total household disposable income is below LVL 90 per capita per month.

17
We first test the differences in medians between users and non-users of capital markets for a set of firm attributes by pooling all firm-year observations (Table 1). The results show that firms that use capital markets are indeed very different from non-users. Firms that raise capital through either equity or bonds are significantly larger (in terms of total assets, sales, or the number of employees) than the publicly listed firms that do not raise capital. The median equity (bond) user firm in China has total assets of $443 million ($1.2 billion), whereas non-users have $214 million in total assets.<sup>23</sup> In India, the typical firm in our sample is much smaller than that in China, though the differences between users and non-users are also large. For instance, total assets for the median firm without an issue are $9 million, which stands in stark contrast with the $56 or $597 million observed for users of equity and bond markets, respectively.<sup>24</sup> We obtain qualitatively similar differences between users and non-users of capital market financing if we focus on sales or the number of employees.

In terms of growth, the firms in our data set have had a performance that mirrors that observed for the whole economy. For instance, the firms in the data set had average total assets and sales growth of 11 and 17 percent per year in China and 7 and 12 percent per year in India between 2004 and 2010. GDP growth over the same period stood at 11 in China and 8 in India. Moreover, equity and bond market users grow faster than non-users and the difference in growth rates between these firms is statistically significant. In China, total assets growth (sales growth) for equity and bond users is on average 18 (20) percent per year, but only 9 (16) percent for nonusers. In India, total assets growth (sales growth) is 16 (17) percent per year for users of equity markets, 18 (18) percent for users of bond markets, and less than 6 (10) percent among nonusers. Notwithstanding the large differences in the median firm size, the growth rates of equity

> 23 Throughout the paper, we report the data in constant 2011 US dollars.

> 24 Although the size of total assets for the median firm without an issue seems small, this figure is consistent with that calculated by Allen et al. (2012) using the Prowess CMIE database.

20
-20 -

Upper change in bronchitis 0.00238 * change in PMlO Central change in bronchitis = 0.00169 * change in PM10 Lower change in bronchitis 0.0008 * change in PM10 Asthma Attacks. Several studies have related air pollution to increases in exacerbation of asthma. For example, in a study of asthmatics in Los Angeles, Whittemore and Korn34 reported a relationship between exacerbations of asthma and daily concentrations of TSP and ozone using logistic regression analysis. Also, Ostro et al. recently reported an association between several different air pollutants, including sulfates, and inreases in asthma atacks among adults residing in Denver. Additional evidence for an effect of particulate matter on asthmatic children is provided by Pope et al.3? This study examines the effects of air pollution on a clinic-based sample and from a school-based (and relatively untreated) sample. Associations were reported between particulate matter, measured as PM10, and both respiratory symptoms and the use of medication.

T'he Ostro et al. study took place during the winter months when asthma attacks were influenced by respiraotyr infections. Thus, this study is used to help derive the upper estimate. Specifically, the central estimate of this study is averaged with the upper estimate of Whittenore and Kom study to generate the upper bound. The central and lower bound estimates are denved from Whittemore and Kom's central regression estimate and minus one standard error. For our calculations, their reported asthma attack incidence of 0.26 was halved to better represent the general population of asthmatics, since many asthmatics with low attack prevalence were dropped from the analysis. The regression model includes both TSP and ozone so the total effects on asthma are apportioned to these two pollutants in accordance with their findings.

As described in greater detail in Section IV, based on available Indonesian data, we have assumed that 8.25 percent of the population of Jakarta are astlunatic versus appwximately 5% in the U.S.'. The
To preserve the same universe of inference across all countries, residual strata are usually used such as rest of manufacturing and rest of services.

The primary sampling unit of every ES is the establishment. Sampling frames are evaluated at the onset of every project and if necessary, new frames are constructed. Special attention is placed on questionnaire translation, and in every country pretesting and pilot interviews are conducted prior to main field work to reduce measurement errors. Measurement error may be particularly concerning with some sensitive questions, in particular those regarding corruption and firm’s accounting results. Experience and anecdotal evidence witnessed during pilot surveys suggest that some facts may be intentionally underreported due to fears of repercussions and/or due to the sensitive nature of the questions. Assuming such underreporting is common and systematic across respondents there could be potential discrepancies between the average response and the actual true mean of the sample. While, unfortunately, there is no ready solution of this particular issue, over time, the ES questionnaire has been adjusted to minimize this effect. Questions are simple and direct as opposed to indirect and wordy; respondents are specially assured of the confidentiality of their answers; enumerators are specially trained to circumvent difficult situations and a special code for “refuse to answer” was introduced to deal with very sensitive questions.

Another issue when dealing with survey data in the developing world is coverage bias emerging from outdated or incomplete firm listings. For the ES, as a general principle, the most updated and complete sampling frame for each economy is used. Additionally, systematic efforts are undertaken to purge ineligible elements from the frame prior to the selection of the sample. Unfortunately, some ineligible elements are practically impossible to identify mechanically due to incomplete or missing fields, outdated firm level information etc. Consequently, field work is 8