metadata: authors: 0: author: Pape,Utz Johann
1: author: Verme,Paolo
count: World
docna: 0: docna: Measuring Poverty in Forced Displacement Contexts
docty: Policy Research Working Paper
theme: Inclusive Growth,Data Development and Capacity Building,Social Development and Protection,Fragility, Conflict and Violence,Public Sector Management,Data production, accessibility and use,Forced Displacement,Mitigation,Gender,Human Development and Gender,Data Development and Capacity Building,Economic Policy,Rural Development,Social Development and Protection,Economic Growth and Planning,Environment and Natural Resource Management,Disease Control,Pandemic Response,Fragility, Conflict and Violence,Public Sector Management,Climate change,Urban and Rural Development,Adaptation,Geospatial Services,Data production, accessibility and use
abstracts: cdata!: Poverty measurement among forcibly
            displaced populations, including refugees and internally
            displaced persons, has been, for long, neglected by the
            economics profession and by poverty specialists working
            across the social sciences. This has changed since the
            beginning of the Syrian conflict in 2011 and the peak of the
            European migration crisis in 2015. This paper reviews the
            evolution, current status, and future prospects of the
            poverty measurement literature on forcibly displaced
            populations; discusses the main data and measurement
            challenges associated with this type of population;
            illustrates selected empirical findings that have emerged
            from the recent literature; and provides an overview of the
            substantial effort that humanitarian and development
            organizations are currently undertaking to close this
            historical gap in poverty measurement.
display_title: Measuring Poverty in Forced
            Displacement Contexts
listing_relative_url: /research/2023/02/33998032/measuring-poverty-forced-displacement-contexts
topicv3: Forced Displacement m1326032 740,Poverty m1326687 695,Poverty Measurement and Analysis m1326941 707,Refugees and Jobs m1504221 2894,Internally Displaced Persons m1939428 4819
publishtoextweb_dt: 2023-02-08T00:00:00Z
dois: 10.1596/1813-9450-10302
origu: Off of Sr VP Dev Econ/Chief Econ (DECVP)
no_of_pages: 29
author_keys: 0: author_key: 1154467
1: author_key: 439872
model_extractions: Policy Research Working Paper 10302 Measuring Poverty in Forced Displacement Contexts _Utz Pape Paolo Verme_ Poverty and Equity Global Practice & Fragility, Conflict and Violence Global Theme February 2023
Policy Research Working Paper 10302

# **Abstract**

Poverty measurement among forcibly displaced populations, including refugees and internally displaced persons, has been, for long, neglected by the economics profession and by poverty specialists working across the social sciences. This has changed since the beginning of the Syrian conflict in 2011 and the peak of the European migration crisis in 2015. This paper reviews the evolution, current status, and future prospects of the poverty measurement literature on forcibly displaced populations; discusses the main data and measurement challenges associated with this type of population; illustrates selected empirical findings that have emerged from the recent literature; and provides an overview of the substantial effort that humanitarian and development organizations are currently undertaking to close this historical gap in poverty measurement.

This paper is a product of the Poverty and Equity Global Practice and the Fragility, Conflict and Violence Global Theme. It is part of a larger effort by the World Bank to provide open access to its research and make a contribution to development policy discussions around the world. Policy Research Working Papers are also posted on the Web at http://www.worldbank. org/prwp. The authors may be contacted at upape@worldbank.org and pverme@worldbank.org.

_The Policy Research Working Paper Series disseminates the findings of work in progress to encourage the exchange of ideas about development issues. An objective of the series is to get the findings out quickly, even if the presentations are less than fully polished. The papers carry the names of the authors and should be cited accordingly. The findings, interpretations, and conclusions expressed in this paper are entirely those of the authors. They do not necessarily represent the views of the International Bank for Reconstruction and Development/World Bank and its affiliated organizations, or those of the Executive Directors of the World Bank or the governments they represent._ Produced by the Research Support Team
# Measuring Poverty in Forced Displacement Contexts<sup>1</sup>

Utz Pape and Paolo Verme<sup>2</sup> <u>Keywords: poverty, refugees, IDPs.</u> JEL: I30 I32.

1 This is an expanded version of the previously published short note ‘Poverty Measurement for Forcibly Displaced Populations: Challenges and Prospects of a New Field’.

2 Authors listed alphabetically. UP: The World Bank and Georg-August-University Göttingen, upape@worldbank.org. PV: World Bank, pverme@worldbank.org. The authors would like to thank Theresa Beltramo, Felix Schmieding, Ciara Silke and Christina Wieser for their helpful comments as well as Benu Bidani for overall guidance. The findings, interpretations and conclusions expressed in this paper are entirely those of the author, and do not necessarily represent the views of the World Bank, its Executive Directors, or the governments of the countries they represent. This work is part of the program 'Building the Evidence on Protracted Forced Displacement: A Multi-Stakeholder Partnership'. The program is funded by UK aid and was established in partnership with the United Nations High Commissioner for Refugees (UNHCR). The scope of the program is to expand the global knowledge on forced displacement by funding quality research and disseminating results for the use of practitioners and policy makers. This work does not necessarily reflect the views of the UK, WBG or UNHCR.
# Introduction

The United Nations High Commissioner for Refugees (UNHCR) estimated that the global number of forcibly displaced persons (FDPs) in the world surpassed 84 million in 2021, up from around 40 million in 2010 and accounting for over 1 percent of the global population.<sup>3</sup> This sharp growth in displaced people during the past decade can be largely attributed to the Syrian conflict that started in 2011, the displacement of the Rohingya people since 2017, and the intensification of several conflicts in Sub-Saharan Africa, particularly along the Sahel region. These numbers are unprecedented in the history of displacement when recording started with the establishment of the UNHCR in 1950 and the signature of the Geneva Refugee Convention in 1951.

FDPs are not a homogenous group. They include internally displaced persons (IDPs – defined by the UNHCR as citizens of a country that have been displaced within the boundaries of their own country due to conflict or security reasons), asylum seekers (displaced people outside their own countries who formally ask for asylum), refugees (people who have obtained asylum in the host country), and other displaced groups that defy simple categorizations. These categories of people fall under the mandate of the UNHCR because they have been displaced “forcibly” due to conflict or violence and because they are in need of international protection. They exclude other categories of displaced people who were not forced to move because of conflict or violence, such as economic migrants and victims of natural or environmental disasters.<sup>4</sup> Of course, many people cannot be simply categorized in these groups and this makes statistics on FDPs gross estimates, but the growth and relevance of these numbers are undisputed.

The growth in the number of FDPs poses a challenge to the measurement of global and national poverty. Those who are forcibly displaced and in need of international protection tend to be persons who have lost their assets, financial resources, and social networks. They are typically very poor with no obvious path out of poverty. For refugees, their number vanishes from poverty statistics of their own country because they are no longer counted in the place of origin. Both IDPs and refugees are also not properly accounted for in the country in which they reside. Their numbers – even though high in absolute terms – are often low relative to the non-displaced population (with some exceptions like Lebanon and South Sudan). Hence, they do not explicitly show up in official statistics. Even if – as in some but not all countries – their locations are appropriately included in the sampling frame, they are unlikely to be sampled due to their small proportion relative to the population and high clustering in specific locations.

> 3 https://www.unhcr.org/refugee-statistics/ 4 Note that the IASC definition of IDPs explicitly includes those fleeing from disasters.

2
The number of FDPs is not small in terms of the absolute global poverty count. Before the COVID-19 pandemic, around 10 percent of the global population (780 million people) was estimated to be extremely poor, below 1.9 USD/day in purchasing power parity (PPP) terms. Using a conservative estimate based on empirical studies of about 1 in 2 FDPs being extremely poor, this translates into 41.2 million poor or about 5 percent of the global poor. The UN population division uses UNHCR’s FDP estimates to adjust national population figures by adding FDPs to population statistics. However, when the national and global poverty figures are estimated, FDPs are often missing from the poverty count.

Low interest, lack of proper microdata on FDPs, measurement issues and political reasons are some of the obstacles that prevent a proper measurement of poverty among FDPs. Before the Syrian conflict, poverty measurement among FDPs was largely confined to occasional exercises – often focusing on vulnerability rather than poverty – carried out by local or international NGOs on behalf of humanitarian agencies such as the UNHCR and the World Food Programme (WFP). These exercises often lacked the data sophistication and academic rigor that characterizes poverty measurement for regular populations in high-, middle-, or low-income countries, typically undertaken by development agencies such as the World Bank, UNDP or regional development banks. The UNHCR, for example, did not collect survey data on income, monetary consumption, or expenditure for refugees systematically as the focus and mandate of the agency is on humanitarian protection rather than poverty alleviation. It only started to collect data on income more systematically when budget restrictions forced the agency to start targeting cash programs. WFP did collect data on consumption regularly, but this effort was largely focused on food consumption for food security and nutritional assessments rather than poverty measurement. With little data and visibility, the economics profession and poverty specialists across the social sciences have also largely neglected FDPs (Verme, 2016; Dionigi and Tabasso, 2020). It is only in the past decade that development agencies have taken an interest in FDPs and, together with humanitarian agencies, started to consider how to gather data and measure poverty among FDPs continuously and rigorously.

This has changed over the past decade. For example, the UNHCR has significantly invested in improving the measurement of income and employment using the ILO standard approach and has adopted the poverty - headcount to measure progress towards the Global Compact for Refugees (https://www.unhcr.org/global <u>compact-refugees-indicator-report/). This organization also works on embedding refugees and IDPs into</u> national statistics exercises via the International Expert Group on Refugee and IDP statistics (EGRIS) process and has established, together with the World Bank, a Joint Data Centre focused on improving the data infrastructure for refugees. The World Bank has also taken important steps towards FDPs with the adoption of a special window for IDA projects, joint data and research initiatives with the UNHCR, and the establishment of FD coordinators across its global practices.

3
Measuring poverty among displaced populations has its own specificities. FDPs are not a representative sample of the population of origin or destination country. They tend to be poorer than most host populations, tend to have higher shares of females and children, have very few assets and only occasional incomes, they may not be allowed to work, have more limited access to public services, have a higher incidence of people with physical and psychological disorders and tend to rely almost entirely on aid provided by the international community (Verme et al. 2016). Concepts such as income, expenditure, and consumption - the three monetary metrics that are typically used to measure poverty – have a very different content and meaning in these contexts. FDPs may also live in camps where shelter and services are provided entirely by the international community. In some very poor countries, FDPs living in camps may be better off than locals, but measuring poverty among FDPs living in camps is hard from a theoretical and empirical perspective. For example, it is not obvious whether hand-outs such as food stamps should be considered as income or consumption, or how to quantify housing, education and health services provided in camps free of charge. Simply adopting standard measurement practices for FDP populations will not be sufficient to properly measure the well-being of FDPs.

The paper is organized as follows. The next section explains why we consider poverty measurement among FDPs as a new field of study. The following section digs into the multitude of measurement issues that arise from this new field of study. We then discuss the prospects for future research before concluding with some considerations.

# A New Field of Study

Economics has traditionally paid very little attention to forced displacement (FD), generally considering this issue a sub-set of migration studies. Partly because of lack of microdata, partly because of the small size of the population studied, and partly because refugees were relegated to the sphere of humanitarian issues, economists rarely invested in researching this topic. This changed in the aftermath of the Syrian displacement crisis.

Consider the literature on the measurement of the impact of forced displacement on host communities. This question has been the main preoccupation of European and North American countries for decades, and the first studies on forced displacement were largely focused on this question. A seminal study in this respect is David Card’s 1990 study on the impact of the 1981 Mariel boatlift of Cubans from Cuba to Miami (Card, 1990), a study that focused on the impact of the newly arrived Cubans on the wages and employment of the local Miami population. Despite this interest, between 1990 and 2010, only an average of one article per year appeared in economics journals on this topic and none of these articles measured the impact on poverty 4
of refugees or host communities. A few studies on poverty among FDPs appeared before 2010, including a few studies on the well-being of Palestinian refugees (Hejoj 2007), and occasional studies on refugees in upper income countries (Bollinger and Hagstrom, 2004, Kriechbaum 2006), but these studies were extremely rare and relegated to specialized journals, or specialized agencies’ reports. This changed after 2010 with tens of studies now published on refugee poverty and household well-being of host populations in top economics journals (Verme and Schuettler, 2021).

The surge in general interest on the topic, the media exposure of the Syrian and European crises and the growing preoccupation of donor countries with the political and economic implications of refugees led development organizations and poverty specialists to start considering FDPs more closely and collaborate with humanitarian organizations. The UNHCR and the World Bank, for example, started to collaborate very closely around 2014 with the first joint study on the welfare of Syrian refugees in Jordan and Lebanon (Verme et al, 2016). This study used UNHCR’s administrative and survey data to measure poverty among refugees, contributing to improving the targeting system of the UNHRC’s cash assistance program. It found that - using the international extreme poverty line of 1.9 USD/day in purchasing power parity (PPP) – 7 in 10 refugees were poor, a figure that rose to 9 in 10 refugees if the national poverty lines of the host countries were used.

This study led the World Bank to reconsider its role in working with refugee populations and encouraged this organization to strengthen cooperation with the UNHCR, leading to the establishment of a joint research program (“Building the Evidence on Forced Displacement”), country-level cooperation on data collection in several countries, a first round of analytical studies on refugees’ well-being (Pape et al., 2018, Pape et al., 2019a, Cuevas et al., 2019, Beltramo et al., 2021), the establishment of a Joint Data Centre between the two organizations, and the implementation of joint rapid poverty assessments for FDPs in the course of the COVID-19 pandemic (see World Bank, 2021 for example). This collaboration was instrumental in equipping the World Bank with improved knowledge on refugee populations and the UNHCR with improved knowledge on data collection and poverty measurement among FDPs.

Better and more data and knowledge on FDPs, in turn, attracted significant investment in FD research on the part of donors, generating a real boom in research on the topic which quickly became mainstream, even in disciplines that traditionally disregarded FD such as economics. By 2021, most of the top economics journals had published articles on FD, including the Quarterly Journal of Economics, the Journal of Political Economy, and the Review of Economics and Statistics, with special issues being published by development journals such as the Journal of Development Economics. This literature, in turn, is providing hard evidence on highly debated topics such as the impact of FD on host communities. A review of this specific literature 5
has shown, for example, that most studies find a positive or non-significant effect of FD on hosts’ employment, wages and household well-being, a finding that disputes much of the popular perceptions on this question (Verme and Schuettler, 2021).

One of the important factors that has limited research on FDPs in the past was the chronic shortage of quality microdata, something that is quickly changing. Today, microdata on these populations can be found in two main publicly available repositories: The World Bank microdata library and the UNHCR microdata library recently established in collaboration with the World Bank. An analysis of these data repositories as of October 2022 shows that the WB microdata library has 576 data sets on refugees (454 dated after 2011) and 206 on Internally Displaced (142 dated after 2011), whereas the UNHCR microdata library has 274 data sets related to refugees (263 dated after 2011) and 34 data sets on IDPs (all dated after 2010).<sup>5</sup> This new research area is also generating significant innovations with the potential to expand research methods in the poverty measurement field. In the area of targeting based on means-tests, for example, a study has shown that Receiving Operations Characteristics (ROC) curves can be an effective decision making tool for humanitarian assistance programs (Verme and Gigliarano, 2019) while another study found that poverty differences in prediction methods for targeting purposes among refugees are attributable to few data fields suggesting that refugee homogeneity can make poverty predictions and targeting easier as compared to regular populations (Altindag et Al., 2021, Beltramo et al., 2019). The existence of the UNHCR refugee registration system, which can be regarded as a live census of refugees, has encouraged others to use cross-survey imputation techniques to estimate poverty among refugees even in the absence of income or consumption data (Dang and Verme, 2021, Beltramo et al., 2021). The mobile nature of refugees and IDPs also lends itself to experimenting with new methodologies to measure poverty with alternative methods such as mobile phones (Blumenstock et al., 2015; Pape et al. 2020, Wieser et al., 2021), or satellite imagery and remote sensing data (Abelson et al., 2014; Neal et al., 2016). For example, night lights or types of infrastructures captured by satellite imagery can provide gross estimates of poverty. In essence, poverty measurement among FD populations benefits from decades of developments in the poverty measurement field, but also provides new opportunities to expand the field because of the atypical characteristics of these populations.

> 5 Note that many microdata sets are present in both the WB and the UNHCR microdata libraries.

6
# Measurement Issues

## Sampling strategies

There are some technical reasons that make surveying refugees and IDPs complex. Sampling is one of these reasons. Refugees and especially IDPs are mobile populations which can move into host areas, even if they are initially residing in camps, and might settle outside camps in previously uninhabited areas. These factors make the inclusion of these populations in national master samples for surveys difficult. For refugees, this problem is partly overcome by the UNHCR registration system (proGres) which requires all registered refugees to provide a set of basic information including location and socio-economic characteristics of the household’s members. However, using the proGres registration system to sample refugees has its own challenges. This system is available in some countries but not in others where a system may be missing or managed by the local authorities and not available to others. Where the system exists, many refugees are not registered, some are registered but their information is outdated or missing, while identifying the unit of observation (household, family, case) may be challenging although in other contexts such as Bangladesh, the registration system may be up to date and almost universal. In other words, extracting a representative sample of refugees in many countries, even when the proGres registration system is available, is not simple.

Sampling IDPs can be even more complex because they no longer reside in the place where they are registered in the census. If they reside in previously uninhabited places, they will be excluded from the sampling frame. International organizations that collect information on IDPs, such as the International Organization for Migration (IOM), tend to collect information on IDP communities, but sampling frames are often not updated accordingly. Security reasons also hamper proper sampling and data collection as many destination areas for IDPs are themselves unsecure locations. In essence, the sampling infrastructure that is typically available for national populations, and even the basic infrastructure that is available for refugees, is lacking for IDPs. Definitions of IDPs can also be problematic as shown by a recent JIIPS study.<sup>6</sup> Former IDPs deciding to remain in their location of displacement but no longer being displaced can easily be mis-classified as IDPs, while IDPs returning to their place of origin without overcoming their specific displacement vulnerabilities can be mis-classified as non-IDPs.

In general, two classes of sampling approaches are used for FPDs: area-based sampling and list-based sampling. The area-based sampling partitions the area of interest into smaller enumeration areas. In the first stage of the sampling, enumeration areas are randomly selected. All households in selected enumeration

> 6https://www.jips.org/jips-publication/identifying-internally-displaced-persons-in-surveys-approaches-andrecommended-questionnaire-modules-jips-2021/

7
areas are listed so that a given number of households can be randomly chosen from the list to be interviewed. The advantage of this approach is that it only requires knowledge of any FDP residing in an enumeration area, so that this person can be included in the sampling frame. The more accurate the number of FDPs in an enumeration area before listing households, the more efficient is the resulting sample. This approach requires appropriate enumeration area maps that can be easily defined through satellite images for camps, but might not be available across the country for FDPs living in host communities. Implementation can be expensive especially with limited knowledge about the number of FDPs in enumeration areas. However, area-based sampling has the big advantage of not requiring FDPs to register. In contrast, list-based sampling uses an existing list of all FDPs and randomly chooses a sample among them. If additional characteristics are available, like location or country of origin, the sample can be stratified. However, the sample will only be representative of registered FDPs. Hence, it is rarely used in the context of IDPs unless a proper registration system is in place.

To partly address sampling issues, the UNHCR and the World Bank have been cooperating to use the UNHCR or national refugee registration systems as initial master samples to conduct consumption surveys, similarly to what is done with censuses of regular populations. Initial experiments in this respect have been conducted in Jordan, Lebanon, Iraq, Uganda, Kenya, Ethiopia, and South Sudan, resulting in consumption surveys and poverty analyses of refugees and IDPs. Both the UNHCR and the World Bank are now also working with statistical agencies in multiple countries across the Middle East and North Africa and SubSaharan Africa regions to include FDPs in national sampling frames with initial efforts conducted in countries such as Chad, Jordan, Kenya, Niger and Uganda (e.g., World Bank 2021b).

However, the political economy and data privacy can make these efforts challenging. For host countries, FDPs may be only one of the many marginalized groups they may be concerned with. With limited budgets, statistical agencies need to justify prioritizing one group over another. Furthermore, data privacy is particularly relevant for FDPs. National sampling frames are constructed based on census information, including personal information such as addresses, phone numbers, names as well as GPS locations in some cases. The UNHCR and national governments generally work together to collect registration data but there is some information that may not be shared between the two sides. For example, national statistical agencies protect the national sampling frame and cannot share the underlying cartography that would be required for UNCHR to amend the sampling frame with counts of FDPs. Even if both agencies would be able to create a trusting relationship allowing close collaboration, questions remain on whether FDPs would need to agree with sharing their information with a government agency. This becomes even more relevant in the context 8
where a government might be a real or perceived contributor to displacement or is harboring an overt or covert policy to reduce the number of refugees in the country.

With limited results in including FDPs in national sampling frames, alternative approaches remain necessary. The UNHCR, the World Bank and numerous scholars and practitioners worldwide are experimenting with satellite images and phone surveys to try detecting refugee and IDP populations that may escape the UNHCR and national registers with some initial encouraging results. In Lebanon, Jordan and the Kurdistan region of Iraq, Aguilera et al. (2020) designed sampling strategies for Syrian refugees with known ex-ante selection probabilities. They used a variety of data sources, including data collected by humanitarian agencies, and also employed geospatial segmenting to create enumeration areas where they did not exist. Systematic field experiments are also underway to test different sampling approaches for IDPs living in camps. For example, Himelein, Pape and Wild (forthcoming) compare the performance of five alternative sampling approaches (satellite mapping, segmentation, grid squares, “Qibla method,” and random walk). Different indicators are assessed including household size, consumption, poverty and ownership of assets. Using empirical evidence from a field experiment in an IDP camp in South Sudan, the total survey error of each sampling approach is compared to a census, allowing for the disaggregation of the total error into sampling and non-sampling components. One of the main findings is that the implementation of all approaches suffers from over-estimation of household sizes caused by a systematic tendency of enumerators to select larger households because they are more likely to find an adult respondent. Such studies with a focus on ground truthing remain important to validate sampling approaches for displaced communities.

Sampling FDPs outside camps faces a different challenge, especially when a list-based approach is not possible. FDPs can be spread in low density across a larger population requiring an area-based adaptive sampling approach. The approach continues to list additional enumeration areas systematically selected, e.g., by selecting neighboring enumeration areas, until a sufficient number of FDPs are covered. Adaptive sampling creates analytical complexities for the calculation of sampling weights. It also adds to the cost as more enumeration areas must be listed – often with only very few or possibly no FDPs present – and it is more difficult to implement given the uncertainty about the number of required enumeration areas that need to be visited to satisfy a given sample size.

# Unit of observation

The definition of the unit of observation in a survey (household, family, case, individual) is a crucial choice that has direct impacts on the measurement of poverty because it determines how household income, 9
consumption or expenditure is measured in per-capita terms and has implications for other indicators of well-being such as housing, rents, or crowding. The Demographic and Health Surveys (DHS), Multiple Indicator Cluster Surveys (MICS) and Living Standards Measurement Surveys (LSMS) are the most common household surveys used to produce comparative statistics on well-being across time and countries. The DHS and MICSs define household members as (i) usual residents or people who slept in the dwelling the previous night and who (ii) share living arrangements and (iii) share food (ICF International, 2012; UNICEF, 2013). The LSMSs define household members as (i) people who slept in the dwelling three or more months of the last 12 months and (ii) share food (Grosh and Munoz, 1996).<sup>7</sup> While the definitions differ, they are defined on similar concepts and are unlikely to lead to major differences in key household characteristics.

The UNHCR has definitions for household that resemble the definitions used by MICSs and LSMSs but uses the concept of “case” as unit of observation. The UNHCR defines a case as: “A processing unit similar to a family headed by a Principal Applicant. It comprises (biological and non-biological) sons and daughters up to the age 18 (or 21) years, but also includes first degree family members emotionally and/or economically dependent and for whom a living on their own and whose ability to function independently in society/in the community and/or to pursue an occupation is not granted, and/or who require assistance from a caregiver.” (Verme et al., 2016; see also https://www.unhcr.org/5ea81c114). This definition is different from the DHS, MICS and LSMS, may pose challenges when one compares cases with households in surveys, and lends itself to exploitation on the part of users, for example by spreading different household members across different cases to maximize benefits.

Some socio-economic surveys for FDPs do not rely on the UNHCR family definition as the unit of socioeconomic analysis – in addition to the individual. Instead, they employ a household definition either from DHS, MICS or LSMS, or from the established national household survey implemented by the national statistical agency, which often is similar to the traditional definitions. When using a list-based sampling approach with UNHCR’s registration data, this creates the challenge of translating the family (case) definition used as a sampling unit to a traditional household definition for the interviews and the analysis. In most cases, not all members of a proGres family are members of the household, while the household usually also has members from other proGres families. Different approaches can be used to overcome this challenge.<sup>8</sup>

> 7 While the DHS and MICS define households as mutually exclusive, the LSMS definition suffers from individuals potentially belonging to multiple households leading to double-counting.

> 8 If not explicitly mentioned, we assume that the universe of FDPs is defined as FPDs registered in proGres.

10
First, the sampling can be based on individuals rather than families. By definition, each household with at least one FDP will have a positive probability to be part of the sample creating a representative sampling frame. Since larger households have a higher chance of being sampled, sampling weights need to correct for that. The sample will be representative but not very efficient as estimates for smaller households have lower accuracy than larger households.

Second, the sampling can be done based on proGres families. In this case, the interview must determine the household for each proGres family member. If family members in one proGres family are from multiple households, the household to be interviewed must be selected randomly among them. The household interview must include all household members, also if they are not part of the proGres family. Since the sample is drawn randomly among all proGres families, all households with FDPs have a positive likelihood to be selected. However, a household with members from several proGres families has a higher chance to be selected. Hence, sampling weights must be adjusted accordingly by the number of members from different proGres families.

A correct implementation of the household definition is crucial even though difficult. Enumerators and respondents can reduce interview time by reporting fewer household members. However, household-level information like assets or consumption will usually still be reported across the household. Hence, households will appear richer as consumption and rooms are shared by fewer reported household members. This is potentially an important issue that would be important to test empirically. Selection of larger households creates a bias by moving the average estimate towards larger households, e.g. increasing the average poverty rate if larger households tend to be poorer. However, a misreporting of the number of household members biases the characteristics of households, often making them appear richer than they are as resources are shared by fewer reported members.

Finally, mixed households consist of FDPs and non-FDPs. All household members independent of their displacement status must be equally considered in the interview, to allow accurate estimates for householdlevel indicators that depend on household size. Mixed households are usually more prevalent outside camps and often exhibit distinctly different characteristics, e.g., in terms of deprivation and labor market access. Comparing camp and non-camp households must consider the presence of mixed households outside camps. Results indicating that non-camp households are less deprived and have better access to labor markets might simply be driven by the presence of non-FDPs in non-camp households.

11
# Survey design and administration

Designing a socio-economic survey including consumption measures for non-displaced households is challenging as the interview time is limited and one has to be very selective to restrict the number of questions in surveys. For FDPs, the challenge is exacerbated given the need to understand not only their present socio-economic well-being, but also their displacement trajectory as well as their aid dependency. In this context, spending 90 or more minutes on an elaborate consumption module might not be a priority. Instead, detailed information on displacement and aid is crucial and something that is not properly assessed in non FDPs surveys.

Fatigue from over-surveying is often cited as an anecdotal challenge in the context of FDPs, leading to survey non-response. Even though FDPs are often subject to intensive surveying by multiple agencies, survey non-response is not as high as for regular populations and the likelihood for a household to be interviewed multiple times is limited. Only censuses interview everyone in the population, they are often prohibitively expensive, and suffer from low data quality outstripping their size advantage. Thus, only few censuses – with very short questionnaires – are necessary for verification exercises. Hence, the chance of multiple interviews for the same households in a short period of time is low. However, refugees do complain about excessive surveying, possibly because contacts with interviewers are not just limited to sample surveys but also needed for a wide variety of purposes related to administering services to refugees.

FDPs are not only particularly vulnerable and need specific protection, they also often originate from a different region in the case of IDPs or country in the case of refugees. This creates several challenges in interviewing FDPs. The respondent might not speak the official language of the country or may not trust enumerators that come from different communities. This can be overcome by assigning purposefully enumerators to specific households to ensure that they speak the same language and share the same culture. However, enumerators should also have a similar cultural background as respondents. Di Maio and Fiala (2020) use a large-scale experiment in Uganda in which enumerators and respondents are randomly paired to explore for which types of questions a significant enumerator effect may exist. While the enumerator effect is minimal for many questions, it is large for specific perception questions, for which it can account for over 30 percent of the variation in responses. Such a bias can also occur for sensitive questions to FDPs including questions on their displacement trajectory, current needs as well as perception questions.

Specifically, in the context of a statistical agency conducting a survey including FDPs, enumerators need to be carefully selected. Refugees might see the statistical agency simply as government that might pass information to other government agencies, potentially threatening their livelihoods. IDPs in the context of an internal conflict involving the government might similarly be reluctant – or even feel personally 12
threatened – if interviewed by a government official. These risks can be mitigated if enumerators are hired among FDPs from the same background. However, this can create legal challenges if FDPs do not have work permits. In some cases, it might also be necessary to have UNHCR or IOM officials accompanying enumerators or even conducting the interviews themselves.

The implementation of FDP-specific socio-economic surveys can be done in parallel with UNHCR’s verification exercises. The verification exercise is a census that visits all registered refugees to update their data. In Kenya, an additional socio-economic questionnaire was administered to a random subset of refugees participating in the verification exercise (Pape et al., 2019b). The parallel implementation reduced costs substantially but the implementation requires close coordination between the verification exercise and the socio-economic survey. Ideally, no redundant data would be collected but in reality most questions from the verification exercise are asked slightly differently than in the socio-economic survey, which usually aims to be consistent with the latest national survey, necessitating different instruments for the same concept to ensure comparability.

# Poverty measures

Poverty measures the level of deprivation in a population. The most widely used concept defines as poverty headcount the number of individuals living below a threshold, usually called the poverty line (Foster, Greer and Thorbecke, 1984). Since the poverty headcount only reflects the proportion of poor people, it is usually complemented by a measure of the depth of deprivation, the poverty gap, which estimates the average gap between poor individuals and the poverty line. All poverty measures constructed in such a way are based on an underlying welfare metric. In most cases, the metric is defined at the household level and needs to be transformed into an individual measure. The transformation either divides household welfare by household size, providing a per capita measure or, in a more sophisticated way, takes into account differences in household composition (usually age and gender), leading to a per-adult equivalent measure.

To measure household welfare, different metrics have been proposed and can generally be classified either into monetary or non-monetary metrics. Monetary metrics equate levels of well-being to a monetary indicator of utility (Samuelson, 1974), usually income, consumption or expenditure. Non-monetary metrics define normatively dimensions of deprivations, e.g., access to education and clean water, and aggregate them either by using dimension-specific thresholds or an aggregated threshold. The most commonly used approach in this class is the Multidimensional Poverty Index by UNDP and Oxford University (MPI; OPHI, 2018). While it is not uncommon to find experts with a strong preference for one measure over the other, both classes of poverty measures are in most cases complementing each other. While the monetary metric has the advantage of its theory-grounded definition without normative choices, it is complex to measure 13
and does not capture acute deprivations in health or education, as the OPHI MPI does. For example, a third of those experiencing multi-dimensional poverty are not captured by the monetary headcount ratio (World Bank, 2020). Hence, it is not surprising that efforts are underway to combine the advantage of both measures like the World Bank’s Multi-Dimensional Poverty Measure (MPM; Nguyen et al., 2021).

In the case of FDPs, monetary and multi-dimensional measures of poverty are both indicated and, as it is often the case in other settings, they complement each other. In this respect, there is not much difference with other types of populations. However, the money metrics used for monetary indicators may be more challenging to estimate for FDPs than regular populations, and the composition of multi-dimensional indexes should be adapted to FDPs’ characteristics.

Moreover, especially in a context of very high deprivation, a poverty headcount might not be able to reveal the actual gravity of the situation and qualitative information should be sought to complement standard questionnaires and possibly inform the development of future questionnaires. Collection of socio-economic data is a passive process where respondents are asked pre-formulated questions. This constrains the respondents in sharing their own narratives and emphasizing what they feel is important. The implementation of household surveys can be seen as an opportunity to collect qualitative information and transform a one-sided narrative into one that provides voice to the poor. Pape (2020), for example, describes how short, voluntary video testimonials with informed consent from people living in South Sudan and Somalia<sup>9</sup> can be used to empower the poor in voicing their own needs. While this is a practice that can be developed in any context, it is particularly promising in contexts where measuring poverty is still relatively new and the specific challenges associated with this measurement are still little known, as in FDP contexts.

# Poverty metrics

Income, consumption or expenditure can be used as monetary metrics to estimate poverty. The correct measurement and classification of income can be a real problem in low-income countries with large informal economies (Deaton and Zaidi, 2002; Mancini and Vecchi, 2022) and, especially, for FDPs. FDPs rarely have regular labor income and tend to rely on occasional, informal income, or have no labor income at all. They have various forms of in-kind and cash assistance that vary from household items such as blankets and kitchen items to food vouchers and cash assistance. The combination of these income sources is not always well captured in surveys because many of these items are provided occasionally and by multitudes of donors. Food vouchers are often traded and can function more as cash than food. FDPs may also produce products that they exchange as they try to use their crafts to supplement incomes, and some

> 9 <u>www.thepulseofsouthsudan.com</u> and www.thesomalipulse.com.

14
of the items produced may be consumed. Many FDPs store wealth in jewelry or gold to carry these values with them, and it is unknown how much of this wealth is used for regular or occasional expenditures. These facts make the estimation of income for FDPs rather hard so that most often consumption-based measures are used.

However, measuring consumption among FDPs has its own challenges, including misreporting and questionnaire fatigue. It relies on a list of all potential products consumed by a household. For each consumed item, the household needs to provide specific information on the consumed quantity. Usually, additional questions are asked about the amount that was purchased, including the outlet and price, to be able to estimate the monetary value of consumption. However, FDPs have a strong incentive for misreporting if they are aid-dependent and believe that their responses affect future aid. Kaplan, Walsh and Pape (2018) find in numerous rounds of data collection in Somalia and South Sudan that IDPs report significantly lower levels of consumption than non-IDP households. For example, 45 percent of Somalia IDP households report food consumption below subsistence levels and approximately 80 percent below recommended levels. While the data may be accurate, there are two reasons to suspect that it is not. Such high levels of below-subsistence consumption would be associated with high rates of mortality due to starvation, which is not borne out in mortality data. Kaplan, Walsh and Pape (2018) use a randomized experiment to test the effectiveness of a bundled nudge including a truth primer and more investigative reporting controls. They find that the bundled nudge induces higher reporting in lower quintiles of reported consumption. This treatment pattern is driven by aid reliant IDPs and vanishes when considering the comparison group of non-IDPs. This suggests that IDPs – and possibly more generally aid-dependent populations – are indeed misreporting. However, the study does not yield a ‘true’ estimate of consumption, which is needed to assess the overall level of misreporting and the different approaches that could be used to mitigate misreporting.

Questionnaire fatigue is another important issue. Administering consumption modules often takes more than 90 minutes and can require multiple household visits. Particularly in a deprived setting like an IDP or refugee camp, it can be challenging for enumerators to justify spending so much time on asking about items that the respondent cannot afford. However, reducing the number of items either by aggregating or by removing items will bias estimated consumption (Beegle et al., 2012). This explains the frequent use of imputed consumption methodologies.

Scholars have experimented with alternative methods to do that while striving to maintain accuracy. For example, Pape (2021) refines a methodology (Pape and Mistiaen, 2018) that combines an innovative questionnaire design with standard survey imputation techniques without relying on a previous baseline survey.

15
Consumption items are partitioned into modules. Each household is administered only one randomly assigned module, creating significant time savings and making it possible to administer a full questionnaire including household and individual questions in less than 60 minutes. The full consumption estimate is obtained by imputing the deliberately absent consumption values for items that are not explicitly asked for a specific household, but was administered to other households. The methodology makes it possible to derive poverty estimates without compromising the credibility of the resulting estimate, and it performs considerably better than alternative approaches based on reduced consumption aggregates and cross-survey imputations. The methodology has been widely applied especially in the context of FDPs in Ethiopia, Kenya, Nigeria, Somalia, South Sudan and Sudan (Pape et al., 2019a; Pape et al., 2019b) to allow for shorter interview time, creating space for in-depth displacement-specific questions while reducing enumerator and respondent fatigue.

An alternative way to obtain accurate consumption estimates without limiting the time of the interview can be borrowed from small-area estimation methods (Elbers, Lanjouw and Lanjouw, 2002; Molina and Rao, 2010). This approach administers the full consumption module only to a small subset of households, while the majority of households do not receive any consumption questions. Instead, within-survey imputations are used to impute consumption for this part of the full sample. This approach avoids the pitfalls of crosssurvey imputations while improving accuracy as compared to assigning subsets of consumption items across all households.<sup>10</sup> However, the subset of households with full consumption modules might suffer from questionnaire fatigue increasing measurement error, which can ultimately exceed the model error in approaches like Pape (2021).

After consumption data is obtained, questions arise on how to value consumption. FDPs often receive aid in the form of in-kind transfers as well as cash vouchers. The standard consumption questions will capture consumption of in-kind transfers like food as well as goods and services that were purchased using cash vouchers. Usually, market prices are used to value consumption. However, market prices in a camp can be distorted or absent for specific products if they are mainly bartered. Traders accepting food vouchers might add a premium on prices given their monopoly power in accepting food vouchers. The premium will be reflected in a higher value of consumption leading to higher welfare, which is – of course – grossly misleading. More generally, households in camps can face significantly different prices than households outside camps. In other cases, such as the Rohingya refugee camps in Bangladesh, prices may not be available as proper markets do not exist. In non-FDP contexts, a similar challenge occurs between rural and

> 10 Accuracy improves compared to Pape (2021) because distributing consumption modules across households ignores correlation between consumption modules in the imputation. The small-area estimation approach, in contrast, estimates full consumption for a subset of households automatically considering any within correlations.

16
urban households or other regionally disparate populations facing different prices (e.g., Ravallion and Lokshin, 2006; Boom, Halsema and Molini, 2015). Deflators can be used to adjust for such price differences. However, they are sensitive to methodological choices (Ravallion and Bidani, 1994) and, usually, require large amounts of high-quality data on quantities and prices.

If income or consumption data for FDPs are missing from the data at hand but available in other data representative of the same population, one can also use cross-survey imputation techniques to estimate consumption using proxies of well-being. This method uses a baseline survey inclusive of consumption data to model consumption using a regression model inclusive of easily measurable household characteristics as independent variables. The estimated coefficients from the model are then used to predict consumption for FDPs using census data (usually FDPs registration data) that lack consumption but include the same household characteristics used with the baseline survey model. This method has been tested with refugees in Jordan and Chad (Dang and Verme, 2021, Beltramo et al., 2021) providing encouraging preliminary results. These works showed that accurate poverty estimations for refugees can be obtained with a relatively small number of proxies of well-being which are usually already available in the UNHCR proGres registration system. The quality of the imputed poverty estimates depends on the similarity of the population surveyed at baseline and the population used for imputations. Both populations might differ because of the time passed between baseline and imputation survey, and because of different characteristics if both surveys are not representative for the same populations. It is therefore essential for these types of exercises to use survey and census data from the same time period and with very similar predictors.

The treatment of aid is also particularly challenging. By definition, aid is included in consumption aggregates and, hence, will reduce estimated poverty among FDPs. This provides a fair assessment of the current situation and is particularly helpful in the context of comparison with host communities to ensure that they are not receiving less assistance while being more deprived. However, it can also easily be misinterpreted that FDPs do not require assistance as they are not poor. To avoid this misinterpretation, a second consumption estimate can be constructed that excludes aid from consumption to reflect livelihoods of FDPs in the absence of aid. A precise estimate of consumption excluding aid is not easily feasible. Even if consumed quantities for each item are split into aid vs. non-aid, in-kind assistance that is sold would make attribution difficult, while an income-focused angle would have similar problems as discussed before. A light-handed approach that asks for each item whether it was mainly provided for free can produce a consumption aggregate excluding aid that can be useful in re-thinking targeting of aid (Pape et al., 2019a).

17
# Poverty lines

How to establish a relevant poverty line for FDPs is another important question. The choice is between international poverty lines such as 1.90 USD/day at PPP value (Ravallion, Chen and Sangraula, 2009), national poverty lines of host countries or contextual poverty lines specific to FDPs. International and national poverty lines can be considered after the use of appropriate deflators. These choices can be controversial for regular populations and carry additional complexities when FDPs are considered.

One question is how to adapt poverty lines for FDPs living inside and outside camps. FDPs in camps often receive shelter, education and health assistance that is not – or not in the same way – provided to the population residing outside camps. This can create a paradox that refugees that pay rents in urban areas would appear much richer than refugees living in camps as their shelter is free of charge and not necessarily accounted for in the consumption aggregate. In traditional poverty estimations, a similar challenge occurs when one considers poverty lines for urban and rural populations since the rental market is often absent in rural areas making it impossible to estimate or impute rent. To allow for the inclusion of rent for urban populations, separate urban and rural poverty lines are estimated, called contextual or regional poverty lines. Only the urban poverty line and its corresponding consumption aggregate would include rent.

Similarly, other services like education and health are extremely hard to value given that cost data for camps’ services are incomplete and grossly available only in aggregate form. Hence, they would also need to be excluded in the contextual poverty line for camps. Given the difficulty to estimate contextual poverty lines, however, this approach is usually not feasible in FDP contexts, while it also adversely affects the comparability of estimated poverty rates. Furthermore, a comparison between camp and non-camp population would ignore differences in housing, health and education between these populations, substantially limiting the value of a comparison. These questions are still largely unresolved and the search for poverty lines adapted to FDP contexts is still in its infancy.

# Comparisons across time and populations

Understanding how poverty among FDPs evolves over time and how this compares to other populations is another essential pillar of poverty measurement.

Tracking FDPs over time can be particularly helpful to monitor changes in livelihoods and changes in displacement status, location as well as the end of displacement.<sup>11</sup> Panel surveys in this respect are useful

> 11 While IOM uses the Displacement Tracking Matrix (DTM), it only provides an aggregated view of the location of FDPs.

18
instruments for FDPs as they would allow to track the same individuals over time, understand their livelihood and residency trajectories, and understand how FDPs take decisions. However, classic panel surveys typically rely on home addresses and they are particularly difficult to administer when populations are highly mobile. This has encouraged scholars working on FDPs to develop new instruments to track people over time. Etang and Hoogeveen (2020) developed a survey known as the “Listening to Displaced People Survey (LDPS)”, a survey that tracked living conditions of displaced people over time in Mali with a face-to-face baseline survey complemented by monthly follow-up mobile phone interviews for a period of 12 months. These data have been used by Hoogeveen, Rossi and Sansone (2019) to study patterns of return of the displaced and understand the factors that contribute to return.

Phone interviews have also increased in popularity with the COVID-19 pandemic, which made it necessary to conduct interviews without face-to-face contact. During this period, the UNHCR and World Bank have launched bi-monthly monitoring surveys of the impact of COVID-19 on the well-being of refugees in several countries across the MENA, SSA and Latin America regions using phone interviews. These resulted in panel surveys that now offer the possibility to assess the impact of COVID-19 on refugees over time and across countries in a comparable manner. Vintar et al. (forthcoming), for example, provide an example of how to use these data to understand the differential labor impacts of COVID-19 on refugees and nonrefugees (see also the report “Answering the Call: Forcibly Displaced During the Pandemic”<sup>i</sup> ). UNCHR’s proGres database includes phone numbers for refugee family heads that can be utilized as a sampling frame. However, data privacy concerns need to be addressed if the phone survey is conducted by a firm. A possible solution is sending text messages to selected respondents asking for permission to share phone numbers with a contractor.

Comparisons between FDPs and host populations are also an essential exercise to conduct in the context of FDP poverty measurement. These comparisons are important for FDPs, host countries and international organizations given that resentment against FDPs is often fueled by a perception that FDPs receive special assistance that is not available to locals. Several surveys have now been conducted in Jordan, Lebanon, Iraq and a few Sub-Saharan African countries to compare the well-being of FDPs and their hosts. These comparisons, while important, are complex because host populations have full access to the labor and consumer markets and government services that are often not available to FDPs, whereas FDPs rely on aid from the international community that is not available to local residents. It is difficult to compare health services in camps, for example, to those provided to the host population by the government, or other social protection services such as unemployment insurance or paid leave that do not exist for FDPs. Again, these are new and largely under-researched issues among poverty specialists.

19
# Future prospects

There are now a wide variety of survey instruments that are being designed for or adapted to measure income or consumption among FDP populations. Home visits initially designed by the UNHCR to question FDPs for protection purposes are being revised to ask questions on income and consumption, becoming in this way viable instruments for poverty measurement. Large home visit exercises are conducted every year in countries such as Jordan and Lebanon and these data have been used to conduct poverty assessments of Syrian refugees (Verme et al., 2016). The WFP conducts vulnerability assessments that have been used by the WFP, UNHCR and World Bank to make gross poverty estimates using the consumption modules of these surveys, even if these modules are typically very short, with few items. The UNHCR conducts Multisector Needs Assessment such as the one conducted in Cox’s Bazar, Bangladesh, in 2018, Socio-economic assessments such as the one conducted in Zimbabwe in 2017, or nutrition surveys such as the one conducted in Tanzania in 2017.<sup>12</sup> All these surveys contain some information on income, consumption, or expenditure that is being used to assess the well-being of FDPs. The World Bank has conducted welfare assessments of Venezuelans in various Latin American countries including Colombia, Peru and Ecuador, with a forthcoming study expected for Chile. The welfare of Afghan refugees has now been studied in their main host country, Pakistan, and in Afghanistan upon return, providing elements to compare the living conditions of this population in the two locations. Again, all these surveys and studies are either very recent or were not used before for poverty/welfare analyses.

Whereas individual and household data on refugees are now more systematically collected, data collection for IDPs remains extremely scarce when compared to refugees, mostly limited to head counting. Refugees and IDPs are very different in that they have a different legal status, which leads to different access to public services, labor markets, government, and international assistance. Surveys for IDPs are often more difficult to conduct because the host country might be linked to the cause of displacement, and a registration system like UNHCR’s proGres is absent, making it more challenging to obtain a representative sample. Given the large number of IDPs (58 percent of all FDPs), the real challenge will be to collect microdata on IDPs systematically in all those countries that are home to large numbers of IDPs. So far, the only country that collects data on IDPs systematically is Colombia and this country has shown that, when quality microdata are available, research on IDPs flourishes.

While many of the discussed issues need more attention and research, some processes are in place that should lead to the establishment of guidelines that address some of these questions. A recent process

> 12 All these surveys can be found in the World Bank microdata library at: https://microdata.worldbank.org/index.php/catalog?sort_by=rank&sort_order=desc&sk=refugees.

20
initiated by the United Nations led to the preparation of guidelines for refugee and IDP statistics including poverty statistics. The United Nations Statistical Commission (UNSC) established in 2016 an international Expert Group on Refugee and Internally Displaced Persons Statistics (EGRIS) made of national and international organizations and several expert statisticians. This process has produced two main documents: The International Recommendations on Refugee Statistics (EGRIS, 2018), and the International Recommendations on IDP Statistics (EGRIS, 2020).<sup>13</sup> The first document recommends measuring the proportion of population below the international poverty line among the social inclusion indicators (indicator 1.1.1), and the second document recommends measuring the proportion of population living below the national poverty line among the livelihoods and economic self-reliance indicators (indicator 1.2.1). While not specifically focused on poverty, these documents recognize the importance of measuring poverty among FDPs and provide general indications on the indexes to measure and the data necessary to measure such indexes. In 2022, monetary poverty was identified as one of the 12 priority SDG indicators that should be measured and disaggregated for both refugees and IDPs (UNHCR, 2020), with the recommendation being picked up by the IAEG-SDG (2019). As already mentioned, the UNHCR has started to introduce standards for the measurement of well-being of FDPs while the new World Bank-UNHCR Joint Data Center established in Copenhagen is expected to develop and fund further methodological advancements as well as playing a pivotal role in the production of poverty data for displaced populations especially through National Statistics Offices. Also other organizations such as WFP and IOM are developing their own standards for surveying FDPs and measuring various well-being indicators.

# Conclusion

The paper provided a first insight into the state of the literature on the measurement of poverty among FDPs. We argued that the economics profession and poverty specialists across the social sciences largely neglected these populations for a combination of factors including lack of interest and microdata. This changed with the beginning of the Syrian conflict in 2011 and the peak of the European migration crisis in 2015. These events have generated partnerships between development and humanitarian organizations that contributed to boost microdata collection among FDPs, poverty and welfare studies worldwide, and is also sparking a virtuous cycle of poverty measurement innovations. As FDPs require special solutions to questions such as sampling, consumption measurement, and targeting, this search is also producing innovative approaches that can serve the poverty measurement community at large.

> 13 Since 2022, the group is renamed the Expert Group on Refugee, IDP and Statelessness Statistics (EGRISS).

21
Poverty measurement for FDPs remains at a very early stage. One should remember that the global poverty measurement spearheaded by the World Bank in the 1980s has required several decades to develop into a structured system of microdata collection at the country level and a theoretical and empirical body of knowledge that could make use of these data. Microdata collection for FDPs can build on this infrastructure but also requires its own specific surveys, measurement methods, and theoretical and empirical adjustments. Multilateral organizations such as the World Bank and UNHCR have started this process and this has encouraged the research community to follow, but this is only the beginning of a long process.

22