6

- a. Use a natural language processing tool called nlp4dev<sup>14</sup> to tag each dataset in the WB MDL by thematic areas and keep those ones that are flagged as relevant to the issues of migration, displacement and refugees;<sup>15</sup>

- b. Keep those datasets that mention “refugee(s)”, “internally displaced”, “internal displacement”, “IDP(s)”, or “Venezuelan(s)” in either the title or sampling section of a dataset in the metadata;

- c. Keep those datasets that have UNHCR as one of the data producers in the metadata.

If a given dataset passes at least one of these filters, it is flagged as “potentially relevant.”<sup>16</sup> It is worth highlighting that these filters have been developed in an iterative process. We consulted with a number of country experts both from UNHCR and the World Bank to avoid _false negatives_ (or those datasets that do in fact sample from refugees or IDPs but are not correctly identified).<sup>17</sup> Applying these filters identifies 358 potentially relevant datasets from the WB MDL.

Fourth, we perform a manual verification to remove those datasets that are also cross-listed in the UNHCR MDL. It is worth noting that most of the potentially relevant datasets taken from the WB MDL are also already cross-listed in the UNHCR MDL and thus are removed to avoid double-counting. After removing these cross-listed datasets, we have 62 datasets left from the WB MDL. Fifth, we apply the same manual verification to identify false positive datasets from the WB MDL following the same procedure as done for the datasets from the UNHCR MDL, which ultimately leaves us with 6 datasets from the WB MDL.<sup>18</sup> Finally, we combine the identified 369 datasets from the UNHCR MDL and 6 datasets from the WB MDL to arrive at our final list of 375 FDP micro-level datasets.

After identifying the corpus of FDP datasets – or micro-level datasets that have a representative sample of refugees and IDPs – we then also manually code various attributes of each dataset that are not readily available from the UNHCR and WB MDLs. First, we evaluate whether a given dataset is _project-specific_ . More specifically, we look at whether the sampling frame in a given dataset is narrowly defined to only include beneficiaries of a certain project or program. While project-specific datasets are useful in their 14 The NLP4Dev application – developed by the WB Development Data Group – is a tool for applying Natural Language Processing (NLP) topic and word embedding models to improve document and data discoverability. These models are used to implement a semantic search engine and to generate summary, structured information on the thematic and geographic composition of a large corpus of unstructured documents. See more details in <u>https://www.nlp4dev.org/.</u> 15 nlp4dev allows users to tag texts by 75 different topic areas and reports what share of the texts cover which topic areas. We use topic 39 – which includes a cluster of issues related to the issues of our concern including refugee, program, country, migration, migrant, labor, remittance, population, international, asylum – to tag potentially relevant datasets. More specifically, we extract the entire metadata from each dataset, turn it into a text file, pass it through nlp4dev to report the coverage rate of each topic and then if the coverage rate is above 10 percent, flag the dataset as potentially relevant.

16 Of the datasets from the WB MDL, 11 percent have titles or sampling sections in languages other than English within the metadata. In these instances, we utilize the DeepL API (using the deeplr package in R) to translate them into English. Subsequently, we apply the same filters. Furthermore, 5.5 percent of datasets ( _N_ =107) have missing sampling sections and for these instances, we manually verify whether refugees or IDPs are included as a sample based on desk research.

17 For example, just applying nlp4dev as a filter turned out to be insufficient for correctly identifying many of the IDP datasets and additional filters were added after consulting with country experts from UNHCR and the World Bank Poverty & Equity Global Practices.

18 Most of the datasets flagged as relevant in the nlp4dev tool turn out to be false positive. They are mostly flagged as relevant because they either have a module specifically on migration or mentioned some key terms related to migration but turn out not to sample specifically from refugees or IDPs.