metadata: authors: 0: author: Ekhator-Mobayode,Uche Eseosa
1: author: Hoogeveen,Johannes G.
count: Middle East and North Africa
docna: 0: docna: Microdata Collection and Openness in the
            Middle East and North Africa (MENA) : Introducing the MENA
            Microdata Access Indicator
docty: Policy Research Working Paper
subtopic: ICT Policy and Strategies,ICT Legal and Regulatory Framework,Health Care Services Industry,Inequality,Employment and Unemployment,Labor & Employment Law
teratopic: Information and Communication Technologies,Industry,Poverty Reduction,Law and Development,Social Protections and Labor
abstracts: cdata!: This paper uses a “mystery client”
            approach and visits the websites of national statistical
            offices and international microdata libraries to assess
            whether foundational microdata sets for countries in the
            Middle East and North Africa region are collected, up to
            date, and made available to researchers. The focus is on
            population and  economic censuses, price data and
            consumption, labor, health, and establishment surveys.
            Following the exercise, a new microdata access indicator
            that measures the degree of opennes of microdata and the
            ease with which microdata users can understand and navigate
            the websites of national statistical offices is presented.
            The results show that about half of the expected core data
            sets are being collected and that only a fraction is made
            available publicly. As a consequence, many summary
            statistics, including national accounts and welfare
            estimates, are outdated and of limited relevance to decision
            makers. Additional investments in microdata collection and
            publication of the data once collected are strongly advised.
            National statistical offices in the region should make
            considerable improvements to the outlook of their websites
            to make them more user friendly. Specifically, microdata
            libraries and updated survey calendars should be a standard
            feature of the websites to ensure easy access to available microdata.
repnme: repnme: Microdata Collection and Openness in the
            Middle East and North Africa (MENA) : Introducing the MENA
            Microdata Access Indicator
display_title: Microdata Collection and Openness in
            the Middle East and North Africa (MENA) : Introducing the
            MENA Microdata Access Indicator
listing_relative_url: /research/2021/12/33693936/microdata-collection-openness-middle-east-north-africa-mena-introducing-mena-microdata-access-indicator
topicv3: Open Data Policy m1326174 679,Poverty m1326687 695,Household Surveys m1504071 2761,Census Data m1504073 2763,Health m1504661 3480
publishtoextweb_dt: 2021-12-29T00:00:00Z
dois: 10.1596/1813-9450-9892
historic_topic: Information and Communication Technologies,Industry,Poverty Reduction,Law and Development,Social Protections and Labor
origu: EFI-MNA-POV-Poverty and Equity (EMNPV)
no_of_pages: 55
author_keys: 0: author_key: 1662654
1: author_key: 435354
model_extractions: Policy Research Working Paper 9892 Microdata Collection and Openness in the Middle East and North Africa (MENA) Introducing the MENA Microdata Access Indicator _Uche Eseosa Ekhator-Mobayode Johannes Hoogeveen_ Poverty and Equity Global Practice December 2021
Policy Research Working Paper 9892

# **Abstract**

This paper uses a “mystery client” approach and visits the websites of national statistical offices and international microdata libraries to assess whether foundational microdata sets for countries in the Middle East and North Africa region are collected, up to date, and made available to researchers. The focus is on population and economic censuses, price data and consumption, labor, health, and establishment surveys. Following the exercise, a new microdata access indicator that measures the degree of opennes of microdata and the ease with which microdata users can understand and navigate the websites of national statistical offices is presented. The results show that about half of the expected core data sets are being collected and that only a fraction is made available publicly. As a consequence, many summary statistics, including national accounts and welfare estimates, are outdated and of limited relevance to decision makers. Additional investments in microdata collection and publication of the data once collected are strongly advised. National statistical offices in the region should make considerable improvements to the outlook of their websites to make them more user friendly. Specifically, microdata libraries and updated survey calendars should be a standard feature of the websites to ensure easy access to available microdata.

This paper is a product of the Poverty and Equity Global Practice. It is part of a larger effort by the World Bank to provide open access to its research and make a contribution to development policy discussions around the world. Policy Research Working Papers are also posted on the Web at http://www.worldbank.org/prwp. The authors may be contacted at uekhator@worldbank.org and jhoogeveen@worldbank.org.

_The Policy Research Working Paper Series disseminates the findings of work in progress to encourage the exchange of ideas about development issues. An objective of the series is to get the findings out quickly, even if the presentations are less than fully polished. The papers carry the names of the authors and should be cited accordingly. The findings, interpretations, and conclusions expressed in this paper are entirely those of the authors. They do not necessarily represent the views of the International Bank for Reconstruction and Development/World Bank and its affiliated organizations, or those of the Executive Directors of the World Bank or the governments they represent._ Produced by the Research Support Team
# **Microdata Collection and Openness in the Middle East and North Africa (MENA): Introducing the MENA Microdata Access Indicator**

Uche Eseosa Ekhator-Mobayode<sup>*1</sup> and Johannes Hoogeveen<sup>*</sup>

**Key words** : National Statistical Offices, Statistical Indicators, Data Openness, Microdata, MENA

**JEL** : C18, H00, I00, O1, O53

> * MENA Poverty and Equity Global Practice. The authors would like to thank Henry Gannat and Federica Alfani for excellent research assistance. The authors would also like to thank Umar Serajuddin, Hai-Anh Dang, Brian Stacy and Daniel Mahler for providing suggestions on earlier drafts of this paper. Special thanks to several colleagues from the Poverty and Equity Global Practice at the World Bank who participated in the peer review process to validate results from the data collection exercise.

> 1 Corresponding author. Email: uekhator@worldbank.org
# 1. Introduction

Timely and consistent statistics are essential to inform and monitor economic, environmental, and social development. Yet to be used in decision making, statistics need to be more than of good quality. They need to be timely and trusted. <mark>Trust in official statistics comes, broadly speaking, from two sources (Brackfield, 2011). The statistics themselves must be trustworthy and credible. Next, the institution producing the statistics needs to be trusted. Openness and transparency affect trust in official statistics through both pathways. Transparency allows the public to assess the methods and data used and increases trust in the organization itself. I</mark> n addition to being important for trust in official statistics, statistical transparency also yields an attractive return. Research in middle income contexts demonstrates that the availability of quality, transparent, and timely disseminated macroeconomic and financial data reduces sovereign borrowing costs on international capital markets. Adherence to the Special Data Dissemination Standards (SDDS), for instance, lowers borrowing costs by 50 basis points as it reassures international investors on the reliability and serviceability of a country’s economic and financial data (Cady, 2005).

In this paper, we examine two aspects of statistical quality, microdata collection and access. We focus on microdata for three reasons. They are an important source of data, especially for researchers, who without it often would not have the ability to carry out their work on nationally representative samples. The <mark>demand for readily available microdata can be illustrated with the 2017 Djibouti Household Survey. Its data have been downloaded 2078 times even though the data was only uploaded on the World Bank microdata library in June 2019. After 20 months since the data have been publicly released, Google scholar already gives 290 hits of academic articles that have been prepared using this data set (checked on 23 Feb 2021). The inflow of research with new data strengthens the analytical capacity of the national statistical system and has huge marginal gains especially for lower income countries that are less likely to conduct household surveys.</mark><sup>2</sup> <mark>The second reason to focus on publicly releasing microdata is that by not doing so, the public use value of the data in research is foregone. This value can be significant. The cost of collecting the data is sunk (taxpayers have already paid for it) and the marginal cost of creating another copy of the data base is negligible. The benefits on the other hand can be substantial. I</mark> ncreased accessibility to data has been related to the MENA region’s chronic low-growth syndrome and Arezki et al. (2020) estimate that the region’s lack of data transparency has resulted in losses of income per person ranging between 7% and 14%.

The third reason to focus on the availability of microdata is because it demonstrates a credible commitment to transparency. Between 2005 and 2008 MENA, was the only region globally to

> 2 Dang et al. (2019) provide evidence that countries with higher incomes more frequently implement household surveys.

2
experience an absolute decline in the “statistical capacity index”- an index of data transparency (Arezki et al. 2020). More data transparency may improve political trust and create more social cohesion. Releasing micro data to the public requires balancing two fundamental principles of statistics: confidentiality and access. An agency not committed to data transparency could argue (erroneously) that privacy considerations –captured in every Statistics Act, prevent it from releasing anonymized micro data.

To assess access to microdata we take the perspective of an everyday data user and visit the public facing websites of all National Statistical Offices (NSOs) in the MENA region as well as microdata libraries maintained by the World Bank, International Household Survey Network (IHSN), IPUMS and Eurostat. We also visit the web-portals of the MICS and DHS surveys. Though, as World Bank staff, we often already have access as part of our official duties, we opt to follow a ‘mystery client approach’ and explore which data can be accessed through public channels. We verify if up to date microdata across several data categories is available for download either immediately or after a request is made by the user.

Informed by this exercise, we do two things. First, we make suggestions aimed at improving NSOs’ ability to provide up to date microdata online. Second, we present a new Microdata Access Indicator (MAI) that measures the degree of openness of available microdata across MENA countries and the degree to which microdata users can <mark>access, understand, and navigate the websites of MENA NSOs.</mark> The new MENA MAI provides insights on microdata accessibility in MENA and complements the Open Data Access Indicator (ODIN)<sup>3</sup> published by Open Data Watch that focuses on NSOs’ ability to provide access to produced statistics and indicators (as opposed to anonymized raw data). Together, both the MENA MAI and the ODIN provide a robust picture of data accessibility in MENA and can over time serve as a useful tool for NSOs and development partners to measure the development of the statistical system and advocate for greater data transparency. They can also be used to encourage dialogue between NSOs, development partners and data users. The findings from the MENA MAI demonstrate that many microdata sets are out of date or not collected at all. Since one cannot publish what is not collected, we strongly advocate for additional investments in microdata collection as well as publication of the data. The findings also show that there is room for improvement with regards to the outlook of the websites of many MENA NSOs – NSOs should make microdata libraries and updated survey calendars a standard feature of their websites so that microdata users do not spend unnecessary time searching for available microdata.

The rest of the paper is structured as follows: the next section explores in greater depth the intersection between public trust in official statistics and data transparency. Section 3 describes the exercise of visiting MENA NSOs’ websites, discusses the results from the exercise and offers

> 3 Open Data Watch - Open Data Inventory http://www.opendatawatch.com

3
some suggestions for progress based on observations made by the research team. Section 4 examines existing indicators measuring data accessibility and discusses results from section 3 <mark>visà-vis</mark> the Open Data Access Indicator (ODIN). Section 5 presents the new microdata access indicator for MENA while section 6 concludes.

# 2. <mark>Transparency and trust in statistics</mark>

<mark>Public trust in official statistics is anchored in professional independence and impartiality of statisticians, their use of scientific methods and equal access for all to official statistical information. T</mark> o operationalize these ideas, the international statistics community has adopted a professional code comprising of ten principles, the _Fundamental Principle of Official Statistics,_ and a set of “ _Good Practices_ ”. Together they emphasize accessibility, impartiality, transparency, accuracy, relevance, cost-effectiveness, confidentiality, professionalism, coordination, and cooperation. At times, the Principles and Practices have conflicting requirements. Confidentiality, for instance, captured in Principle no 6 necessitates measures to <mark>prevent the direct or indirect disclosure of data on persons, households, businesses, and other individual respondents. As this could be interpreted as a prohibition to release source data, statisticians also commit themselves to “</mark> _<mark>a</mark> framework describing methods and procedures to provide sets of anonymous micro-data for further analysis by bona fide researchers, maintaining the requirements of confidentiality_ .”<sup>4</sup> In this way the Good Practices forge a compromise between confidentiality on the one hand, and transparency and access on the other.

Access to microdata is typically offered in two ways. Some agencies make anonymized microdata directly available to the public. India’s statistical agency for instance, the <mark>Ministry of Statistics and Programme Implementation</mark> MOSPI, has a long history of running national sample surveys dating back to the 1950s when they were initiated by Professor <mark>Mahalanobis, the father of Indian statistics, and of publicly releasing the anonymized microdata. On MOSPI’s website microdata sets are available for download dating back to as far as 1975. Other known sources of downloadable microdata sets are the World Bank’s (WBs) microdata library,</mark><sup>5</sup> <mark>the DHS</mark><sup>6</sup> <mark>and MICS</mark><sup>7</sup> <mark>websites, the labor force surveys curated by the ILO,</mark><sup>8</sup> <mark>and IPUMS</mark><sup>9</sup> <mark>which publishes (samples of) population censuses.</mark> Others, like EUROSTAT, make microdata available in two formats: Public and Scientific Use Files. The Public Use Files (PUFs) can be downloaded immediately. They are subsamples of the Scientific Use Files (SUFs) which allow researchers to explore data sets and build their code. These PUFs cannot be used for publications. For this the SUF files are needed. SUF files are also made

> 4 See https://unstats.un.org/unsd/dnss/gp/FP-Rev2013-E.pdf

> 5 https://microdata.worldbank.org/

> 6 https://www.dhsprogram.com/

> 7 https://mics.unicef.org/surveys

> 8 https://www.ilo.org/surveyLib/index.php/catalog/LFS/about

> 9 https://usa.ipums.org/usa/

4
available but require a stricter two-step application process in which the organization of a researcher first has to be recognized as a research entity – a university, research institution or research department in a public administration, bank, statistical institute etc., after which a researcher can submit an application to receive the full microdata set.

In the MENA region there is less of a tradition of making microdata available and few countries seem to provide public access to (anonymized) microdata. For example, Atamov et al. (2020) report that in 2019 only 7 of the 20 countries in the region provided public or licensed access to household budget surveys which provide the source data on the basis of which the World Bank calculates its estimates of poverty (Table 1). To help with the advocacy of accessibility to microdata in MENA, it is important to have a more complete understanding of the state of microdata access, beyond the availability of household budget surveys. We do so in the remainder of this paper.

**Table 1. Status of public and WB access to household budget surveys in MENA as of August 2019**

**Public or licensed access** Djibouti Egypt, Arab Rep. Iran, Islamic Rep. Iraq Tunisia West Bank and Gaza Yemen, Rep. Source: Atamanov et al. 2020

5
<!-- Start of picture text --> Data categories for source/survey data/micro data<br>¥ |<br>Establishment data Individual/ Household data<br>v J<br>Consumption Labor Health Population Economic<br>data Force data data census data census data<br>Representative Representative<br>dataset(s): dataset(s):<br>R tati Household<br>dataset(s):epresentative’ Representative budget,g' hold Representative; . ||Demographic. ; Representative. i<br>Enterprise Househo dataset(s): and Health Representative || dataset:<br>survey, Annual dataset(s):Consumer price incomeLivi Labor Force. Survey (DHS), dataset:: Economic5 /<br>survey. of survey" or ,StandardLiving survey. or Multiple. Population Business/:<br>industry or equivalent Measurementandards equivalent Indicator census data Establishment<br>equivalent Cluster Survey census<br>Survey (LSMS) (MICS) or<br>or equivalent. equivalent.<br><!-- End of picture text -->
(used to calculate the producer price index, PPI). In this category we do not look for the availability of each data point, though such information would be informative, but for the availability of item level price series.

We divide the _individual/household data category_ into three subcategories as follows: _consumption (welfare) data; labor force data and health data_ and consider various possibilities under each subcategory. For the _consumption data sub- category_ , we look for household budget surveys, household income surveys and/or living standard measurement survey – these surveys are typically used to measure household spending and income and are the underlying source data used to estimate poverty statistics. For the _labor force data sub-category_ , we consider labor force surveys which are the underlying source data used to monitor labor supply and estimate various labor market statistics including labor force participation rate and employment rate. For the _health data sub-category_ , we consider two possibilities, demographic and health surveys and multiple indicator cluster survey or any equivalent which provides the source data to estimate key health statistics including fertility, mortality, nutritional status, and various disease incidences.

Finally, we divide the _census data category_ into two subcategories namely _: population and economic censuses_ . Census data help define the structure and key characteristics of the population and economy and provide the <mark>framework needed for sampling different surveys. Censuses are rarely published in their entirety but many NSOs, i</mark> ncluding in the United States, Canada and Britain, publish randomized 5%-10% samples from their censuses.

3.2 Definition of recent microdata and classification of microdata accessibility To allow for the possibility that microdata are not released because they have not been collected recently, we first establish the availability of recently collected data in each category, whereby recently is defined based on the data at hand. For establishment, consumption, labor force and health surveys, we expect data to be collected at least once every 5 years. This is lenient: the 2016 State of Development Data Funding (SDDF) report published by the Global Partnership for Sustainable Development Data proposes a frequency of 2-3 years for health surveys, 5 years for consumption surveys and annually for labor force and establishment surveys.<sup>12</sup> The World Bank expects welfare surveys to be updated every three years. We expect price survey data to be collected multiple times annually – typically monthly- but examine NSOs for data within the past year. Census data is expected to be collected at least once every decade. Although the exercise of examining NSO websites for recent microdata was carried out between February and April 2021, we use year 2019 as the reference year. This because of COVID-19 related disruption in data collection which often prevented face-to-face interviews from being conducted. Hence recent establishment, consumption, labor force and health surveys are considered to be those

> 12 See <mark>http://opendatawatch.com/knowledge-partnership/state-of-development-data-funding-2016/</mark>

7
carried out between 2014 and 2019 or later; recent price data are collected between 2018 and 2019 or later; recent census data are collected between 2009 and 2019 or later.

Once we have established that data has been collected recently, we assess whether the data is publicly accessible. For each data category, we classify microdata accessibility into 4 groups as follows:

1. **No coverage:** if no representative microdata was recently collected.

2. **No openness:** if representative microdata was recently collected but the data or a link to the data is not available on the website.

3. **Satisfactory openness:** if representative microdata was recently collected and the data (or link) is available on website but is restricted i.e., users need to submit a request and/or register to be granted access to the data.

4. **Excellent openness:** if representative microdata was recently collected and the data (or link) is publicly available on website in machine readable format for immediate download.

We differentiate between “satisfactory” and “excellent” openness because microdata openness is examined from the perspective of the data user. From this perspective “excellent openness” is ideal because there is no wait time for data users to access available data. However, “satisfactory openness” is acceptable because it is okay for data guardians to require registration, authorization, and clearance before releasing data to prevent unauthorized access. The best scenario being where following satisfactory registration, access to the data is granted automatically.

3.3 Implementation exercise of microdata classification in MENA The exercise of visiting the websites of the NSOs<sup>13</sup> and international organizations<sup>14</sup> to examine microdata accessibility was designed to be cost effective and easy to apply to countries in MENA and beyond. <mark>To prevent bias and ensure accuracy and replicability, the exercise is implemented in a 3-step process by a</mark> team comprised of 3 core researchers with language competencies in English, Arabic and French – major languages in the MENA region.

**<mark>Step 1:</mark>** <mark>Each of the three researchers in the research team independently visits the websites to classify m</mark> icrodata accessibility for all data categories into one of the 4 groups discussed in section 3.2.

**Step 2:** Researchers <mark>meet to discuss their independent findings from step 1 and reconcile any differences.</mark> When a researcher finds a representative microdata for the categories covered on a public facing website that other researchers do not, the reconciliation process involves providing

> 13 See Annex table A1 for list of NSOs in MENA and their websites.

> 14 See Annex table A2 for list of the websites of international microdata repositories.

8
a link to the portion of the website where the data was found. The research team visits the link as a group to verify the data and updates their result.

**<mark>Step 3:</mark>** <mark>The updated result from the research team in step 2 is sent for peer review. The peer review is done by World Bank colleagues who work as country/poverty economists and are familiar with the coverage of microdata in the MENA region. Like in step 2, when country economists are aware about</mark> representative microdata for the data categories covered on a public channel not captured by the research team, they provide the link to the data. The research team then verifies the data and updates the result.

Although the methodology described here has only been implemented for MENA countries, it can be scaled globally. To minimize cost, the implementation exercise for a global scale up may be modified. Since step 3 of the implementation exercise involves a review by credible peers to validate the results from steps 1 and 2, only 1 researcher may implement step 1. In this case, step 2 will be eliminated. If this modification occurs, it will be preferable that the researcher chosen to implement the classification exercise for a given region is multilingual in the major languages in the region.

# 3.4 Microdata coverage in MENA

Before data can be made available, it must be collected. Hence, we first determine the collection of recent data for each data category. On NSO websites, we do this by searching explicitly in the “survey/ data section” and/or microdata dashboard/library or implicitly like checking for any mention or reference to the data in a report, summary table, survey calendar/ event schedule and/or announcement page. We also check international microdata libraries to determine recent collection of representative data for each data category. In table 2, we summarize results from the exercise. At the start, we expected to be able to identify a total of 140 micro data sets – 7 data subcategories across 20 countries: eventually we could verify that around half (81) of these microdata sets had actually been collected.

9
**Table 2. Status of survey data (with year collected) in MENA on NSOs website and other public channels**

|**Economy/data**<br>**category**|**Establish**<br>**ment**<br>**survey**|**Price**<br>**survey**|**Consumption**<br>**survey**|**Labor**<br>**force**<br>**survey**|**Health survey**|**Populatio**<br>**n census**|**Economic**<br>**census**|**Total data**<br>**categories with**<br>**recently**<br>**collected data**<br>**per country**|
|---|---|---|---|---|---|---|---|---|
|Algeria|No|Yes (2021)|No|No|No|No|Yes (2011)|2/7|
|Bahrain|No|Yes (2021)|Yes (2014/15)|No|Yes (2018)|Yes (2020)|No|4/7|
|Djibouti|No|Yes (2021)|Yes (2017/18)|No|No|Yes (2011)|No|3/7|
|Egypt, Arab Rep.|No|Yes (2021)|Yes (2017/18)|Yes (2020)|Yes (2014) *|Yes (2017)|Yes (2017)|6/7|
|Iran, Islamic Rep.|No|Yes (2020)|Yes (2019/20)|Yes (2018)|No|Yes (2016)|No|4/7|
|Iraq|No|Yes (2019)|Yes (2017/2018) *|No|Yes (2018) *|No|No|3/7|
|Jordan|No|Yes (2021)|Yes (2017)|Yes (2020)|Yes (2017/18)|Yes (2015)|Yes (2018)|6/7|
|Kuwait|Yes (2018)|Yes (2020)|Yes (2019/21)|Yes (2015)|No|No|No|4/7|
|Lebanon|No|Yes (2020)|No|Yes (2018/19)|No|No|No|2/7|
|Libya|No|Yes (2020)|No|No|Yes (2014)|Yes (2012)|No|3/7|
|Malta|Yes (2016)|Yes (2021)|Yes (2015)|Yes (2020)|No|Yes (2011)|No|5/7|
|Morocco|Yes (2019)|Yes (2021)|Yes (2014)|Yes (2019)|No|Yes (2014)|No|5/7|
|Oman|No|Yes (2020)|No|No|Yes (2014) *|Yes (2020)|Yes (2020)|4/7|
|Qatar|No|Yes (2020)|No|Yes (2019)|No|Yes (2015)|Yes (2015)|4/7|
|Saudi Arabia|Yes (2019)|Yes (2021)|Yes (2018)|Yes (2020)|Yes (2017/18)|Yes (2010)|Yes (2010)|7/7|
|Syrian Arab Republic|No|Yes (2019)|No|No|No|No|Yes (2019)|2/7|
|Tunisia|No|Yes (2021)|Yes (2015)|Yes (2017)|Yes (2018) *|Yes (2014)|No|5/7|
|United Arab Emirates|No|Yes (2020)|Yes (2019)|No|No|No|No|2/7|
|West Bank and Gaza|Yes (2018)|Yes (2021)|Yes (2017)|Yes (2019)|Yes (2019/20) *|Yes (2017)|Yes (2017)|7/7|
|Yemen,Rep.|No|No|Yes(2014)|No|No|Yes(2014)|Yes(2014)|3/7|
|Total economies|||||||||
|indicating collection<br>of recent data for<br>each data category|5/20|19/20|14/20|11/20|9/20|14/20|9/20||
|Total recent data<br>indicated to have|||||||||
|been collected across|||||||||
|all economies||||81/140|||||

Note: Evidence that a survey was collected can be explicit like in a “survey section” of the website or “implicit” like in a report, summary table and/or any mention or reference to the data on the website.

* indicates instances where collection of recent microdata was not indicated on NSOs website, but the research team discovered it on an external website. These include Iraq: Rapid welfare monitoring survey SWIFT 2017/2018 downloadable from <u>https://microdata.worldbank.org/,</u> Egypt (2014) downloadable from http://www.dhsprogram.com/ and Iraq MICS 2018, Oman MICS 2014, Tunisia: MICS 2018, West Bank and Gaza (Palestine) MICS 2019/20 downloadable from <u>https://mics.unicef.org/surveys.</u>

All MENA NSOs except the Republic of Yemen collect price data for their CPI and or PPI and about half are up to date with respect to their labor force, consumption, and census data. Eleven NSOs report recent surveys in the Labor Force microdata category and 14 recent surveys are found in the consumption data category. For establishment data, only a quarter of NSOs (5) collected such data recently: the 2018 Kuwait’s Annual Survey of Establishments, 2016 Malta’s Labor Cost Survey, 2019 Morocco’s National Business Survey, 2019 Saudi Arabia’s Economic Indicator Survey and the 2018 Palestinian Economic Survey Series.

The NSOs of Saudi Arabia and West Bank and Gaza are up to date with their micro data collection across all data categories- 7 out of 7 recent microdata sets expected. They are closely followed by the Arab Republic of Egypt and Jordan which collected data for 6 out of the 7 recent microdata 10
sets expected. By contrast Algeria, Lebanon, the Syrian Arab Republic and the United Arab Emirates, only report 1 or 2 recent microdata sets.

# 3.5 Accessibility of Microdata Nationally

Having collected data does not necessarily imply that the (anonymized) microdata is publicly accessible. For all the data categories, we examine NSO websites<sup>15</sup> for accessibility of the microdata indicated to have been collected. This is reported in Table 3, where entries are only provided where in Table 2 it was indicated that a recent microdata set has been collected. Of the 81 microdata sets, only 16 are accessible to a user visiting NSO websites. Of these, only 5 can be downloaded immediately: the 2018-19 Lebanon Labor Force and Household Conditions Survey (LFHLCS), the <mark>2014 Morocco National survey on Household Consumption and Expenditure, the 2015 Tunisia National survey on budget, consumption and household living standard</mark><sup>16</sup> <mark>,</mark> the 2017 Tunisia National Population and Employment Survey and a subset of the 2014 population census microdata for Morocco. All others require prior registration.

We conclude that NSOs in the MENA region face two major challenges with respect to microdata. Except for price data which are up to date across the board, in all other data categories only about half the countries have up to date microdata sets on which they can draw. Note that this is a very lenient interpretation as microdata sets collected as far back as 2014 are counted towards being up to date. If a stricter definition of up to date were used, the number of countries with recent data would fall lower.

With respect to making the data that has been collected publicly available, NSOs in the region face even more challenges. Only 16 microdata sets, out of a potential 140 that ideally would have been collected, and 81 that have been collected, are downloadable from NSO websites. Consequently, and depending of the definition used, only 10%-20% of the expected microdata are available to the public on NSO websites. Within the price and health data categories none of the NSOs makes microdata publicly available.

> 15 Microdata available nationally may also be on the platforms of other national agencies besides the NSO. If this is the case, we examine the website of the national agency as well.

> 16 For the <mark>2015 Tunisia Budget survey,</mark> it is important to note that not all variables are included in the microdata set available for immediate download.

11
**Table 3: Publicly accessible microdata sets on website of MENA NSOs**

|**Economy/data category**|**Establishment**<br>**survey**|**Price survey**|**Consumption**<br>**survey**|**Labor force**<br>**survey**|**Health**<br>**survey**|**Population**<br>**census**|**Economic**<br>**census**|
|---|---|---|---|---|---|---|---|
|Algeria|-|No openness|-|-|-|-|No openness|
|Bahrain|-|No openness|No openness|-|No openness|No openness|-|
|Djibouti|-|No openness|Satisfactory|-|-|No openness|-|
|Egypt, Arab Rep.|-|No openness|Satisfactory|No openness|No openness|No openness|Satisfactory|
|Iran, Islamic Rep.|-|No openness|Satisfactory|Satisfactory|-|Satisfactory|-|
|Iraq|-|No openness|No openness|-|No openness|No openness|-|
|Jordan|-|No openness|No openness|No openness|No openness|No openness|No openness|
|Kuwait|No openness|No openness|No openness|No openness|-|-|-|
|Lebanon|-|No openness|-|Excellent|-|-|-|
|Libya|-|No openness|-|-|No openness|No openness|-|
|Malta|No openness|No openness|No openness|No openness|-|No openness|-|
|Morocco|No openness|No openness|Excellent|No openness|-|Excellent|-|
|Oman|-|No openness|-|No openness|No openness|No openness|-|
|Qatar|-|No openness|-|-|-|No openness|No openness|
|Saudi Arabia|No openness|No openness|No openness|No openness|No openness|No openness|No openness|
|Syrian Arab Republic|-|No openness|-|-|-|-|No openness|
|Tunisia|-|No openness|Excellent|Excellent|No openness|No openness|No openness|
|United Arab Emirates|-|No openness|No openness|-|-|-|-|
|West Bank and Gaza|Satisfactory|No openness|Satisfactory|Satisfactory|No openness|Satisfactory|Satisfactory|
|Yemen,Rep.|-|-|No openness|-|-|No openness|No openness|
|Total economies with some<br>degree<br>of<br>accessibility<br>of|1/20|0/20|6/20|4/20|0/20|3/20|2/20|
|microdata for data category<br>Total surveys/ census with some<br>degree<br>of<br>accessibility<br>of<br>microdata on NSOs website.||||16/140||||

Note: - indicates up to date microdata have not been collected.

# 3.6 Accessibility of Microdata Internationally

We have not (yet) considered non-NSO websites and/or repositories from which a country’s data could be available. We excluded these on purpose in table 3 as data users –most of whom would be nationals, should be able to access data for their country from their national NSO (or other national agencies: health surveys, for instance, are at times collected and published by Ministries of Health). Yet there are instances where microdata sets are available in international repositories, even while they are unavailable locally. For example, the National Statistics Office of Malta makes microdata from some surveys available to Eurostat who then makes it available to data users upon successful registration and application for the data – these data may not be available on the website of Malta’s National Statistical Office.<sup>17</sup> To complete the picture of microdata accessibility for each country, we explore what is available in international microdata libraries. We do so by visiting the WB microdata library, the web-portals of the MICS and DHS surveys as well as the microdata libraries maintained by the International Household Survey Network (IHSN), IPUMS and Eurostat.<sup>18</sup> The results from this exercise are summarized in table 4.

> 17 For example, Malta National Statistics Office sends microdata from its <mark>European Statistics on Income and Living Conditions Survey (EU-SILC), Household Budgetary Survey as well as Labor Cost Survey – Enterprise survey- to Eurostat where it can be requested by data users.</mark>

> 18 See annex table A2 for the links to these microdata libraries.

12
Overall microdata accessibility improves by around 50% when we consider international accessibility in addition to national accessibility – from 16/140 to 25/140. Some countries like Iraq and Jordan who had _no microdata openness_ for all data categories when we examined only NSOs website now have _satisfactory data openness_ for some data categories. However, despite these improvements, microdata accessibility in MENA remains poor.

**Table 4: Openness of recent source/survey/micro data on public facing website of MENA NSOs and international microdata libraries.**<sup>**19**</sup>

|**Economy/data category**|**Establishment**<br>**survey**|**Price survey**|**Consumption**<br>**survey**|**Labor force**<br>**survey**|**Health survey**|**Population**<br>**census**|**Economic**<br>**census**|
|---|---|---|---|---|---|---|---|
|Algeria|-|No openness|-|-|-|-|No openness|
|Bahrain|-|No openness|No openness<br>|-|No openness|No openness|-|
|Djibouti|-|No openness|Satisfactory<br>(NSO, WB)|-|-|No openness|-|
|Egypt, Arab Rep.|-|No openness|Satisfactory<br>(NSO)|No openness|Satisfactory<br>(WB, DHS)|No openness|Satisfactory<br>(NSO)|
|Iran, Islamic Rep.|-|No openness|Satisfactory<br>(NSO)<br>|Satisfactory<br>(NSO)|-<br>|Satisfactory<br>(NSO)|-|
|Iraq|-|No openness|Satisfactory<br>(WB)|-|Satisfactory<br>(WB, MICS)|-|-|
|Jordan|-|No openness|No openness|No openness|Satisfactory<br>(WB, DHS)|No openness|No openness|
|Kuwait|No openness|No openness|No openness|No openness<br>|<br>-|-|-|
|Lebanon|-|No openness|-|Excellent<br>(NSO)|-|-|-|
|Libya|-|No openness|-<br>|-<br>|No openness|No openness|-|
|Malta|No openness|No openness|Satisfactory<br>(Eurostat)|Satisfactory<br>(Eurostat)|-|No openness|-|
|Morocco|No openness|No openness|<br>Excellent<br>(NSO)|<br>No openness|-|Excellent<br>(NSO, IPUMS)|-|
|Oman|-|No openness|-|No openness|Satisfactory<br>(WB, MICS)|No openness|-|
|Qatar|-|No openness|-|-|<br>-|No openness|No openness|
|Saudi Arabia|No openness|No openness|No openness|No openness|No openness|No openness|No openness|
|Syrian Arab Republic|-|No openness|-|-|-||No openness|
|Tunisia|-|-|Excellent<br>(NSO)|Excellent<br>(NSO)|Satisfactory<br>(MICS)|No openness|No openness|
|United Arab Emirates|-<br>|No openness|No openness<br>||-<br>|-<br>|-<br>|
|West Bank and Gaza|Satisfactory<br>(NSO)|No openness|Satisfactory<br>(NSO)|Satisfactory<br>(NSO)|Satisfactory<br>(MICS)|Satisfactory<br>(NSO, IPUMS)|Satisfactory<br>(NSO)|
|Yemen,Rep.|-|-|No openness|-|-|No openness|No openness|
|Total economies with some||||||||
|degree<br>of<br>microdata<br>accessibility for each data|1/20|0/20|8/20|5/20|6/20|3/20|2/20|
|<br>category||||||||
|Total surveys/ census with<br>some degree of microdata<br>accessibility on NSO, WB,||||||||
|IHSN, IPUMS, Eurostat and/or||||||||
|DHS and MICS websites.||||25/140||||
|**Legend**||||||||
|-<br>Microdata|Microdata avail|able|Microdata availa|ble|Microdat|a available both n|ationally|
|not collected|nationally only||internationally o|nly|and inter|nationally||

> 19 WB, IHSN microdata library as well as IPUMS, Eurostat, DHS and MICS data was accessed on 8th August 2021.

13
<mark>3.7 Opportunities for NSOs to improve microdata accessibility</mark> Collecting microdata is costly, which may be one reason why relatively few microdata sets are collected in the MENA region. While the frequency with which microdata are collected may not change overnight, our search for microdata revealed opportunities for NSOs to improve their data accessibility at almost no additional cost. No MENA NSO makes price surveys (or at least item level price indices) publicly available even though such information would be relevant to a host of users. Almost all MENA NSOs possess recent population census data, but few make them publicly available. The exceptions are Morocco and the Islamic Republic of Iran where a sample of anonymized individual and household level data is available for download. <mark>Additional suggested practices that can improve accessibility of microdata on NSO websites are outlined below.</mark> Suggested practice 1: Provide an English version of the website <mark>While the primary audience for NSO statistics are nationals, many potential data users live abroad. Since English is understood by majority of people in almost every region of the world, it is best practice for NSOs to make available an adequate English version/translation of their website. At present, not all MENA NSOs have an adequate English version of their website. For instance, an English version of the website of the Islamic Republic of Iran NSO exists, but several data sets available on the Persian version of the website are not available on the English version. This includes the consumption (welfare), labor force survey as well as the population census data reported to have satisfactory openness in table 4. Consequently, non-Persian speakers would have difficulty identifying the wealth of data that is available, particularly as the Islamic Republic of Iran is exemplary in providing data access. All recent, available microdata sets are downloadable from the website, some like the household budget surveys at an annual frequency.</mark> Suggested practice 2: Provide a microdata catalog, data tab and a search button on website landing page Given the multiplicity of information that is typically available on an NSO’s website, ensuring a good routing through the website is critical. For primary microdata users, a data tab and/or microdata catalog that present all data available on the website is a useful tool. This will make microdata on the website easy to find and download. Egypt for instance has a tab “MetaData” on its landing page that leads visitors to a central data catalog. This is very helpful for website visitors interested in the country’s data. Some countries go even further. UAE’s open data portal allows users to search for data by the organization within UAE that owns the data. More generally, a search button is important to facilitate finding relevant information on the website and ultimately ensuring a favorable user experience. To date, not all MENA NSOs include a data tab, search button and/or microdata catalog on the landing page of their website. <mark>There is a</mark> 14
<mark>freely available, World Bank approved microdata cataloging tool available at</mark> <u><mark>http://nada.ihsn.org/</mark></u> <mark>that can serve as a guideline for NSOs.</mark> Suggested practice 3: Provide links to other websites with country’s data Earlier we reported that not all microdata sets are hosted on the NSOs websites and about half of the publicly available microdata sets are accessible through international repositories. Where this is the case, providing a link to the websites with the relevant country data is best practice. Microdata available in other Microdata Library of the WB, IHSN, Eurostat, MICS and IPUMS data can be easily linked on the NSO’s website whether the NSO owns all the data available on these websites. Djibouti sets a good example for providing external links to its country’s data. At the time of the study, on the landing page of the website of the National Institute of Statistics of Djibouti, there is a tab named “database” with 3 dropdown tabs as follows (1) survey data. (2) Open data (3) key indicators. The survey data tab links to the World Bank's microdata library.

Suggested practice 4: Provide clarity for requesting restricted data In the classification of microdata accessibility in section 3.2, we differentiate between two classes of microdata accessibility - “satisfactory openness” and “excellent openness” where the former involves a situation where authorization and/or registration is required before a data user can access available data and the later a situation where microdata is available for immediate download on the website. As discussed earlier, “excellent openness” is ideal from the perspective of a data user, however requiring registration, authorization, and clearance before data is released by data guardians is acceptable. When microdata has “satisfactory openness”, it is important that NSO’s provide clarity regarding the steps that need to be followed to gain access, that access is granted within a reasonable period of time and that granting permission is ‘rule based’ and not dependent on ad-hoc criteria. However, for some MENA NSOs for which satisfactory openness is reported in table 3, the website indicates that the data is available upon request without clear instructions about the steps needed to obtain the data. The best scenario for “satisfactory openness” where following satisfactory registration, access to the data is granted automatically is standard practice for international organizations such as the WB, MICS, DHS, IPUMS reported in table 4.

Apart from these best practices that could be implemented by any NSO at a negligible expense, we also strongly advocate to close the microdata gap by investing in regular microdata collection.

15
# <mark>4. Existing indicators measuring data accessibility in MENA</mark>

## 4.1 Existing indicators

The evidence in section 3 shows that the availability and accessibility of microdata in the MENA region is very constrained. Yet the existing indicators measuring data openness do not capture this reality. These indicators include the Open Data Inventory (ODIN)<sup>20</sup> published by Open Data Watch; Open Data Barometer (ODB)<sup>21</sup> by World Wide Web Foundation and Global Open Data Index (GODI)<sup>22</sup> by Open Knowledge Foundation<sup>23</sup> (see table 5 for a summary).<sup>24</sup> Of these indicators, the <mark>ODIN ranks highest with regards to country coverage.</mark> <mark>The ODIN covers 178 countries in its 2018/19 version including 17 MENA countries and 187 countries in its 2020/21 version including all 20 MENA countries. Additionally, ODIN also has a substantial proportion of its elements assessing data accessibility or openness - it assesses the coverage and openness of data available on National Statistics Offices (NSOs) websites based on ten elements across two dimensions - coverage and openness. Five of the ten elements measure data coverage i.e., the degree to which data is available and while the others measure access/openness i.e., the degree to which available data is accessible. Each of the five elements in the</mark> coverage dimension is assessed as follows: representative indicators are available and are disaggregated appropriately; data are available for the preceding five years; data are available for the preceding ten years; data are disaggregated at the first administrative level and data are disaggregated at the second administrative level. <mark>Each of the five elements assessed in the ODIN</mark> data accessibility/ openness dimension are assessed as follows: machine readability; nonproprietary; download options; metadata available and terms of use. All th <mark>e elements in the ODIN coverage and openness dimensions are assessed across several data categories and data dimensions (</mark> see annex table A3 and A4 for the full list of data categories, representative indicators in each category and scoring option for the 2018 ODIN). The default ODIN overall score gives equal weights to the three dimensions <mark>. ODIN scores are calculated as a percentage of the maximum score obtainable.</mark>

> 20 Open Data Watch - Open Data Inventory http://www.opendatawatch.com

> 21 World Wide Web Foundation- https://opendatabarometer.org/barometer/

> 22 Global Open Data Index - https://index.okfn.org/

> 23 The Statistical Access Indicator (SAI) by Almeida and Hoogeveen (2015) with 72% (36 out of 50) of its elements assessing data access and covering 49 African countries would have made the fifth indicator. However, it is unpublished, hence excluded from the review.

> 24 The Statistical Access Indicator (SAI) by Almeida and Hoogeveen (2015) with 72% (36 out of 50) of its elements assessing data access and covering 49 African countries would have made the fifth indicator. However, it is unpublished, hence excluded from the review.

16
**Table 5: Indicators/ data sets measuring data openness**

|**Indicators/Data sets**|**Author**|**% of elements focusing on**<br>**data access**|**Number of countries**<br>**covered(year)**|**MENA countries coverage**|
|---|---|---|---|---|
|1. Open Data Inventory (ODIN)<br>-2018/2019 and 2020|Open Data Watch|50% (5 out of 10)|178 (2018/2019)<br>187 (2020)|17 of 20 in 2018/2019 -<br>Algeria, Djibouti, Egypt, Arab<br>Rep., Iraq, Iran, Islamic Rep.,<br>Jordan, Kuwait, Lebanon,<br>Libya, Malta, Morocco,<br>Oman, Qatar, Saudi Arabia,<br>Syrian Arab Republic, Tunisia,<br>United Arab Emirates<br>All 20 MENA countries in<br>2020|
|2. Open Data Barometer (ODB)<br>– The leadership edition|World Wide Web<br>Foundation|Around 33.3 %|30 (2017)<br>Note: Scope of study was<br>reduced to 30 countries<br>in 2017 to include only<br>countries thatpublicly<br>committed to adopt<br>the International Open<br>Data Charter<br>Principles or the<br>equivalent G20 Anti-<br>Corruption Open Data<br>Principles.|1 of 20 – Saudi Arabia|
|3. Global Open Data Index<br>(GODI)|Open Knowledge<br>Foundation|About 50%|94 (2015)|3 of 20- Iran, Islamic Rep.,<br>Tunisia,Oman|

**Source: Author’s compilation**

17
<mark>Unlike the ODIN, the ODB assesses not just NSOs but overall government data, i.e., all government data regardless of whether the NSO contributes to making the data available. The most recent edition (the fifth edition) known as the leaders’ edition of the ODB covers far fewer countries than the ODIN. Although previous editions of the ODB covered over 100 governments, the</mark> _<mark>Leaders Edition</mark>_ <mark>covers only 30 governments that have publicly committed to adopt the International Open Data Charter Principles or the equivalent G20 Anti-Corruption Open Data Principles – only one of the 30 countries – Saudi Arabia- is in the MENA region. The ODB scores countries on three sub-indexes as follows: readiness; implementation and; impact. Each of the sub-indexes are weighted equally and further subdivided into various components weighted equally within each sub index. The readiness sub index measures the ability of governments to secure positive outcomes from open government data initiative. This is measured through three components focusing on: (1) Government; (2) Citizens and Civil Society; and (3) Entrepreneurs and Business. These components include measures relating to the existence of open data, and a range of interventions that support engagement with and re-use of open data. For the implementation sub index, the ODB asks researchers to complete a checklist of 10 questions</mark><sup>25</sup> <mark>assessing data openness for various data categories (see annex table A5 for the various data categories).</mark><sup>26</sup> <mark>Finally, for the impacts sub index, the ODB takes the approach of treating online, mainstream media and academic publications about open data impacts as a proxy for existence of impacts. Researchers were asked to score the extent of impact on a 0 – 10 scale. The scoring guidance directed assignment of the highest scores for peer-reviewed studies showing impact and emphasized the importance of sources making a direct connection between open data and observed impacts. For scores over 5, researchers were asked to cite at least two separate examples in the given category. To calculate each score for each component an average of the variables in that component is taken and the average of components is used to generate each sub-index. The weighted average of the sub-indexes is used to generate the overall ODB score.</mark> Like the ODB, the GODI also assesses government data based on the open definition and the <mark>open data charter. However, unlike the ODB, GODI limits its inquiry to the publication of national government data and ignores other aspects of the common open data assessment framework such as context, use or impact. It covers 94 countries including 3 MENA countries- the Islamic Republic of Iran, Tunisia, and Oman- in its most recent version. GODI data categories are refined</mark>

> 25 These 10 questions are as follows: <mark>Does the data exist? Is it available online from government in any form? Is the data set provided in machine-readable formats?; Is the machine-readable data available in bulk?; Is the data set available free of charge?; Is the data openly licensed?; Is the data set up to date?;  Is the publication of the data set sustainable?; Was it easy to find information about this data set and;</mark> ; <mark>Are (linked) data URIs provided for key elements of the data?</mark>

> 26 Excerpt from ODB methodology report- ‘”By putting forward categories of data, rather than specific named data sets, we allowed researchers to exercise judgement as to the extent to which countries were making data of this kind available, whilst also sourcing specific examples of data sets that fit into these categories in different countries, and generating a rich collection of qualitative information about the reasons that certain data may or may not be available in different countries, and the extent to which certain data sets tend to exist at national or federal levels.”

18
<mark>each year to reflect key data that is relevant for civil society at large and are developed in partnership with domain experts, including organizations championing open data in their respective fields (see annex table A6 for most recent data categories). Each data set is evaluated using a set of questions in a survey that examines the openness of the data sets. Each survey question measures a crucial aspect of either the legal, technical, or practical ‘openness’ of data (see annex table A7 for detailed questions and scoring guidelines). As shown in annex table A7, GODI survey questions check different aspects of data access and usability. The scoring guidelines for GODI’s survey questions are such that fairly high scores may not always mean open data, but access-controlled data, or public data in poorly structured, or not machine-readable formats. Specifically, GODI does not add many filters, such as exclusively considering data that is machine-readable - even though it might give a more realistic image of open data. For example, budget data can be assessed in PDF form which may be in public domain, available online for free, but in a format making it practically unusable. This data is presented as 80% open. The score suggests a fairly high degree of openness, but in fact, the data is not open. Only 100% means that the data is open. GODI claims that this approach, seeks to demonstrate which data is already available and how it can be further improved.</mark> In comparison to the ODB and GODI, the ODIN has comprehensive country coverage and covers the most MENA countries. It <mark>is also easier to understand b</mark> ecause its measure of data accessibility credits data when it is machine readable as opposed to the GODI that assigns scores to data in formats like PDF which is not really open at least from the perspective of a researcher seeking machine readable data for analytical work. However, ODIN’s methodological guide mention that the terms “data,” “statistics,” and “indicators” are used interchangeably.<sup>27</sup> These three terms are clearly not synonyms, as data in the ODIN context does not include microdata. ODIN’s measure only captures access to generated statistics and indicators. Hence it is possible that ODIN suggests data accessibility where in fact there is no access to microdata. <mark>In the next section, we discuss recent ODIN scores for MENA vis-à-vis the results of microdata accessibility presented in section 3.</mark>

# 4.2 Performance of MENA countries on the ODIN

Countries in the MENA region perform rather well on the Open Data Access Indicator (ODIN). As shown in figure 2, in the 2018/2019 ODIN, <mark>MENA generally does better than Sub-Saharan Africa, and is on par with South Asia, Latin America and East Asia and the Pacific.</mark>

> 27 See https://docs.google.com/document/d/1MBK0hN6MoQrii7_E1bmRXmsUcE8Fbb-Q32nxm8d8qTw/edit Accessed 18 October 2021.

19
<!-- Start of picture text --> North America 76.2 North America 70.3<br>ECA s9.0 ECA 55.0<br>EAP 46.0 South Asia 44.2<br>MNA 41.8 EAP 41.8<br>South Asia 41.0 MNA 40.5<br>LAC 40.6 LAC 40.3<br>SSA 32.93 SSA 32.2<br>o 20 mean40of OverallsO 8Oo Oo 20mean of40Coverage60 so<br>North America 80.8<br>ECA 62.5<br>EAP 49.4<br>MNA 42.9<br>LAC 40.8<br>South Asia 38.2<br>SSA 33.5<br>fe} 20mean of40OpennesssO so<br><!-- End of picture text -->
ODIN’s measures are of value because of the meticulous and transparent way in which it documents its scores. As discussed earlier, <mark>it is based on ten elements across two dimensions - coverage and openness assessed across several data categories and data dimensions. But the data under these categories are not required to be a</mark> t a high level of disaggregation i.e., they are not required to be individual/household level data- regional and sub regional level data satisfies ODINs scoring guidelines. ODIN’s scores present an excellent basis for data users interested in summary statistics, scoring countries on data availability, degree of disaggregation and the ability to download data in machine readable format. However, given the evidence presented in section 3.1 and 3.2, the usefulness of the ODIN by NSOs and development partners as part of a measuring rod for the development of the statistical system, improving data access and encouraging dialogue with data users is limited for MENA without a complementary indicator measuring microdata access. Given the foregoing, we build on the methodology discussed in section 3 and present an indicator focusing on microdata openness in the next section. This indicator together with the ODIN will give a more balanced view of data opennes <mark>s in MENA.</mark> 21
# Microdata Access Indicator (MAI)

- ‘ Establishment data

- * Price data

- of website website * Consumption data ¢ Labor force data * Health data * Population census * Economic census

<!-- Start of picture text -->
* English translation/version of website website<br>* Micro data dashboard/library<br>* Survey calendar.<br><!-- End of picture text -->
translation/version of the landing page of NSOs website; (2) a micro data dashboard/library and (3) a survey calendar <mark>. These features are chosen for several reasons. First, English is the most widely spoken language globally a</mark> nd has a dominant presence in the media. Hence a <mark>larger group of international audience including content creators and researchers will be able to better use an English version of NSOs’ websites. When an English version is unavailable, it is possible for international users to leverage other translation resources such as Google translate. However, it is still vital for NSOs to maintain an outstanding English translation of its website so that the information disseminated in the local language is translated appropriately from the original material and retain its intended meaning. S</mark> imilarly, organizing available microdata in a microdata dashboard/library/tab on NSOs website makes it easy to find. A survey calendar also helps microdata users know and track available and forthcoming surveys by the NSO or other agencies. Combined, a microdata library and survey calendar on an NSO’s website helps microdata users navigate NSOs website in search of microdata without wasting unnecessary time.

<mark>Following the implementation procedure outlined in section 3.3, countries are assigned scores for each of the 3 key features u</mark> sing the scoring criteria outlined in table 6.

**Table 6: Scoring criteria for the user experience element**

|Element|Representative indicators|Scoringcriteria|
|---|---|---|
|User<br>experience|English translation|0 if no English translation/version is available on NSO’s website<br>landing page.|
|||50 if an English translation/version for NSO’s website is available but<br>does not reasonably reflect the local language(s)and/ or not available<br>for other pages besides the landing page.<br>100 if English translation/version for NSO’s website is available,<br>reasonably reflects the local language and available for other pages<br>besides the landing page.|
||Micro data tab/dashboard/library|0 if no micro data tab/dashboard/library is available on NSO’s website<br>100 if micro data tab/dashboard/libraryis available on NSO’s website|
||Survey calendar|0 if no survey calendar is available on NSO’s website<br>50 if survey calendar is available on NSO’s website but not up to date.|
|||100 if surveycalendar is available on NSO’s website and upto date.|

# 5.2 Data openness element

In section 3, we describe and present results of an exercise that examines both the website of NSOs and international repositories such as web-portals of the MICS and DHS surveys and microdata libraries maintained by the World Bank (WB), International Household Survey Network (IHSN), IPUMS and Eurostat to determine the degree of accessibility of microdata. For each data 23
category, the data openness element assigns scores to countries based on the classification of microdata data accessibility summarized in table 4 using the scoring criteria outlined in table 7.

**Table 7: Scoring criteria for the data openness element**

|**Data**<br>**category**|**Data sub-**<br>**category**|**Representative indicator/**<br>**data sets**|**Scoring**criteria|
|---|---|---|---|
|**Firm data**|N/A|Enterprise survey or<br>Annual survey of industry<br>or equivalent.|0 if up to date representative microdata accessibility is<br>classified as “No coverage” or “No openness”.|
|**Price data**|N/A|Commodity price survey<br>or equivalent.|50 if up to date representative microdata accessibility is<br>classified as “Satisfactory openness”.|
|**Individual/**<br>**Household**<br>**data**|**Consumption**<br>**data**|Household budget survey,<br>Household income and<br>expenditure survey,<br>Living Standards<br>Measurement Survey<br>(LSMS) or equivalent.|100 if up to date representative microdata accessibility is<br>classified as “Excellent openness”.|
||**Labor Force**<br>**data**|Labor Force survey or<br>equivalent.||
||**Health data**|Demographic and Health<br>Survey (DHS),<br>Multiple Indicator Cluster<br>Survey (MICS) or<br>equivalent.||
|**Census data**|**Population**|Population census||
||**Economic**|Economic / Business/<br>Establishment census|0 if up to date representative microdata accessibility is<br>classified as “No coverage” or “No openness”.<br>**If available for one or more sectors**<br>25 if up to date representative microdata accessibility is<br>classified as “Satisfactory openness”.<br>50 if up to date representative microdata accessibility is<br>classified as “Excellent openness”.<br>**If available for all sectors**<br>50 if up to date representative microdata accessibility is<br>classified as “Satisfactory openness”.<br>100 if up to date representative microdata accessibility is<br>classified as “Excellent openness”.|

24
# 5.3 Constructing scores for the MENA MAI

The MENA MAI uses equal nested weights with its 2 elements. This weighting structure is symmetric, monotonic, and decomposable by subgroup. It is based on Atkinson’s (2003) counting method and has been used to construct several indices including the Multi-dimensional Poverty Index, Social Exclusion Index and Statistical Performance Index (see Alkire and Foster, 2011; Cameron et al., 2021; Chakravarty & D’Ambrosio, 2006; Dang et al. 2021). The equal nested weighting structure is such that the indicator is sequentially aggregated at each level so that all features within an element receive an equal share of the total weight of that element (see annex table A8 for weights of features within the MENA MAI). Since, the MENA MAI has a two-level structure – the first level comprises of the two elements and the second comprises of the features within each element-, this implies the following.

1. The sub-score for each of the two elements in each country is derived by taking the average of the scores for features within the element.

2. The overall score in each country is derived by taking the average of the sub-scores for the 2 elements.

Hence the scores for each country are given as follows: 1 **User experience sub-score** = ∑ F F F nn Where: ii=1 ii nn FF n=3 (total number of features in the user experience element) and i represents each feature. 1 **Data openness sub-score** = ∑ mm Where: kk=1 kk mm DDF FFDDF FFD FFDDFFDD m=7 (total number of data categories in the data openness element) and k represents each data category 1 1 **Overall MENA SAI score** = (user experience sub score) + (data openness sub score). 2 2 The data openness cross-element cut-off score is fixed at 50. This is because when countries receive a score of at least 50 in the data openness element, it implies that on average they have at least satisfactory openness in all data categories examined. Similarly, receiving at least a score of 50 on the user experience element is equivalent to receiving the midpoint score across all 3 features of the element. This suggests that microdata users can easily understand and navigate NSOs website at an acceptable level. The scores for each country are summarized in the next section.

# 5.4 Results

<mark>The raw scores of the MENA MAI based on the scoring criteria in table 6 and 7 for MENA countries are reported in</mark> Annex table A9. Only Tunisia and West Bank and Gaza obtain the cut off score or 25
better (see figure 4). NSOs perform lower than average on the user experience element. Although most NSOs obtain a perfect score for providing an English translation of their websites landing page, they fall considerably short with regards to providing updated survey calendars and micro data libraries on their websites. Majority of NSOs falling below the cut off score for the user experience element implies that they can make considerable improvements to the experience of microdata users who visit their websites (see panel A of figure 5). In general, all NSOs perform poorly on the overall score of the MENA MAI. The data openness element contributes significantly to this poor performance (see panel B of figure 5) since a lot of countries obtain no score for data openness for the various data categories assessed because recent data is not available on their NSOs websites and/or internationally.

**Figure 4: MENA micro data access indicator – overall score**

<!-- Start of picture text --> Algeria 16.7<br>Bahrain 16.7<br>Djibouti 20.2<br>Egypt, Arab Rep. 44.1<br>Iran, Islamic Rep. 27.4<br>Iraq 23.8<br>Jordan 36.9<br>Kuwait 25.0<br>Lebanon 23.8<br>Libya 8.4<br>Malta 40.5<br>Morocco 14.3<br>Oman 36.9<br>Qatar 33.4<br>Saudi Arabia 33.4<br>Syrian Arab Republic 8.4<br>Tunisia 67.9<br>United Arab Emirates 33.4<br>West Bank and Gaza 71.4<br>Yemen, Rep 16.7<br>0.0 50.0 100.0<br><!-- End of picture text --> <mark>Source: Author’s compilation using own data</mark> 26
Panel A: User experience element Panel B: Data openness element <!-- Start of picture text --> Score Ct OFF a Score yt OF A<br>Aeris —3=3]s0e Algeria 0,0<br>Sahin DSS Bahrain 0,0<br>Djibouti ATs Djibouti maa<br>ZY Pt, lll 7s Egypt, Arab_. NNNDIU<br>(ran, SSS Iran, blamic..<br>[rq SSS Inq =e<br>Jorcheary _Seleteeetsc7 Jordan vag<br>Kuvecit SOL Kuwait 0,0<br>Lebanon ESS Lebanon MGS<br>Libya MT Libya 0,0<br>Malta iiss Mata —aaIse<br>Morocco 0,0Less than average user experiencerange Acceptable user experience range Morocco satisfactoryPee than openness range Acceptable openness range<br>Oman 7ee Oman =a<br>Cate Sl ss7 Qatar 0.0<br>Saudi Arabia Ss7 Saudi Arabia 0.0<br>Syrian... 7 Syrian Arab.9,0<br>Tunes LL Tusa Ss 7<br>United...Sse United Arab.9.0<br>West Sank. West Sark_.gz]<br>Yernen SsSSE Yemen 0.0<br>0.0 50.0 100.0 eo 50.0 1000<br>m User experience sub score m< average user experience mAcceptable user experience @ Microdata openness sub score §< satisfactory openness @ >= Satisfactory openness<br><!-- End of picture text -->
<!-- Start of picture text --> Algeria Bahrain Djibouti Egypt Iran<br>Consumption survey | 0 Consumption survey | 0 Consumption survey 50 Consumption survey [mmm 50 Consumption survey 50<br>Economic census | 0 Economic census | 0 Economic census | 0 Economic census 50 Economic census<br>Establishment survey | 0 Establishment survey | 0 Establishment survey | 0 Establishment survey [| o | Establishment survey<br>Health survey | 0 Health survey | 0 Health survey | 0 Health survey 50 Health survey<br>Labor force survey | 0 Labor force survey | 0 Labor force survey | 0 Labor force survey Labor force survey Jam 50<br>Population census | 0 Population census | 0 Population census | 0 Population census Population census 50<br>Price survey | 0 Price survey | 0 Price survey | 0 Price survey Price survey | ° |<br>° 8 8 oc 8 8 oc 8 8 a) 8 oc 8 8<br>Openness score Openness score Openness score Openness score Openness score<br>Iraq Jordan Kuwait Lebanon Libya<br>Consumption survey 50 Consumption survey | 0 Consumption survey | 0 Consumption survey | 9 Consumption survey | 0<br>Economic census Economic census | 0 Economic census | 0 Economic census | ° Economic census | 0<br>Establishment survey Establishment survey | 0 Establishment survey | 0 Establishment survey | ° Establishment survey | 0<br>Health survey 50 Health survey 50 Health survey | 0 Health survey | ° Health survey | 0<br>Labor force survey Labor force survey Labor force survey | 0 Labor force survey 100 Labor force survey | 0<br>Population census Population census Population census | 0 Population census Population census | 0<br>Price survey Price survey Price survey | 0 Price survey Price survey | 0<br>° 8 8 oc 8 8 oc 8 8 an) 8 sc 8 8<br>Openness score Openness score Openness score Openness score Openness score<br>Malta Morocco Oman Qatar SaudiArabia<br>Consumption survey 50 Consumption survey 100 Consumption survey | 0 Consumption survey | 0 Consumption survey | 0<br>Economic census Economic census | ° Economic census | 0 Economic census | 0 Economic census | 0<br>Establishment survey Establishment survey | © Establishment survey | 0 Establishment survey | 0 Establishment survey | 0<br>Health survey Health survey | ° Health survey 50 Health survey | 0 Health survey | 0<br>Labor force survey 50 Labor force survey | ° Labor force survey Labor force survey | 0 Labor force survey | 0<br>Population census Population census 100 Population census Population census | 0 Population census | 0<br>Price survey Price survey 1° | Price survey Price survey | 0 Price survey | 0<br>© 8 8 c 8 8 co 8 8 oc 68 s oc 8 8<br>Openness score Openness score Openness score Openness score Openness score<br>Syria Tunisia UnitedArabEmirates WestBankandGaza Yemen<br>Consumption survey | 0 Consumption survey 100 Consumption survey | 0 Consumption survey mmm 50 Consumption survey | 0<br>Economic census | 0 Economic census Economic census | 0 Economic census am 50 Economic census | 0<br>Establishment survey | 0 Establishment survey Establishment survey | 0 Establishment survey J 50 Establishment survey | 0<br>Health survey | 0 Health survey mmm 50 Health survey | 0 Health survey Sm 50 Health survey | 0<br>Labor force survey | 0 Labor force survey 100 Labor force survey | 0 Labor force survey SI 50 Labor force survey | 0<br>Population census | 0 Population census Population census | 0 Population census 50 Population census | 0<br>Price survey | 0 Price survey Price survey | 0 Price survey 1° | Price survey | 0<br>° 8 8 o 68 8 ° 8 8 © 8 8 cs 8 8<br>Openness score Openness score Openness score Openness score Openness score<br>| Acceptable microdata openness score cut off (0- recent microdata is unavailable. 50 - recent microdata is available but restricted. 100 - recent microdata is available for immediate download).<br><!-- End of picture text -->
**Figure 7: MENA SAI micro data openness sub score vs. ODIN 2020 openness sub score.**

<!-- Start of picture text --> Algeria 0.0 26.3<br>Bahrain 0.0 66.7<br>Djibouti 7.1 29.1<br>Egypt , Arab Rep. 21.4 33.8<br>Iran, Islamic Rep. 21.4 37.9<br>Iraq 14.3 52.1<br>Jordan 7.1 48.7<br>Kuwait 0.0 45.5<br>Lebanon 14.3 48.2<br>Libya 0.0 14<br>Malta 14.3 49.1<br>Morocco 28.6 71.8<br>Oman 7.1 93.2<br>Qatar 0.0 43.6<br>Saudi Arabia 0.0 44.1<br>Syrian Arab Republic 0.0 25.6<br>Tunisia 35.7 61.8<br>United Arab Emirates 0.0 85.9<br>West Bank and Gaza 42.9 75.4<br>Yemen 0.0 32.5<br>0.0 50.0 100.0<br>2020 ODIN openness score Microdata openness sub score<br><!-- End of picture text --> <mark>Source: Author’s compilation using own data and 2020 ODIN data from Open Data Watch -- Open Data Inventory</mark> <u><mark>http://www.opendatawatch.com</mark></u>

**<mark>Figure 8: ODINs poverty and income openness scores vs MENA MAI consumption/welfare data openness score</mark>**

<!-- Start of picture text --> 100 100100 100 100<br>100 80<br>7070<br>80 60 60 60 6060 60<br>50 50 50 5050 50 50 50 50 50<br>60 4040 40 40 40<br>40 20<br>20 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0<br>0<br>2018 ODIN openness score (Poverty and Income category)<br>2020 ODIN openness score (Poverty and Income category)<br>MENA MAI (Consumption data openness score)<br><!-- End of picture text --> <mark>Source: Author’s compilation using own data, 2018/19 and 2020 ODIN data from Open Data Watch -- Open Data Inventory</mark> <u><mark>http://www.opendatawatch.com</mark></u> Note: Countries without poverty and income openness score for the 2018/19 and 2020 ODIN are not shown.

29
<!-- Start of picture text --> a SPI = 0.56*** ODIN + 27.57 200<br>. * R*2—= 0.33 SPI = 0.71*** MENA MAI + 33.67<br>R2=0.52<br>_..<br>S a 8 ee<br>2& ~ . & we<br>.<br>5:5<br>2& : nq2 20 ..<br>o<br>° ° ° ° ° ° “ oO 20 40 60 80<br>2020 ODIN Overall Score for MENA Countries MEN|A Microdata Access Indicator<br><!-- End of picture text -->
epidemic, during which face-to-face data collection came to a standstill, we used 2019 as benchmark year, implying that any censuses done after 2009 and surveys done after 2014 were considered up to date.

Our findings are threefold. First price data are typically collected (often at a monthly basis), but census and survey data are often out of date. Only 14 out of 20 countries are current on their population census; 9 out of 20 are up to date on their economic census. Only 5 out of the 20 countries carried out an establishment survey recently and about half the countries are up to date with respect to their health, labor force and consumption surveys (having been completed in 9, 11, and 14 countries respectively). The implication is that in almost half the cases, no or outdated microdata are used to produce core statistics including National Accounts and SDG reporting.

Our second finding is that only in few instances where microdata has been collected, they are made publicly accessible. Of the 140 potential microdata sets we looked for (7 data categories in 20 countries) 81 had been collected and as few as 25 were accessible. Remarkably, of these 25 about a third are not accessible through the website of the NSO; they can only be downloaded from international microdata repositories. Our third finding is that recent microdata is scarce in MENA. Summary statistics are generally available –as evidenced by the Open Data Inventory (ODIN). However, many of these statistics are necessarily based on outdated microdata and decision makers relying on such information would need to consider them with care.

Hence, we present a new indicator to complement the ODIN. The new indicator assesses the degree of openness of recent microdata as well as the ease with which microdata users can access this data on NSOs’ websites. Together with the ODIN, it gives a robust picture of data openness in MENA. It is cost effective, takes less time to collect and can be updated periodically as it only requires visits by the research team to the websites of all NSOs and international microdata libraries to determine the scores of its various elements with no input from the NSOs. The scores of MENA countries on the new microdata access indicator show that there is a strong case to be made to invest in the collection and release of microdata sets in the MENA region. The availability of recent microdata would make avoidable the situation where decision makers are informed by summary statistics that no longer reflect their economic and social realities. If such microdata were also made publicly available, it would further improve statistical transparency while also soliciting researchers to contribute their knowledge to help answer the pressing development questions of our time.

31
# References

Alkire, S., Foster J. (2011). _Counting and Multidimensional Poverty Measurement_ . Journal of Public Economics, 95(7): 476-487.

- Arezki, R., Lederman, D., Abou Harb, A., El-Mallakh, N., Fan, R. Y., Islam, A.,  Zouaidi, M. (2020). _How Transparency Can Help the Middle East and North Africa._ Washington, DC: World Bank. doi:10.1596/978-1-4648-1561-4.

- Atamanov, A., Tandon,S., Lopez-Acevedo, G., Vergara B., Mexico. A., (2020). _Measuring Monetary Poverty in the Middle East and North Africa (MENA) Region. Data Gaps and Different Options to Address Them._ World Bank Policy Research Paper # 9259, May 2020.

- Atkinson, A. B. (2003). _Multidimensional Deprivation: Contrasting Social Welfare and Counting Approaches_ . Journal of Economic Inequality, 1(1): 51-65

- Brackfield, D. (2011). _OECD Work on Measuring Trust in Official Statistics_ . Int Statistical Inst.: Proc 58th World Statistical Congress, Dublin (session STS070).

- Cady, J. (2005). _Does SDDS Subscription Reduce Borrowing Costs for Emerging Market Economies? IMF Staff Papers, 52_ (3), 503-17.

Cameron, G.J., Dang, H.H, Dinc, M., Foster, J., Lokshin, M.M. (2021).

- _Measuring the Statistical Capacity of Nations_ . Oxford Bulletin of Economics and Statistics. Doi: https://doi.org/10.1111/obes.12421

Chakravarty, S.R., D'Ambrosio. C. (2006). _The Measurement of Social Exclusion_ .

- Review of Income and Wealth. 52(3): 377-398.

<mark>Dang, H.H., Pullinger, J., Serajuddin, U., Stacy, B. (2021).</mark>

- _<mark>Statistical Performance Indicators and Index: A New Tool to Measure Country Statistical Capacity</mark>_ <mark>. Policy Research Working Paper; No. 9570. World Bank, Washington, DC. © World Bank. https://openknowledge.worldbank.org/handle/10986/35301 License: CC BY 3.0 IGO.</mark>

- <mark>Dang H., Jolliffe D., & Carletto C. (2019). Data Gaps, Data Incomparability, and Data Imputation: A Review of Poverty Measurement Methods for Data Scarce Environments. Journal of Economic Surveys. 3</mark> 3(3). Pg 757-797 <u>https://doi.org/10.1111/joes.12307</u>

Sauro, J. (2016). SUPR-Q: A Comprehensive Measure of the Quality of the Website User Experience. Journal of Usability Studies. Vol. 10, Issue 2, February 2015, pp. 68-86. - <u>https://uxpajournal.org/wp content/uploads/sites/7/pdf/JUS_Sauro_Feb2015.pdf</u> 32
# Annex

|Table<br>**S/N**|A1: Websites of Nation<br>**Economy**|al Statistical Agencies in MENA<br>**National Statistical Agencies**|**Website**|
|---|---|---|---|
|1|Algeria|The National Office of Statistics|https://www.ons.dz/|
|2|Bahrain|National statistical office of Bahrain<br>Information and eGovernment Authority|http://www.data.gov.bh;<br>http://www.iga.gov.bh|
|3|Djibouti|National Institute of Statistics of Djibouti|www.instad.dj|
|4|Egypt, Arab Rep.|Central Agency for Public Mobilization<br>and Statistics(CAPMAS)|https://www.capmas.gov.eg/|
|5|Iraq|Central Statistical Organization|http://cosit.gov.iq/en/|
|6|Iran, Islamic Rep.|Statistical Centre of Iran|https://www.amar.org.ir/|
|7|Jordan|Department of Statistics|http://dosweb.dos.gov.jo/ar|
|8|Kuwait|Central Statistical Bureau|https://www.csb.gov.kw/|
|9|Lebanon|Central Administration for Statistics|http://www.cas.gov.lb/|
|10|Libya|Bureau of Statistics and Census Libya|http://www.bsc.ly/|
|11|Malta|National Statistics Office|https://nso.gov.mt/en/|
|12|Morocco|Direction de la Statistique|https://www.hcp.ma/Direction-de-la-<br>statistique_a716.html|
|13|Oman|National Centre for Statistics and<br>Information|https://www.ncsi.gov.om/Pages/NCSI.aspx|
|14|Qatar|Planning and Statistics Authority|https://www.psa.gov.qa/en/Pages/default.aspx|
|15|Saudi Arabia|General Authority for Statistics|https://www.stats.gov.sa/en|
|16|Syrian Arab Republic|Central Bureau of Statistics|http://cbssyr.sy/index-EN.htm|
|17|Tunisia|National Institute of Statistics (INS)|http://www.ins.tn/en/statistics-tunisia-<br>national-institute-statistics|
|18|United Arab<br>Emirates|Federal competitiveness and Statistical<br>Center|https://fcsc.gov.ae/en-us|
|19|West Bank and Gaza|Palestinian Central Bureau of Statistics|http://www.pcbs.gov.ps/default.aspx|
|20|Yemen, Rep.|Central Statistical Organization|http://www.cso-yemen.com/|

33
<mark>Table A2: I</mark> nternational organizations and their microdata library websites

|**S/N **|**Organization**|**Online microdata library**|
|---|---|---|
|1|World Bank|https://microdata.worldbank.org/|
|2|IPUMS|https://ipums.org/|
|3|International Household SurveyNetwork(IHSN)|https://www.ihsn.org/|
|4|Multiple Indicator Cluster Survey (MICS)|https://mics.unicef.org/surveys|
|5|Eurostat|https://ec.europa.eu/eurostat/web/microdata|
|6|Demographic and Health Surveys (DHS)|https://dhsprogram.com/data/available-<br>datasets.cfm|

34
|S/N|Table A3: ODIN d<br>Data dimension|ata dimension,catego<br>Data category|ryand indicators<br>Representative indicators|Categorical disaggregation|Coverage element 1 scoring guidelines*|
|---|---|---|---|---|---|
|1|Social Statistics|Population and Vital<br>Statistics|(1.1) Population by 5-year age<br>groups<br>(1.2) Birth rate<br>(1.3) Death rate|(1.1) Sex; marital status<br>(1.2) Sex; marital status<br>(1.3) Sex|To receive full point: Must have all indicators disaggregated<br>by sex.<br>To receive half point: Must have at (1.1) with one<br>disaggregation, or (1.2) and (1.3) with one disaggregation<br>each.|
|2||Education Facilities|(2.1) Number of schools or<br>classrooms;<br>(2.2) Number of teaching<br>staff;<br>(2.3) Education budget data.|(2.1) School stage; school type.<br>(2.2) School stage; school type;<br>(2.3) School stage; functional<br>categories.|To receive full point:Must have all three indicators<br>disaggregated as follows: (2.1) and (2.2) by school stage (see<br>notes) and one other disaggregation and (2.3) with one<br>disaggregation.<br>To receive half point:Must have one indicator with two<br>disaggregation or two indicators with one disaggregation<br>each.|
|3||Education Outcomes|(3.1) Enrollment rate;<br>(3.2) Completion or<br>graduation rate;<br>(3.3) Competency exam<br>results.|(3.1, 3.2) Sex; school stage;<br>age; school type.<br>(3.3) Sex.|To receive full point: Must have all indicators<br>disaggregated as follows: (3.1) and (3.2) by sex and one<br>other disaggregation, and (3.3) by sex.<br>To receive half point: Must have (3.1) or (3.2) with two<br>disaggregations, or (3.3) disaggregated by sex. Two indicators<br>with only one disaggregation each is not enough.|
|4||Health Facilities|(4.1) Number of health<br>facilities;<br>(4.2) Number of beds or data<br>on health care staff;<br>(4.3) Health budget data.|(4.1) Facility type;<br>(4.2) Facility type; department<br>type; staff type<br>(4.3) Functional categories|To receive full point: Must have two indicators with one<br>disaggregation each.<br>To receive half point:Must have one indicator with one<br>disaggregation.|

**<mark>Table A3 continued</mark>**

35
|S/N|**Table A3 continued**<br>Data dimension|<br>Data category|Representative indicators|Categorical disaggregation|Coverage element 1 scoring guidelines*|
|---|---|---|---|---|---|
|5|Social Statistics|Health Outcomes|(5.1) Immunization rate.<br>(5.2) Diseases prevalence or<br>incidence.<br>(5.3) Stunting, wasting, or obesity<br>|(5.1) Age; sex.<br>(5.2) Age; sex; disease type.<br>(5.3) Age; sex.|To receive full point: Must have (5.1), (5.2) and (5.3)<br>by sex, and (5.2) by disease type.<br>To receive half point:Must have at least one indicator<br>with one disaggregation.|
||||rate.|||
|6||Reproductive Health|(6.1) Maternal mortality ratio or<br>rate.<br>(6.2) Infant mortality rate or<br>neonatal mortality rate;<br>(6.3) Under-5 mortality rate;|(6.1) None;<br>(6.2, 6.3) Sex;<br>(6.4-6.6) None.|To receive full point: Must have five indicators with one<br>disaggregation each.<br>To receive half point:Must have two indicators with one<br>disaggregation each.. One must be a mortality rate.|
||||(6.4) Fertility rate;|||
||||(6.5) Contraceptive prevalence rate;|||
||||(6.6)Adolescent birth rate.|||
|7||Gender Statistics|(7.1) Proportion or number of<br>women who are victims of physical,<br>sexual, or psychological violence;<br>(7.2) Proportion of women in<br>government, management or senior<br>positions;|(7.1) Age; disability status;<br>relationship to perpetrator;<br>(7.2) None;<br>(7.3) None.|To receive full point: Must have all indicators with one<br>disaggregation.<br>To receive half point:Must have at least one indicator<br>with one disaggregation.|
||||(7.3)Data on child marriages.<br>||**Table A3 continued**|

36
|S/N|**Table A3 cont**<br>Data dimension|**inued**<br>Data category|Representative indicators|Categorical disaggregation|Coverage element 1 scoring guidelines*|
|---|---|---|---|---|---|
|8|Social Statistics|Crime and Justice Statistics|(8.1) Homicide rate or count;<br>(8.2) Crime rate or count;|(8.1) Sex of victim; age of<br>victim; sex of perpetrator; age<br>of perpetrator;<br>victim/perpetrator<br>relationship.<br>(8.2) Crime type; age of victim;<br>sex of victim; sex of<br>perpetrator; age of perpetrator;<br>victim/perpetrator relationship.|To receive a full point:Must have all three indicators with one<br>disaggregation each or two indicators with two disaggregations<br>each. (8.2) must be disaggregated by crime type.<br>To receive half point: Must have one indicator with two<br>disaggregations or two indicators with one disaggregation each.|
||||(8.3) Persons in prison or<br>incarceration rate.|(8.3) Sentencing status; age;<br>sex.||
|9||Poverty Statistics|(9.1) Poverty rate;|(9.1, 9.2) None|To receive full point:Must have two indicators.|
||||(9.2) Distribution of income by<br>||To receive half point:Must have one indicator.|
||||deciles or Gini coefficient.|||
|10|Economic statistics|National Accounts|(10.1) GDP (production<br>approach) or gross value<br>added;<br>(10.2) GDP (expenditure<br>approach).|(10.1) Industrial classification;<br>(10.2) Major expenditure<br>categories.|To receive full point: Must have all indicators disaggregated<br>as follows: (10.1) by industrial classification and (10.2) by major<br>expenditure categories. Any data in the most recent 5 years<br>(2013 onward) must be presented on at least quarterly basis to<br>receive a full point.<br>To receive half point: Must have at least one indicator with<br>one disaggregation. Data can be presented on a monthly,<br>quarterly,or annual basis.|
|11<br>||Labor Statistics<br>|(11.1) Employment rate;<br>(11.2) Employment<br>distribution;<br>(11.3) Unemployment rate.<br>|(11.1) Sex; age;<br>(11.2) Industry; Occupation<br>type; Sex;<br>(11.3) Sex; age.<br>|To receive full point: Must have: (11.1) by sex, or (11.2) by<br>sex and one other disaggregation, and (11.3) by sex.<br>To receive half point:Must have at one indicator with one<br>disaggregation.<br> **Table A3 continued**|

37
|S/N|**Table A3 continued**<br>Data dimension|<br>Data category|Representative indicators|Categorical disaggregation|Coverage element 1 scoring guidelines*|
|---|---|---|---|---|---|
|12|Economic statistics|Price Indexes|(12.1) Consumer price index<br>(CPI);<br>(12.2) Producers price index<br>(PPI)|(12.1, 12.2) None|To receive full point: Must have all indicators. Any data in the<br>most recent 5 years (2013 onward) must be presented on at<br>least quarterly basis to receive a full point.<br>To receive half point:Must have at least one indicator. Data can<br>bepresented on a monthly, quarterly,or annual basis.|
|13||Government Finance|(13.1) Actual revenues;<br>(13.2) Actual expenditures.|(13.1) Revenue source;<br>(13.2) Administrative<br>classification; economic<br>classification; functional<br>classification.|To receive full point:Must have all indicators with one<br>disaggregation each.<br>To receive half point: Must have one indicator with one<br>disaggregation.|
|14||Money and Banking|(14.1) Money supply;<br>(14.2) Interest rates.|(14.1) M1; M2; M3;<br>(14.2) Rate type.|To receive full point: Must have all indicators with one<br>disaggregation. Must have at least three rates.<br>To receive half point:Must have one disaggregated indicator. If<br>indicator is(14.2),onlyone rate is needed.|
|15||International Trade|(15.1) Merchandise exports;<br>(15.2) Merchandise imports.|(15.1,<br>15.2)<br>Major<br>product<br>categories<br>(agricultural<br>products,<br>fuels,<br>mining,<br>manufactures, etc.).|To receive full point: Must have (15.1) and (15.2) imports by<br>major product categories. Any data in the most recent 5 years<br>(2013 onward) must be presented on at least quarterly basis to<br>receive a full point.|
||||||To receive half point: Must have one indicator with one<br>disaggregation. Data can be presented on a monthly, quarterly,<br>or annual basis.<br> **Table A3 continued**|

38
|S/N|**Table A3 continued**<br>Data dimension|Data category|Representative indicators|Categorical disaggregation|Coverage element 1 scoring guidelines*|
|---|---|---|---|---|---|
|16|Economic statistics|Balance of Payments|(16.1) Current account;<br>(16.2) Capital and Financial<br>account.|(16.1) Goods and services,<br>income, and current transfers<br>(or secondary income);<br>(16.2) Direct investment or<br>international investment<br>position.|To receive full point: Must have (16.1) and (16.2) with one<br>disaggregation each.<br>To receive half point: Must have (16.1) or (16.2) with one<br>disaggregation.|
|17|Environmental statistics|Land Use|(17.1) Data on land use or land<br>cover.<br>(17.2) Data on protected lands.|(17.1) Urban and rural;<br>agricultural use (crop type);<br>environmental zones.<br>(17.2) None|To receive full point: Must have (17.1) with two<br>disaggregations, as well as (17.2). Land use data with one<br>disaggregation and land cover data with one disaggregation is<br>accepted.<br>To receive half point: Must have one indicator with one<br>disaggregation.|
|18||Resource Use|(18.1) Data on fishery harvests;<br>(18.2) Data on timber harvests<br>or deforestation;<br>(18.3) Data on major mining or<br>extractive activities;<br>(18.4) Water supply and/ or<br>consumption.|(18.1, 18.2) None<br>(18.3) Type of mining activity;<br>(18.4) None|To receive full point: Must have three indicators,<br>disaggregated<br>To receive half point: Must have two indicators, disaggregated|
|19<br>||Energy Use<br>|(19.1) Energy consumption<br>|(19.1) Energy type; end-use<br>sector; industrial sector.<br>|To receive full point: Must have (19.1) by energy type and<br>one other disaggregation. Three energy types must be present.<br>To receive half point: Must have (19.1) by energy type and one<br>other disaggregation. Two energytypes must bepresent.<br> **Table A3 continued**|

39
|S/N|**Table A3 continu**<br>Data dimension|**ed**<br>Data category|Representative indicators|Categorical disaggregation|Coverage element 1 scoring guidelines*|
|---|---|---|---|---|---|
|20|Environmental<br>statistics|Pollution|(20.1) CO2 or other<br>greenhouse gas (GHG)<br>emissions;<br>(20.2) Emissions of air or<br>water pollutants.|(20.1,20.2) None|To receive full point: Must have all indicators. CO2 must<br>be specified.<br>To receive half point:Must have at least one indicator.|
|21||Built Environment|(21.1) Proportion of<br>households with access to<br>water;<br>(21.2) Proportion of<br>households with access to<br>sanitation;<br>(21.3) Housing quality<br>indicators.|(21.1) Water supply type;<br>(21.2) Sanitation facility type;<br>(21.3) Number of houses by<br>type; number of rooms;<br>Houses by construction<br>material; Houses by piping<br>type; other.|To receive full point: Must have (21.1) and (21.2); as well as<br>(21.3) with one disaggregation.<br>To receive half point: Must have either (21.1) and (21.2), or<br>(21.3) with at least one disaggregation.|

<mark>Source: Authors own compilation from ODIN 2018/19 Methodology Report.</mark> <u><mark>https://docs.google.com/document/d/1ubPL1l_3im9bjlCVZ6W2ICAy6UAiXl1hGeA1aXImkxI/edit</mark></u> <mark>*Scoring options for elements 2 to 10 are presented in Table A4.</mark> For the elements of data openness, scoring is calculated independent of the data coverage.

40
|Table A4: ODIN 2018<br>Dimension<br>Element|/2019 scoringoptions<br>Scoringoptions|Notes|
|---|---|---|
|Coverage*<br>2. Data are available for<br>the preceding five years|1 point if all published data are available for 3 of<br>the last 5 years.<br>0.5 points if some published data are available for<br>1-2 of the last 5 years.|Scores for this element cannot be greater than the<br>score for coverage element 1. Scores are given by data<br>category, not indicator.|
|3. Data are available for<br>the preceding ten years|1 point if all published data are available for 6 of<br>the last 10 years.<br>0.5 points if some published data are available for<br>3-5 of the last 5 years.|Scores for this element cannot be greater than the<br>score for coverage element 1. Scores are given by data<br>category, not indicator.|
||0 points if all published data are unavailable for 2<br>or fewer of the last 10years.||
|4. Data are<br>disaggregated at the<br>first administrative<br>level|1 point if all published data in a data category are<br>available at first administrative level.<br>0.5 points if some published data in a data<br>category are available at first administrative level.<br>0points if no data are available at this level|Scores for this element cannot be greater than the<br>score for coverage element 1. Scores are given by data<br>category,<br>not<br>indicator.<br>Additionally,<br>data<br>disaggregated at the first administrative level is only<br>scored if national level data also exists for that<br>indicator.|
|5. Data are<br>disaggregated at the<br>second administrative<br>level|1 point if all data in a data category are available<br>at second administrative level.<br>0.5 points if some published data are available at<br>second administrative level.<br>|Scores for this element cannot be greater than the<br>score for coverage element 1. Scores are given by data<br>category,<br>not<br>indicator.<br>Additionally,<br>data<br>disaggregated at the second administrative level are<br>only scored if national level data also exists for that<br>indicator.|
||0points if no data are available at this level.||
|Openness<br>1. Machine readability|1 point if all published data are available in a<br>machine-readable format (such as XLS, XLSX, CSV,<br>Stata, SAS, SPSS, JSON and so forth).<br>0.5 points if some published data are available in<br>machine-readable format.<br>0 points if all published data are not available in<br>machine-readable format.|Scores are not penalized for having identical data sets<br>in both machine readable and non-readable formats.<br>Compression formats do not affect machine<br>readability scores, only non-proprietary scores (see<br>next page). Scores are given by data category, not<br>indicator.|
|2. Non-proprietary|1 point if all published data are available in non-<br>proprietary format (such as XLSX, DOCX, CSV, XML,<br>HTML, and JSON).<br>0.5 points if some published data are available in<br>non-proprietary format.|If data files are compressed in RAR format (which is<br>proprietary), data for that indicator should be<br>considered proprietary even if the enclosing files are<br>in a non-proprietary format. Files compressed in ZIP<br>format are not affected.|
||0 points if no data are available in a non-<br>proprietaryformat.<br>|**Table A4 continued**|

41
**Table A4 continued**

|Dimension|Element|Scoringoptions|Notes|
|---|---|---|---|
|Openness|3. Download options|1 point if all published data has a bulk download<br>option and an API or user-selectable download<br>option.<br>0.5 points if some published data has an API, bulk<br>download, or user-selectable download options.<br>0 points if no published data have any download<br>options|A bulk download is defined at the indicator level<br>as: The ability to download all data recorded in<br>ODIN for a particular indicator (all years,<br>disaggregations, and subnational data) in one file,<br>or multiple files that can be downloaded<br>simultaneously. Bulk downloads are a key<br>component of the Open Definition, which requires<br>data to be “provided as a whole . . . and<br>downloadable via the internet.”<br>User-selectable download options are defined as:<br>Users must be able to select an indicator and at<br>least one other dimension to create a download or<br>table. These dimensions could include time<br>periods, geographic disaggregations, or other<br>recommended disaggregations. An option to<br>choose the file export format is not enough.<br>API stands for Application Programming Interface.<br>Ideally, APIs should be clearly displayed on the<br>website. ODIN assumes APIs are available for the<br>NSOs entire data collection used in ODIN, unless<br>clearly stated. ODIN assessors do not register for<br>use or test API functionality. Scores are given by<br>data category,not indicator.|
||4. Metadata available|1 point if all published data have complete metadata.<br>0.5 points if some published data have complete or<br>incomplete metadata, or all published data have<br>incomplete metadata.<br>0 points if all published data have no metadata.|ODIN classifies metadata into three categories: (1)<br>Not Available, (2) Incomplete, and (3) Complete.<br>The following must be available to classify<br>metadata as complete:<br>• Definition of the indicator, or definition of key<br>terms used in the indicator description (as<br>applicable), or how the indicator was calculated.<br>• Publication (date of upload), compilation date<br>(date on front of report is not sufficient), or date<br>data set was last updated.<br>• Name of data source (what agency collected the<br>data).<br>If the metadata only have one or two of the above<br>elements, they are scored as incomplete|

**<mark>Table A4 continued</mark>**

42
# **Table A4 continued**

|Element<br>category|Element|Scoring options|Notes|
|---|---|---|---|
|Openness|5. Terms of use|1 point if all published data have a terms of use<br>classified as open.<br>0.5 points if some published data have a terms of use<br>classified as open, or if some published data has a terms<br>of use classified as semi-restrictive or if all published<br>data has a terms of use classified as semi-restrictive.<br>0 points If no terms of use are found or if all published<br>data have a terms of use classified as restrictive.|Generally, terms of use (TOU) will apply to an<br>entire website or data portal (unless otherwise<br>specified). In these cases, all data found on the<br>same website and/or portal will receive the same<br>score. If a portal is located on the same domain as<br>the NSO website, the terms of use on the NSO site<br>will apply. If the data are located on a portal or<br>website on a different domain, another terms of<br>use will need to be present. For a policy/ license to<br>be accepted as a terms of use, it must clearly refer<br>to the data found on the website. Terms of use<br>that refer to nondata content (such as pictures,<br>logos, etc.) of the website are not considered. A<br>copyright symbol at the bottom of the page is not<br>sufficient. A sentence indicating a recommended<br>citation format is not sufficient.<br>Terms of use are classified the following ways: (1)<br>Not Available, (2) Restrictive, (3) Semi-Restrictive,<br>and (4) Open.|
|Source:<br>https://docs.<br>*Scoring opti<br>it is presente<br>For the elem|Authors<br>own<br>google.com/document/d/1<br>ons for coverage element 1<br>d in Annex Table A2.<br>ents of data openness, scori|compilation<br>from<br>ODIN<br>ubPL1l_3im9bjlCVZ6W2ICAy6UAiXl1hGeA1aXImkxI/edit<br>- Representative indicators are available and are disaggre<br>ng is calculated independent of the data coverage.|2018/19<br>Methodology<br>Report.<br>gated appropriately - varies by each indicator. Hence|

43
|Table A5: O<br>**Variable**|DB data ca<br>**Short**<br>**Name**|tegories covered in<br>**Long Name**|technical survey<br>**Description**|
|---|---|---|---|
|**ODB.2013.D1**|Map|Mapping data|A detailed digital map of the country provided by a national mapping agency<br>and kept updated with key features such as official administrative borders,<br>roads and other important infrastructure. Please look for maps of at least a<br>scale of 1:250,000 or better (1cm = 2.5km).|
|**ODB.2013.D2**|Land|Land ownership data|A data set that provides national level information on land ownership. This<br>will usually be held by a land registration agency, and usually relies on the<br>existence of a national land registration database.|
|**ODB.2013.D4**|Stats|National statistics|Key national statistics such as demographic and economic indicators (GDP,<br>unemployment, population, etc), often provided by a National Statistics<br>Agency. Aggregate data (e.g. GDP for whole country at a quarterly level, or<br>population at an annual level)is considered acceptable for this category.|
|**ODB.2013.D5**|Budget|Detailed budget data|National government budget at a high level (e.g. spending by sector,<br>department etc). Budgets are government plans for expenditure, (not<br>details of actual expenditure in the past which is covered in the spend<br>category).|
|**ODB.2013.D6**|Spend|Government spend data|Records of actual (past) national government spending at a detailed<br>transactional level; at the level of month to month government expenditure<br>on specific items (usually this means individual records of spending<br>amounts under $1m or even under $100k). Note: A database of contracts<br>awarded or similar is not sufficient for this category, which refers to detailed<br>ongoingdata on actual expenditure.|
|**ODB.2013.D7**|Company|Company registration<br>data|A list of registered (limited liability) companies in the country including<br>name, unique identifier and additional information such as address,<br>registered activities. The data in this category does not need to include<br>detailed financial data such as balance sheet etc.|
|**ODB.2013.D8**|Legislation|Legislation data|The constitution and laws of a country, including national laws and statutes<br>but excluding case-law and administrative regulations.|
|**ODB.2013.D9**|Transport|Public transport<br>timetable data|Details of when and where public transport services such as buses and rail<br>services are expected to run. Please provide details for both bus and rail<br>services if applicable. If no national data is available, please check and<br>provide details related to the capital city.|
|**ODB.2013.D10**|Trade|International trade data|Details of the import and export of specific commodities and/or balance of<br>trade data against other countries.|
|**ODB.2013.D11**|Health|Health sector<br>performance data|Statistics generated from administrative data that could be used to indicate<br>performance of specific services, or the healthcare system as a whole. The<br>performance of health services in a country has a significant impact on the<br>welfare of citizens. Look for ongoing statistics generated from<br>administrative data that could be used to indicate performance of specific<br>services, or the healthcare system as a whole. Health performance data<br>might include: Levels of vaccination; Levels of access to health care; Health<br>care outcomes for particular groups; Patient satisfaction with health<br>services|
|**ODB.2013.D12**|Education|Primary and secondary<br>education performance<br>data|.<br>The performance of education services in a country has a significant impact<br>on the welfare of citizens. Look for ongoing statistics generated from<br>administrative data that could be used to indicate performance of specific<br>services, or the education system as a whole. Performance data might<br>include: Test scores for pupils in national examinations; School attendance<br>rates; Teacher attendance rates. Simple lists of schools do not qualify as<br>educationperformance data.|

44
# **Table A5 continued.**

|**Variable**|**Short Name**|**Long Name**|**Description**|
|---|---|---|---|
|**ODB.2013.D13**|Crime|Crime statistics data|Annual returns on levels of crime and/or detailed crime reports.Crime<br>statistics can be provided at a variety of levels of granularity, from annual<br>returns on levels of crime, to detailed real-time crime-by-crime reports<br>published online andgeolocated,allowingthe creation of crime maps.|
|**ODB.2013.D14**|Environment|National environmental<br>statistics data|Data on one or more of: carbon emissions, emission of pollutants (e.g.<br>carbon monoxides, nitrogen oxides, particulate matter etc.), and<br>deforestation. Pleaseprovide links to sources for each if available.|
|**ODB.2013.D15**|Elections|National election results<br>data|Results by constituency / district for the most all national electoral contests<br>over the last ten years.|
|**ODB.2013.D16**|Contracting|Public contracting data|Details of the contracts issued by the national government.|

45
|Table A6: GO<br>**Category**|DI data categories covered in techni<br>**What we look at?**|cal survey<br>**Why we look at it?**|**Characteristics**|
|---|---|---|---|
|Budget|National government budget at a high<br>level. This is planned government<br>expenditure for the upcoming year, and<br>not the actual expenditure.To develop<br>this category the Index drew on work<br>fromOpen Spending.|Open budget data allows for<br>well-informed publics. It<br>showing what money is spent<br>on, how public funds develop<br>over time, and why certain<br>activities are funded. See<br>here a list of cases how<br>budget data has been used in<br>the past.|Following data must be online to qualify for<br>assessment:<br>•<br>Budget for each national<br>government department, ministry,<br>or agency<br>•<br>Descriptions for budget sections<br>•<br>Level of granularityBudget<br>separated into sub-department,<br>political program, or expenditure<br>type|
|Spending|Records of actual (past) national<br>government spending at a detailed<br>transactional level. Data must display<br>ongoing expenditure, including<br>transactions. A database of contracts<br>awarded or similar will not be<br>considered sufficient. Also, a database<br>only showing subsidies will not be<br>sufficient. To develop this category the<br>Index drew on work fromOpen<br>Spending.|Open spending data shows<br>whether public money is<br>efficiently and effectively<br>used. It helps to understand<br>spending patterns and to<br>display corruption, misuse,<br>and waste.|Following data must be online to qualify for<br>assessment:<br>•<br>Government office which had the<br>transaction<br>•<br>Date of transaction<br>•<br>Name of vendor<br>•<br>Nominal amount of individual<br>transactionLevel of granularity<br>•<br>Individual record of each<br>transaction|
|Procurement|All tenders and awards of the<br>national/federal government<br>aggregated by an office. It does not look<br>into procurement planning or other<br>procurement phases such as<br>implementation (i.e. actual money<br>transfers, which are part of our<br>spending category). To develop this<br>category the Index drew on work from<br>theOpen Contracting Partnership.|Open procurement data may<br>enable fairer competition<br>among companies, allow to<br>detect fraud, as well as<br>deliver better services for<br>governments and citizens.<br>Monitoring tenders helps<br>new groups to participate in<br>tenders and to increase<br>government compliance.|Following data must be online to qualify for<br>assessment:<br>**Tender phase**<br>•<br>Tenders per government office<br>•<br>Tender name<br>•<br>Tender description<br>•<br>Tender status<br>**Award phase**<br>•<br>Awards per government office<br>•<br>Award title<br>•<br>Award description<br>•<br>Value of the award<br>•<br>Supplier's name|
|Election<br>results|This data category looks at results for<br>the latest national electoral contest.<br>Election data informs about voting<br>outcomes and voting process. What are<br>electoral majorities and minorities?<br>How many votes are registered, invalid,<br>or spoilt? The Index consulted the<br>National Democratic Institute (NDI) to<br>develop this data category, but did not<br>take their latest recommendation which<br>will be considered for the next edition.<br>For more information, see the<br>NDI’sOpen Elections Data Initiative.|To enable the highest level of<br>transparency, the Index<br>assesses polling station-level<br>data. Polling stations are the<br>locations at which voters cast<br>their vote. Having this data<br>allows for independent<br>scrutiny of each stage of the<br>voting and counting process.<br>It also helps electoral<br>stakeholders better target<br>their voter education and<br>mobilization efforts for the<br>next elections.|Following data must be online to qualify for<br>assessment:<br>•<br>Results for major national<br>electoral contests (such as general<br>elections)<br>•<br>Number of registered votes<br>•<br>Number of invalid votes<br>•<br>Number of spoiled votes (not<br>required, if a digital voting system<br>is assessed, that does not<br>recognize spoiled votes) Level of<br>granularity<br>•<br>Data available at polling station<br>level|

**Table A6 continued.**

46
# **Table A6 continued**

|**Category**|**What we look at?**|**Why we look at it?**|**Characteristics**|
|---|---|---|---|
|Company<br>register|Lists of registered (limited liability)<br>companies. The submissions in this data<br>category do not need to include<br>detailed financial data such as balance<br>sheets.This category draws on the work<br>ofOpenCorporates.|Open data from company<br>registers may be used to<br>many ends: enabling<br>customers and businesses to<br>see with whom they deal, or<br>to see where a company has<br>registered offices.|Following data must be online to qualify for<br>assessment:<br>•<br>Name of company<br>•<br>Company address<br>•<br>Unique identifier of the company<br>•<br>Register available for entire<br>country (usually assessed through<br>sample: it is answered with „Yes“<br>if a register indicates companies in<br>|
||||different regions)|
|Land<br>ownership|Maps of lands with parcel layer that<br>displays boundaries. Also a land registry<br>with information on registered parcels<br>of land.The assessment criteria were<br>developed in collaboration with Cadasta<br>Foundation. For more information on<br>land ownership data sets, see Cadasta<br>Foundation's Data Overview.|The Index focuses on<br>assessing open land tenure<br>data (describing the rules and<br>processes of land property).<br>Responsible use may enable<br>tenure security and increase<br>the transparency of land<br>transactions.|The following characteristics must be<br>included in cadastral and registry<br>information submitted.<br>•<br>Parcel boundaries<br>•<br>Parcel ID<br>•<br>Property Value (price paid for<br>transaction or tax value)<br>•<br>Tenure Type (public, private,<br>customary,etc.)|
|National maps|A geographical map of the country<br>including national traffic routes,<br>stretches of water, and markings of<br>heights. The map must at least be<br>provided at a scale of 1:250,000 (1 cm =<br>2.5km), a scale feasible for most<br>countries. The Index developed this<br>category basedon a landmark report of<br>the United Nations Committee of<br>Experts on Global Geospatial<br>Information Management (UNGGIM).|Geographic information is<br>instrumental for many use<br>cases, including journey<br>planning, the mapping of<br>topography, as well as<br>demographic indicators.|Following data must be online to qualify for<br>assessment:<br>•<br>Markings of national traffic routes<br>•<br>Markings of relief/heights<br>•<br>Markings of water stretches<br>•<br>National borders Coordinates -<br>Note: To qualify, data must<br>contain geographic projections<br>that enable to interpret<br>coordinates|
|Administrative<br>Boundaries|Data on administrative units or areas<br>defined for the purpose of<br>administration by a (local)<br>government.The development of this<br>category draws on work ofFAO Global<br>Administrative Unit Layers<br>(GAUL)project, as well as theUNGIWG.|Open data about<br>administrative zones has<br>many use cases: Who are the<br>candidates in my region?<br>Which government bodies<br>administer my region? How is<br>wealth distributed across<br>regions? The Index assesses<br>two administrative boundary<br>levels (e.g. federal states =<br>level 1, and municipalities =<br>level 2).|Following data must be online to qualify for<br>assessment:<br>•<br>Boundary level 1<br>•<br>Boundary level 2 (not required, if<br>country has only one level)<br>•<br>Coordinates of administrative<br>zone (latitude, longitude)<br>•<br>Name of polygon<br>•<br>Borders of polygon - Note: To<br>qualify, data must contain<br>geographic projections that enable<br>to interpret coordinates|

**Table A6 continued.**

47
**Table A6 continued**

|**Category**|**What we look at?**|**Why we look at it?**|**Characteristics**|
|---|---|---|---|
|Locations|A database of postcodes/zipcodes and<br>the corresponding spatial locations<br>regarding latitude and longitude (or<br>similar coordinates in an openly<br>published coordinate system). The data<br>has to be available for the entire<br>country. The Index drew on work of<br>theUniversal Postal Union to develop<br>this category.|Open location data shows<br>the addresses of public and<br>private buildings. While<br>mainly used to route postal<br>services, this data has many<br>use cases: to calculate the<br>number of persons in a city<br>district, to provide homes<br>with services, or for direct<br>mailing and marketing.|Following data must be online to qualify for<br>assessment:<br>•<br>Zipcodes Addresses (required, if<br>zip code does not include the<br>address)<br>•<br>Coordinates (latitude, longitude)<br>•<br>Data available for entire country -<br>Note: To qualify, data must<br>contain geographic projections<br>that enable to interpret<br>coordinates|
|National<br>statistics|Key national statistics on demographic<br>and economic indicators such as Gross<br>Domestic Product (GDP), or<br>unemployment and population<br>statistics. These statistics can be<br>published as aggregates for the entire<br>country.|As Open Data Watch states<br>"Official statistics provide an<br>indispensable element in the<br>information system of a<br>democratic society, serving<br>the Government, the<br>economy and the public with<br>data about the economic,<br>demographic, social and<br>environmental situation."|Following data must be online to qualify for<br>assessment:<br>•<br>Country Population (Required:<br>census data, updated every year,<br>Optional: vital statistics of birth<br>and death)<br>•<br>Gross Domestic Product<br>(measured in current or constant<br>prices, updated quarterly, last<br>update must not be more than 3<br>months ago)<br>•<br>National unemployment (absolute<br>numbers, or expressed as<br>percentage of entire population,<br>updated quarterly, last update<br>must not be more than 3 months<br>ago)|
|Draft<br>legislation|Data about the bills discussed within<br>national parliament as well as votes on<br>bills (not to be confused with passed<br>national law). Data on bills must be<br>available for the current legislation<br>period. This data category draws on<br>work by the National Democratic<br>Institute (NDI) and theDeclaration of<br>Parliamentary Openness.|Open data on the law-making<br>process is crucial for<br>parliamentary transparency:<br>What does a bill text say and<br>how does it change over<br>time? Who introduces a bill?<br>Who votes for and against it?<br>Where is a bill discussed next<br>so that the public can<br>participate in debates?|Following data is required. It must be online<br>for the data to qualify for assessment:<br>•<br>Content of bill<br>•<br>Author of bill<br>•<br>Status of bill<br>•<br>Available for current legislation<br>period<br>Following data is assessed optionally (only if<br>available):<br>•<br>Votes on bill per member of<br>parliament<br>•<br>Transcripts of debates on bill<br>•<br>Note on optional data: This category is<br>newly added in 2016. Not all data needs to<br>be available online to qualify. The Index<br>team used minimum requirements to<br>explores how much data is currently<br>available online. In future editions, the<br>categorymayrequire more data elements.|

**Table A6 continued.**

48
# **Table A6 continued**

|**Category**|**What we look at?**|**Why we look at it?**|**Characteristics**|
|---|---|---|---|
|National law|This data category requires all national<br>laws and statutes to be available online,<br>although it is not a requirement that<br>information on legislative behaviour e.g.<br>voting records is available.This data<br>category draws on work by the National<br>Democratic Institute (NDI) and<br>theDeclaration of Parliamentary<br>Openness.|Access to open data on a<br>country's legal code (i.e.<br>national law) supports<br>compliance with law, enables<br>to keep track of legal<br>changes, and also enables<br>public deliberation around a<br>law.|Following data must be online to qualify for<br>assessment:<br>•<br>Content of the law / status<br>•<br>Date of last amendment<br>•<br>Amendments to the law (if<br>applicable)|
|Air quality|Data about the daily mean<br>concentration of air pollutants,<br>especially those potentially harmful to<br>human health. Data should be available<br>for all air monitoring stations or zones in<br>a country, including at least 3 major<br>cities. The Index evaluates the openness<br>of key pollutants as defined by<br>theWorld Health Organisation (WHO).|Air quality is a key factor for<br>human health and<br>environment.|Following data must be online to qualify for<br>assessment:<br>•<br>Particulate matter (PM)<br>•<br>Sulphur oxides (SOx)<br>•<br>Nitrogen oxides (NOx)<br>•<br>Carbon monoxide (CO)<br>•<br>Ozone (O3)<br>•<br>Available per air monitoring<br>station (at least for 3 major cities<br>Following data is assessed optionally (if<br>available):Volatile organic compounds<br>(VOCs)|
|Water quality|Water quality data by water source. The<br>data category regards the quality of<br>designated drinking water sources. If<br>data on designated drinking water<br>sources is not available, it refers to<br>environmental water sources (lakes,<br>rivers, groundwater). Data per each<br>water source is desirable. But for this<br>year the Index also accepted if a country<br>only published country-wide aggregated<br>reports. As the review shows, we either<br>find local and granular data or<br>aggregated national reports.|This information is essential<br>for both the delivery of<br>services and the prevention<br>of diseases.|In order to satisfy the minimum<br>requirements for this category, data should<br>be available on level of the following<br>chemicals:<br>•<br>Fecal coliform<br>•<br>Arsenic<br>•<br>Fluoride levels<br>•<br>Nitrates<br>•<br>Total Dissolved Solids<br>•<br>Data per water source<br>•<br>Available for the entire country|

49
|Table A7: GOD<br>**Question**|I survey question and scoring<br>**Description**|**Score**|**Rationale**|
|---|---|---|---|
|Is the data<br>collected by<br>government (or a<br>third-party<br>related or linked<br>to government)?|Answer “Yes” if the chosen data is collected<br>by the government, or a third party is<br>officially representing the government. This is<br>the case for state-owned-enterprises or<br>contractors delivering public services for<br>government. Answer “No” if one of the<br>following cases apply: i) The data is collected<br>by organisations that do not represent<br>government; ii) The data is collected but not<br>for the relevant government level; iii) The<br>data is not collected at all|Not<br>scored|Data collection by itself is not a characteristic of ‘open’<br>data. Our knowledge of edge cases or exceptions from<br>the rule (such as legal arrangements of data publication in<br>cases of public-private partnerships) is too limited to<br>develop valid statements about a reasonable scoring.|
|Is the data<br>available online<br>without the need<br>to register or<br>request access to<br>the data?|Answer “Yes”, if the data is made available by<br>the government on a public website. Answer<br>“No” if the data are NOT available online or<br>are available online only after registering,<br>requesting the data from a civil servant via<br>email, completing a contact form or another<br>similar administrativeprocess.|15<br>points|Online availability is a requirement for openness:<br>everyone has to have online access to specific data.<br>Furthermore, it is a condition for all following questions<br>and mandatory registration can deter people from using<br>data (focus on user perspective). We put emphasis on the<br>additional requirement that data must also be available<br>without mandatoryregistration|
|Is the data<br>available online<br>at all?|Tell us if the data is available online at all<br>(after registering, after getting authentication.|Not<br>scored|We currently do not aim to reward mandatory<br>registration. Administrative processes may entail terms of<br>use that contradict open data: such as agreeing to terms<br>of use. A zero score is a indicates to governments that<br>their way of online publication is not ideal for all user<br>groups.|
|Is the data<br>available free of<br>charge?|The data is free if you don’t have to pay for it.|15<br>Points|Data has to be for free in order to be accessible to<br>everyone. We cannot expect users to pay for data sets in<br>order to evaluate them for us. Some data (especially<br>when provided in machine-readable file formats) have to<br>bepaid for.|
|Where did you<br>find the data?|Indicate a URL and a description of the URL.<br>Example: If you find data on a financial<br>department website, please fill in: “Website<br>of National Department of Finances”.<br>Sometimes you can find data in a lot of places<br>in the web. To limit your search, tell us the<br>first 5 URLs you can easily find for each source<br>type. Make sure the URLs are from an official<br>government source.|Not<br>scored|This is a subjective assessment. The results may be<br>affected by a submitter's topical expertise or familiarity<br>with government websites.|
|How much do<br>you agree with<br>the following<br>statement: “It<br>was easy for<br>me to find the<br>data.”<br>|Submitters answer with a Likert scale.<br>|Not<br>scored<br>|This is a subjective assessment. The results may be<br>affected by a submitter's topical expertise or<br>familiarity with government websites. We<br>experiment with the results to develop a better<br>findabillity assessment.<br>**Table A7 continued.**|

50
# **Table A7 continued**

|**Question**|**Description**|**Score**|**Rationale**|
|---|---|---|---|
|Is the data<br>downloadable at<br>once?|Answer “Yes”, if you can download all data at<br>once from the URL at which you found them.<br>In case that downloadable data files are very<br>large, their downloads may also be organised<br>by month or year or broken down into sub-<br>files. Answer “No” if you have to do many<br>manual steps to download the data, or if you<br>can only retrieve very few parts of a large<br>data set at a time (for instance through a<br>search interface).|15<br>Points|We score if a data set can be downloaded at once. This<br>question therefore rewards the technical possibility to<br>retrieve all data from the internet without having to<br>download dozens of small pieces of information, getting<br>access to data through a search interface only, sending<br>requests, having captchas or other limits to download.<br>Important note: data may be split into smaller sub-sets.<br>This applies for instance for long time series, or large<br>geospatial data. It is important that these sub-sets are<br>logically linked, and that it is possible to retrieve data<br>automaticallyfrom one or several URLs.|
|Data should be<br>updated every<br>[Time Interval]: Is<br>the data up-to-<br>date?|Please base your answer on the date at which<br>you answer this question. Answer “No” if you<br>cannot determine a date, or if the data are<br>outdated.|15<br>Points|Some of the data we assess are most valuable right after<br>their releases such as short-term weather forecasts,<br>election results or budget data. Timely provision of these<br>data is crucial. - Some data is not as time sensitive as<br>others. Our scoring wants to strike a balance between<br>both cases and therefore amounts to 15 points, in order<br>|
||||not to avoid an over-emphasis of this category.|
|Is the data openly<br>licensed/in public<br>domain?<br>|This question measures if anyone is legally<br>allowed to use, modify and redistribute data<br>for any purpose. Only then data is considered<br>truly "open" (see Open Definition).<br>•<br>Answer ”Yes” if the data are openly<br>licensed. The Open Definition<br>provides a list of conformant<br>licenses. Also, consult the terms of<br>use which often indicate whether<br>data can be freely reused<br>•<br>Answer “Yes” if there is no open<br>license, but a statement that the<br>data set is in “public domain”. To<br>count as public domain, the data set<br>must not be protected by copyright,<br>patents or similar restrictions.<br>•<br>If you are not sure whether an open<br>licence or public domain notice is<br>compliant with the Open Definition<br>2.1, seek feedback on the Open<br>Data Index discussion forum.<br>Answer “No” whenever it is not fully evident<br>that the license or terms of use are compliant<br>with the Open Definition.<br>|20<br>Points<br>|Legal usability of data is a core requirement of the open<br>definition. It is a prerequisite for unrestricted usability for<br>everyone. Our old scoring was fairly high, emphasizing the<br>legal usability of data. The current scoring is lowered to<br>give us some space to stress other aspects of openness.<br>This question will not lose its significance for openness<br>(still scored higher than in the Open Data Barometer)<br>**Table A7 continued.**|

51
**Table A7 continued**

|**Question**|**Description**|**Score**|**Rationale**|
|---|---|---|---|
|Is the data in<br>open and<br>machine-<br>readable file<br>formats?|We automatically compare them against a list<br>of file formats that are considered machine-<br>readable and open. A file format is called<br>machine-readable if your computer can<br>process, access, and modify single elements in<br>a data file.The Index considers formats to be<br>“open” if they can be fully processed with at<br>least one free and open-source software tool.<br>Potentially these formats allow more people<br>to use the data because people do not need<br>to buy specific software to open it. The source<br>code of these format does not have to be<br>open.|20<br>points|Both features (machine-readable and open format) are<br>key aspects of the open definition. Machine-readability is<br>a major enhancement of technical usability. However, if a<br>file is only usable with proprietary software (such as<br>ArcGIS) ‘normal’ users are exempt from using them. Open<br>formats put no copyright, monetary restrictions or other<br>restrictions on their use (important for people who<br>cannot / do not want to afford proprietary software).|
|How much<br>human effort is<br>required to use<br>the data. (1 =<br>little to no effort<br>is required, 3 =<br>extensive effort is<br>required)|The submitters tell us their use case and the<br>steps they took to make the data usable<br>(example: “I have to reformat the data”).|Not<br>scored|The question is a subjective assessment. Furthermore,<br>usability depends on context and the purposes for which<br>a person wants to use the data.|

Table A8: Weights of Features of the MENA Micro Access Indicator (MENA MAI)

|**Element**|**Feature**|**Weight in overall score**|
|---|---|---|
|User experience|English translation|1/6<br>1/2|
||Micro data tab/dashboard/library|1/6|
||Survey calendar|1/6|
|Data openness|Firm data|1/14<br>1/2|
||Price data|1/14|
||Consumption data|1/14|
||Labor Force data|1/14|
||Health data|1/14|
||Population Census|1/14|
||Economic Census|1/14|

52