metadata: authors: 0: author: Seitz,William Hutchins
count: Central Asia
docna: 0: docna: Where They Live : District-Level Measures of
            Poverty, Average Consumption, and the Middle Class in
            Central Asia
docty: Policy Research Working Paper
subtopic: Inequality,Industrial Economics,Economic Theory & Research,Economic Growth,Employment and Unemployment,Educational Sciences,Social Protections & Assistance
teratopic: Poverty Reduction,Industry,Macroeconomics and Economic Growth,Education,Social Protections and Labor
abstracts: cdata!: Rapid economic growth over the past two
            decades lifted millions of people out of poverty in Central
            Asia. But the uneven spread of prosperity left many
            communities struggling to catch up. To support lagging
            regions within countries, each of the region's five
            national governments has made convergence a pillar of their
            development strategies. An imperfect patchwork of household
            surveys allows policy makers to monitor progress and
            identify some spatial disparities. But these share an
            important weakness: none of the official surveys in the
            region is representative when disaggregated to the level of
            districts. Islands of poverty and prosperity are thus lost
            in the averages -- leading to targeting inaccuracies that
            can slow the pace of poverty reduction. This study partially
            addresses the challenge. The accuracy of key welfare
            indicators is sharpened well beyond what could be achieved
            for any country alone by: i) unifying survey data from
            across the region and ii) applying the techniques of
            small-area estimation. The results provide detailed measures
            of welfare that in turn can be disaggregated for each
            district in Central Asia. Comprehensive maps of where the
            poor and the middle class live are presented, for the entire
            region and individually for each country.
alt_title: 0: alt_title: Где они живут : Показатели уровней
            бедности, среднего потребления и среднего класса в
            Центральной Азии на районном уровне
repnme: repnme: Where They Live : District-Level Measures
            of Poverty, Average Consumption, and the Middle Class in
            Central Asia
display_title: Where They Live : District-Level
            Measures of Poverty, Average Consumption, and the Middle
            Class in Central Asia
listing_relative_url: /research/2019/07/31240980/live-district-level-measures-poverty-average-consumption-middle-class-central-asia
topicv3: Economic Growth m1326241 471,Poverty m1326687 695,Education m1327048 470,Social Protection and Labor m1327194 697,Census Data m1504073 2763
publishtoextweb_dt: 2019-07-11T00:00:00Z
historic_topic: Poverty Reduction,Industry,Macroeconomics and Economic Growth,Education,Social Protections and Labor
no_of_pages: 82
author_keys: 0: author_key: 1298530
fullavailablein: node_id: 31240980
model_extractions: Policy Research Working Paper 8940

# Where They Live

District-Level Measures of Poverty, Average Consumption, and the Middle Class in Central Asia _William Seitz_ Poverty and Equity Global Practice July 2019
Policy Research Working Paper 8940

# **Abstract**

Rapid economic growth over the past two decades lifted millions of people out of poverty in Central Asia. But the uneven spread of prosperity left many communities struggling to catch up. To support lagging regions within countries, each of the region’s five national governments has made convergence a pillar of their development strategies. An imperfect patchwork of household surveys allows policy makers to monitor progress and identify some spatial disparities. But these share an important weakness: none of the official surveys in the region is representative when disaggregated to the level of districts. Islands of poverty and prosperity are thus lost in the averages—leading to targeting inaccuracies that can slow the pace of poverty reduction. This study partially addresses the challenge. The accuracy of key welfare indicators is sharpened well beyond what could be achieved for any country alone by: i) unifying survey data from across the region and ii) applying the techniques of small-area estimation. The results provide detailed measures of welfare that in turn can be disaggregated for each district in Central Asia. Comprehensive maps of where the poor and the middle class live are presented, for the entire region and individually for each country.

This paper is a product of the Poverty and Equity Global Practice. It is part of a larger effort by the World Bank to provide open access to its research and make a contribution to development policy discussions around the world. Policy Research Working Papers are also posted on the Web at http://www.worldbank.org/prwp. The author may be contacted at wseitz@worldbank.org.

_The Policy Research Working Paper Series disseminates the findings of work in progress to encourage the exchange of ideas about development issues. An objective of the series is to get the findings out quickly, even if the presentations are less than fully polished. The papers carry the names of the authors and should be cited accordingly. The findings, interpretations, and conclusions expressed in this paper are entirely those of the authors. They do not necessarily represent the views of the International Bank for Reconstruction and Development/World Bank and its affiliated organizations, or those of the Executive Directors of the World Bank or the governments they represent._ Produced by the Research Support Team
**Where They Live** District-Level Measures of Poverty, Average Consumption, and the Middle Class in Central Asia

William Seitz<sup>1</sup> Keywords: Poverty mapping. Fay-Herriot, Small Area Estimation, Central Asia, Regional Development JEL Codes: I30, I32 R12

> 1 Poverty and Equity Global Practice; the World Bank; wseitz@worldbank.org
# **ACKNOWLEDGMENTS**

This report is a product of the World Bank Poverty and Equity Program for Central Asia, led by William Seitz (Economist, GPV03). Maps and data files were generated with assistance from AvraltOd Purevjav (Consultant, GPV03). The approach used to derive the small area estimates in this report relies on a Stata command developed by Paul Andres Corral Rodas (Data Scientist, GPV03), William Seitz, Minh Cong Nguyen (Economist, GPV03) and Joao Pedro Azevedo (Lead Economist). Reviewers include: Bob Fay (Senior Scientist, Westat), Alexandru Cojocaru (Senior Economist, GPV03), Kenneth Simler (Senior Economist, GPV02), Kristen Himelein (Senior Economist, GPV02) and David Newhouse (Senior Economist, GPV07). Additional project support was provided by Alisher Rajabov (Economist, GPV03), and Saida Ismailakhunova (Senior Economist, GPV03). This task was completed under the guidance of Carlos Silva-Jauregui (Lead Economist and Acting Practice Manager, GPV03) 2
# Contents

|I – Introduction............................................................................................................................................. 7|
|---|
|**I.I – Definitions of Poverty in Central Asia**........................................................................................... 8|
|**I.II – The Middle Class in Kazakhstan, and Average Per Capita Consumption**.............................. 15|
|II – Data...................................................................................................................................................... 17|
|**II.I – Districts by Country**..................................................................................................................... 18|
|**II.II – Kazakhstan HBS data for 2017**.................................................................................................. 18|
|**II.III – Uzbekistan L2CU Baseline Survey 2018**................................................................................. 19|
|**II.IV – Kyrgyz Republic HBS 2016**....................................................................................................... 19|
|**II.V – Tajikistan L2T Baseline Survey for 2015**................................................................................... 20|
|**II.VI – Satellite and Administrative Data**............................................................................................ 20|
|III – Fay-Herriot Small Area Estimation Model........................................................................................... 21|
|IV – Results................................................................................................................................................. 23|
|V – Concluding Remarks and Example Applications.................................................................................... 28|
|VI – References........................................................................................................................................... 32|
|VII – Annexes.............................................................................................................................................. 33|
|**Annex A – Kazakhstan District-level Results 2017**.............................................................................. 33|
|**Annex B – Uzbekistan District-level Results 2018**.............................................................................. 40|
|**Annex C – Kyrgyz Republic District-level Results 2016**..................................................................... 46|
|**Annex D – Tajikistan District-level Results 2015**................................................................................ 48|
|**Annex E – Country-Level Maps**........................................................................................................... 51|
|**Annex F – Validation and Precision**..................................................................................................... 66|
|**Annex G – Comparison of Domain Variance Estimation Alternatives**............................................. 72|
|**Annex H – Regression Model Details**.................................................................................................. 74|
|**Annex I – Sensitivity to Model Selection Technique**.......................................................................... 76|
|**Annex J – K-fold Model Selection Sensitivity Analysis**....................................................................... 78|

3
# Tables

|**Table 1: Survey Data Used**........................................................................................................................ 18|
|---|
|**Table 2: Kazakhstan District-level Results 2017**...................................................................................... 33|
|**Table 3:Uzbekistan District-level Results 2018**....................................................................................... 40|
|**Table 4: Kyrgyz Republic District-level Results 2016**............................................................................. 46|
|**Table 5:Tajikistan District-level Results 2015**......................................................................................... 48|
|**Table 6: Fay-Herriot Model for Poverty Rate at $5.5/day and $3.2/day**............................................... 74|
|**Table 7: Fay-Herriot Model for share below middle-class line, and average consumption in 2011 PPP**<br>.................................................................................................................................................................... 75|

4
# Figures

|**Figure 1: Cross-Country Comparison of Income Class Poverty Rates**.................................................. 10|
|---|
|**Figure 2: Poverty rates in Kazakhstan using international poverty lines**.............................................. 11|
|**Figure 3: National Poverty Rates in Tajikistan (left-note not comparable to international poverty**<br>**estimates); Number of quarters in poverty (right)**.................................................................................. 12|
|**Figure 4: Poverty rates in the Kyrgyz Republic**....................................................................................... 13|
|**Figure 5: Poverty Rates by Region of Uzbekistan (2018)**....................................................................... 14|
|**Figure 6: UMIC Poverty Rate by Regions (Oblasts) in Central Asia**.................................................... 15|
|**Figure 7: The middle class share of the population, by regions (oblasts) of Central Asia**................... 16|
|**Figure 8: Average Daily Per Capita Consumption in 2011 PPP**............................................................. 17|
|**Figure 9: Poverty headcount ratio at $3.2-a-day PPP per person, rates for Central Asia**..................... 24|
|**Figure 10: Poverty at $5.5-a-day PPP per person, rates for Central Asia**............................................... 25|
|**Figure 11: Average Per Capita Consumption 2011 PPP**.......................................................................... 26|
|**Figure 12: Share of Population in the Middle Class in Central Asia**...................................................... 27|
|**Figure 13: Share of Households with at Least One Member Abroad for Work**.................................... 28|
|**Figure 14: District Share of HHs with Migrants by District Per Capita Consumption Quintile**......... 29|
|**Figure 15: Number of Early Learning Centers per 1000 Children**.......................................................... 30|
|**Figure 16: Number of Kindergartens per 1000 Children**......................................................................... 30|
|**Figure 17: Poverty headcount ratio at $5.5-a-day PPP per person, rates for Kazakhstan 2017**............ 51|
|**Figure 18: Share of the population in the middle class, rates for Kazakhstan 2017**.............................. 52|
|**Figure 19: Average Per Capita Daily Consumption in 2011 PPP for Kazakhstan 2017**......................... 53|
|**Figure 20: Poverty headcount ratio at $3.2-a-day PPP per person, rates for Uzbekistan 2018**............. 54|
|**Figure 21: Poverty headcount ratio at $5.5-a-day PPP per person, rates for Uzbekistan 2018**............. 55|
|**Figure 22: Share of population in the middle class, rates for Uzbekistan 2018**..................................... 56|
|**Figure 23: Average Per Capita Daily Consumption in 2011 PPP for Uzbekistan 2018**......................... 57|
|**Figure 24: Poverty headcount ratio at $3.2-a-day PPP per person, rates for Kyrgyzstan 2016**............. 58|
|**Figure 25: Poverty headcount ratio at $5.5-a-day PPP per person, rates for Kyrgyzstan 2016**............. 59|
|**Figure 26: Share of the population in the middle class, rates for Kyrgyzstan 2016**............................... 60|
|**Figure 27: Average Per Capita Daily Consumption in 2011 PPP for Kyrgyzstan 2016**.......................... 61|
|**Figure 28: Poverty headcount ratio at $3.2-a-day PPP per person, rates for Tajikistan 2015**............... 62|
|**Figure 29: Poverty headcount ratio at $5.5-a-day PPP per person, rates for Tajikistan 2015**............... 63|
|**Figure 30: Share of the population in the middle class, rates for Tajikistan 2015**................................. 64|
|**Figure 31: Average Per Capita Daily Consumption in 2011 PPP for Tajikistan 2015**............................ 65|
|**Figure 32: Upper and Lower-bound Confidence Intervals, Including Out-of-Sample Predictions**.... 66|
|**Figure 33: Comparison between ELL and FH**........................................................................................ 67|
|**Figure 34: Root Mean Square Error Before and After FH Approach (Poverty Headcount Ratio**|
|**$5.5/day - Left; Average per-capita consumption 2011 PPP – Right)**................................................... 68|
|**Figure 35: Comparison of Individual Models vs. Pooled Model (Only in sample - Left; Including out**|
|**of sample - Right)**...................................................................................................................................... 68|
|**Figure 36: Left: Strict VIF Threshold (5) vs. Adopted Threshold (10); Right: No VIF Threshold vs.**|
|**Adopted Threshold (1))**............................................................................................................................. 69|
|**Figure 37: Direct vs. Fay Herriot Estimates for in-sample Poverty $5.5day (left) and average**|
|**consumption (right)**.................................................................................................................................. 70|
|**Figure 38: Improvement of Coefficient of Variation in average consumption**..................................... 71|

5
|**Figure 39 comparison of variance estimates for average consumption**................................................. 72|
|---|
|**Figure 40: Comparison of variance estimates for poverty at $5.5/day**.................................................. 72|
|**Figure 41: Poverty Headcount Ratio (5.5/day) for Taylor Linearized Method vs. HT Method (Left),**|
|**Taylor Liberalized Method vs. Smoothed (Right)**.................................................................................. 73|
|**Figure 42: Lasso AIC vs. Stepwise**........................................................................................................... 76|
|**Figure 43: Lasso EBIC vs. Stepwise**........................................................................................................ 77|
|**Figure 44: Lasso BIC vs. Stepwise**........................................................................................................... 77|
|**Figure 45: Fold A Full Sample (Left); Withheld Sample (Right)**........................................................... 78|
|**Figure 46: Fold B Full Sample (Left); Withheld Sample (Right)**........................................................... 79|
|**Figure 47: Fold C Full Sample (Left); Withheld Sample (Right)**........................................................... 79|
|**Figure 48: Fold D Full Sample (Left); Withheld Sample (Right)**........................................................... 80|
|**Figure 49: Fold E Full Sample (Left); Withheld Sample (Right)**........................................................... 80|

6
# **I – Introduction**

Measurement of poverty rates, average per capita consumption, the size of the middle class, and other welfare indicators is traditionally conducted using survey data. To allow for frequent monitoring and to contain the costs of gathering detailed information, such surveys usually visit only a small sample of the population. When this sample of the population is representative, welfare surveys provide reliable estimates of poverty incidence for the entire population at a small fraction of the cost that would be required to survey each person in the country.

However, this approach necessarily leads to sampling errors, and consequently, a typical household income or expenditure survey cannot produce statistically reliable welfare estimates for small geographic units. This study starts with nationally (and regionally) representative data from each of the four Central Asian countries (Kazakhstan, Kyrgyzstan, Tajikistan and Uzbekistan) for which they are available. The analysis then proceeds to sharpen the reliability of the survey estimates to allow reporting at a level below what is traditionally reported (moving from “oblast” level estimates, to “rayon” level estimates). Because no such survey data are available for Turkmenistan, the country is excluded from the analysis.

There are two survey types used in the approach described in this report. In Kazakhstan and the Kyrgyz Republic, welfare estimates are derived from the Household Budget Surveys (HBS) conducted by the national statistical agencies. In Tajikistan and Uzbekistan, HBS data were either inappropriate for this analysis, or unavailable. Instead, the welfare estimates for these countries are derived from the baseline surveys for two World Bank studies: Listening to Tajikistan (L2T) and Listening to the Citizens of Uzbekistan (L2CU).

Each of these four surveys are designed to be representative at the national level in the respective country, for rural and urban areas, and at the region (oblast) level. For this reason, the national official rates of poverty monitored in each country are not usually published for administrative units below the regional level (with some exceptions). However, many government administrative activities are undertaken at the district level, and regions are a relatively high level of spatial aggregation for targeting policies that are sensitive to the needs of poor and vulnerable people. Likewise, policies that are intended to grow the middle class and average income could benefit from greater disaggregation than is directly available.

Small area estimation (SAE) techniques sharpen the precision of welfare measures to enable reporting at highly disaggregated geographic units. Using statistical models for imputation, SAE approaches provide estimates of indicators for small areas that would be impossible to reliably construct with traditional survey data alone. The results are often used to target policies and assign resources to have greater poverty-reducing impact or are intended to address the concerns of specific welfare groups at the local level. In many countries and regions, poverty maps have been used to highlight geographic variations, simultaneously display different dimensions of well-being, understand income determinants, and to both design and select interventions.

A variety of such methods have been devised to overcome the increasing imprecision of welfare estimates as they are disaggregated. The standard approach to SAE, used by the World Bank and applied in most cases when enough data are available, is described in Elbers, Lanjouw, and Lanjouw 7
(2003) and is often referred to as the “ELL” poverty mapping method. The assumptions and data employed for ELL maps are further elaborated upon in Bedi, Coudouel, and Simler (2007). However, a pre-requisite for using the ELL approach is access to micro-level census data. In Central Asia, census data either do not exist (for instance, in Uzbekistan), are relatively dated (for instance, in the Kyrgyz Republic and Tajikistan), or are not made available for these purposes (for instance, in Kazakhstan). In such cases, the most common alternative approach is the Fay-Herriot (FH) method, which is adopted to generate the results described in this report.

The FH method allows estimation of indicators and rates using a combination of survey data and district-level indicators from available sources that are less subject to imprecision, such as administrative data or remote sensing. In this report, most of the publicly available sources used are derived from satellite imagery. The FH approach proceeds by matching accurate area-based information with indicators that are aggregated to the level of interest in the survey (the district, in this case). Starting from the relatively imprecise estimates from the survey, a statistical model is developed, which attempts to explain the variation of the welfare indicator at the district level (in this case, either the poverty rate, average consumption per capita, or the share of the middle class at the district level). Once the model is estimated, the direct survey estimates also enter into the final area-level results: the final estimated area-level poverty rate is a weighted average of the observed and model-based estimates for cases in which both estimates are present. For areas that do not appear in the survey data, the results rely entirely on estimates derived from the statistical model.

The report is structured as follows. The remainder of this section reviews the definitions of the key indicators to be estimated. Section II reviews the data used in the study (relying on both survey microdata, and complementary satellite/administrative data). The approach used for the maps described in this report is discussed in more detail in section III. Section IV includes the district-level maps for Central Asia. Section V briefly discusses the results and gives two examples of uses for the maps (comparing migration rates to poverty rates and locating program locations on maps). The annexes include detailed tables of the results at the level of districts (rayons) throughout Central Asia, validation and robustness exercises, and the regression models used.

# **I.I – Definitions of Poverty in Central Asia**

The World Bank regularly produces internationally comparable estimates of poverty as part of its mandate. In 2017, the World Bank updated all such estimates to a new set of poverty lines, and at the same time, began applying up-to-date purchasing power parity (PPP) conversion factors for each country (based on estimates of price differences in 2011). The objective of calculating the World Bank’s internationally comparable poverty estimates is to estimate the prevalence of poverty in terms of a single global standard. Measures created on this basis are in turn used to monitor progress towards development goals set by the World Bank, the member states of the United Nations, and other partners.

The World Bank poverty estimates contrast with national official poverty estimates in Central Asia in several ways. The most common official poverty approaches in the region consider local patterns of 8
income or consumption and are often more appropriate for country-specific analysis. However, despite their many advantages, national official poverty measures are not comparable with the approaches used in other counties, in part because they are so tailored the specific country context. Thus, comparisons across countries (for instance, for the purposes of monitoring or benchmarking) require a harmonized approach, such as that conducted by the World Bank.

An additional reason that the World Bank uses a stand-alone approach to measure internationally comparable poverty rates is to account for differences in the cost of living between countries. To address this issue, the World Bank participates in the International Comparisons Program (ICP), a global effort to measure differences in the amount of goods and services a unit of one country’s currency can purchase in another country. The ICP exercise was most recently completed in 2011, which led to important updates to previous PPP measures.

The original dollar-a-day line was created in 1991 using 1985 PPPs and was estimated by taking an average of the national poverty lines in the world’s poorest countries. When a new set of PPPs was published in 1993, the line changed to $1.08 per day. PPPs were revised again in 2005, and the new line was estimated at $1.25.<sup>2</sup> For the most recent revision in 2011, the new line was estimated again using the national poverty lines of the poorest countries, as was the case in previous rounds. The result was a line set at $1.90 a-day. However, poverty by this measure is quite rare in Central Asia and is therefore excluded from the analysis that follows.

But in addition to the international “low income” poverty line, the World Bank also uses income class poverty lines which facilitate comparisons between countries at similar stages of development. The income class poverty lines are defined for the lower middle-income and upper middle-income countries and are based on the national poverty lines of the countries in each group. As such, they provide a more appropriate threshold to measure poverty for countries in each income class. The lines are defined at $3.2 (for lower middle-income countries) and $5.5 (for upper middle-income countries). The welfare measures of income or consumption used are the same as those used for the international poverty line. Cross-country comparisons of poverty measured at these lines are presented in Figure 1.

> 2 For more details on the updated lines from Francisco Ferreira, Dean Joliffe and Espen Prydz, follow this link

9
**Figure 1: Cross-Country Comparison of Income Class Poverty Rates**

<!-- Start of picture text --> 2002/3 2014/15<br>100<br>90<br>80<br>70<br>60<br>50<br>40<br>30<br>20<br>10<br>0<br>international poverty line lower middle income class poverty upper middle income class poverty<br>line line<br>Poverty Headcount<br>Armenia Georgia Kyrgyz Republic Kazakhstan Russian Federation Tajikistan Turkey Ukraine Armenia Georgia Kyrgyz Republic Kazakhstan Russian Federation Tajikistan Turkey Ukraine Armenia Georgia Kyrgyz Republic Kazakhstan Russian Federation Tajikistan Turkey Ukraine<br><!-- End of picture text --> _Source: Official Survey Data by Country, Author’s calculations_

**Kazakhstan** has achieved large and enduring reductions in poverty since independence. The rate of extreme poverty measured at the international poverty line of $1.9-a-day is now largely indistinguishable from zero. Despite setbacks in 2009 and 2015, poverty measured using all common lines indicate substantially improving welfare over the long-term. The most recent data available suggest that the poverty rate in Kazakhstan fell between 2016 and 2017. Measured against the highest poverty line the World Bank uses – which is usually reserved for upper middle-income countries like Kazakhstan – poverty has fallen from about 65 percent in 2011 down to about 8.5 percent in 2017. But many challenges remain, and Kazakhstan is vulnerable to economic shocks that periodically reverse the country’s progress on poverty reduction. The two most notable cases of this were in 2005 and from about 2013 to 2016. During those spells of low growth or contraction, many people fell into poverty (though not often extreme poverty), and only a return to growth in the following years pulled the poverty rate down again. Indeed, measured at the highest poverty line, Kazakhstan has still yet to return to the low rate it achieved in 2013.

Even at the regional level, welfare and resilience to shocks have strong spatial dimensions in Kazakhstan. During the most recent economic downturn starting in late 2014 and continuing into 2015, regions with lower average incomes and histories of higher poverty rates experienced larger increases in poverty in 2015.

10
**Figure 2: Poverty rates in Kazakhstan using international poverty lines**

<!-- Start of picture text --> 70.0%<br>60.0%<br>50.0%<br>40.0%<br>30.0%<br>20.0%<br>10.0%<br>0.0%<br>$1.9 PPP $3.2 PPP $5.5 PPP<br>2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011 2012 2013 2014 2015 2016 2017<br><!-- End of picture text --> _Source: the Kazakhstan HBS, Author’s calculations_ Among the Central Asian countries, **Tajikistan** traditionally has the highest rates of poverty, though in recent years the gap with Kyrgyzstan has been falling. As is the case with Kazakhstan, the country monitors poverty using a national official definition that is distinct from the World Bank’s poverty line. According to the national series, the poverty rate fell from over 72 percent in 2003 to 47 percent in 2009. The rate of extreme poverty measured at the international poverty line of $1.9-a-day is below 2 percent. According to a different method of measuring national poverty which began in 2012, poverty continued to decline from 37.4 percent in 2012 to 29 percent in 2017. The poverty rate in Tajikistan fluctuates strongly between seasons and about half of the population falls into poverty (by the national definition) at least once during the year (figure 3).

11
**Figure 3: National Poverty Rates in Tajikistan (left-note not comparable to international poverty estimates); Number of quarters in poverty (right)**

<!-- Start of picture text --> 45% Number of Quarters a Household is<br>40% Poor During the Year<br>35% 100%<br>90%<br>30%<br>80%<br>25%<br>70%<br>20%<br>60%<br>15% 50%<br>10% 40%<br>30%<br>5%<br>20%<br>10%<br>0%<br>2013 2014 2015 2016 2017<br>Poverty Ext. Poverty No Poor Qtr One Poor Qtr Two Poor Qtr<br>Qtr. Poverty Qtr. Ext. Poverty Three Poor Qtr Four Poor Qtr<br><!-- End of picture text --> _Source: the Tajikistan HBS, Author’s calculations_

**Kyrgyzstan** struggles with poverty rates nearly as high as those in Tajikistan, and in some cases higher. Although poverty fell rapidly for the first half of the past decade, progress stagnated thereafter. Measured at $3.2-a-day, the poverty rate fell from about 79 percent of the population in 2000 to 20 percent in 2008. However, the declining trend nearly halted at that point, and over the following years has remained nearly flat. Changes in the poverty rate are strongly associated with economic growth in the Kyrgyzstan, and recent slow economic growth has translated into little poverty reduction. The country is also relatively undiversified, and in the past, boom-bust cycles in economic activity have been closely linked to periods of progress and stagnation in poverty reduction.

12
<!-- Start of picture text --> 100 4<br>%° 3.5<br>50 7 * 2<br>30 Yee @ “qe,tw | o<br>20<br>10 0.5<br>9 0<br>2000 2002 2004 2006 2008 2010 2012 2014 2016<br>Poverty —#— International Poverty Line —® Lower Middle IC Line GDP per capita<br>rate (Thousand)<br>(%) —®— Upper Middle IC Line +@+» National Poverty Line<br><!-- End of picture text -->
**Figure 5: Poverty Rates by Region of Uzbekistan (2018)**

<!-- Start of picture text --> 40%<br>35%<br>30%<br>25%<br>20%<br>15%<br>10%<br>5%<br>0%<br>$3.2/Day (Inc) $3.2/Day (Cons)<br><!-- End of picture text --> _Source: The Listening to the Citizens of Uzbekistan baseline survey, Author’s calculations_

**Turkmenistan** alone has no data available at all to estimate internationally comparable estimates. For this reason, it is excluded from the analyses that follow.

In each of the four countries for which data are available, the survey data that are used to generate poverty estimates are usually considered to be representative at the level of regions (rayons) only. When disaggregated further, sampling errors are sufficiently large that confidence intervals substantially overlap. The resulting maps can potentially be misleading. Point estimates indeed suggest a specific value for the poverty rate in a given district, but if the precision of these estimates is ignored, it is easy to miss that the true value of the poverty rate may be much higher or lower. Such interpretations can lead to inappropriate policy decisions. Figure (6) presents harmonized poverty estimates for the four countries in which survey data are available at the level that the underlying surveys are usually considered to be representative. The task for the remainder of this report is to estimate and report welfare indicators, including poverty rates, that are representative at the district level, one step lower than those presented here.

14
<!-- Start of picture text --> aoe a<br>——al— Poverty<br>BB rate(69,76) (%)<br>— Bees<br>. yyom a as\ (43,49](49,56)<br>ws (36,43)<br>(29,36]<br>(23,29)<br>© (16,23)<br>.<br>= sn = (9,16)[3,9]<br>F a No data<br>ss<br>.<br><!-- End of picture text -->
<!-- Start of picture text --> classpsa!(%)<br>BB (33,36) :<br>. || feared23,26]<br>yy BB (29,33)(20,23]<br>o ree13,17]<br>’ N o-, a© (10,13)(7,10)<br>5 a BS<br>>.' | Lydwn?oe on (4.7)No data<br>a, ree2<br>Ea<br>( —s ot<br>—— tw<br><!-- End of picture text -->
<!-- Start of picture text --> “a , _— ~<a<br>‘ ¥<br>AS<br>>. Seeey “ay<br>4 Pale= a HBOS (12.94,13.77](12.11,12.94]<br>~~a ) (41.29,12.11](10.46,11.29]<br>‘ (9.63,10.46]<br>( we~ © (7.97,8.80](8.80,9.63](7.15,7.97]<br>MS (6.32,7.15]<br>BB (5.49,6.32]<br>TB (4.66,5.49]<br>No data<br><!-- End of picture text -->
sample. Any sampling for the second source leads to additional error and should be avoided if possible. For the area-based estimates described in this report, publicly available satellite, station, and geographic data aggregated to the district-level are used, none of which suffers from additional constraints due to sampling.

To ensure that the definitions of key indicators defined in the microdata are as similar across countries as possible, the harmonization standard applied by the World Bank in its ECAPOV initiative in generating official World Bank estimates of poverty is used here.

# **II.I – Districts by Country**

Presenting the results and matching the survey data to satellite imagery require identifying the cartography of each country at the district level. This was accomplished using standardized map shapefiles and matching each to the disaggregated location data available for each survey.

## **Table 1: Survey Data Used**

|Country|Year|Hhlds|Svy<br>Dist.|Map<br>Dist.|Impute|
|---|---|---|---|---|---|
|KAZ|2017|<sup>47445</sup>|178|202|24|
|KGZ|2016|<sup>19575</sup>|40|42|2|
|TJK|2015|2999|149|159|10|
|UZB|2018|4013|126|190|64|
|TKM|None|.|.|.|.|

The analyses discussed in this report primarily use the data from the diary/list of food consumed, non-food expenditure, and household composition.

# **II.II – Kazakhstan HBS data for 2017**

Poverty estimates and other welfare indicators for Kazakhstan rely on the 2017 round of the HBS, conducted by the Statistics Agency of the Republic of Kazakhstan. For household consumption and expenditure, the survey is usually considered nationally representative, representative at the oblast (region) level, and separately representative for rural and urban areas. The survey uses a stratified sample design with strata corresponding to 14 regions + 2 large cities crossed by their urban and rural areas (except for Almaty and Astana cities, which are entirely urban). A complete consumption module is gathered in the HBS, covering both food and non-food items. Information on household composition, income, employment, and related topics is also collected. All data for the HBS in Kazakhstan are collected in person, except for consumption, which is collected using a diary-type instrument. The survey is continuous, and representative quarterly within the year for most welfare indicators.

18
# **II.III – Uzbekistan L2CU Baseline Survey 2018**

Official HBS data for Uzbekistan had not been made available by the National Statistical Agency to the World Bank until recently. However, the L2CU activity, which is conducted by a private firm on behalf of the World Bank and in cooperation with Ministries and the Statistical Agency, included a comprehensive baseline survey appropriate to the needs of poverty mapping. The survey design closely followed the Living Standards Measurement Study (LSMS) surveys and was conducted using a standard two-stage sampling design, in which 200 clusters (administrative territories called mahallas, in the case of Uzbekistan) were randomly selected with probability proportionate to size. The national sample was stratified by region and by urban areas.

Though a national household census would be a preferable sample frame for the first selection step; no such census has been conducted in Uzbekistan since independence from the Soviet Union (a census is currently scheduled for 2022). Instead, the universe of mahallas, the smallest official jurisdiction in Uzbekistan, was used. These data include official population statistics provided by the National Mahalla Committee from official registration data (current as of the first quarter of 2018). The data were re-weighted based on observed population totals within the mahallas at the time of the survey fieldwork. The second stage procedure was conducted using simple random selection with equal probability within selected mahallas. A separate stratification level for households that receive social assistance was included, totaling 4 households per mahalla. The final sample included 4,000 households in total (20 households per mahalla), 800 of which were social protection recipients by design.

The baseline survey included a full consumption and expenditure module using a list/recall approach. The resulting estimates are representative for 12 regions, 1 autonomous republic, and 1 independent city (Tashkent), crossed with their urban areas (except for the City of Tashkent, which is entirely urban). The survey was conducted entirely on tablet devices (CAPI), enabling validation using crossreferencing and other techniques to ensure accuracy. The survey was conducted over the course of a 1.5-month period in May/June 2018. Although the L2CU study collected data continuously, the baseline is the only source that provides a comprehensive consumption and expenditure module.

# **II.IV – Kyrgyz Republic HBS 2016**

Poverty rates and other welfare estimates for Kyrgyzstan rely on the 2016 round of the Kyrgyzstan HBS, conducted by the National Statistical Agency. For household consumption, the survey is usually considered nationally representative, representative at the oblast (region) level, and separately representative for rural and urban areas. The survey uses a stratified sample design with strata corresponding to seven regions + 1 large city (Bishkek) crossed by their urban and rural areas. A complete consumption module (using a diary approach) is gathered in the HBS, covering both food and non-food items. Information on household composition, income, employment, and related topics is also collected. All data are collected in-person, and in most recent years using a CAPI system.

19
# **II.V – Tajikistan L2T Baseline Survey for 2015**

Although the Tajikistan HBS survey is made available to the World Bank and other development partners, the sampling design used is not well-suited to the purposes of poverty mapping because the geographic coverage of the survey is limited in comparison to other options. Geographic coverage is crucial in the context of poverty mapping, and as an alternative, a survey with a more widely distributed sample was used. The baseline for the L2T Survey was selected because it: i) used the 2010 national census sample frame for the L2T baseline survey conducted by the national statistical agency, and ii) used a widely distributed traditional stratified two-stage clustered sample design. In the first stage, 150 clusters were selected, with a probability of selection proportional to size. In the second stage, 3,000 households were selected to participate in the survey.

The sample was designed to be nationally representative for consumption and expenditure. The survey was conducted over a two-month period beginning in March 2015. CAPI systems were used and all data were collected in person. As with the L2CU study, continuous data collection was conducted following the baseline survey, however, no comprehensive consumption or expenditure model was included in follow-up rounds. The interviews were implemented under the supervision of the Ministry of Health and Social Protection of Tajikistan and the World Bank.

# **II.VI – Satellite and Administrative Data**

The main source of satellite and administrative data for this study is the AidData project at the college of William and Mary.<sup>3</sup> The spatially aggregated data available for Kazakhstan from this source include:

- 1) World Bank project locations

- 2) Yearly VIIRS day night band nighttime lights data (without stray light correction)

- 3) Version 4 DMSP-OLS Nighttime Lights composites. The lights from cities, towns, and other sites with persistent lighting, including gas flares. Ephemeral events, such as fires have been discarded. Calibrated across sensors and years using Elvidge 2014 coefficients

- 4) Average precipitation per year, created using UDel Precipitation data set (v4.01)

- 5) Average air temperature per year, created using UDel Air Temperature dataset (v4.01)

- 6) Global slope (in degrees) derived from Shuttle Radar Topography Mission (SRTM) data set (v4.1) at 500m resolution

- 7) Global elevation (in meters) from Shuttle Radar Topography Mission (SRTM) data set (v4.1) at 500m resolution

- 8) Binary indicating locations with deposits of known on-shore oil and gas deposits

- 9) Standard MODIS land cover type data product (MCD12Q1) in the IGBP Land Cover Type Classification

- 10) Yearly value for Normalized Difference Vegetation Index (NDVI). Created using the NASA Long Term Data Record (v4) AVHRR data

> 3 Data available at <u>http://geo.aiddata.org/query/#!/</u>

20
|i<br>i=Pite;<br>i<br>i<br>i<br>i<br>i<br>i~N(0,9;)<br>i<br>i =X<br>P+ Uj<br>i<br>i<br>E(u,;) =0<br>Var(u,) = 02<br>i<br>i =X<br>P+uj t e;<br>i=XB + uj;<br>MSE(B) = E(P, —P,)”<br>“BLUP<br>yg4y,|
|---|
where u� �γ��p� �X�β�, and γ� � ��<sup>�</sup> is referred to as a “shrinkage factor”. Given that σ�� is ��� ��<sup>�</sup> unknown, the Best Linear Unbiased Predictor (BLUP) is replaced with its empirical counterpart EBLUP: P<sup>�</sup> � ����� �P�������σ����, which can be rewritten as:

**�** �<sup>�</sup> � where β<sup>�</sup> is the Feasible GLS estimator for β and γ��� .  Thus,  p� is a weighted average of the ��� **�** �<sup>�</sup> direct survey estimate p� and the synthetic (model-based) estimate X�β<sup>�</sup> , and the weights are given by γ��. For p� with smaller sampling variances φ� the shrinkage factor gives higher weight to the direct estimate, while for p� with higher sampling variances a higher weight is assigned to the synthetic estimate. In areas that are not part of the survey sample, the prediction is based on the synthetic estimate X�β<sup>�</sup> , where β<sup>�</sup> �β<sup>�</sup> �σ����. The prediction error associated with p̂� takes account of the sampling � variance associated with p�, as well as the uncertainty associated with the estimate of β and σ� (see Rao, 2003 for more details).

When calculating the term φ� needed for the Fay-Herriot approach, there are several potential methods for considering the stratified and clustered two stage sample designs of the surveys used in this application. Perhaps the simplest approach is to use the Horvitz-Thompson (HT) estimator of P�, which can be expressed as: The term s� denotes the set of sampled units in domain _d_ . However, the HT estimator is often not preferred in SAE applications (National Research Council, 2013; Eurostat, 2013). It is commonly unstable, especially for small domains with few survey observations (a common challenge in this context). To address this, there are various alternative direct estimators that may out-perform the HT estimator.

Common practice in the World Bank has been to obtain sampling variances associated with the district-level poverty rates by taking the variance estimate from the survey data source and dividing it by the sample size for each domain to obtain a set of “smoothed” sampling variance estimates. This ignores components of the clustered sample design; however, these smoothed sampling variances are commonly less volatile than simple weighted direct HT estimates.

A final approach is to compute variance and the associated root mean square error of the mean using the linearized variance estimator approach—based on a first-order Taylor series (Wolter 2007). In sensitivity analyses this was the most stable variance measure of the three described here, and the preferred approach for this application. Annex G reports comparisons and analysis of sensitivity to the choice of domain variance measure. Final results are quite similar when comparing the second (“smoothed”) and third (“linearized”) options described, though as expected, the HT approach yields results that are more distant from those of the remaining two options. For more detail on the tradeoff between approaches for domain variance estimation, see Heeringa et. al., (2017); Molina and Rao (2010); Wolter (2007); and Kolenikov (2010).

22
The results of the SAE estimates are presented graphically in the following section. Separate modeling exercises are conducted for each welfare indicator. The model variables that are part of the _X_ vector in the estimation procedure were chosen to maximize the ratio of explained variance to the total variance, as captured by the adjusted R<sup>2</sup> .<sup>4</sup> There is no pre-set group of variables that are guaranteed to achieve that objective. Instead, automated variable selection using the stepwise approach was used. Sensitivity analysis comparing the results to alternative model development approaches (including several variations of the lasso approach) are described in annex I.

# **IV – Results**

Summarizing poverty rates by region (at which the underlying surveys are themselves considered representative) yields a range of between 0 and 27.7 percent at the poverty line for lower middleincome countries ($3.2 PPP per person, per day), and between 0 and 76 percent for the line appropriate for upper middle-income countries ($5.5 PPP per person, per day). But this level of aggregation hides significant variation within regions. When disaggregated to the district level, the resulting poverty estimates for the LMIC line range from 0 percent to 70 percent, and many individual districts register nearly 100 percent poverty rates at the UMIC line, despite a lower average poverty rate in the surrounding region.<sup>5</sup> Poverty rates at the LMIC line are relatively low throughout Kazakhstan and in much of Uzbekistan. However, pockets of high poverty rates remain in the Kyrgyz Republic and in Tajikistan. The parts of the Ferghana Valley within Uzbekistan’s borders have relatively low rates of poverty but nearby districts in the Kyrgyz Republic and Tajikistan with somewhat higher rates of poverty.

Even stronger regional disparities are apparent when using the UMIC line. Poverty by this definition was still less than 10 percent in Kazakhstan in 2017, but this stands in contrast to much higher rates in the Kyrgyz Republic, Tajikistan, and to a lesser extent, Uzbekistan. In Tajikistan and the Kyrgyz Republic there are districts in which nearly all people are poor by the UMIC line, whereas the highest rate in Kazakhstan is 44 percent.

Throughout Central Asia, most of the poor live in the most populated districts, despite the higher poverty rates prevalent in some of the sparsely populated and mountainous regions. The density of the population below the poverty threshold (i.e. the absolute number of poor individuals, obtained as the product of the predicted poverty rate and population of the district) is concentrated in the more densely populated areas in each country. Rural areas are much poorer than urban areas on average, and remote areas have higher average poverty rates. Districts that border on Afghanistan also tend to be poorer. Figures 9-12 (and 17-31) in the following depict each of the welfare indicators derived using the Fay-Herriot approach in this application.

> 4 Adjusted R is chosen instead of (unadjusted) R because the latter is non-decreasing in the number of explanatory variables in the model.

> 5 The district-level estimates are reported in Annexes A-D.

23
<!-- Start of picture text --> Cas Nig? ae ee NS<br>eae een ant:<br><!-- End of picture text -->
<!-- Start of picture text --> My wer - on<br><!-- End of picture text -->
<!-- Start of picture text --> yr<br>“ ad ~<br>a Ss<br>ty<br>es<br>Ag 5 lee Ml (14.57,15.79]<br>< AP (13.35,14.57]<br>y MO) (12.12,13.35]<br>{ (10.90,12.12]<br>(9.68, 10.90]<br>(8.46,9.68]<br>ie 1 (6.02,7.24](7.24,8.46]<br> (4.80,6.02]<br>HB (3.57,4.80]<br>Wl (2.35,3.57]<br><!-- End of picture text -->
<!-- Start of picture text --> a mae Soe<br>~*~ a<br>Siesd !- ie<br>w fase<br><!-- End of picture text -->
<!-- Start of picture text --> am<br>(50,53)<br>. *<br>ASS + (47,50]<br>iv fo)al (44,47)TB (41,44)<br>HB (35,38)<br>7 Jes Yaees. i ceca<br>im eran<br>um. eee (24,27)<br><ang ® 21,24)<br>aq > © (18,21)<br>Bay a Mm1) (12,15)iztsl<br>of / (9,12)<br>(6,9)<br>(3,6)<br>(0,3)<br><!-- End of picture text -->
**Figure 14: District Share of HHs with Migrants by District Per Capita Consumption Quintile**

<!-- Start of picture text --> 25%<br>20%<br>15%<br>10%<br>5%<br>0%<br>Bottom Quint 2 3 4 Top Quint<br>Share of HHs w/Migrants<br><!-- End of picture text --> _Source: Author’s calculations on the basis of the baseline survey for Listening to the Citizens of Uzbekistan (2018), the baseline survey for Listening to Tajikistan (2015), the household budget survey of Kazakhstan (2017), and the household budget survey for Kyrgyzstan (2016)._ These results complement other analyses that focus on migration; providing useful regional context to this analysis. Where household-level determinants of migration have been analyzed in more detail, there is a strong relationship between migration and local economic challenges, including low labor force participation, household welfare shocks, and living in an area with greater dependence on social protection benefits. Absent financial support from migrants, poverty rates and unemployment rates in struggling regions would be significantly higher, while average incomes would be much lower.

Another example is country specific: in Tajikistan, the government and World Bank are in advanced discussions of expanding the number of kindergartens and early childhood learning centers. But what areas need more assistance? The following figures (15-16) overlay the poverty maps for the country with details on the number of educational facilities per 1,000 children, highlighting areas with significant co-incidence of high poverty rates and low coverage.

29
<!-- Start of picture text --> HB Consumption(9.14,9.78]<br>[33] 27 (7.87,8.50]B® (8.50,9.14]<br>(22) (6.59,7.23](7.23,7.87]<br>a- : fs) (6.95,6.59]<br>rawa: p25, 3 1) (5.31,5.95](4.68,5.31]<br>we = (4.04,4.68]<br>° a BB (3.40,4.04](2.76,3.40]<br>39 56 4 4a<br>60<br>@o° oO. 6 = r::)<br>& ><br>31Ove or On s<br>xe52oy5° ofPam2 48 °<br>me JT 26 7<br>)me. “ ©er °°<br>proportional to number of number of of ELCs per per thousand children children<br>HB Consumption 0399.78)<br>B oe . = (9.00,9.39](8.61,9.00]<br>BB 22861<br>6 Fo BBHB css223caszes<br><a WB 7057.44<br>is . 3 ke HB 66,7.05)<br>bad DDBB eese2627.600)<br>n 6 BB 6495.28)<br>1 e:105.49)<br>WD) @715.10<br>- it Cc—~— 725 2 16 - 30 1)/ @s2473)(3.93,4.32)(3.54,3.93]<br>25 < A917 =~ (3.15,3.54)(2.76.15)<br>5 11 {10}7 10 ‘ ° cf —_ A \ i 8OFj 32 he13 re. | 10 cs= v7 10 4<br>é0 | =-<br>~<br>‘ 6 13<br>lgA25 \ 9 Q 13 22<br>) 10<br><!-- End of picture text -->

NOTE: Point size proportional to number of number of of ELCs per per thousand children children NOTE: Point size proportional to number of kindergartens per thousand children
These results thus provide detailed measures of welfare that in turn can be disaggregated for each district in Central Asia. The results highlight the heterogeneity of progress in the region. Poverty estimates range from 0 to nearly 100 percent by both the LMIC poverty line and the UMIC line. The results are robust to many separate validation approaches.

The maps created with these data can be used to identify areas where progress is being achieved most rapidly, as well as those areas that are lagging. With better and more detailed information at the ready, policies and interventions can be better targeted to support people in need where they live.

31
# **VI – References**

Ahrens, A., Hansen, C.B., Schaffer, M.E. 2018. lasso2: Program for lasso, square-root lasso, elastic net, ridge, adaptive lasso and post-estimation OLS.

- Bedi, T., A. Coudouel and K. Simler, (2007) _More than a pretty picture: using poverty maps to design better policies and interventions._ Washington DC: The World Bank Group.

- Elbers C., J. O. Lanjouw and P. Lanjouw (2003) “Micro-Level Estimation of Poverty and Inequality”. _Econometrica_ 71(1): 355–364.

- Fay, R. and R. Herriot. (1979) “Estimates of income for small places: an application of James-Stein procedures to census data.” _Journal of the American Statistical Association_ 74 (1979): 269–277.

- Goodman, S., Ben Yishay, A., Runfola, D., 2016. Overview of the geo Framework. AidData. Available online at geo.aiddata.org. DOI: 10.13140/RG.2.2.28363.59686

- Heeringa, Steven G., Brady T. West, and Patricia A. Berglund. _Applied survey data analysis_ . Chapman and Hall/CRC, 2017.

- Molina I. and Rao J. (2010) “Small area estimation of poverty indicators.” _Canadian Journal of Statistics_ , 38(3), 369-385.

- National Research Council. _National Patterns of R&D Resources: Future Directions for Content and Methods: Summary of a Workshop_ . National Academies Press, 2013.

- Office for Official Publications of the European Communities, Luxembourg.  Eurostat (2013). Handbook on precision requirements and variance estimation for ESS household surveys

Rao, J. N. K. (2003) _Small Area Estimation._ 1st ed. Wiley-Interscience.

- Statistical Office of the Republic of Kazakhstan (2015) “Income and Living conditions in the Republic of Kazakhstan – 2013”. Final Report. Belgrade, Republic of Kazakhstan: Statistical Office

- Kolenikov, Stanislav. "Resampling variance estimation for complex survey data." _The Stata Journal_ 10.2 (2010): 165-199.

- Wolter, Kirk. _Introduction to variance estimation_ . Springer Science & Business Media, 2007.

- World Bank (2012) “Pilot Study of Small Area Poverty Estimation Methods for the New Member States of the European Union,” Report prepared for the Scientific Steering Committee of the World Bank and European Commission Project on Small Area Poverty Estimation, Washington, DC: The World Bank Group.

32
# **VII – Annexes**

## **Annex A – Kazakhstan District-level Results 2017**

**Table 2: Kazakhstan District-level Results 2017**

|Province|District|Not<br>Precise|Rate<br>($3.2)|S.E.|<br>Rate<br>($5.5)|S.E.|<br>Below<br>MClass|S.E.|<br>Mean<br>Cons.|S.E.|
|---|---|---|---|---|---|---|---|---|---|---|
|Almaty|Aksuskiy||0.000|0.000|0.056|0.021|0.571|0.039|14.896|0.643|
|Almaty|Alakolskiy||0.000|0.000|0.132|0.035|0.657|0.043|13.741|0.838|
|Almaty|Almaty (Alma-Ata)||0.001|0.001|0.020|0.004|0.574|0.013|14.828|0.221|
|Almaty|Balkhashskiy|*|0.000|0.083|0.143|0.144|0.846|0.077|9.897|1.832|
|Almaty|Enbekshikazakhskiy||0.000|0.000|0.092|0.013|0.777|0.018|11.332|0.303|
|Almaty|Iliyskiy||0.000|0.000|0.020|0.010|0.728|0.029|12.578|0.722|
|Almaty|Karasayskiy||0.000|0.000|0.048|0.009|0.610|0.019|14.202|0.338|
|Almaty|Karatal`Skiy|*|0.053|0.081|0.201|0.141|0.871|0.075|9.595|1.794|
|Almaty|Kerbulakskiy|*|0.044|0.082|0.125|0.142|0.821|0.076|10.706|1.801|
|Almaty|Koksuskiy||0.000|0.000|0.024|0.015|0.639|0.043|14.114|0.915|
|Almaty|Panfilovskiy||0.000|0.000|0.039|0.013|0.881|0.020|9.824|0.276|
|Almaty|Raiymbekskiy||0.000|0.000|0.000|0.000|0.594|0.038|14.369|0.350|
|Almaty|Sarkandskiy||0.000|0.082|0.043|0.142|0.784|0.076|11.538|1.811|
|Almaty|Taldyqorghan||0.000|0.000|0.013|0.007|0.620|0.027|14.151|0.483|
|Almaty|Talgarskiy||0.000|0.000|0.026|0.011|0.570|0.032|15.789|0.698|
|Almaty|Uygurskiy||0.000|0.000|0.016|0.011|0.862|0.028|10.246|0.396|
|Almaty|Zhambylskiy||0.000|0.000|0.020|0.011|0.680|0.033|12.491|0.511|
|Aqmola|Akkol`Skiy||0.008|0.008|0.111|0.027|0.810|0.031|10.340|0.456|
|Aqmola|Arshalynskiy||0.049|0.034|0.153|0.054|0.842|0.047|9.804|0.855|
|Aqmola|Astrakhansiy||0.000|0.000|0.019|0.024|0.749|0.062|12.307|0.857|
|Aqmola|Atbasarskiy||0.020|0.017|0.236|0.050|0.787|0.043|10.093|0.689|
|Aqmola|Bulandynskiy||0.000|0.000|0.066|0.043|0.778|0.058|11.137|0.943|

33
|Province|District|Not<br>Precise|Rate<br>($3.2)|S.E.|<br>Rate<br>($5.5)|S.E.|<br>Below<br>MClass|S.E.|<br>Mean<br>Cons.|S.E.|
|---|---|---|---|---|---|---|---|---|---|---|
|Aqmola|Egindykol`Skiy|*|0.022|0.082|0.193|0.141|0.811|0.079|9.715|1.790|
|Aqmola|Enbekshil`Derskiy|*|0.019|0.081|0.123|0.141|0.823|0.079|10.239|1.792|
|Aqmola|Ereymengauskiy||0.022|0.018|0.116|0.038|0.798|0.042|10.584|0.651|
|Aqmola|Esil`Skiy||0.000|0.000|0.058|0.041|0.782|0.059|11.156|0.941|
|Aqmola|Korgalzhynskiy|*|0.008|0.082|0.175|0.141|0.783|0.080|9.842|1.791|
|Aqmola|Sandyktauskiy||0.000|0.000|0.106|0.043|0.788|0.050|10.505|0.665|
|Aqmola|Shortandinskiy||0.000|0.000|0.000|0.000|0.760|0.061|12.406|1.044|
|Aqmola|Shuchinskiy||0.010|0.008|0.134|0.029|0.827|0.030|10.104|0.467|
|Aqmola|Tselinogradskiy||0.000|0.000|0.015|0.004|0.681|0.014|12.983|0.166|
|Aqmola|Zerendinskiy||0.010|0.007|0.111|0.023|0.739|0.030|11.893|0.514|
|Aqmola|Zhaksynskiy||0.048|0.035|0.228|0.065|0.808|0.052|10.163|1.022|
|Aqmola|Zharkainskiy||0.000|0.000|0.126|0.059|0.884|0.046|8.444|0.612|
|AqtöBe|Alginskiy||0.000|0.000|0.084|0.053|0.891|0.052|9.661|0.761|
|AqtöBe|Aqtobe||0.002|0.002|0.072|0.011|0.791|0.017|10.940|0.235|
|AqtöBe|Aytekebiyskiy||0.007|0.012|0.107|0.043|0.868|0.044|10.050|0.887|
|AqtöBe|Bayganinskiy||0.014|0.021|0.191|0.068|0.986|0.019|7.216|0.399|
|AqtöBe|Irgizskiy||0.018|0.026|0.225|0.071|0.986|0.017|7.460|0.476|
|AqtöBe|Kargalinskiy||0.000|0.000|0.000|0.000|0.812|0.065|12.026|1.072|
|AqtöBe|Khobdinskiy||0.000|0.000|0.068|0.038|0.837|0.049|10.658|1.254|
|AqtöBe|Khromtauskiy||0.000|0.000|0.090|0.039|0.852|0.042|10.173|0.645|
|AqtöBe|Martukskiy||0.000|0.000|0.275|0.083|0.881|0.052|9.308|1.077|
|AqtöBe|Mugalzharskiy||0.018|0.013|0.186|0.039|0.947|0.021|7.563|0.320|
|AqtöBe|Shalkarskiy||0.000|0.000|0.133|0.043|0.878|0.038|9.500|0.536|
|AqtöBe|Temirskiy||0.000|0.000|0.176|0.072|0.942|0.038|8.237|0.607|
|AqtöBe|Uilskiy||0.000|0.000|0.000|0.000|0.792|0.062|12.295|1.017|
|Atyrau|Atyrau||0.000|0.000|0.044|0.011|0.861|0.018|10.118|0.245|
|Atyrau|Inderskiy||0.000|0.000|0.137|0.047|0.978|0.019|7.207|0.287|
|Atyrau|Isatayskiy||0.000|0.000|0.078|0.036|0.910|0.035|8.853|0.513|

34
|Province|District|Not<br>Precise|Rate<br>($3.2)|S.E.|<br>Rate<br>($5.5)|S.E.|<br>Below<br>MClass|S.E.|<br>Mean<br>Cons.|S.E.|
|---|---|---|---|---|---|---|---|---|---|---|
|Atyrau|Kurmangazinskiy||0.000|0.000|0.182|0.037|0.935|0.022|8.339|0.346|
|Atyrau|Kzylkoginskiy||0.000|0.000|0.012|0.016|0.851|0.045|10.426|0.605|
|Atyrau|Makatskiy|*|0.022|0.081|0.146|0.141|0.921|0.076|9.540|1.800|
|Atyrau|Makhambetskiy||0.000|0.000|0.049|0.033|0.842|0.047|10.796|0.643|
|Atyrau|Zhylyoyskiy||0.000|0.000|0.000|0.000|0.759|0.040|12.393|0.538|
|East Ka|Abayskiy|*|0.010|0.081|0.143|0.141|0.888|0.076|10.045|1.794|
|East Kaz|Ayagozskiy||0.098|0.040|0.344|0.062|0.873|0.041|8.913|0.933|
|East Kaz|Beskaragayskiy|*|0.006|0.081|0.143|0.141|0.873|0.075|10.074|1.790|
|East Kaz|Borodulikhinskiy||0.000|0.000|0.109|0.035|0.818|0.040|10.704|0.637|
|East Kaz|Glubokovskiy||0.000|0.000|0.044|0.026|0.704|0.046|12.924|0.904|
|East Kaz|Katon-Karagayskiy||0.000|0.000|0.066|0.034|0.699|0.051|12.409|1.470|
|East Kaz|Kokpektinskiy||0.000|0.000|0.048|0.031|0.908|0.035|9.349|0.461|
|East Kaz|Kurchumskiy||0.013|0.016|0.201|0.057|0.811|0.047|10.762|0.920|
|East Kaz|Leninogorsk||0.000|0.000|0.036|0.021|0.679|0.045|12.968|0.681|
|East Kaz|Semipalatinskiy||0.003|0.003|0.050|0.011|0.721|0.021|12.106|0.313|
|East Kaz|Shemonaikhinskiy||0.047|0.019|0.153|0.032|0.753|0.035|11.659|0.684|
|East Kaz|Tarbagatayskiy||0.000|0.000|0.235|0.061|0.870|0.041|9.030|0.585|
|East Kaz|Ulanskiy||0.004|0.003|0.045|0.010|0.577|0.023|15.149|0.500|
|East Kaz|Urdzharskiy||0.008|0.010|0.192|0.042|0.895|0.029|8.548|0.404|
|East Kaz|Zaysanskiy|*|0.040|0.081|0.128|0.141|0.825|0.076|10.270|1.802|
|East Kaz|Zharminskiy||0.000|0.000|0.026|0.021|0.895|0.038|9.180|0.550|
|East Kaz|Zyryanovsk||0.000|0.000|0.019|0.014|0.644|0.042|13.886|0.793|
|Mangghystau|Aqtau||0.000|0.000|0.012|0.007|0.901|0.019|10.202|0.200|
|Mangghystau|Beyneuskiy||0.000|0.000|0.086|0.029|0.982|0.013|8.478|0.233|
|Mangghystau|Karakiyanskiy||0.000|0.000|0.033|0.012|0.916|0.018|10.036|0.217|
|Mangghystau|Manghystauskiy||0.000|0.000|0.015|0.012|0.900|0.029|10.194|0.337|
|Mangghystau|Tupkaraganskiy||0.000|0.000|0.024|0.016|0.969|0.016|8.760|0.187|
|North Kaz|Akzharskiy|*|0.000|0.082|0.108|0.141|0.775|0.081|10.602|1.793|

35
|Province|District|Not<br>Precise|Rate<br>($3.2)|S.E.|<br>Rate<br>($5.5)|S.E.|<br>Below<br>MClass|S.E.|<br>Mean<br>Cons.|S.E.|
|---|---|---|---|---|---|---|---|---|---|---|
|North Kaz|Ayyrtauskiy||0.006|0.011|0.138|0.048|0.817|0.045|9.967|0.684|
|North Kaz|Bulaevskiy||0.000|0.000|0.000|0.000|0.853|0.047|10.210|0.507|
|North Kaz|Esil`Skiy||0.027|0.030|0.194|0.068|0.788|0.056|10.422|0.993|
|North Kaz|Kyzylzharskiy||0.018|0.008|0.109|0.018|0.671|0.026|12.500|0.388|
|North Kaz|Mamlyutskiy|*|0.000|0.082|0.128|0.141|0.777|0.081|10.380|1.794|
|North Kaz|Shal Akyna||0.009|0.011|0.236|0.048|0.779|0.042|10.346|0.745|
|North Kaz|Sovetskiy||0.025|0.028|0.144|0.058|0.788|0.054|10.565|0.903|
|North Kaz|Taiynshinskiy||0.000|0.000|0.000|0.000|0.744|0.058|12.593|0.645|
|North Kaz|Timiryazevskiy||0.000|0.000|0.075|0.042|0.760|0.056|11.553|0.886|
|North Kaz|Tselinniy||0.000|0.000|0.110|0.043|0.814|0.046|10.038|0.638|
|North Kaz|Ualikhanovskiy|*|0.000|0.082|0.098|0.141|0.764|0.082|10.502|1.794|
|North Kaz|Zhambylskiy||0.000|0.000|0.019|0.023|0.788|0.055|11.125|0.804|
|Pavlodar|Aksuskiy||0.000|0.000|0.030|0.016|0.722|0.038|12.534|0.530|
|Pavlodar|Aktogayskiy|*|0.000|0.082|0.131|0.141|0.808|0.081|10.092|1.793|
|Pavlodar|Bayanaul`Skiy||0.000|0.000|0.080|0.043|0.804|0.052|10.150|0.657|
|Pavlodar|Ekibastuz||0.000|0.000|0.061|0.019|0.716|0.032|11.743|0.414|
|Pavlodar|Irtyshskiy||0.000|0.000|0.000|0.000|0.748|0.066|11.877|0.994|
|Pavlodar|Kachirskiy||0.011|0.025|0.152|0.074|0.803|0.060|10.242|1.061|
|Pavlodar|Lebyazhinskiy||0.000|0.000|0.176|0.078|0.809|0.062|10.180|1.073|
|Pavlodar|Mayskiy||0.000|0.000|0.032|0.036|0.899|0.045|8.921|0.621|
|Pavlodar|Pavlodarskiy||0.001|0.001|0.044|0.010|0.764|0.020|11.590|0.305|
|Pavlodar|Sherbaktinskiy||0.000|0.000|0.041|0.035|0.800|0.057|10.568|0.837|
|Pavlodar|Uspenskiy||0.014|0.030|0.116|0.069|0.723|0.069|11.141|1.546|
|Pavlodar|Zhelezinskiy||0.007|0.015|0.081|0.045|0.733|0.059|11.939|0.844|
|Qaraghandy|Abayskiy||0.007|0.006|0.095|0.022|0.671|0.033|13.244|0.607|
|Qaraghandy|Aktogayskiy||0.000|0.000|0.075|0.030|0.775|0.041|11.383|0.618|
|Qaraghandy|Bukhar-Zhyrauskiy||0.001|0.001|0.055|0.008|0.652|0.016|13.576|0.278|
|Qaraghandy|Karkaralinskiy||0.000|0.000|0.081|0.046|0.839|0.049|9.695|0.706|

36
|Province|District|Not<br>Precise|Rate<br>($3.2)|S.E.|<br>Rate<br>($5.5)|S.E.|<br>Below<br>MClass|S.E.|<br>Mean<br>Cons.|S.E.|
|---|---|---|---|---|---|---|---|---|---|---|
|Qaraghandy|Nurinskiy||0.010|0.020|0.104|0.058|0.777|0.062|11.091|1.346|
|Qaraghandy|Osakarovskiy||0.000|0.000|0.137|0.046|0.732|0.052|11.835|0.872|
|Qaraghandy|Shetskiy||0.011|0.012|0.179|0.044|0.878|0.033|8.692|0.446|
|Qaraghandy|Ulytauskiy||0.018|0.008|0.152|0.021|0.833|0.022|10.363|0.592|
|Qaraghandy|Zhanaarkinskiy||0.023|0.025|0.171|0.059|0.810|0.050|9.690|0.709|
|Qostanay|Altynsarinskiy||0.013|0.022|0.161|0.062|0.842|0.051|9.348|0.728|
|Qostanay|Amangel`Dinskiy|*|0.000|0.081|0.127|0.141|0.912|0.076|9.746|1.798|
|Qostanay|Arkalyk||0.000|0.000|0.000|0.000|0.897|0.037|10.732|0.377|
|Qostanay|Auliekol`Skiy||0.000|0.000|0.145|0.042|0.853|0.038|9.792|0.588|
|Qostanay|Denisovskiy||0.000|0.000|0.063|0.038|0.793|0.055|12.289|1.023|
|Qostanay|Dzhangil`Dinskiy|*|0.000|0.081|0.128|0.141|0.915|0.076|9.445|1.808|
|Qostanay|Fyodorovskiy||0.000|0.000|0.198|0.070|0.828|0.055|10.047|0.941|
|Qostanay|Kamystinskiy||0.053|0.036|0.237|0.065|0.915|0.039|8.195|0.579|
|Qostanay|Karabalykskiy||0.037|0.031|0.184|0.060|0.929|0.034|8.596|0.698|
|Qostanay|Karasuskiy||0.000|0.000|0.000|0.000|0.887|0.044|10.149|0.573|
|Qostanay|Mendykarinskiy||0.000|0.000|0.102|0.039|0.807|0.045|11.066|0.718|
|Qostanay|Naurzumskiy|*|0.000|0.081|0.128|0.141|0.896|0.075|10.052|1.792|
|Qostanay|Qostanay||0.000|0.000|0.030|0.008|0.757|0.019|11.501|0.259|
|Qostanay|Sarykol`Skiy||0.000|0.000|0.124|0.050|0.962|0.024|8.185|0.399|
|Qostanay|Taranovskiy||0.000|0.000|0.190|0.043|0.904|0.029|8.755|0.450|
|Qostanay|Uzunkol`Skiy||0.000|0.000|0.000|0.000|0.863|0.048|10.605|0.557|
|Qostanay|Zhitikarinskiy||0.000|0.000|0.000|0.000|0.783|0.052|12.712|0.763|
|Qyzylorda|Aral`Skiy||0.000|0.000|0.080|0.025|0.884|0.026|9.547|0.480|
|Qyzylorda|Karmakchinskiy||0.000|0.000|0.000|0.000|0.907|0.042|9.823|0.543|
|Qyzylorda|Kazalinskiy||0.000|0.000|0.162|0.039|0.903|0.031|8.920|0.441|
|Qyzylorda|Qyzylorda||0.002|0.003|0.166|0.020|0.881|0.017|9.558|0.360|
|Qyzylorda|Shieliyskiy||0.000|0.000|0.164|0.039|0.889|0.031|8.916|0.486|
|Qyzylorda|Terenozekskiy||0.000|0.000|0.160|0.040|0.962|0.020|7.819|0.299|

37
|Province|District|Not<br>Precise|Rate<br>($3.2)|S.E.|<br>Rate<br>($5.5)|S.E.|<br>Below<br>MClass|S.E.|<br>Mean<br>Cons.|S.E.|
|---|---|---|---|---|---|---|---|---|---|---|
|Qyzylorda|Zhalagashskiy||0.000|0.000|0.100|0.041|0.984|0.016|7.784|0.314|
|Qyzylorda|Zhanakorganskiy||0.000|0.000|0.126|0.032|0.956|0.020|7.889|0.325|
|South Kaz|Arysskiy||0.000|0.000|0.000|0.000|0.817|0.029|11.340|0.325|
|South Kaz|Baydibekskiy||0.000|0.000|0.016|0.012|0.866|0.032|10.730|0.353|
|South Kaz|Chardarinskiy||0.000|0.000|0.311|0.045|1.000|0.000|7.146|0.224|
|South Kaz|Kazygurtskiy||0.000|0.000|0.043|0.019|0.981|0.012|8.289|0.212|
|South Kaz|Maktaaral`Skiy||0.000|0.000|0.056|0.014|0.968|0.011|8.212|0.171|
|South Kaz|Ordabasynskiy||0.015|0.011|0.388|0.042|0.968|0.015|6.722|0.296|
|South Kaz|Otrarskiy||0.000|0.000|0.000|0.000|0.781|0.041|12.435|0.501|
|South Kaz|Saryagashskiy||0.014|0.006|0.332|0.024|0.981|0.007|6.859|0.144|
|South Kaz|Sayramskiy||0.000|0.000|0.033|0.009|0.964|0.010|8.558|0.135|
|South Kaz|Shymkent||0.000|0.000|0.016|0.005|0.783|0.015|11.774|0.151|
|South Kaz|Suzakskiy|*|0.000|0.085|0.000|0.147|1.014|0.079|9.984|1.813|
|South Kaz|Tolebiyskiy||0.000|0.000|0.272|0.027|0.943|0.014|7.338|0.193|
|South Kaz|Turkestan||0.000|0.000|0.061|0.010|0.950|0.009|8.447|0.106|
|South Kaz|Tyul`Kubaskiy||0.000|0.000|0.132|0.034|0.926|0.025|8.841|0.340|
|West Kaz|Akzhaikskiy||0.045|0.026|0.226|0.050|0.945|0.025|8.044|0.435|
|West Kaz|Burlinskiy||0.025|0.018|0.091|0.033|0.823|0.040|10.478|0.552|
|West Kaz|Chingirlauskiy||0.000|0.000|0.232|0.072|0.924|0.039|8.168|0.547|
|West Kaz|Dzhangalinskiy|*|0.009|0.081|0.143|0.141|0.897|0.075|9.563|1.794|
|West Kaz|Dzhanybekskiy||0.000|0.000|0.209|0.065|0.953|0.029|7.578|0.468|
|West Kaz|Karatobinskiy||0.000|0.000|0.104|0.052|0.880|0.048|9.280|0.643|
|West Kaz|Kaztalovskiy||0.011|0.018|0.202|0.067|0.858|0.051|9.062|0.772|
|West Kaz|Syrymskiy|*|0.014|0.081|0.140|0.141|0.883|0.075|9.890|1.791|
|West Kaz|Taskalinskiy||0.000|0.000|0.092|0.058|0.827|0.061|11.160|1.263|
|West Kaz|Terektinskiy||0.034|0.025|0.253|0.059|0.909|0.034|8.028|0.651|
|West Kaz|Urdinskiy||0.031|0.030|0.266|0.074|0.966|0.026|7.481|0.559|
|West Kaz|Zelenovskiy||0.000|0.000|0.079|0.013|0.760|0.021|11.343|0.305|

38
|Province|District|Not<br>Precise|Rate<br>($3.2)|S.E.|<br>Rate<br>($5.5)|S.E.|<br>Below<br>MClass|S.E.|<br>Mean<br>Cons.|S.E.|
|---|---|---|---|---|---|---|---|---|---|---|
|Zhambyl|Bayzakskiy||0.000|0.000|0.079|0.012|0.844|0.017|9.874|0.220|
|Zhambyl|Kordayskiy||0.036|0.017|0.267|0.040|0.955|0.018|7.385|0.292|
|Zhambyl|Lugovskoy||0.006|0.007|0.215|0.037|0.956|0.018|7.767|0.272|
|Zhambyl|Merkenskiy||0.025|0.015|0.162|0.035|0.921|0.025|8.382|0.373|
|Zhambyl|Moyynkumskiy||0.000|0.000|0.042|0.030|0.850|0.051|10.315|0.695|
|Zhambyl|Sarysuskiy||0.029|0.021|0.268|0.052|0.973|0.020|6.744|0.340|
|Zhambyl|Shuskiy||0.000|0.000|0.031|0.013|0.878|0.025|9.710|0.372|
|Zhambyl|Talasskiy|*|0.049|0.086|0.249|0.141|0.935|0.079|8.062|1.887|
|Zhambyl|Zhamb.||0.033|0.016|0.443|0.042|0.994|0.007|6.188|0.194|
|Zhambyl|Zhambylskiy|*|0.062|0.086|0.231|0.142|0.905|0.079|8.108|1.885|
|Zhambyl|Zhualy|*|0.039|0.086|0.310|0.141|0.980|0.080|7.159|1.888|
|Zhambyl|Zhualynskiy||0.000|0.000|0.032|0.016|<br>0.956|0.018|<br>8.663|0.261|

39
# **Annex B – Uzbekistan District-level Results 2018**

## **Table 3:Uzbekistan District-level Results 2018**

|Province|District|Not<br>Precise|Rate<br>($3.2)|S.E.|<br>Rate<br>($5.5)|S.E.|<br>Below<br>MClass|S.E.|<br>Mean<br>Cons.|S.E.|
|---|---|---|---|---|---|---|---|---|---|---|
|Andijon|Andijon||0.126|0.015|0.688|0.021|0.984|0.005|5.645|0.150|
|Andijon|Asaka||0.160|0.014|0.559|0.019|0.960|0.007|6.290|0.150|
|Andijon|Baliqchi||0.127|0.015|0.689|0.021|0.945|0.011|5.673|0.159|
|Andijon|Bo'Zsuv|*|0.105|0.087|0.640|0.150|0.935|0.078|6.731|1.837|
|Andijon|Buloqboshi|*|0.140|0.087|0.566|0.144|0.906|0.076|7.109|1.813|
|Andijon|Izboskan|*|0.108|0.087|0.523|0.146|0.912|0.077|7.099|1.834|
|Andijon|Jalolquduq||0.095|0.011|0.559|0.019|0.902|0.011|7.192|0.218|
|Andijon|Marhamat|*|0.013|0.091|0.368|0.142|0.858|0.076|8.045|1.797|
|Andijon|Oltinko'L|*|0.129|0.087|0.692|0.157|0.935|0.078|6.664|1.861|
|Andijon|Paxtaobod||0.057|0.011|0.482|0.023|0.949|0.010|8.387|0.475|
|Andijon|Qo'Rg'Ontepa|*|0.032|0.091|0.385|0.142|0.834|0.076|8.602|1.804|
|Andijon|Shahrixon||0.175|0.018|0.645|0.022|0.882|0.015|7.724|0.456|
|Andijon|Ulug'Nor|*|0.016|0.090|0.455|0.145|0.890|0.076|7.622|1.798|
|Andijon|Xo'Jaobod||0.048|0.014|0.513|0.032|0.942|0.015|7.389|0.287|
|Bukhoro|Buxoro||0.065|0.009|0.328|0.017|0.872|0.012|8.816|0.252|
|Bukhoro|G'Ijduvon|*|0.126|0.082|0.297|0.142|0.859|0.081|10.581|1.942|
|Bukhoro|Jondor|*|0.085|0.082|0.275|0.142|0.829|0.081|10.953|1.941|
|Bukhoro|Kogon||0.055|0.014|0.318|0.030|0.817|0.024|10.174|0.630|
|Bukhoro|Olot||0.024|0.013|0.162|0.032|0.737|0.037|11.651|1.565|
|Bukhoro|Peshku||0.043|0.013|0.158|0.023|0.773|0.026|13.428|0.842|
|Bukhoro|Qorako'L|*|0.088|0.082|0.255|0.142|0.832|0.081|10.949|1.943|
|Bukhoro|Qorovulbozor|*|0.117|0.082|0.304|0.142|0.830|0.081|10.592|1.940|
|Bukhoro|Romitan||0.158|0.032|0.496|0.043|0.800|0.033|10.814|1.028|
|Bukhoro|Shofirkon||0.106|0.019|0.342|0.030|0.828|0.023|11.649|0.808|
|Bukhoro|Vobkent||0.110|0.014|0.345|0.021|0.851|0.016|9.248|0.367|

40
|Province|District|Not<br>Precise|Rate<br>($3.2)|S.E.|<br>Rate<br>($5.5)|S.E.|<br>Below<br>MClass|S.E.|<br>Mean<br>Cons.|S.E.|
|---|---|---|---|---|---|---|---|---|---|---|
|Ferghana|Beshariq||0.231|0.026|0.574|0.031|0.905|0.018|7.450|0.373|
|Ferghana|Bog'Dod||0.107|0.010|0.431|0.016|0.882|0.010|7.923|0.175|
|Ferghana|Buvayda||0.092|0.018|0.439|0.032|0.883|0.020|8.144|0.551|
|Ferghana|Dang'Ara||0.064|0.009|0.293|0.018|0.884|0.012|8.895|0.229|
|Ferghana|Farg'Ona||0.066|0.010|0.304|0.018|0.858|0.013|9.612|0.274|
|Ferghana|Furqat|*|0.124|0.082|0.337|0.142|0.865|0.076|8.002|1.800|
|Ferghana|O'Zbekiston||0.048|0.008|0.303|0.017|0.867|0.013|8.558|0.178|
|Ferghana|Oltiariq||0.134|0.021|0.351|0.030|0.912|0.018|8.314|0.302|
|Ferghana|Oxunboboev||0.000|0.000|0.281|0.028|0.660|0.029|10.205|0.368|
|Ferghana|Quva||0.004|0.004|0.170|0.024|0.693|0.028|11.834|0.448|
|Ferghana|Rishton|*|0.145|0.082|0.353|0.142|0.873|0.075|7.945|1.798|
|Ferghana|So’X|*|0.076|0.082|0.234|0.142|0.889|0.075|9.312|1.825|
|Ferghana|Toshloq|*|0.218|0.084|0.584|0.149|0.902|0.077|7.211|1.821|
|Ferghana|Uchko'Prik|*|0.121|0.082|0.436|0.151|0.848|0.076|8.333|1.798|
|Ferghana|Yozyovon|*|0.139|0.082|0.485|0.150|0.867|0.076|7.788|1.798|
|Jizzakh|Arnasoy|*|0.009|0.089|0.254|0.154|0.796|0.083|10.417|1.964|
|Jizzakh|Baxmal|*|0.093|0.089|0.353|0.155|0.772|0.083|9.719|1.976|
|Jizzakh|Do'Stlik|*|0.009|0.089|0.253|0.154|0.797|0.083|10.406|1.964|
|Jizzakh|Forish||0.000|0.000|0.324|0.033|0.768|0.029|11.639|0.667|
|Jizzakh|G'Allaorol|*|0.033|0.089|0.257|0.154|0.785|0.083|10.616|1.966|
|Jizzakh|Jizzax||0.045|0.009|0.152|0.015|0.756|0.018|11.013|0.264|
|Jizzakh|Mirzacho'L|*|0.010|0.089|0.254|0.154|0.806|0.083|10.230|1.964|
|Jizzakh|Paxtakor||0.030|0.012|0.211|0.029|0.870|0.023|9.430|0.330|
|Jizzakh|Yangiobod|*|0.000|0.089|0.233|0.154|0.812|0.083|10.510|1.966|
|Jizzakh|Zafarobod|*|0.051|0.089|0.273|0.154|0.799|0.083|10.209|1.965|
|Jizzakh|Zarbdor||0.000|0.000|0.149|0.025|0.776|0.029|9.280|0.273|
|Jizzakh|Zomin||0.051|0.016|0.235|0.030|0.729|0.030|12.476|0.755|
|Karakalpakstan|Amudaryo||0.376|0.043|0.689|0.044|0.952|0.021|5.839|0.533|
|Karakalpakstan|Beruniy||0.135|0.032|0.586|0.047|0.952|0.021|6.176|0.317|

41
|||Not|Rate||Rate||Below||Mean||
|---|---|---|---|---|---|---|---|---|---|---|
|Province|District|Precise|($3.2)|S.E.|<br>($5.5)|S.E.|<br>MClass|S.E.|<br>Cons.|S.E.|
|Karakalpakstan|Chimboy||0.273|0.025|0.494|0.029|0.947|0.013|7.196|0.432|
|Karakalpakstan|Ellikqala|*|0.099|0.087|0.460|0.151|0.946|0.081|7.467|1.924|
|Karakalpakstan|Kegeyli||0.104|0.021|0.494|0.035|0.981|0.010|6.741|0.299|
|Karakalpakstan|Mo'Ynoq|*|0.001|0.091|0.334|0.158|0.880|0.086|8.739|2.038|
|Karakalpakstan|Nukus|*|0.185|0.086|0.555|0.149|0.957|0.079|6.032|1.885|
|Karakalpakstan|Qanliko'L|*|0.180|0.086|0.568|0.149|0.958|0.079|6.009|1.882|
|Karakalpakstan|Qo'Ng'Irot||0.055|0.021|0.434|0.047|1.000|0.000|5.981|0.224|
|Karakalpakstan|Qorao'Zak|*|0.147|0.086|0.517|0.149|0.968|0.079|6.713|1.886|
|Karakalpakstan|Shumanay||0.146|0.032|0.415|0.047|1.000|0.000|6.314|0.263|
|Karakalpakstan|Taxtako'Pir||0.117|0.031|0.682|0.044|1.000|0.000|5.152|0.219|
|Karakalpakstan|To'Rtko'L||0.085|0.014|0.453|0.025|0.953|0.011|7.085|0.244|
|Karakalpakstan|Xo'Jayli||0.192|0.016|0.517|0.021|0.962|0.008|6.252|0.147|
|Kashkadarya|Chiroqchi|*|0.096|0.085|0.369|0.142|0.848|0.075|8.831|1.802|
|Kashkadarya|Dehqonobod||0.000|0.000|0.257|0.031|0.865|0.023|9.192|0.449|
|Kashkadarya|G'Uzor||0.174|0.027|0.371|0.034|0.895|0.021|7.487|0.423|
|Kashkadarya|Kasbi||0.109|0.013|0.329|0.019|0.800|0.016|8.608|0.204|
|Kashkadarya|Kitob|*|0.081|0.085|0.385|0.142|0.879|0.075|7.893|1.799|
|Kashkadarya|Koson||0.032|0.009|0.371|0.024|0.830|0.019|10.298|0.498|
|Kashkadarya|Muborak|*|0.030|0.085|0.295|0.141|0.887|0.076|8.802|1.795|
|Kashkadarya|Nishon||0.032|0.009|0.286|0.023|0.771|0.021|12.201|0.622|
|Kashkadarya|Qamashi||0.055|0.010|0.241|0.018|0.772|0.017|11.096|0.349|
|Kashkadarya|Qarshi||0.008|0.004|0.114|0.013|0.726|0.017|11.498|0.279|
|Kashkadarya|Shahrisabz||0.087|0.020|0.382|0.034|0.960|0.013|7.575|0.288|
|Kashkadarya|Usmon Yusupov||0.144|0.025|0.309|0.033|0.778|0.029|9.466|0.381|
|Kashkadarya|Yakkabog'||0.103|0.016|0.358|0.025|0.773|0.021|10.554|0.483|
|Khorezm|Bog'Ot|*|0.059|0.087|0.385|0.150|0.893|0.075|8.587|1.796|
|Khorezm|Gurlan||0.110|0.018|0.417|0.028|0.965|0.010|6.795|0.195|
|Khorezm|Hazorasp||0.030|0.008|0.341|0.022|0.972|0.008|7.053|0.146|
|Khorezm|Qo'Shko'Pir||0.086|0.022|0.309|0.036|0.727|0.034|12.658|0.959|

42
|||Not|Rate||Rate||Below||Mean||
|---|---|---|---|---|---|---|---|---|---|---|
|Province|District|Precise|($3.2)|S.E.|<br>($5.5)|S.E.|<br>MClass|S.E.|<br>Cons.|S.E.|
|Khorezm|Shovot||0.046|0.012|0.475|0.028|0.944|0.013|6.463|0.203|
|Khorezm|Urganch||0.075|0.009|0.361|0.017|0.885|0.011|9.767|0.366|
|Khorezm|Xiva||0.000|0.000|0.353|0.040|0.921|0.022|7.088|0.264|
|Khorezm|Xonqa|*|0.066|0.087|0.397|0.150|0.897|0.076|8.468|1.797|
|Khorezm|Yangiariq|*|0.065|0.087|0.395|0.150|0.894|0.076|8.492|1.796|
|Khorezm|Yangibozor||0.091|0.023|0.448|0.039|0.929|0.020|7.587|0.585|
|Namangan|Chortoq||0.076|0.011|0.213|0.017|0.777|0.017|12.201|0.598|
|Namangan|Chust||0.093|0.015|0.372|0.025|0.839|0.018|9.445|0.426|
|Namangan|Kosonsoy||0.006|0.005|0.184|0.027|0.811|0.027|9.361|0.313|
|Namangan|Mingbuloq||0.157|0.025|0.473|0.035|0.909|0.020|7.183|0.356|
|Namangan|Namangan||0.191|0.014|0.434|0.017|0.872|0.012|8.271|0.252|
|Namangan|Norin|*|0.273|0.091|0.647|0.150|0.949|0.080|6.457|1.887|
|Namangan|Pop||0.113|0.022|0.255|0.031|0.838|0.025|12.802|0.963|
|Namangan|To'Raqo'Rg'On|*|0.193|0.084|0.500|0.145|0.904|0.076|7.754|1.814|
|Namangan|Uchqo'Rg'On||0.000|0.000|0.251|0.031|0.890|0.022|9.144|0.353|
|Namangan|Uychi||0.124|0.023|0.322|0.033|0.873|0.023|9.521|0.797|
|Namangan|Yagiqo'Rg'On||0.114|0.016|0.376|0.025|0.868|0.017|7.673|0.221|
|Navoi|Karmana|*|0.149|0.082|0.309|0.142|0.907|0.076|8.486|1.791|
|Navoi|Konimex|*|0.115|0.082|0.269|0.142|0.939|0.077|8.726|1.792|
|Navoi|Navbahor|*|0.096|0.082|0.265|0.142|0.840|0.076|8.850|1.799|
|Navoi|Nurota||0.030|0.007|0.300|0.019|0.857|0.015|9.306|0.335|
|Navoi|Qiziltepa||0.029|0.011|0.276|0.029|0.754|0.028|10.507|1.606|
|Navoi|Tomdi||0.094|0.018|0.352|0.030|0.924|0.016|7.939|0.449|
|Navoi|Uchquduq|*|0.000|0.083|0.094|0.144|0.916|0.077|9.760|1.840|
|Navoi|Xatirchi||0.138|0.018|0.486|0.026|0.918|0.014|7.579|0.471|
|Samarkand|Bulung'Ur|*|0.181|0.082|0.424|0.142|0.826|0.076|8.012|1.804|
|Samarkand|Ishtixon||0.144|0.011|0.374|0.015|0.848|0.011|10.228|0.311|
|Samarkand|Jomboy|*|0.258|0.084|0.558|0.144|0.883|0.076|6.894|1.819|
|Samarkand|Kattaqo'Rg'On||0.230|0.023|0.483|0.028|0.915|0.015|8.686|0.696|

43
|Province|District|Not<br>Precise|Rate<br>($3.2)|S.E.|<br>Rate<br>($5.5)|S.E.|<br>Below<br>MClass|S.E.|<br>Mean<br>Cons.|S.E.|
|---|---|---|---|---|---|---|---|---|---|---|
|Samarkand|Narpay|*|0.190|0.082|0.440|0.142|0.900|0.075|7.998|1.797|
|Samarkand|Nurobod||0.352|0.028|0.716|0.027|1.000|0.000|4.643|0.140|
|Samarkand|Oqdaryo||0.246|0.024|0.348|0.027|0.794|0.023|10.520|0.547|
|Samarkand|Pastdarg'Om||0.080|0.012|0.226|0.018|0.744|0.019|10.954|0.416|
|Samarkand|Paxtachi|*|0.147|0.082|0.343|0.142|0.907|0.076|8.550|1.792|
|Samarkand|Payariq|*|0.213|0.082|0.469|0.143|0.869|0.075|7.874|1.806|
|Samarkand|Qo'Shrabot|*|0.182|0.082|0.414|0.142|0.877|0.075|8.291|1.792|
|Samarkand|Samarqand||0.098|0.018|0.450|0.030|0.906|0.017|7.208|0.250|
|Samarkand|Toyloq||0.165|0.010|0.470|0.014|0.901|0.008|7.525|0.185|
|Samarkand|Urgut|*|0.200|0.082|0.387|0.142|0.850|0.075|8.469|1.797|
|Sirdaryo|Boyovut||0.134|0.024|0.417|0.035|0.847|0.025|7.723|0.388|
|Sirdaryo|Guliston|*|0.124|0.082|0.348|0.142|0.871|0.076|8.579|1.794|
|Sirdaryo|Mirzaobod||0.087|0.020|0.317|0.033|0.944|0.016|7.457|0.344|
|Sirdaryo|Oqoltin||0.117|0.016|0.464|0.025|0.980|0.007|6.899|0.190|
|Sirdaryo|Sayxunobod|*|0.141|0.082|0.361|0.142|0.887|0.075|8.172|1.795|
|Sirdaryo|Sharof Rashidov|*|0.125|0.082|0.360|0.142|0.865|0.076|8.553|1.793|
|Sirdaryo|Sirdaryo||0.162|0.026|0.498|0.035|0.955|0.014|6.361|0.264|
|Sirdaryo|Xovos||0.134|0.024|0.623|0.034|0.967|0.012|6.071|0.264|
|Surkhandarya|Angor||0.079|0.015|0.278|0.026|0.926|0.015|7.286|0.213|
|Surkhandarya|Bandixon|*|0.240|0.085|0.688|0.152|0.927|0.076|6.827|1.820|
|Surkhandarya|Boysun|*|0.162|0.082|0.464|0.150|0.858|0.075|9.433|1.796|
|Surkhandarya|Denov||0.260|0.024|0.488|0.028|0.915|0.015|6.344|0.254|
|Surkhandarya|Jarqo'Rg'On|*|0.168|0.082|0.533|0.149|0.890|0.075|7.980|1.798|
|Surkhandarya|Muzrabot||0.154|0.014|0.427|0.020|0.792|0.016|9.724|0.351|
|Surkhandarya|Oltinsoy|*|0.171|0.082|0.542|0.150|0.845|0.075|8.315|1.805|
|Surkhandarya|Qiziriq|*|0.178|0.082|0.579|0.149|0.906|0.075|7.624|1.804|
|Surkhandarya|Qumqo'Rg'On||0.281|0.025|0.371|0.027|0.821|0.021|8.914|0.472|
|Surkhandarya|Sariosiyo||0.077|0.015|0.474|0.028|0.947|0.012|6.193|0.159|
|Surkhandarya|Sherobod||0.107|0.017|0.743|0.025|0.963|0.011|5.862|0.196|

44
|Province|District|Not<br>Precise|Rate<br>($3.2)|S.E.|<br>Rate<br>($5.5)|S.E.|<br>Below<br>MClass|S.E.|<br>Mean<br>Cons.|S.E.|
|---|---|---|---|---|---|---|---|---|---|---|
|Surkhandarya|Sho'Rchi|*|0.159|0.082|0.531|0.149|0.891|0.075|8.372|1.796|
|Surkhandarya|Termiz||0.162|0.017|0.605|0.022|0.904|0.013|9.731|1.367|
|Surkhandarya|Uzun||0.120|0.018|0.480|0.028|0.997|0.003|5.912|0.152|
|Tashkent|Bekobod||0.102|0.020|0.288|0.029|0.798|0.025|8.585|0.304|
|Tashkent|Bo'Ka|*|0.062|0.087|0.343|0.142|0.857|0.076|8.602|1.793|
|Tashkent|Bo'Stonliq||0.020|0.009|0.034|0.012|0.610|0.030|14.101|0.384|
|Tashkent|Chinoz|*|0.138|0.087|0.435|0.144|0.891|0.076|7.830|1.812|
|Tashkent|O'Rtachirchiq||0.000|0.000|0.209|0.026|0.699|0.029|10.584|0.354|
|Tashkent|Ohangaron||0.079|0.013|0.228|0.021|0.914|0.013|8.870|0.211|
|Tashkent|Oqqo'Rg'On||0.000|0.000|0.254|0.028|0.883|0.020|8.595|0.365|
|Tashkent|Parkent|*|0.045|0.087|0.288|0.142|0.791|0.076|10.377|1.809|
|Tashkent|Piskent||0.050|0.008|0.310|0.018|0.758|0.016|12.693|0.622|
|Tashkent|Qibray||0.025|0.004|0.144|0.009|0.697|0.012|11.555|0.187|
|Tashkent|Quyichirchiq|*|0.137|0.087|0.471|0.143|0.897|0.076|7.801|1.815|
|Tashkent|Toshkent||0.390|0.031|0.832|0.024|0.979|0.009|4.871|0.297|
|Tashkent|Yangiyo'L|*|0.129|0.087|0.427|0.143|0.885|0.076|7.975|1.808|
|Tashkent|Yuqorichirchiq|*|0.068|0.087|0.364|0.142|0.820|0.076|9.076|1.802|
|Tashkent|Zangiota|*|0.162|0.088|0.487|0.146|0.887|0.077|7.549|1.833|
|Tashkent City|Tashkent City||0.000|0.000|<br>0.165|0.007|<br>0.736|0.009|<br>11.953|0.172|

45
# **Annex C – Kyrgyz Republic District-level Results 2016**

## **Table 4: Kyrgyz Republic District-level Results 2016**

|Province|District|Not<br>Precise|Rate<br>($3.2)|S.E.|<br>Rate<br>($5.5)|S.E.|<br>Below<br>MClass|S.E.|<br>Mean<br>Cons.|S.E.|
|---|---|---|---|---|---|---|---|---|---|---|
|Batken|Batken||0.089|0.025|0.669|0.044|0.982|0.012|5.664|0.376|
|Batken|Kadamjai||0.297|0.029|0.785|0.027|0.995|0.005|4.385|0.150|
|Batken|Lailak||0.310|0.033|0.787|0.030|0.994|0.006|4.388|0.159|
|BišKek|BišKek||0.165|0.011|0.516|0.015|0.958|0.006|6.392|0.110|
|ChüY|AlamüDüN||0.102|0.019|0.567|0.032|0.980|0.009|5.712|0.183|
|ChüY|Chui||0.279|0.040|0.725|0.043|0.995|0.007|4.607|0.227|
|ChüY|Jaiyl||0.069|0.021|0.367|0.041|0.940|0.021|7.410|0.394|
|ChüY|Kemin||0.136|0.037|0.823|0.042|0.999|0.004|4.435|0.169|
|ChüY|Moskovsky||0.186|0.035|0.765|0.038|0.974|0.015|4.807|0.283|
|ChüY|Panfilov||0.000|0.000|0.237|0.061|0.968|0.032|7.705|0.651|
|ChüY|Sokuluk||0.270|0.028|0.630|0.031|0.987|0.007|5.156|0.194|
|ChüY|Ysyk-Ata||0.245|0.034|0.641|0.039|0.987|0.009|5.147|0.242|
|Jalal-Abad|Aksyi||0.301|0.030|0.833|0.025|1.000|0.001|4.156|0.107|
|Jalal-Abad|Ala-Buka||0.048|0.018|0.676|0.042|0.992|0.009|5.299|0.189|
|Jalal-Abad|Bazar-Korgon||0.164|0.027|0.599|0.037|0.995|0.005|5.138|0.143|
|Jalal-Abad|Chatkal||0.079|0.024|0.593|0.044|0.997|0.005|5.581|0.201|
|Jalal-Abad|Nooken||0.229|0.031|0.729|0.033|0.986|0.009|5.027|0.396|
|Jalal-Abad|Suzak||0.226|0.019|0.781|0.019|0.993|0.004|4.614|0.105|
|Jalal-Abad|Togus-Toro|*|.|.|.|.|.|.|.|.|
|Jalal-Abad|Toktogul||0.141|0.029|0.716|0.039|0.996|0.005|4.809|0.166|
|Naryn|Ak-Talaa||0.421|0.054|0.912|0.033|0.997|0.006|3.253|0.229|
|Naryn|At-Bashi||0.107|0.037|0.481|0.067|0.941|0.028|6.868|0.768|
|Naryn|Jumgal||0.150|0.038|0.795|0.042|0.995|0.007|4.560|0.244|
|Naryn|Kochkor||0.313|0.047|0.718|0.049|0.952|0.021|5.166|0.442|
|Naryn|Naryn||0.210|0.038|0.624|0.047|0.971|0.015|5.558|0.361|
|Osh|Alai||0.036|0.015|0.543|0.042|0.975|0.014|6.236|0.253|

46
|Province|District|Not<br>Precise|Rate<br>($3.2)|S.E.|<br>Rate<br>($5.5)|S.E.|<br>Below<br>MClass|S.E.|<br>Mean<br>Cons.|S.E.|
|---|---|---|---|---|---|---|---|---|---|---|
|Osh|Aravan||0.333|0.034|0.837|0.028|0.995|0.006|4.189|0.156|
|Osh|Chong-Alay|*|.|.|.|.|.|.|.|.|
|Osh|Kara-Kuldja||0.093|0.026|0.775|0.040|0.973|0.016|4.941|0.275|
|Osh|Kara-Suu||0.174|0.017|0.745|0.020|0.986|0.005|4.833|0.106|
|Osh|Nookat||0.064|0.014|0.716|0.025|0.991|0.005|5.264|0.125|
|Osh|Uzgen||0.278|0.025|0.812|0.022|0.996|0.003|4.331|0.117|
|Osh (City)|Osh||0.171|0.021|0.704|0.025|0.978|0.008|5.318|0.176|
|Talas|Bakai-Ata||0.018|0.016|0.351|0.057|0.991|0.013|7.025|0.322|
|Talas|Kara-Buura||0.231|0.043|0.826|0.039|0.995|0.007|4.208|0.195|
|Talas|Manas||0.012|0.015|0.583|0.070|0.996|0.011|5.645|0.335|
|Talas|Talas||0.053|0.019|0.668|0.042|0.994|0.007|5.379|0.226|
|Ysyk-KöL|Ak-Suu||0.090|0.021|0.499|0.038|0.953|0.016|6.678|0.310|
|Ysyk-KöL|Djety-Oguz||0.047|0.019|0.582|0.047|0.970|0.017|6.135|0.338|
|Ysyk-KöL|Ton||0.350|0.049|0.884|0.036|0.999|0.005|3.732|0.185|
|Ysyk-KöL|TüP||0.296|0.050|0.825|0.047|0.997|0.006|4.297|0.242|
|Ysyk-KöL|Ysyk-KöL||0.170|0.028|0.632|0.037|<br>0.973|0.013|<br>5.587|0.289|

47
# **Annex D – Tajikistan District-level Results 2015**

## **Table 5:Tajikistan District-level Results 2015**

|||Not|Rate||Rate||Below||Mean||
|---|---|---|---|---|---|---|---|---|---|---|
|Province|District|Precise|($3.2)|S.E.|<br>($5.5)|S.E.|<br>MClass|S.E.|<br>Cons.|S.E.|
|Dushanbe|Rudaki||0.053|0.008|0.305|0.016|0.841|0.013|9.121|0.212|
|GBAO|Darvoz|*|0.188|0.082|0.369|0.142|0.816|0.076|9.096|1.808|
|GBAO|Ishkoshim|*|0.296|0.085|0.577|0.152|0.923|0.077|6.435|1.960|
|GBAO|Murghob|*|0.245|0.091|0.632|0.161|0.835|0.080|7.530|2.052|
|GBAO|Roshtqala|*|0.294|0.085|0.575|0.151|0.927|0.077|6.455|1.958|
|GBAO|Rushon||0.246|0.034|0.707|0.036|1.000|0.000|4.641|0.166|
|GBAO|Shughnon||0.199|0.035|0.685|0.044|0.989|0.010|5.470|0.284|
|GBAO|Vanj|*|0.153|0.083|0.348|0.145|0.854|0.080|<sup>10.010</sup>|1.849|
|Khatlon|Baljuvon||0.070|0.028|0.347|0.055|0.935|0.030|7.610|0.461|
|Khatlon|Bokhtar||0.339|0.020|0.702|0.020|0.996|0.003|4.644|0.108|
|Khatlon|Danghara|*|0.199|0.082|0.592|0.144|0.954|0.077|6.322|1.837|
|Khatlon|Farkhor||0.174|0.027|0.609|0.036|0.971|0.013|5.599|0.238|
|Khatlon|Jilikul||0.375|0.051|0.956|0.023|1.000|0.000|3.330|0.150|
|Khatlon|Jomi||0.327|0.033|0.641|0.035|0.995|0.005|4.777|0.243|
|Khatlon|Khovaling||0.000|0.000|0.253|0.037|0.845|0.032|9.152|0.745|
|Khatlon|Khuroson||0.258|0.037|0.633|0.043|1.000|0.000|5.130|0.272|
|Khatlon|Kolkhozobod||0.137|0.029|0.593|0.043|0.941|0.022|6.325|0.392|
|Khatlon|Kulob||0.375|0.025|0.728|0.023|0.979|0.007|4.687|0.177|
|Khatlon|Moskva|*|0.168|0.082|0.552|0.144|0.955|0.077|5.965|1.831|
|Khatlon|Muminobod|*|.|.|.|.|.|.|.|.|
|Khatlon|Norak||0.048|0.017|0.236|0.036|0.892|0.028|8.091|0.378|
|Khatlon|Nosir Khusrav||0.212|0.046|0.625|0.058|0.986|0.016|5.954|0.440|
|Khatlon|Panj||0.058|0.016|0.504|0.037|0.963|0.014|6.452|0.250|
|Khatlon|Qabodiyon||0.000|0.000|0.358|0.056|1.000|0.000|7.107|0.309|
|Khatlon|Qumsangir||0.335|0.029|0.763|0.027|0.974|0.010|4.658|0.236|

48
|||Not|Rate||Rate||Below||Mean||
|---|---|---|---|---|---|---|---|---|---|---|
|Province|District|Precise|($3.2)|S.E.|<br>($5.5)|S.E.|<br>MClass|S.E.|<br>Cons.|S.E.|
|Khatlon|Sarband|*|.|.|.|.|.|.|.|.|
|Khatlon|Shahrituz||0.089|0.020|0.473|0.036|0.928|0.019|6.832|0.298|
|Khatlon|Shurobod||0.222|0.046|0.770|0.049|1.000|0.000|4.249|0.205|
|Khatlon|Sovet||0.434|0.034|0.793|0.030|1.000|0.000|4.062|0.162|
|Khatlon|Vakhsh|*|.|.|.|.|.|.|.|.|
|Khatlon|Vose||0.406|0.034|0.768|0.031|0.996|0.005|4.042|0.144|
|Khatlon|Yovon||0.298|0.032|0.634|0.035|0.987|0.008|5.071|0.212|
|Sogd|Asht||0.132|0.025|0.474|0.036|0.940|0.017|7.743|0.573|
|Sogd|Ayni||0.000|0.000|0.285|0.052|0.824|0.045|9.771|0.951|
|Sogd|Ghafurov||0.111|0.010|0.451|0.017|0.896|0.010|7.423|0.171|
|Sogd|Ghonchi|*|0.049|0.082|0.404|0.148|0.933|0.078|7.912|1.875|
|Sogd|Isfara||0.080|0.023|0.472|0.044|0.953|0.019|6.861|0.403|
|Sogd|Istaravshan||0.328|0.020|0.724|0.019|0.977|0.006|4.635|0.125|
|Sogd|Jabor Rasulov||0.211|0.048|0.852|0.040|1.000|0.000|4.664|0.328|
|Sogd|Konibodom<br>Kuhistoni||0.167|0.032|0.547|0.044|0.964|0.017|6.117|0.301|
|Sogd|Mastchoh||0.184|0.041|0.311|0.052|0.800|0.047|9.967|0.903|
|Sogd|Matchin||0.029|0.014|0.605|0.043|1.000|0.000|5.359|0.176|
|Sogd|Pandjakent||0.250|0.024|0.617|0.028|0.953|0.012|5.547|0.228|
|Sogd|Shahriston||0.101|0.036|0.641|0.057|0.971|0.020|5.768|0.412|
|Sogd|Spitamen||0.228|0.048|0.747|0.051|1.000|0.000|4.552|0.249|
|Sogd|Zafarobod||0.266|0.049|0.801|0.047|1.000|0.000|3.762|0.178|
|RRS|Fayzobod||0.049|0.015|0.450|0.037|0.974|0.012|6.481|0.209|
|RRS|Hissor||0.149|0.020|0.444|0.029|0.922|0.016|6.862|0.258|
|RRS|Jirgatol||0.000|0.000|0.197|0.035|0.833|0.034|9.882|0.483|
|RRS|Nurobod||0.707|0.034|0.933|0.022|0.997|0.005|2.725|0.157|
|RRS|Rasht||0.431|0.031|0.666|0.030|0.982|0.009|4.697|0.243|
|RRS|Roghun|*|.|.|.|.|.|.|.|.|
|RRS|Rudaki||0.153|0.023|0.522|0.032|0.961|0.013|6.581|0.271|

49
|Province|District|Not<br>Precise|Rate<br>($3.2)|S.E.|<br>Rate<br>($5.5)|S.E.|<br>Below<br>MClass|S.E.|<br>Mean<br>Cons.|S.E.|
|---|---|---|---|---|---|---|---|---|---|---|
|RRS|Shahrinav||0.114|0.037|0.596|0.060|0.973|0.020|5.858|0.443|
|RRS|Tavildara|*|0.148|0.083|0.508|0.149|0.960|0.080|6.925|1.890|
|RRS|Tojikobod|*|.|.|.|.|.|.|.|.|
|RRS|Tursunzoda||0.140|0.022|0.594|0.032|0.982|0.009|5.430|0.172|
|RRS|Vahdat||0.036|0.008|0.241|0.019|0.925|0.011|8.299|0.192|
|RRS|Varzob||0.115|0.038|<br>0.423|0.060|<br>1.000|0.000|<br>6.525|0.434|

50
<!-- Start of picture text --> BB (39,43) s<br>HS (35,39)<br>TM 61,35) | .<br>(27,31)<br>(23,27]<br>(19,23]<br>1 (15,19](11,15)<br>1.3) nigra es<br><!-- End of picture text -->
<!-- Start of picture text --> GB (14.92,15.79] :<br>GB (14.04,14.92)<br>1) (13.17,14.04] 3<br>(12.30,13.17] - us J)<br>(11.42,12.30]<br>(10.55,11.42]<br>©) (9.68,10.55] 4<br>DS (8.81,9.68]<br>BB (7.93,8.81)<br>Bl ( 67 . 1906 ,7. 0693 )<br>R [<br>oF : ;<br>: -<br>f-' > ie<br><!-- End of picture text -->
<!-- Start of picture text --> E<br>oS ><br>BB 67339)<br>-_<br>~ .<br>(90,32) fp ? ~~ Se<br>1 (19,22) Cit 6h» ldces pe<br>DD«5,171719) NY - 2. asss<br>1) 03,15)<br>1 1,13) ‘ )<br>ea is 4;<br>69)<br>48) g‘<br>(0,2) g<br><!-- End of picture text -->
<!-- Start of picture text --> :<br>HB= (74,79}79.23) na | . : P :<br>(65,70) |<br>a ( 706165 , 74 ) y y 4<br>BB 67.51 ‘|<br>BD 3,48)<br>69.43)<br>1) 4,39)<br>1) e034<br>| (26,30)<br>1) 1,26)<br>7,21<br>(12,17)<br>(8,12]<br>[3,8]<br><!-- End of picture text -->
- Co

- ‘ ye

- HB (35,39) OS (32,35) 1) (28,32) (25,28] (21,25) (18,21] (14,18)

- 1,14] MS (7,11) BS 47 BS 04)
- MB rs. BS **(1** 23 **.** 3824 **,1** 34 **.** 2410 **]** 1) (11.52,12.38] (10.66,11.52]

- (9.80,10.66]

- (8.94,9.80] (8.08,8.94]

- 1) (7.22,8.08] 1) (.36,7.22] Gl (5.50,6.36] Hl [4.64,5.50)
<!-- Start of picture text -->
|BB (37,40)(40.42)37 | 5 _ ; |<br>|BBBB (33.35)(30.33)202026.20 | : .<br>BB 025<br>BD a2<br>DY oo2n<br>TY) 016.19)<br>DY) 04.16)<br>asa<br>| (9,12)<br>79)<br>(571<br>(2,5)<br>(02)<br><!-- End of picture text -->
<!-- Start of picture text --> BBBB os)08,1023) 4<br>BB con cf<br>BB asso)<br>a (61,85)81) q<br>BB err,<br>| (63.67) =<br>DY1) 59,(69,63)054) ~ MaeasS<br>TY) 46,50) =a<br>(41,48) »<br>eran}237) SS > _—a<br>(28,32) q > S~J5<br>(24,28) ‘x Sftl<br>\ \eemaa<br>pS: (<br>a<br><!-- End of picture text -->
<!-- Start of picture text -->
| 5,6](4,5)<br>44)<br>(3,4)<br>(2,3)<br>1)<br>© (0,1)<br>au<br>i (2-2 , -1]2)<br><!-- End of picture text -->
<!-- Start of picture text --> WB (7.22,7.71)<br>BS 6.73,7.22] SS<br> ©.25,6.73] Sn<br>(5.76,6.25](5.27,5.76] vi \<br>(4.79,5.27] /<br>_<br>= (4.30,4.79] (3.81,4.30] f<br>HB (3.33,3.81](2.84,3.33) ioa J<br>/<br>a<br><!-- End of picture text --> <!-- Start of picture text --> «<br>SP Be<br><!-- End of picture text -->
<!-- Start of picture text --> (63,67)<br>n c (59,63)67,71)<br>a BB 65.59)<br>iE<br>| 0,24)<br>| 16,20)<br>(12,16)<br>48)<br>f [0,4]<br>ee 12)<br>gr<br><!-- End of picture text -->
<!-- Start of picture text --> .<br>at _<br>| )<br>r: zp<br>| ,<br>}P<br>:- |<br>;<br><!-- End of picture text -->

HB e196) & HB 79,83) We 7479) HE 70,74) WE 66,70 cess I) os,62 I)ID) 49,53)63.59) 1) 45,49) mar.) (87,41) (32,37) (28,32) (24,28) [20,24]
<!-- Start of picture text --> BB (18,20)<br>WS (16,18)<br>1 (14,16)<br>(12,14)<br>(10,12]<br>(8,10]<br>> 66](6,8] = a<br>— ( 31 , 53 )<br>mS -t1) Uae<br><!-- End of picture text -->
<!-- Start of picture text --> HE (9.35,10.01] \<br>WW (8.69,9.35]<br>1) (8.02,8.69]<br>(7.36,8.02]<br>(6.70,7.36]<br>(6.04,6.70]<br>(5.37,6.04]<br>1 (4.71,5.37] /<br> (4.05,4.71]<br>GM (3.39,4.05]<br>WB (2.72,3.39] SP<br>|<br><!-- End of picture text -->
# **Annex F – Validation and Precision**

One concern often raised regarding poverty map results is that the poverty estimates may not be precise enough for the purposes to which they are put. One way to assess this is to measure whether there are clear and statistically significant differences between the resulting poverty rates at the district level that can be usefully integrated into the work of the World Bank, counterparts, and other partners. This can be demonstrated graphically by reporting the poverty rates with the associated 95% confidence interval.

Because there are some districts in which there are no survey observations, these must be imputed out of sample when using the FH approach. In this minority of cases, the root mean square errors are larger than average, and should be used with caution. These cases of relatively high imprecision are noted in the district level tables with red stars (*). However, it is also important to note that most cases in this application are in sample do not suffer from this limitation of the FH approach.

**Figure 32: Upper and Lower-bound Confidence Intervals, Including Out-of-Sample Predictions**

<!-- Start of picture text --> 1<br>0.9<br>0.8<br>0.7<br>0.6<br>0.5<br>0.4<br>0.3<br>0.2<br>0.1<br>0<br>fh_55 upper_kaz lower_kaz upper_kgz lower_kgz<br>upper_tjk lower_tjk upper_uzb lower_uzb<br><!-- End of picture text --> Another way to demonstrate performance is to benchmark against “gold standard” results using the Elbers, Lanjouw, and Lanjouw (ELL) method previously published by the World Bank. The following graph demonstrates the comparison visually for 17 previously published maps that used the ELL approach. The results from the Central Asia poverty map perform quite well in this comparison for those cases that are in-sample. The 95% confidence intervals are usually well within those of other
published results. However, there are notable exceptions for cases in which the estimates are derived entirely from the FH model (i.e. those districts in which there were no observations in the underlying survey). The root mean square errors for these cases are higher than in the case for most ELL maps and this is one of the key advantages of the ELL approach (because such analyses use census microdata, there are no missing districts that need to be estimated out-of-sample). However, unfortunately, in this application the ELL approach is not possible.

**Figure 33: Comparison between ELL and FH**

<!-- Start of picture text --> 100%<br>90%<br>80%<br>70%<br>60%<br>50%<br>40%<br>30%<br>20%<br>10%<br>0%<br>ELL Maps Lower ELL Maps Upper CA FH Lower CA FH Upper<br><!-- End of picture text --> The improvements in precision (as measured by root mean square error) over direct survey estimates from the Fay Herriot approach are also substantial. Improvements are concentrated in those locations that had the least precise estimates before the exercise. The following figure (34) illustrates the improvement in terms of the root mean square error of the mean for poverty at $5.5/day.

67
**Figure 34: Root Mean Square Error Before and After FH Approach (Poverty Headcount Ratio $5.5/day - Left; Average per-capita consumption 2011 PPP – Right)**

<!-- Start of picture text --> 0.12 4<br>3.5<br>0.1<br>3<br>0.08<br>2.5<br>0.06 2<br>1.5<br>0.04<br>1<br>0.02<br>0.5<br>0 0<br>Standard Error Fay‐Herriot Standard Error Fay‐Herriot<br>Standard Error No Correction Standard Error No Correction<br><!-- End of picture text --> An additional robustness check estimates FH models for each country individually, comparing the results with those using pooled data across countries. The results presented in figure (35) show that this makes little substantive difference in either the point estimates or in the root mean square error for within-sample districts. There is a very strong relationship between the estimates regardless of the pooled vs. individual approach. The relationship remains strong but weakens somewhat with respect to out-of-sample districts (figure 35 - right). Using the pooled approach likely improves on the outof-sample prediction: this method improves the training of the model with more data than is available at the national level.

**Figure 35: Comparison of Individual Models vs. Pooled Model (Only in sample - Left; Including out of sample - Right)**

<!-- Start of picture text --> 1 1<br>0.9 R2 = 0.9852 0.9 R2 = 0.9216<br>0.8 0.8<br>0.7 0.7<br>0.6 0.6<br>0.5 0.5<br>0.4 0.4<br>0.3 0.3<br>0.2 0.2<br>0.1 0.1<br>0 0<br>0 0.2 0.4 0.6 0.8 1 0 0.2 0.4 0.6 0.8 1<br>Individual Country Maps Individual Country Maps<br>Pooled Pooled<br><!-- End of picture text --> 68
These results suggest that there is little concern that country-specific idiosyncratic relationships between y and the explanatory variables are contaminating the results for other countries. Because the right-hand side variables in the model are almost entirely based on remote-sensing data, and are not country-specific in nature, the approach is more amenable for such cross-country modeling than other potential methods that are more sensitive to these concerns. None of the explanatory variables that are used in the modeling stage (reviewed in annex H) in the final regression models would be expected to vary directly with policy regimes in the countries under study.

The model selection adopted in this approach uses the stepwise approach, which is vulnerable to the inclusion of variables with high variance inflation factors. Comparing a model that screens for high VIFs is reported in figure (36). The results show that limiting the variance inflation factor (VIF) of the variables included in the stepwise selection approach to standard levels has some small effect on the model and consequently the estimated results, and a reduction in the adjusted R2 (for the $5.5/day line, down from .72 to .67, when estimated over all Central Asia). Though this change leads to only small differences in terms of the point estimates or the root mean square error of the estimation results, the VIF minimizing strategy is clearly superior, and adopted in this report.

**Figure 36: Left: Strict VIF Threshold (5) vs. Adopted Threshold (10); Right: No VIF Threshold vs. Adopted Threshold (1))**

<!-- Start of picture text --> 1 1<br>R2 = 0.9758 R2 = 0.9628<br>0.9 0.9<br>0.8 0.8<br>0.7 0.7<br>0.6 0.6<br>0.5 0.5<br>0.4 0.4<br>0.3 0.3<br>0.2 0.2<br>0.1 0.1<br>0 0<br>0 0.2 0.4 0.6 0.8 1 0 0.2 0.4 0.6 0.8 1<br><!-- End of picture text --> To ensure that the map faithfully represents the underlying poverty dynamics of the country, it is important to ensure that the results are internally consistent. One method of assessing the stability of the estimates and the robustness of the results is to compare the relationship between the direct and adjusted estimates. This relationshipcan be presented visually as a scatter plot, with direct survey estimates along the x-axis, and model estimates along the y-axis. With perfect correlation, the estimates would lie along the 45 ̊ line, and as figure (37) shows, there is indeed a strong relationship between the model predictions and those of the observed values in the survey data. The cases in which the FH most differs from the direct estimate, the direct estimate was relatively imprecise, and the synthetic result thus relies more on the model-based estimate of the indicator. The overall strong correspondence suggests that the model-based estimates improve the precision of estimates without 69
dramatically changing the point estimates derived from the survey. In some cases, the estimate lies exactly on the 45 ̊ which signifies that the model-based estimates did not provide additional information enough to revise the direct estimates.

**Figure 37: Direct vs. Fay Herriot Estimates for in-sample Poverty $5.5day (left) and average consumption (right)**

<!-- Start of picture text --> 1 20<br>R2 = 0.9975 R2 = 0.9628<br>0.9 18<br>0.8 16<br>0.7 14<br>0.6 12<br>0.5 10<br>0.4 8<br>0.3 6<br>0.2 4<br>0.1 2<br>0 0<br>0 0.2 0.4 0.6 0.8 1 0 5 10 15 20<br>Direct Estimate Direct Estimate<br>Fay Herriot Fay Herriot<br><!-- End of picture text --> Coefficients of variation (CVs) are a useful guideline with respect to precision when estimates are far away from zero, and when the sample size is small (for instance, the surveys in UZB and TJK are much smaller than for the other two countries). In the case of per-capita consumption, no resulting in-sample estimates are above the 20% threshold that some statistical agencies adopt for reporting (Eurostat, 2013). Furthermore, the R2 between district poverty rates at $5.5/day and average consumption is very high (.74) suggesting that the estimates of poverty and average consumption are very consistent. This strong accordance should give greater credence to the poverty estimates.

70
**Figure 38: Improvement of Coefficient of Variation in average consumption**

<!-- Start of picture text --> 18 25<br>16<br>20<br>14<br>12<br>15<br>10<br>8<br>10<br>6<br>4<br>5<br>2<br>0 0<br>Avg Consumption Day/PPP (Left) Fay Herriot CV (%)<br>Direct CV (%) Linear (Fay Herriot CV (%))<br>Linear (Direct CV (%))<br><!-- End of picture text --> 71
# **Annex G – Comparison of Domain Variance Estimation Alternatives**

The preceding results use the linearized variance estimator approach—based on a first-order Taylor series. The two other options are contrasted with this approach in figures (39) and (40) for average consumption and the poverty rate at $5.5/day respectively. Figure (41) provides comparisons of poverty rate results comparing across methods of variance estimation, highlighting the relatively small differences between the adopted approach (though the largest differences are present for the HTbased approach).

**Figure 39 comparison of variance estimates for average consumption**

<!-- Start of picture text --> 100 100<br>90 90<br>80 80<br>70 70<br>60 60<br>50 50<br>40 40<br>30 30<br>20 20<br>10 10<br>0 0<br>0 100 200 300 0 100 200 300<br>HT Variance Est. Taylor Linearization Smoothed WB Taylor Linearization<br><!-- End of picture text -->

**Figure 40: Comparison of variance estimates for poverty at $5.5/day**

<!-- Start of picture text --> 0.08 0.08<br>0.07 0.07<br>0.06 0.06<br>0.05 0.05<br>0.04 0.04<br>0.03 0.03<br>0.02 0.02<br>0.01 0.01<br>0 0<br>0 100 200 300 0 100 200 300<br>Taylor Linearization HT Variance Est. Taylor Linearization Smoothed WB<br><!-- End of picture text --> 72
**Figure 41: Poverty Headcount Ratio (5.5/day) for Taylor Linearized Method vs. HT Method (Left), Taylor Liberalized Method vs. Smoothed (Right)**

<!-- Start of picture text --> 100% 100%<br>R2 = 0.9083 R2 = 0.9909<br>90% 90%<br>80% 80%<br>70% 70%<br>60% 60%<br>50% 50%<br>40% 40%<br>30% 30%<br>20% 20%<br>10% 10%<br>0% 0%<br>0% 20% 40% 60% 80% 100% 0% 20% 40% 60% 80% 100%<br><!-- End of picture text --> 73
# **Annex H – Regression Model Details**

## **Table 6: Fay-Herriot Model for Poverty Rate at $5.5/day and $3.2/day**

|Poor<br>$5.5/day||Poor<br>$3.2/day|
|---|---|---|
|(1)||(2)|
|No local service road max value<br>1.0000***|No local service road max value|1.0000***|
|(0.0000)||(0.0000)|
|Grid Mean GDP<br>0.9559***|Minimum air temperature|1.0088***|
|(0.0103)||(0.0012)|
|Grid Mean Max<br>0.9981***|Maximum elevation slope|1.0024***|
|(0.0005)||(0.0006)|
|Air temperature max<br>1.0115***|Grid minimum GDP|1.4635***|
|(0.0017)||(0.1916)|
|Grid Minimum GDP<br>1.7417***|500m Elevation|1.0001***|
|(0.3025)||(0.0000)|
|Minimum population count<br>1.0002**|Grid mean GDP|0.9702***|
|(0.0001)||(0.0083)|
|500m Elevation<br>1.0003***|Constant|1.0155|
|(0.0001)||(0.0155)|
|Constant<br>1.1588***<br>(0.0373)|||
|Observations<br>331|Observations|331|
|R-squared<br>0.6984|R-squared|0.4538|
|*** p<0.01, ** p<0.05, * p<0.1; selected<br>country/region dummy variables not shown|*** p<0.01, ** p<0.05, * p<0.1; selected<br>country/region dummy variables not show|n|
**Table 7: Fay-Herriot Model for share below middle-class line, and average consumption in 2011 PPP**

||Below<br>Middle<br>Class||Average<br>Cons.|
|---|---|---|---|
||(1)||(2)|
|Population count|1.0000**|Air temperature max|0.9071***|
||(0.0000)||(0.0270)|
|Grid Sum GDP|1.0000***|Max precipitation|1.0278***|
||(0.0000)||(0.0104)|
|Grid GDP Min|1.2266***|Nighttime lights|1.0000***|
||(0.0940)||(0.0000)|
|Vegetation Index max|1.0000**|Grid GDP Min|0.0142***|
||(0.0000)||(0.0225)|
|Max precipitation|0.9987***|No local service road max value|1.0000***|
||(0.0004)||(0.0000)|
|Vegetation Index average|0.9999***|Grid GDP Mean|1.5187***|
||(0.0000)||(0.1485)|
|No local service road max|1.0000***|Population count|1.0000**|
||(0.0000)||(0.0000)|
|||Grid GDP Sum|1.0002***<br>(0.0001)|
|||500m Elevation|0.9977***|
||||(0.0006)|
|Constant|2.8220***|Constant|9,759***|
||(0.1133)||(5755)|
|Observations|331|Observations|331|
|R-squared|0.4902|R-squared|0.5593|

*** p<0.01, ** p<0.05, * p<0.1; selected country/region dummy variables not shown

*** p<0.01, ** p<0.05, * p<0.1; selected country/region dummy variables not shown

75
# **Annex I – Sensitivity to Model Selection Technique**

Sensitivity analysis was conducted with respect to the method of automated model selection used in the FH application. The main alternative assessed was the lasso technique implemented using the _lassopack_ suite of commands in Stata (Ahrens et al., 2018). In contrast to stepwise approaches which iteratively add variables to the model and assess the p-value of the indicator (keeping the variable in the model if it is below a certain threshold), the lasso approach minimizes the residual sum of squares subject to a constraint on the absolute size of coefficient estimates. There are thought to be two major advantages of lasso approaches over least squares. First the lasso tends to produce sparse solutions and thus facilitates model interpretation. Secondly, lasso can at times outperform least squares in terms of prediction due to lower variance. However, the stepwise models adopted in this paper were already quite sparse, and there were no great advantages observed in terms of minimizing the variance.

Each of three lasso options are contrasted with simple step-wise selection. In no case are the differences remarkable. In this context, stepwise was maintained as the preferred approach due to its greater replication simplicity and ease of explanation to non-expert audiences.

## **Figure 42: Lasso AIC vs. Stepwise**

<!-- Start of picture text --> 1<br>R2 = 0.9445<br>0.9<br>0.8<br>0.7<br>0.6<br>0.5<br>0.4<br>0.3<br>0.2<br>0.1<br>0<br>0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1<br><!-- End of picture text -->
# **Figure 43: Lasso EBIC vs.  Stepwise**

<!-- Start of picture text --> 1<br>R2 = 0.9232<br>0.9<br>0.8<br>0.7<br>0.6<br>0.5<br>0.4<br>0.3<br>0.2<br>0.1<br>0<br>0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1<br><!-- End of picture text -->

# **Figure 44: Lasso BIC vs. Stepwise**

<!-- Start of picture text --> 1<br>R2 = 0.9232<br>0.9<br>0.8<br>0.7<br>0.6<br>0.5<br>0.4<br>0.3<br>0.2<br>0.1<br>0<br>0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1<br><!-- End of picture text --> 77
# **Annex J – K-fold Model Selection Sensitivity Analysis**

An additional concern of automated variable selection techniques is that overfitting will generate spurious models that perform poorly in out-of-sample prediction. One method of assessing the risk of overfitting is to train models while withholding a portion of the data and checking that the automated model development technique performs similarly well despite the missing data. This assessment is described in this annex (J), and the results (presented graphically in figures 45-49) show that there is little change in the estimates of interest when withholding a subset of data to validate performance and the presence of overfitting.

In practice, this assessment was conducted by splitting the sample into 5 randomly assigned groups (called “folds”), and using 4 folds to develop the model, imputing by using the FH model to adjust the missing observations.

**Figure 45: Fold A Full Sample (Left); Withheld Sample (Right)**

<!-- Start of picture text --> 1 1<br>R2 = 0.983 R2 = 0.9795<br>0.9 0.9<br>0.8 0.8<br>0.7 0.7<br>0.6 0.6<br>0.5 0.5<br>0.4 0.4<br>0.3 0.3<br>0.2 0.2<br>0.1 0.1<br>0 0<br>0 0.2 0.4 0.6 0.8 1 0 0.2 0.4 0.6 0.8 1<br><!-- End of picture text --> 78
# **Figure 46: Fold B Full Sample (Left); Withheld Sample (Right)**

<!-- Start of picture text --> 1 1<br>R2 = 0.9784 R2 = 0.9782<br>0.9 0.9<br>0.8 0.8<br>0.7 0.7<br>0.6 0.6<br>0.5 0.5<br>0.4 0.4<br>0.3 0.3<br>0.2 0.2<br>0.1 0.1<br>0 0<br>0 0.2 0.4 0.6 0.8 1 0 0.2 0.4 0.6 0.8 1<br><!-- End of picture text -->

# **Figure 47: Fold C Full Sample (Left); Withheld Sample (Right)**

<!-- Start of picture text --> 1 1<br>R2 = 0.9754 R2 = 0.9893<br>0.9 0.9<br>0.8 0.8<br>0.7 0.7<br>0.6 0.6<br>0.5 0.5<br>0.4 0.4<br>0.3 0.3<br>0.2 0.2<br>0.1 0.1<br>0 0<br>0 0.2 0.4 0.6 0.8 1 0 0.2 0.4 0.6 0.8 1<br><!-- End of picture text --> 79
# **Figure 48: Fold D Full Sample (Left); Withheld Sample (Right)**

<!-- Start of picture text --> 1 1<br>R2 = 0.9809<br>0.9 0.9 R2 = 0.9804<br>0.8 0.8<br>0.7 0.7<br>0.6 0.6<br>0.5 0.5<br>0.4 0.4<br>0.3 0.3<br>0.2 0.2<br>0.1 0.1<br>0 0<br>0 0.2 0.4 0.6 0.8 1 0 0.2 0.4 0.6 0.8 1<br><!-- End of picture text -->

**Figure 49: Fold E Full Sample (Left); Withheld Sample (Right)**

<!-- Start of picture text --> 1 1<br>R2 = 0.978 R2 = 0.9693<br>0.9 0.9<br>0.8 0.8<br>0.7 0.7<br>0.6 0.6<br>0.5 0.5<br>0.4 0.4<br>0.3 0.3<br>0.2 0.2<br>0.1 0.1<br>0 0<br>0 0.2 0.4 0.6 0.8 1 0 0.2 0.4 0.6 0.8 1<br><!-- End of picture text --> 80