metadata: authors: 0: author: Engstrom,Ryan
1: author: Newhouse,David Locke
2: author: Soundararajan,Vidhya
count: Sri Lanka
docna: 0: docna: Estimating Small Area Population Density
            Using Survey Data and Satellite Imagery : An Application to
            Sri Lanka
docty: Policy Research Working Paper
subtopic: Employment and Unemployment,Inequality,ICT Applications,Democratic Government,De Facto Governments,Public Sector Administrative & Civil
            Service Reform,Economics and Finance of Public
            Institution Development,Public Sector Administrative and Civil
            Service Reform,State Owned Enterprise Reform
teratopic: Social Protections and Labor,Poverty Reduction,Information and Communication Technologies,Governance,Public Sector Development
abstracts: cdata!: Country-level census data are typically
            collected once every 10 years. However, conflict, migration,
            urbanization, and natural disasters can cause rapid shifts
            in local population patterns. This study uses Sri Lankan
            data to demonstrate the feasibility of a bottom-up method
            that combines household survey data with contemporaneous
            satellite imagery to track frequent changes in local
            population density. A Poisson regression model based on
            indicators derived from satellite data, selected using the
            least absolute shrinkage and selection operator, accurately
            predicts village-level population density. The model is
            estimated in villages sampled in the 2012/13 Household
            Income and Expenditure Survey to obtain out-of-sample
            density predictions in the nonsurveyed villages. The
            predictions approximate the 2012 census density well and are
            more accurate than other bottom-up studies based on
            lower-resolution satellite data. The predictions are also
            more accurate than most publicly available population
            products, which rely on areal interpolation of census data
            to redistribute population at the local level. The
            accuracies are similar when estimated using a random forest
            model, and when density estimates are expressed in terms of
            population counts. The collective evidence suggests that
            combining surveys with satellite data is a cost-effective
            method to track local population changes at more frequent intervals.
geo_reg_key: 517198,517191,517212
repnme: repnme: Estimating Small Area Population Density
            Using Survey Data and Satellite Imagery : An Application to
            Sri Lanka
display_title: Estimating Small Area Population
            Density Using Survey Data and Satellite Imagery : An
            Application to Sri Lanka
listing_relative_url: /research/2019/03/30895987/estimating-small-area-population-density-using-survey-data-satellite-imagery-application-sri-lanka
topicv3: State Owned Enterprises m1326323 721,Poverty m1326687 695,Big Data m1327333 661,Population, Aging and Health m1327736 640,Census Data m1504073 2763
publishtoextweb_dt: 2019-03-12T00:00:00Z
historic_topic: Social Protections and Labor,Poverty Reduction,Information and Communication Technologies,Governance,Public Sector Development
no_of_pages: 43
author_keys: 0: author_key: 1161777
1: author_key: 450079
2: author_key: 1057141
model_extractions: Policy Research Working Paper 8776 Estimating Small Area Population Density Using Survey Data and Satellite Imagery An Application to Sri Lanka _Ryan Engstrom David Newhouse Vidhya Soundararajan_ Poverty and Equity Global Practice March 2019
Policy Research Working Paper 8776

# **Abstract**

Country-level census data are typically collected once every 10 years. However, conflict, migration, urbanization, and natural disasters can cause rapid shifts in local population patterns. This study uses Sri Lankan data to demonstrate the feasibility of a bottom-up method that combines household survey data with contemporaneous satellite imagery to track frequent changes in local population density. A Poisson regression model based on indicators derived from satellite data, selected using the least absolute shrinkage and selection operator, accurately predicts village-level population density. The model is estimated in villages sampled in the 2012/13 Household Income and Expenditure Survey to obtain out-of-sample density predictions in the nonsurveyed villages. The predictions approximate the 2012 census density well and are more accurate than other bottom-up studies based on lower-resolution satellite data. The predictions are also more accurate than most publicly available population products, which rely on areal interpolation of census data to redistribute population at the local level. The accuracies are similar when estimated using a random forest model, and when density estimates are expressed in terms of population counts. The collective evidence suggests that combining surveys with satellite data is a cost-effective method to track local population changes at more frequent intervals.

This paper is a product of the Poverty and Equity Global Practice. It is part of a larger effort by the World Bank to provide open access to its research and make a contribution to development policy discussions around the world. Policy Research Working Papers are also posted on the Web at http://www.worldbank.org/research. The authors may be contacted at vidhyasrajan@iimb.ac.in.

_The Policy Research Working Paper Series disseminates the findings of work in progress to encourage the exchange of ideas about development issues. An objective of the series is to get the findings out quickly, even if the presentations are less than fully polished. The papers carry the names of the authors and should be cited accordingly. The findings, interpretations, and conclusions expressed in this paper are entirely those of the authors. They do not necessarily represent the views of the International Bank for Reconstruction and Development/World Bank and its affiliated organizations, or those of the Executive Directors of the World Bank or the governments they represent._ Produced by the Research Support Team
**Estimating Small Area Population Density Using Survey Data and Satellite Imagery: An Application to Sri Lanka**<sup>∗</sup>

Ryan Engstrom<sup>†</sup> David Newhouse<sup>‡</sup> Vidhya Soundararajan<sup>§</sup>

**Keywords:** Population density predictions, Satellite imagery, Machine learning.

**JEL Code:** J19, C52, C53.

> ∗ We thank Sarah Antos for help with the data, and Jonathan Hersh for comments and discussions, and presenting an earlier version of this paper. We also thank Sarosh Sattar, Ani Silwal, and Walker Bradley for detailed comments, and the participants of Geo4dev symposium 2017, Population Association of America Annual conference 2018, and seminar participants of the World Bank for comments and suggestions. We acknowledge the funding for this project from the World Bank group. All remaining errors are our own.

> † George Washington University, Washington, DC. Email: rengstro@gwu.edu

> ‡ The World Bank, Washington DC. Email: dnewhouse@worldbank.org

> § Indian Institute of Management Bangalore. Email: vidhyasrajan@iimb.ernet.in
# **1 Introduction**

Up-to-date estimates of population density in small areas is a valuable input for policymakers (Stevens et al., 2015; Wardrop et al., 2018). They could, for example, facilitate efficient delivery of public goods and services and infrastructure projects (Guiteras et al., 2018); track net migration patterns, especially in response to civilian conflict, political upheavals, and climate tragedies; and help better understand the impact of geographically-targeted economic policy interventions such as Special Economic Zones. Traditional population data sources do not meet these requirements, as censuses provide local population measurements infrequently, typically decennially. Although household surveys can yield more frequent population estimates, they are not representative at small administrative levels. The challenge of tracking population is particularly exacerbated in low- and lower-middle- income countries where population growth rates are high and net migration patterns are rapid (Figure 1).

Satellite imagery, in combination with survey data, has the potential to fill this gap by generating more frequent estimates of population that are representative in small areas. Satellites offer several advantages as data collection instruments. They systematically and universally capture large geographic areas, produce imagery at reasonably regular intervals, and are unaffected by topography, political climate, or conflict. The technology and market for both collecting and processing satellite-based imagery are developing rapidly, and with ever more satellites being launched, high-spatial resolution imagery is increasingly available at reasonable cost.

Although existing publicly available population products incorporate satellite data, this does not imply that they are up to date. Existing products typically rely on dasymetric mapping approaches to distribute population from the census to lower administrative units, either using equal weights, or weights based on satellite indicators and advanced statistical techniques. These “top-down” methods may not provide accurate or up-to-date population estimates for two reasons. First, the accuracy of distributing population relies heavily on the input population data, that is, on the census (Wardrop et al., 2018). However, the census itself may quickly become outdated due to frequent migration patterns, and rapid and uneven population growth.<sup>1</sup> Second, these methods often rely on Night Time Lights (NTL) or other low-spatial resolution or coarse built-up area measures from satellite and other sources to distribute census population, and hence are limited at fine spatial scales (Vogel et al., 2018). This study seeks to overcome these limitations by proposing a “bottom-up” method which

> 1 Wardrop et al. (2018) provide examples of censuses that are outdated in countries such as Lebanon and Somalia due to conflicts, and postponed or cancelled in some others such as Afghanistan, Madagascar and the Republic of Congo. It is especially critical that we are able to obtain local population estimates for conflict ridden and unstable countries not only to track settlements and refugee movements, but also to provide aid and assistance.

1
combines survey and satellite data to generate local estimates of population density, and by extension, population counts. Instead of relying on the census, which becomes outdated over time, the proposed method exploits the updated demographic information in surveyed areas from existing periodic household surveys, and the widespread coverage and granular information offered by satellite imagery. The method predicts population density in non-surveyed areas, and in between-census years by employing updated survey and satellite data. This technique is applied in the context of Sri Lanka using the Household Income and Expenditure Survey (HIES), a nationally-representative household survey. The satellite indicators include those derived from both low- and high-resolution satellite imagery for the entire country, and additionally, object and contextual features derived from very-high resolution imagery for a randomly selected portion of the country. These indicators are used to predict population density at the Gram Niladhari (GN) division, the lowest administrative level in Sri Lanka. The GN division is similar in size to a village in many developing country settings, and we henceforth, for ease of exposition, refer to GN divisions as “villages”.

To motivate this approach, we begin by documenting the inconsistency of existing “topdown” population products at the village level, both with each other and with the census. We then address three questions that shed light on the ability of indicators derived from satellite imagery to predict population density. First, how accurately do satellite data predict census population density at the local level? Second, does including indicators derived from high and very high-resolution satellite imagery substantially improve the predictive power of the model? Third, how does the size of the training data affect prediction accuracy? We then implement the “bottom-up” approach, and finally ask how accurate are out-of-sample predictions that are derived from the HIES and satellite-imagery-based model, compared to the existing “top-down” products?

The “top-down” products considered include, WorldPop for the years 2010 and 2015, the Global Human Settlement Layer (GHSL) for 2014, High Resolution Settlement Layer (HRSL) created by Facebook for 2015, the Center for International Earth Science Information Network’s (CIESIN) Gridded Population of the World (GPW) for 2010 and 2015, and LandScan for 2010. We use simple statistical measures of association to confirm that top-down estimates are poorly correlated with each other and with the census at the village level. WorldPop 2015 and Facebook are the exceptions, because they use high-resolution satellite imagery and are calibrated to the latest census data at geographically fine levels.<sup>2</sup> However, since even the most accurate population products use the census to redistribute population, they may quickly become outdated as the census ages, necessitating “bottom-up” methods to track changes more frequently.

> 2 At Divisional Secretariat level (one level above the village), however, all estimates are highly correlated with each other and with the census, implying that accuracy at the coarser levels is easier to achieve.

2
We obtain satellite indicators from a variety of sources, which we categorize into four types, based on their resolution, availability, and accessibility. The publicly available indicators obtainable for the entire country are categorized into, (i) low-resolution indicators, based on imagery resolution ranging from about 0.5km per pixel to about 30m per pixel; and (ii) highresolution imagery whose resolution ranges from about 30m per pixel to 12m per pixel. The proprietary indicators are taken from Engstrom et al. (2017), and are based on very-high resolution imagery acquired from DigitalGlobe covering 55 Divisional Secretariat divisions (one level higher than the village, henceforth referred to as sub-districts). These are divided into two categories: (iii) contextual feature indicators; (iv) object-based indicators quantifying objects such as roofs and cars.

Our results that use Poisson regressions based on variables selected by the least absolute shrinkage and selection operator (LASSO) indicate that satellite indicators accurately predict village-level census population density. Models using publicly available satellite indicators in a full national sample predict density with an out-of-sample _R_<sup>2</sup> of 0.75. Including the very-high resolution indicators and estimating the model in the 55 sub-district sample, the out-of-sample _R_<sup>2</sup> rises to 0.83. Although the public indicators perform well in predicting density in rural as compared to urban areas, the urban-sector prediction accuracy increases substantially (about 0.13 points) when object and contextual features from Engstrom et al. (2017) are added, reflecting in part the heterogeneity in the relationship between population density and builtup area in urban areas.<sup>3</sup> The accuracy of the predictions does not meaningfully change when the size of the training sample is reduced. Results from random forest models produce similar results to the LASSO-based regressions, and corroborate the results across different types of high-resolution resolution-types and sectors.

Finally, we obtain results from a similar Poisson model that uses estimates of population density obtained from the 2012/13 HIES instead of the census. We then compare the out-ofsample predictions from this model with the actual census population densities in villages not covered by the HIES. This yields an out-of-sample _R_<sup>2</sup> of 0.79, Spearman Rank Correlation (SRC) of 0.91, a mean Relative Error (RE) of 37%, and a median RE of 28%. Based on _R_<sup>2</sup> and mean RE, our model is more accurate than the top-down measures: GPW (both years), GHSL, LandScan, and the 2012 WorldPop, and not as accurate as Facebook and WorldPop 2015.<sup>4</sup> The results on these relative accuracies extend to village-level population-count predictions derived by multiplying density predictions by village area. The prediction accuracy remains similar if only indicators that are publicly available are used in the model.

> 3 Rural buildings are mostly residential, but urban buildings tend to contain a heterogeneous mix of commercial and residential buildings. High resolution built-up area measures and object identifiers are better able to capture these heterogeneities in urban areas.

> 4 Using median RE as the benchmark, we report comparable performances to all 2015 estimates, except Facebook and WorldPop 2015.

3
Two aspects of these results are particularly noteworthy. First, the survey used to calibrate the model covers only 17% of the villages in Sri Lanka. The fact that the combination of satellite imagery with this type of small survey predicts population density as accurately as the estimates that use an entire census demonstrates the cost-effectiveness of this approach.<sup>5</sup> In fact, simulations indicate that combining survey data with satellite data generates sub-district estimates of population density that are as precise as a survey that samples 80 percent of the villages nationwide. Second, although our estimates are not as accurate as the most accurate top-down estimates, the model gives comparable or better performance than most others. However, because surveys are collected much more frequently than censuses, bottom-up methods that combine satellite imagery with surveys have the crucial advantage of remaining up to date even as the census ages.<sup>6</sup> This study fits into a rapidly growing literature on using satellite imagery to predict population counts and density in local areas. Wardrop et al. (2018) provide an overview of these studies. Top-down approaches to distribute population remain popular in the literature. For example, Stevens et al. (2015) use a dasymetric approach to redistribute population counts from the census using a Random Forest model-based weighting scheme in Cambodia, Vietnam, and Kenya. Using a combination of regression and tree-based methods, Anderson et al. (2014) predict population density for districts in Peru where no direct samples were available, using two coarse satellite-based covariates: Normalized Difference Vegetation Index (NDVI) and the daytime Land Surface Temperature (LST).<sup>7</sup> Our contribution to this literature is threefold. First, relatively few studies use “bottomup” techniques (Wardrop et al., 2018). Early studies using bottom-up style methods either use simple area measures and/or coarse satellite imagery, or mostly focus on developed countries where there are fewer data limitations (Sutton et al., 2001; Biljecki et al., 2016; Li and Weng, 2005).<sup>8</sup> Second, we compare prediction accuracies between imageries of various resolutions to ascertain if, and by how much, higher resolution imageries are better at predicting density compared to lower resolution imageries. Prior studies have not examined the trade-offs across

> 5 It is important here to clarify that even though we do not directly rely on the census, the census remains essential to provide the sampling frame for subsequent and periodic surveys. Our method, hence, does not reduce the importance of the census.

> 6 For instance, as we will show in section 4, WorldPop 2010 that utilizes the 2001 Sri Lankan census (a gap of nine years) for redistribution, performs much worse than WorldPop 2015 that uses the 2012 census (a gap of three years).

> 7 We also speak to a parallel literature that uses other sources of big data such as mobile phones to map population distribution (Deville et al., 2014) or estimate population maps based on age structure (Alegana et al., 2015), and to an emerging literature that uses satellite data and machine learning techniques to predict human welfare and demographic variables, and urban market boundaries (McBride and Nichols, 2016; Athey, 2017; Engstrom et al., 2017; Jean et al., 2016; Vogel et al., 2018).

> 8 Hillson et al. (2014) use bottom-up techniques but answer a different but related question of what survey size would support a particular confidence level in the estimating population using rooftop area in Bo, Sierra Leone.

4
different types of resolutions within the same context.

Third, we validate out-of-sample predictions derived from a household survey against the census. Such validations are not commonly performed by studies using “bottom-up” methods (Wardrop et al., 2018), except Biljecki et al. (2016) and Harvey (2002). The estimates are more accurate than previous studies that make use of two-dimensional satellite data. Our out-of-sample _R_<sup>2</sup> of 0.78 is higher than the 0.72 reported for Australia (Harvey, 2002). Our median RE at 28% is lower than those reported by Biljecki et al. (2016) in the Netherlands, which ranges from 42% to 85.4%. These higher errors could be attributed to their usage of fewer satellite indicators, linear (rather than non-linear) regressions and simple (rather than stratified) random sampling for the survey.<sup>9</sup> Overall, the results demonstrate the feasibility of national statistics offices utilizing geospatial data in combination with frequently conducted surveys to produce accurate local population estimates in the between-census years and in areas where surveys are not conducted. Improved availability of such statistics would provide useful inputs into a wide variety of policy decisions. The rest of the paper is organized as follows: section 2 describes the data, section 3 presents the model, section 4 presents the results, and section 5 finally concludes the paper and discusses policy implications.

# **2 Data sources and description**

## **2.1 Population sources**

Our main measures of population density at the village level are the Census of Population and Housing, Sri Lanka, 2012, and the 2012-13 Sri Lankan Household Income and Expenditure Survey (HIES). The HIES is a detailed survey of about 25,000 households in all the districts and sub-districts of Sri Lanka, but only covers about 2,421 of the total 14,103 villages of the country. The HIES follows a two-stage stratification process: the census blocks form the Primary Sampling Units (PSUs), and the households form the Secondary Sampling Units (SSUs) or Final Sampling Units (FSUs).

The PSUs used in the HIES are census blocks, which are portions of villages.<sup>10</sup> Therefore, the HIES can yield direct density estimates only at the PSU-level, and not at the villagelevel. Unfortunately, it is not possible to model population density at the PSU-level due to the lack of a PSU-level Geographical Information System (GIS) boundary file. We therefore

> 9 Interestingly, Biljecki et al. (2016) find lower errors (18.3% or 18.5%) by utilizing floor-space or volumetric 3D data and functional information about buildings. But these 3D models and data are costly to obtain periodically in low-income countries.

> 10 A PSU is contained within the village. There were 62,571 PSUs versus 13,984 GNs (villages) in the 2012 census.

5
indirectly estimate the village population density using the HIES survey weights, under the assumption that the survey weights reflect the inverse probability of housing unit being selected for the sample. The steps used to estimate village level population density from the HIES are described in Appendix A.

# **2.2 Publicly available areal estimates of population density**

We validate several publicly available gridded population data sets against each other and with the census, and compare them with our survey-based population density predictions. We examine the following five sources: (1) Landscan for the year 2010; (2) WorldPop for 2010 and 2015; (3) Facebook HRSL for 2015; (4) Gridded Population of the World v3 (year 2010) and v4 (year 2015), and (5) GHSL for 2015. All sources employ a top-down approach using a combination of areal interpolation techniques, including basic dasymetric approaches in conjunction with ancillary data, and statistical modeling methods to distribute census population to smaller grids. While GPW redistributes data based on an equal weighting technique, other sources use measures such as built-up area, road count and density, elevation, slope, and light intensity (“covariates”) to proportionally redistribute population. Facebook, WorldPop 2015, GPW 2015, and GHSL estimates are based on the 2012 Sri Lankan census; WorldPop 2010 and GPW 2010 are based on the 2001 census; and Landscan uses mid-year population of the country of the past year calculated by the Geographic Studies Branch, US Bureau of Census (Oakridge National Laboratory, 2018).<sup>11</sup>

# **2.3 Satellite data**

We use various sources of geo-spatial data in our models. The low-resolution public indicators are from three sources: (1) Night time lights from the Visible Infrared Imaging Radiometer Suite (VIIRS) at a resolution of 750 m per pixel. We use the maximum and mean intensity of two months, namely March and September, 2014; (2) Global Forest Change data based on Hansen et al. (2013), from which we use mean tree cover in 2000, and gain and loss in forest area between 2000 and 2014, at a resolution of 30 meters per pixel; (3) Advanced Spaceborne Thermal Emission and Reflection Radiometer (ASTER)’s elevation and slope data at a resolution of 30 meters per pixel.

We also use publicly available built-up area measures based on higher resolution imagery. The Global Urban Footprint (GUF) (year 2012) and Global Urban Footprint plus (GUF+)

> 11 Detailed information on the input population, ancillary data, and the redistribution methodology for each source is presented in the supplementary Table S1.

6
(year 2015) provide built-up estimates at a resolution of 12 meters per pixel.<sup>12</sup> Global Human Settlement Layer’s (GHSL) built-up estimates are at approximately 38 meters per pixel. We also use Facebook’s High Resolution Settlement Layer (HRSL) built-up area measure, available at a resolution of 30 meters per pixel.

Additionally, we use contextual and object features from Engstrom et al. (2017) which used proprietary imagery for 1,360 villages within 55 sub-districts obtained from DigitalGlobe. These images cover 3,500 _km_<sup>2</sup> of area in Sri Lanka, and are mostly obtained for the years 2011 and 2012, although some images were also captured for the year 2010. The list of 55 sub-districts is available in Appendix B. Object features extracted include the number of cars, building count and size, roof type, shadow pixels, road length and type, type of farm land (paddy or plantations).<sup>13</sup> Further, seven contextual features are calculated: Fourier transform, Gabor filter, Histogram of Oriented Gradients (HoG), Line Support Regions (LSR), Pantex, Normalized Difference Vegetation Index (NDVI), and Speed-Up Robustness Features (SURF).<sup>14</sup> More details on these objects and features can be found in Engstrom et al. (2017).

Table 1 concisely presents the key variable types, their source, time frame, resolution type, and geographical coverage. Tables 2 and 3 present the summary statistics for the population, geographic, and satellite imagery based variables for the national sample and the 55 subdistricts, respectively.

# **3 Modeling**

## **3.1 Predicting population density**

We use Poisson regressions to model population density at the village level. This takes the following form: where _Pv_ is the population density (persons/km<sup>2</sup> ) of village _v_ . _Xv_ consists of the set of satellite imagery-based indicators defined for village _v_ . We include the natural logarithm of village area ( _Ln areav_ ), and an indicator for urban villages ( _ρS_ ).<sup>15</sup> _ηd_ represents binary indicator

> 12 While GUF was made with TerraSAR-X/TanDEM-X, which are radar data sets, GUF+ adds Sentinel and Landsat which are optical remotely sensed data which substantially improves the estimation.

> 13 Supplementary Figure S1 shows examples of developed area building classification, with the raw image (left) and CNN based building classification (right) in Panel A, and a sample car classification in Panel B. In a test sample, the average precision of the building classification was 71 percent, indicating a moderate share of false positives.

> 14 The contextual and object features are identified using both deep learning-based Convolutional Neural Networks (CNN) and classification of spectral and textural characteristics (Engstrom et al., 2017).

> 15 While the official sectoral classifications in Sri Lanka include three categories: the urban, rural, and estate sectors, we combine the second and the third to represent two categories: (1) Urban (2) Rural and estate.

7
variables for Sri Lanka’s districts.

To prevent model over-fitting in using the entire set of satellite imagery in _Xv_ , we employ the least absolute shrinkage and selection operator (LASSO) regularization which estimates a regression model with an added constraint that enforces parsimony. We follow a two-step procedure. First, a full set of variables is included to conduct LASSO regularization to choose variables. Second, the final Poisson model is estimated using the chosen variables. Equation 2 presents the objective function for LASSO regularization in a model using intercept _β_ 0 and the vector of predictor coefficients _τ_ (of length _p_ ) in step 1.

where _l_ is the value of the log likelihood function of the Poisson model using parameters _β_ 0 and _τ_ , and _λ_ is a non-negative regularization parameter. While setting _λ_ = 0 yields unconstrained Poisson regression estimates, a large _λ_ penalizes the absolute values of the coefficients, _β_ .<sup>16</sup> Fivefold cross validation is applied to choose the value of _λ_ that minimizes the root-mean squared error (RMSE) across the folds. The in-sample _R_<sup>2</sup> from this step indicates the goodness of the model fit. Since LASSO also generally shrinks the magnitude of all coefficients towards zero (Varian, 2014), we avoid biased predictors by running a simple Poisson model based on the Lasso selected variables. From this, we obtain the out-of-sample _R_<sup>2</sup> , mean absolute error (MAE), and RMSE, all of which indicate out-of-sample predictive accuracy.

First, we implement the above model using the village level census population density as the dependent variable. We conduct this exercise for the entire country separately using different imagery resolution types: (1) low-resolution publicly available indicators; (2) higher resolution publicly available indicators plus the indicators mentioned in (1). Doing this enables us to compare prediction accuracy between higher versus lower resolution data in the national sample. In the 55 sub-districts where the features from Engstrom et al. (2017) are available, we implement a model using: (3) spectral and texture-features plus indicators used in (2); and (4) object-features plus the indicators used in (3). For the 55 sub-districts, we compare prediction accuracy successively from including the set of imagery-based indicators in (1)

> 16 For estimations, we use the glmnet package in _R_ that estimates the model parameters by minimizing the penalized log likelihood function. Glmnet provides an option to penalize the minimand based on the sum of the absolute values of the parameters (LASSO), or based on the sum of squares of the parameters (ridge), or a combination of both (Hastie and Qian, 2018). Glmnet’s minimization function is presented in Equation 3. We set _α_ = 1 to use the LASSO method.

8
through (4). We also repeat the analysis separately for urban areas and for rural and estate areas (grouped together) to check if prediction accuracy varies across sectors. Finally, focusing on the 55 sub-districts, we reduce the training sample size by half and quarter, to examine if the prediction accuracy is sensitive to the size of the training sample. As a robustness check, we also implement a flexible random forest algorithm using population density as the response variable, using the four sets of satellite indicators described above. For each model, we report out-of-bag _R_<sup>2</sup> , RMSE, and MAE.<sup>17</sup> Next, we focus on an overlapping sample between the sampled villages in the HIES and those in the 55 sub-districts for which proprietary features are available. These amount to 1,360 villages. In 414 villages of this sample, we model the HIES population density as a function of the all public and proprietary satellite indicators, and apply the procedure described above. Since this is a survey-based model, we use the inverse of village population for village _v_ as weights, which is given by _V illage Population_ <u>1</u> _v_<sup>.Inaddition,therecouldbefactors</sup> correlated with satellite-derived indicators that affect the probability of selection of the village into the HIES. We therefore adjust the weights based on the predicted probability of the village being sampled in HIES, based on satellite indicators available for all villages (Horvitz and Thompson, 1952; Wooldridge, 2002). This probability is obtained from the following probit model: where _INHIESv_ is a binary indicator for whether village _v_ is sampled into HIES, and _lassov_ represents all LASSO-selected variables from the Poisson model. We estimate this model using data for all 1,360 villages in the 55 sub-districts. The predicted probability of the � village selection into the HIES, _INHIESv_ is used for correcting the weights in the following <u>1</u> way : � _V illagePopulation X INHIESv_<sup>.</sup> Using these above two sets of weights, we estimate the model in HIES-villages. Then, we predict out-of-sample densities for the remaining 946 non-HIES villages and report outof-sample accuracy measures by comparing them with the actual census density. To provide additional context, we compare the accuracy of this model to predictions from a similar censusbased model and with density estimates from the publicly available “top-down” population products. Further, we calculate population count predictions by multiplying these density predictions with village-level area, and report their accuracy with respect to the census population.

> 17 Out-of-bag _R_ 2 is an out-of- sample statistic that refers to the average variance explained by the random forest predictions, across all villages, when averaging over the bootstrapped samples that did not contain each village.

9
# **4 Results**

The sub-sections below present (a) the results comparing publicly available “top-down” population estimates with each other and with the census; (b) the predictive accuracy of satellite imagery-based indicators for census density, using LASSO-based Poisson regression models; and (c) out-of-sample density predictions based on a model using the survey (HIES), and comparing their accuracies with other estimates.

## **4.1 Validation of population products**

The validation exercise indicates that publicly available population count estimates are accurate at the sub-district level but not necessarily at the village level. At the sub-district level, except LandScan and WorldPop 2010, the _R_<sup>2</sup> for all other products with the census are above 0.9 (panel A in Table 4). At the village level, panel B in Table 4 shows that WorldPop 2015 has the highest association with the census ( _R_<sup>2</sup> of 0.988), followed by Facebooks HRSL (0.841), GHSL (0.672), GPW 2015 (0.595), GPW 2010 (.404), WorldPop 2010 (0.107) and LandScan (0.0006) in that order. These results qualitatively hold if we use correlation coefficients as a measure of association (see supplementary tables S2 and S3). Clearly, the products using the most recent census and using higher resolution imagery-based covariates are performing better than the others. For example, GPW and WorldPop 2010 estimates use the 2001 census for redistribution, and hence it is not surprising that they perform poorly in comparison to the 2012 census.<sup>18</sup> However, even the best performing products may themselves become inaccurate with the passage of time as their source data in the census becomes outdated.<sup>19</sup> 18 Similarly, an earlier unpublished version of Facebook’s population data which we worked with, only yielded a correlation coefficient of 0.5 with the 2012 Sri Lankan census at the village level because Facebook initially used the 2001 Sri Lankan census for calibration. Their estimates were later updated by using the most recent census, which now gives an _R_<sup>2</sup> of 0.841 and a correlation coefficient of 0.917.

> 19 We also assess the consistency of four publicly available built-up area measures, namely, GUF, GUF+, GHSL, and Facebook, against each other, and the validate the built-up area estimates for the 55 sub-districts by Engstrom et al. (2017) against Facebook’s. All estimates of built-up area at the village level are reasonably consistent with each other. At the national level, the _R_<sup>2</sup> between estimates from Facebook and each of the first three sources are 0.76 or greater (supplementary Table S4). In the 55 sub-districts, the _R_<sup>2</sup> between Facebook estimates and those from Engstrom et al. (2017) is also high at 0.794 (supplementary Table S5). These patterns also hold if correlation coefficients are employed for the analysis (see supplementary tables S6 and S7).

10
# **4.2 Accuracy of population density predictions based on census population counts**

Poisson regression results using the census data for all villages, based on the variables selected from LASSO regularization indicate that satellite indicators have exceptionally strong predictive power in predicting village level population density (Table 5). At the national level, publicly available indicators explain a large amount of variation in village population density. In this case, the value added in using high-resolution indicators (out-of-sample _R_<sup>2</sup> is 0.75) as opposed to low resolution indicators (out-of-sample _R_<sup>2</sup> is 0.702) is minimal. In the 55 sub-districts, adding proprietary contextual features to public imagery-based models does not improve the prediction accuracy (out-of-sample _R_<sup>2</sup> remains at around 0.7). However, adding object classifiers to the model improves the out-of-sample _R_<sup>2</sup> by 10 points (0.83).<sup>20</sup> The results differ across sectors. Using the full national sample, the publicly available satellite indicators explain more variation in population density in the rural and estate sectors (0.808) than the urban sector (0.569) (panel A in Table 6). In the sample with 55 sub-districts, the predictive power of urban population density increases tremendously when adding the proprietary contextual and object features, relative to only including publicly available indicators (0.804 compared to 0.525). Adding proprietary indicators leads only to a limited improvement in rural areas (0.869 compared to 0.838) (panel B in Table 6). This result is consistent with greater uniformity in the relationship between built-up area and population density in rural areas, where simple measures of built-up area are sufficient to predict population density. In urban areas, due to the complex nature of the relationship between population density and buildings where the latter can be both commercial and residential, advanced measures such as object classifiers and contextual measures are required for better prediction.

Random forest (RF) models provide similar results overall and across sector-types (Table 7). The out-of-bag _R_<sup>2</sup> , RMSE, and MAE obtained from the RF models are comparable to the corresponding statistics obtained from the post-LASSO Poisson models. The accuracy pattern across resolution types seen in the LASSO results is also maintained in the RF results. Strikingly, the tremendous improvement in predictive accuracy from using very high resolution indicators in urban areas is corroborated by the RF results. These results collectively indicate that LASSO selection performs as well as random forest models in this setting.<sup>21</sup>

> 20 The out-of-sample _R_ 2 obtained in Table 5 are similar to those obtained in models that do not include log village area (supplementary Table S8), especially in models using higher-resolution indicators. The marginal effects of LASSO-selected variables from the model originally using all public and proprietary indicators are provided in supplementary Table S9. This refers to the model which uses the 55-sub-district sample and _all_ imagery in Table 5.

> 21 While random forest models are known to outperform LASSO models in some contexts (Mullainathan and Spiess, 2017), these results are based on linear LASSO models. The use of Poisson regression in our context may negate the predictive advantage of random forest models, which are traditionally better able to account

11
The size of the training data does not significantly affect the accuracy of the predictions (Table 8). Using a sample of 1,178 villages in the 55 sub-districts, the out-of-sample _R_<sup>2</sup> using all the available satellite data is 0.842, as seen earlier in Table 5. The accuracy increases marginally to 0.843 if we reduce the training sample by half, with a mild reduction to 0.832 if we further reduce the sample size by another one-half. There is only a small increase in the MAE and RMSE with the reduced sample sizes. Overall, these results indicate that changing the training sample size does not significantly change the prediction accuracy.

# **4.3 Survey-Based Predictions and Validation with the Census**

While it is encouraging that satellite data accurately predict census population density, they do not conclusively show that survey data can be used to accurately approximate population density. We now turn to examining this by testing the accuracy of a survey-based model in the 55 sub-districts sample where we estimate a Poisson model of HIES population density in HIES-villages, and obtain out-of-sample density predictions in the non-HIES villages.

The results from a model using both public and proprietary satellite imagery-based indicators are provided in panel A, columns (1)-(3), in Table 9. The _R_<sup>2</sup> , Spearman Rank Correlation (SRC), and Mean Absolute Error (MAE) between the density predictions and the census density are 0.79, 0.91, and 664, respectively. This model uses the inverse of village population as weights. If the weights are corrected based on probability of the village being selected into the HIES, we obtain similar results: 0.78, 0.92, and 665 for the same measures. Considering the _R_<sup>2</sup> and SRC, our model performs equally well compared to the census based model, and better than all “top-down” estimates except Facebook and Worldpop 2015 (panel C).<sup>22</sup> In panel B, we present the same results based on a model using only publicly available satellite imagery. Interestingly, similar out-of-sample prediction accuracies are obtained using only public imagery (0.75, 0.92, and 663, and 0.77, 0.92, and 657 for the three measures using the two types of weights).

We multiplied these density predictions with village-level area to obtain population count predictions, which are reported in columns (4)-(6). The relative performance of the population count predictions, in comparison to the top-down estimates, is similar to those of the population density predictions. The _R_<sup>2</sup> and SRC are generally lower for population counts compared to density predictions, but the MAEs are also lower.

> for non-linear relationships. Further, studies predicting population counts and density have specifically shown mixed results on comparative performances between random forest and LASSO-based predictions (Anderson et al., 2014).

> 22 The out-of-sample _R_ 2, SRC, MAE, in that order, for the “top-down” measures are the following : WorldPop 2015 (0.99, 1.00, and 312) and Facebook’s HRSL (0.91, 0.97, and 417), GHSL (0.84, 0.88, and 639), GPW 2015 (0.68, 0.90, and 1029), GPW 2010 (0.29, 0.28, and 1550), WorldPop 2010 (0.31, 0.81, and 1206), and LandScan (0.04, 0.10, and 2562).

12
The mean and the median relative errors (which are same for population density and count predictions) are reported in columns (7) and (8), respectively.<sup>23</sup> The mean and median relative errors of the model are 37% and 28%, and 35% and 29%, without and with the additional weight correction. Using only public imagery, the mean and median REs are 34% and 28%, and 34% and 28% without and with weights correction, respectively. These mean REs are lower than most top-down estimates, except Facebook and WorldPop 2015. The median REs are higher than only Facebook and Worldpop 2015, and comparable to GHSL and GPW 2015 and much lower compared to GPW, Worldpop, and Landscan (all in 2010).<sup>24</sup> The coefficient of variation (CV) of predicted density estimates for each sub-district based on the HIES model in combination with satellite imagery is low, at about 11%. This is four times lower than the CV of the survey estimates themselves, which stands at 45.2%. The increase in precision due to the incorporation of satellite imagery is tantamount to increasing the share of sampled villages in the population from 17%, which is currently the case, to a remarkable 80%.<sup>25</sup> This is striking but not surprising because the model predicts with an out-of-sample _R_<sup>2</sup> of about 0.80 and the imagery covers all villages.

# **5 Discussion and Conclusion**

Existing methods of estimating local population use a “top-down” dasymetric mapping approach to distribute census data based on a set of covariates derived from satellite imagery. These are only as accurate as the source data—the census—and hence their accuracy declines as the census itself ages. We propose a “bottom-up” technique by pairing survey data with satellite imagery using Poisson regressions employing variables selected based on LASSO regularization. This model can yield population density predictions in areas where the survey was not conducted, and can be repeated frequently using periodic survey and satellite data to frequently track population changes. We apply the method in the context of Sri Lanka to predict population density at the lowest administrative level, the Gram Niladhari (GN) division (village-level). Our model uses the Sri Lankan Household Income and Expenditure Survey (HIES), and predicts density out-of-sample in the non-HIES villages.

Two aspects of our results stand out. First, our performance is better than the only

> 23 The relative error with respect to the census is the same for population density and population count because the only difference between the two is the multiplicative factor, village-area. 24 We conducted a similar exercise using population count as the dependent variable in the model, from which we directly obtained population predictions. The accuracies of these predictions are reported in Table S10. Examining the _R_<sup>2</sup> , SRC, and Mean RE, it is evident that the population predictions derived indirectly using a density-based model by multiplying density predictions with village-area (as in Table 9) performs much better than obtaining population count predictions from a population-based model.

> 25 The CV of 45.2% was obtained by manually calculating the mean CVs from a synthetic 80% sample of the census.

13
two other studies that report out-of-sample validation of “bottom-up” population estimations using similar satellite data (Harvey, 2002; Biljecki et al., 2016). Second, our predictions outperform many “top-down” estimates with the exception of Facebook and WorldPop 2015. Although ours is not the most accurate, it crucially does not _directly_ rely on the census data.

The main reason why this method is useful is because surveys are more frequent and less expensive than censuses, and satellite data can be acquired routinely and with complete spatial coverage, and hence can potentially be more efficiently utilized to track population changes in small areas. Frequent up-to-date small area population estimations are important inputs for governments in applications such as tracking disaster management, delivering policy programs, understanding migration patterns, and many more. The Sri Lankan Department of Census and Statistics conducts the HIES every three years; the most recent was fielded in 2016, and the one before that was in 2012-13, and the next one is planned for 2019. Furthermore, Sri Lanka runs a continual Labor Force Survey that could also be used for this purpose. Therefore, the results support the case for the Department of Census and Statistics to keep careful track of the number of buildings identified in the relisting phase, and use that information to generate revised local population estimates.

14
# **References**

- Alegana, V. A., Atkinson, P. M., Pezzulo, C., Sorichetta, A., Weiss, D., Bird, T., ErbachSchoenberg, E., and Tatem, A. J. (2015). Fine resolution mapping of population agestructures for health and development applications. _Journal of The Royal Society Interface_ , 12(105):20150073.

- Anderson, W., Guikema, S., Zaitchik, B., and Pan, W. (2014). Methods for Estimating Population Density in Data-Limited Areas: Evaluating Regression and Tree-Based Models in Peru. _PloS one_ , 9(7):e100037.

- Athey, S. (2017). Beyond prediction: Using big data for policy problems. _Science_ , 355(6324):483–485.

- Biljecki, F., Ohori, K. A., Ledoux, H., Peters, R., and Stoter, J. (2016). Population estimation using a 3d city model: A multi-scale country-wide study in the netherlands. _PloS one_ , 11(6):e0156808.

- Centre for International Earth Science Information Network (2018). High resolution settlement layer. Retrieved from https://www.ciesin.columbia.edu/data/hrsl/ on August 4th 2018.

- Department of Census and Statistics (2015). Household income and expenditure survey 2012/13. final report. Department of Census and Statistics, Ministry of Policy Planning Economic Affairs, Child Youth and Cultural Affairs, Sri Lanka.

- Deville, P., Linard, C., Martin, S., Gilbert, M., Stevens, F. R., Gaughan, A. E., Blondel, V. D., and Tatem, A. J. (2014). Dynamic population mapping using mobile phone data. _Proceedings of the National Academy of Sciences_ , 111(45):15888–15893.

- Engstrom, R., Hersh, J., and Newhouse, D. (2017). Poverty from space: Using high-resolution satellite imagery for estimating economic well-being. World Bank Working Paper No. 8284.

- European Commission (2018). Global Human Settlement (GHSL). Retrieved from https://ghsl.jrc.ec.europa.eu/data.php on November 19th 2018.

- Guiteras, R., Levinsohn, J., and Mobarak, M. (2018). Demand estimation with strategic complementarities: Sanitation in bangladesh.

- Hansen, M. C., Potapov, P. V., Moore, R., Hancher, M., Turubanova, S., Tyukavina, A., Thau, D., Stehman, S., Goetz, S., Loveland, T., et al. (2013). High-resolution global maps of 21st-century forest cover change. _science_ , 342(6160):850–853.

- Harvey, J. (2002). Estimating census district populations from satellite imagery: some approaches and limitations. _International Journal of Remote Sensing_ , 23(10):2071–2095.

- Hastie, T. and Qian, J. (2018). Glmnet vignette. Retrieved from https://web.stanford.edu/ hastie/glmnet/glmnet _alpha.htmlpoi_ on November 19th 2018.

15
- Hillson, R., Alejandre, J. D., Jacobsen, K. H., Ansumana, R., Bockarie, A. S., Bangura, U., Lamin, J. M., Malanoski, A. P., and Stenger, D. A. (2014). Methods for determining the uncertainty of population estimates derived from satellite imagery and limited survey data: a case study of bo city, sierra leone. _PloS one_ , 9(11):e112241.

- Horvitz, D. G. and Thompson, D. J. (1952). A generalization of sampling without replacement from a finite universe. _Journal of the American statistical Association_ , 47(260):663–685.

- Jean, N., Burke, M., Xie, M., Davis, W. M., Lobell, D. B., and Ermon, S. (2016). Combining satellite imagery and machine learning to predict poverty. _Science_ , 353(6301):790–794.

- Li, G. and Weng, Q. (2005). Using landsat etm+ imagery to measure population density in indianapolis, indiana, usa. _Photogrammetric Engineering & Remote Sensing_ , 71(8):947–958.

- McBride, L. and Nichols, A. (2016). Retooling poverty targeting using out-of-sample validation and machine learning. _The World Bank Economic Review_ .

- Mullainathan, S. and Spiess, J. (2017). Machine learning: an applied econometric approach. _Journal of Economic Perspectives_ , 31(2):87–106.

- Oakridge National Laboratory (2018). Documentation page of the landscan. Retrieved from https://landscan.ornl.gov/documentation on August 3rd 2018.

- Socioeconomic Data and Applications Center (2018). Gridded Population of the World (GPW). Retrieved from http://sedac.ciesin.columbia.edu/data/collection/gpwv4/methods/method1 on November 19th 2018.

- Stevens, F. R., Gaughan, A. E., Linard, C., and Tatem, A. J. (2015). Disaggregating census data for population mapping using random forests with remotely-sensed and ancillary data. _PloS one_ , 10(2):e0107042.

- Sutton, P., Roberts, D., Elvidge, C., and Baugh, K. (2001). Census from heaven: An estimate of the global human population using night-time satellite imagery. _International Journal of Remote Sensing_ , 22(16):3061–3076.

- Varian, H. R. (2014). Big data: New tricks for econometrics. _Journal of Economic Perspectives_ , 28(2):3–28.

- Vogel, K., Goldblatt, R., Hanson, G., and Khandelwal, A. (2018). Detecting urban markets with satellite imagery. International Growth Centre Working Paper C-89448-INC-1.

- Wardrop, N., Jochem, W., Bird, T., Chamberlain, H., Clarke, D., Kerr, D., Bengtsson, L., Juran, S., Seaman, V., and Tatem, A. (2018). Spatially disaggregated population estimates in the absence of national population and housing census data. _Proceedings of the National Academy of Sciences_ , 115(14):3529–3537.

- Wooldridge, J. M. (2002). Inverse probability weighted m-estimators for sample selection, attrition, and stratification. _Portuguese Economic Journal_ , 1(2):117–139.

- WorldPop (2018). World pop website and methods page. Retrieved from http://www.worldpop.org.uk/data/methods/ on August 4th 2018.

16
<!-- Start of picture text --> (e=|<br>oO<br>oO<br>wo<br>oO<br><8<br>ot<br>=<br><<br>Cc<br>22<br>oS<br>Oo<br>o<br>ro<br>=<br>2<br>sO<br>rs) N<br>ao<br>=F} -<br>oO<br>oO<br>=)<br>2020 2040 2060 2080 2100<br>year<br>—+*— High-income countries ——*— _ Upper-middle-income countries<br>——*—_ Lower-middle-income countries ——*—-_ Low-income countries<br><!-- End of picture text -->
Table 1: Satellite Indicators by Resolution and Availability

|Indicator|Source|Time-frame|Coverage|Resolution|Availability|
|---|---|---|---|---|---|
|_A. Population_||||||
|Population|Census||National||public|
|Population|HIES||1421 villages||public|
|Population|Facebook||National||public|
|Population|WorldPop||National||public|
|Population|GPW||National||public|
|Population|LandScan||National||public|
|Population|GHSL||National||public|
|_B. Other sources_||||||
|Night time lights|VIIRS|2014|National|Low|public|
|Elevation|ASTER||National|Low|public|
|Slope|ASTER||National|Low|public|
|Tree cover|Hansen et al. (2013)|2000|National|Low|public|
|Tree cover gain and loss|Hansen et al. (2013)|2000 to 2014|National|Low|public|
|Built-up area|GUF, GUF+|2012, 2015|National|High|public|
|Built-up area|GHSL|2014|National|High|public|
|Built-up area|Facebook|2015|National|High|public|
|Built-up area|Engstrom et al. (2017)|2011, 2012|55 sub-dist.|High|proprietary|
|Cars|Engstrom et al. (2017)|2011, 2012|55 sub-dist.|High|proprietary|
|Shadows|Engstrom et al. (2017)|2011, 2012|55 sub-dist.|High|proprietary|
|Roof type|Engstrom et al. (2017)|2011, 2012|55 sub-dist.|High|proprietary|
|Road type|Engstrom et al. (2017)|2011, 2012|55 sub-dist.|High|proprietary|
|Agricultural land type|Engstrom et al. (2017)|2011, 2012|55 sub-dist.|High|proprietary|
|Paddy land type|Engstrom et al. (2017)|2011, 2012|55 sub-dist.|High|proprietary|
|NDVI|Engstrom et al. (2017)|2011, 2012|55 sub-dist.|High|proprietary|
|Other Contextual Indicators|Engstrom et al. (2017)|2011, 2012|55 sub-dist.|High|proprietary|

Note: HIES=the Household Income and Expenditure Survey; GPW=Gridded Population of the World; GHSL=Global Human Settlement Layer; GUF= Global Urban Footprint; VIIRS= Visible Infrared Imaging Radiometer Suite; ASTER=Advanced Spaceborne Thermal Emission and Reflection Radiometer; NDVI = Normalized Difference Vegetation Index. Engstrom et al. (2017) use proprietary imagery from DigitalGlobe to derive features and textures for 55 sub-district divisions. “Other Contextual indicators” include SURF, Pantex, and Histogram of Oriented Gradients. The time frame refers to the year(s) during which the satellite images were obtained.

18
Table 2: Summary Statistics: Villages in the National Sample

|Indicator|Mean|Std. Dev.|Min|Max|
|---|---|---|---|---|
|_Geographic Descriptors_|||||
|log village area|1.25|0.81|0.04|6.35|
|Share of urban villages|0.092|0.289|0.00|1.00|
|_Population summary_|||||
|Village level|||||
|Village Census Population Density(per km<sup>2</sup>)|1,400|2,703|0|50,126|
|Village HIES Population Density(per km<sup>2</sup>; N=2,348)|1,996|3,394|0.607|49,209|
|Village population- Census|1,455|1,244|12|28,003|
|Village population estimate - Facebook|1,477|1,334|0|30,046|
|Village population estimate - WorldPop 2010|1,451|1,246|0|28,003|
|Village population estimate - WorldPop 2015|1,263|1,081|14|26,870|
|Village Population estimate - GPW 2010|1,387|1,652|0|38,355|
|Village Population estimate - GPW 2015|1,650|1,689|18|37,381|
|Village population estimate - LandScan|1,519|2,873|0|61,275|
|Village population estimate - GHSL|1,468|1,475|0|38,775|
|Sub-district level (N=332)|||||
|Sub-district population- Census|61,322|48,605|0|323,223|
|Sub-district population estimate - Facebook|62,223|49,471|3,217|318,643|
|Sub-district population estimate - WorldPop 2010|61,252|42,621|4,742|261,693|
|Sub-district population estimate - WorldPop 2015|53,119|42,095|281|277,480|
|Sub-district Population estimate - GPW 2010|58,442|45,983|3,505|330,155|
|Sub-district Population estimate - GPW 2015|61,062|48,140|3,291|318,456|
|Sub-district population estimate - LandScan|64,211|44,390|0|268,650|
|Sub-district population estimate - GHSL|61,772|48,865|1,661|310,012|
|Key Satellite Imagery Based indicators|||||
|_Low Resolution Public Indicators_|||||
|Night-time lights March 2014 - Mean|0.98|2.25|0.000|72.57|
|Night-time lights March 2014 - Maximum|1.45|4.13|0.000|274.93|
|Night-time lights September 2014 - Mean|0.95|2.04|0.000|56.61|
|Night-time lights September 2014 - Maximum|1.39|3.28|0.0|170.62|
|Mean Elevation|197.13|313.35|3.4|2,214.55|
|Mean Slope|8.68|5.15|1.2|28.79|
|Mean Tree Cover|46.93|26.91|0.0|97.26|
|Gain in Tree Cover|0.02|0.02|0.000|0.25|
|Loss in Tree Cover|0.005|0.012|0.000|0.330|
|_High Resolution Public Indicators_|||||
|GUF Built-up Area|10.60|20.37|0.0|100.00|
|GUF+ Built-up Area|14.21|25.83|0.0|101.05|
|GHSL Built-up Area|15.41|24.88|0.0|100.00|
|Facebook Built-up Area (N= 13,437)|15.65|19.16|0.0|99.55|
|Observations (N)||13,9|70||

Note: GPW=Gridded Population of the World; GHSL=Global Human Settlement Layer; HIES=Household Income and Expenditure Survey; GUF= Global Urban Footprint; NDVI = Normalized Difference Vegetation Index.

19
Table 3: Summary statistics: Villages in the 55 Sub-districts Sample

||Mean|Std. Dev.|Min|Max|
|---|---|---|---|---|
|_Geographic Descriptors_|||||
|Log village area|0.97|0.60|0.09|4.27|
|Share of urban villages|0.28|0.45|0.00|1.00|
|_Population Summary_|||||
|Village level|||||
|Village Census Population Density|2621|3507|22|43,984|
|Village HIES Population Density(per km<sup>2</sup>; N=414)|2,821|3,576|10.22|38,609|
|Village population- Census|2341|2183|212|28,003|
|Village population estimate - Facebook|2350|2176|82|30,046|
|Village population estimate - WorldPop 2010|2014|1928|0|21,326|
|Village population estimate - WorldPop 2015 (N=1,357)|2047|1919|215|26,870|
|Village Population estimate - GPW 2010|2192|2886|0|36,832|
|Village Population estimate - GPW 2015 (N=1,145)|2747|2813|146|34,058|
|Village Population estimate - LandScan|1359|2655|0|40,639|
|Village Population estimate - GHSL|1359|2655|0|40,639|
|Sub-district level (N=55)|||||
|Sub-district population- Census|57,887|54,117|1,709|2,40,114|
|Sub-district population estimate - Facebook|58,100|55,066|1,714|2,51,530|
|Sub-district population estimate - WorldPop 2010|49,812|46,543|1,222|1,96,328|
|Sub-district population estimate - WorldPop 2015|50,494|47,191|1,436|2,07,748|
|Sub-district Population estimate - GPW 2010|54,212|52,689|2,025|2,28,419|
|Sub-district Population estimate - GPW 2015|57,181|54,766|1,655|2,41,757|
|Sub-district population estimate - LandScan|33,606|37,591|0.00|1,53,304|
|Sub-district population estimate - GHSL|57,799|55,175|1,708|2,46,673|
|_Low Resolution Public Indicators_|||||
|Night-time lights March 2014 - Mean|1.39|3.27|0.00|57.21|
|Night-time lights March 2014 - Maximum|2.10|8.54|0.00|274.93|
|Night-time lights September 2014 - Mean|1.26|2.50|0.00|28.96|
|Night-time lights September 2014 - Maximum|1.76|3.88|0.00|92.13|
|Mean Elevation|209.96|428.52|5.56|2214.55|
|Mean Slope|7.63|5.14|1.15|23.89|
|Mean Tree Cover|44.22|26.35|0.00|92.24|
|Gain in Tree Cover|0.02|0.03|0.00|0.22|
|Loss in Tree Cover|0.01|0.02|0.00|0.33|
|_High Resolution Public Indicators_|||||
|GUF Built-up Area|21.81|26.92|0.00|97.11|
|GUF+ Built-up Area|31.68|36.52|0.00|100.11|
|GHSL Built-up Area|33.59|35.68|0.00|100.00|

_Continued on next page_ 20
Table 3 – _Continued_ _<u>from previous page</u>_

||Mean|Std. Dev.|Min|Max|
|---|---|---|---|---|
|Facebook Built-up Area (N=1,275)|30.16|28.18|0.67|97.43|
|_Very High resolution Proprietary indicators_<br>_Road variables_|||||
|% of roads that are minor paved (4 m width)|6.78|15.50|0.00|100|
|% of roads that are main paved (5 m width)|11.32|15.07|0.00|100|
|% of roads that are paved city (4 m width)|11.74|17.64|0.00|92.76|
|_Building Density and Vegetation_|||||
|% shadow pixels covering valid area (N=1,353)|5.80|5.47|0.27|38.17|
|NDVI, mean scale 32|0.52|0.17|0.00|0.95|
|Total built-up area|69,757|68,699|0|7,08,867|
|_Roof type_|||||
|Fraction of total roofs that are clay|35.80|20.67|0.00|100|
|Fraction of total roofs that are aluminum|14.26|7.27|0.00|71.92|
|Fraction of total roofs are asbestos|7.79|11.66|0.00|71.20|
|_Cars_|||||
|log number of cars (N=1,252)<br>_Agricultural Land_|3.42|1.02|0.88|8.30|
|% of Village agriculture that is paddy|44.46|37.65|0.00|100.00|
|% of Village agriculture that is plantation|55.14|37.58|0.00|100.00|
|_Textural and spectral characteristics_|||||
|Pantex (human settlements) mean, scale 8m|0.56|0.50|0.00|3.95|
|Pantex (human settlements) mean, scale 32m|0.66|0.59|0.00|4.68|
|Gabor flter (scale 64m, features 6), mean|0.67|0.32|0.01|2.03|
|Gabor flter (scale 64m, features 14), mean|0.68|0.31|0.02|1.95|
|Histogram of Oriented Gradients (scale 16m), mean|37.91|9.66|0.00|146.18|
|Observations (N)||1,360|||

Note: GPW=Gridded Population of the World; GHSL=Global Human Settlement Layer. HIES=Household Income and Expenditure Survey; GUF= Global Urban Footprint; NDVI = Normalized Difference Vegetation Index. The NDVI scale refers to the size of the window used to calculate average NDVI. Summary statistics of LASSO-selected indicators across all models are reported.

21
Table 4: Validation of Population Estimates

|_R_<sup>2</sup>|Census|Facebook|WorldPop 2010|A. Sub-district<br>WorldPop 2015|Level<br>GPW 2010|GPW 2015|GHSL|LandScan|
|---|---|---|---|---|---|---|---|---|
|Census|1||||||||
|Facebook|.996|1|||||||
|WorldPop 2010|.753|.758|1||||||
|WorldPop 2015|.999|.995|.757|1|||||
|GPW 2010|.985|.982|.748|.986|1||||
|GPW 2015|.986|.989|.770|.988|.990|1|||
|GHSL|.994|.996|.766|.995|.982|.990|1||
|LandScan|.015|.013|.026|.015|.022|.017|.014|1|
|_Average discrepa_|_ncy relati_|_ve to census_|||||||
|Persons|0<br>|2468<br>|15107<br>|10360<br>|4948<br>|3663<br>|2778<br>|46936<br>|
|Percent|0%|4.02%|21.03%|13.38%|6.72%|5.37%|4.79%|69.58%|
|||||B. Village L|evel||||
|_R_<sup>2</sup>|Census|Facebook|WorldPop 2010|WorldPop 2015|GPW 2010|GPW 2015|GHSL|LandScan|
|Census|1||||||||
|Facebook|.841|1|||||||
|WorldPop 2010|.107|.093|1||||||
|WorldPop 2015|.988|.833|.107|1|||||
|GPW 2010|.404|.466|.049|.420|1||||
|GPW 2015|.595|.634|081|.604|.595|1|||
|GHSL|.672|.801|.076|.671|.408|.527|1||
|LandScan|.0006|.0009|.0008|.00007|.00007|.0009|.0007|1|
|_Average discrepa_|_ncy relati_|_ve to census_|||||||
|Persons|0|292|733|197|669|553|515|1684|
|Percent|0%|22.62%|72.12%|13.5%|51.02%|45.77%|42.42%|197.37%|

Note: GPW=Gridded Population of the World; GHSL=Global Human Settlement Layer. Discrepancy is the absolute value of the difference between population estimates and the census at the village level. Percent relative to the census is (absolute-discrepancy/census-population) X 100.

22
Table 5: Accuracy of Population Density Predictions based on LASSO Selected Poisson Regression of Population Density

|Imagery Type|In-sample _R_<sup>2</sup>|Out-of-sample _R_<sup>2</sup>|Number of <br>Candidate|variables<br>Selected|No. of Villages|
|---|---|---|---|---|---|
|A. National Sample||||||
|1. No satellite imagery|0.806|0.650|35|15|13,970|
|2. Low-resolution, public|0.843|0.702|35|18|13,970|
|3. All public|0.888|0.750|39|13|13,437|
|B. 55 sub-district sample||||||
|1. No satellite imagery|0.731|0.589|35|5|1,360|
|2. Low-resolution, public|0.804|0.677|35|7|1,360|
|3. All public|0.858|0.710|39|9|1,275|
|4. All public + propriety texture|0.874|0.755|46|11|1,275|
|5. All|0.918|0.830|67|23|1,178|

Note: The results are based on a Poisson regression model whose dependent variable is the census village population density, and the independent variables are selected based on LASSO regularization among the following: Relevant satellite imagery sources as described in panel B in Table 1, district fixed effects, a binary indicator for urban villages, and log-village area. The models with “No satellite imagery” use only the district fixed effects, an indicator for urban villages, and log village-area as independent variables. “Lowresolution public” indicators refer to the low-resolution and public indicators as defined in Table 1. “All public” indicators refers to all the public imagery based indicators, including the high-resolution imagery based indicators described in Table 1. “Proprietary texture” indicators include very high-resolution texture features based on Digital Globe from Engstrom et al. (2017) as defined in Table 1. “All” imagery refers to all public and proprietary texture indicators, and object identifiers, such as cars, roofs etc., as defined as in Table 1. The out-of sample _R_<sup>2</sup> is obtained from stata’s crossfold command using five-fold cross-validation.

23
Table 6: Accuracy of Population Density Predictions, by Sector and Nature of Indicators

|Imagery Type|In-sample _R_<sup>2</sup>|Out-of-sample _R_<sup>2</sup><br>A. Nati|Number of <br>Candidate<br>onal Sample|variables<br>Selected|No. of villages|
|---|---|---|---|---|---|
|A. Rural and estate||||||
|1. No satellite imagery|0.757|0.646|35|23|12,686|
|2. Low resolution, public|0.806|0.726|35|25|12,686|
|3. All Public|0.877|0.808|39|19|12,389|
|B. Urban||||||
|1. No satellite imagery|0.595|0.493|35|3|1,284|
|2. Low resolution, public|0.654|0.540|35|7|1,284|
|3. All Public|0.690|0.569|39|6|1,048|
|||A. 55 s|ub-districts|||
|A. Rural and estate||||||
|1. No satellite imagery|0.752|0.688|35|8|978|
|2. Low resolution, public|0.815|0.754|35|8|978|
|3. All Public|0.898|0.838|39|8|935|
|4. All Public + Propriety texture|0.919|0.882|46|24|935|
|5. All|0.942|0.869|67|41|881|
|B. Urban||||||
|1. No satellite imagery|0.487|0.413|35|3|382|
|2. Low resolution, public|0.637|0.511|35|5|382|
|3. All Public|0.665|0.525|39|7|340|
|3. All Public + Propriety texture|0.740|0.668|46|8|340|
|5. All|0.860|0.804|67|26|336|

Note: The results are based on a Poisson regression model whose dependent variable is census village population density, and the independent variables are selected based on LASSO regularization among the following: Relevant satellite imagery sources as described in panel B in Table 1, district fixed effects, a binary indicator for urban villages, and log village area. The models with “No satellite imagery” use only the district fixed effects, an indicator for urban villages, and log village-area as independent variables. “Low-resolution public” indicators refer to the low-resolution and public indicators as defined in Table 1. “All public” indicators refers to all the public imagery based indicators, including the high-resolution imagery based indicators described in Table 1. “Proprietary texture” indicators include very high-resolution texture features based on Digital Globe from Engstrom et al. (2017) as defined in Table 1. “All” imagery refers to all public and proprietary texture indicators, and object identifiers, such as cars, roofs etc., as defined as in Table 1. The out-of sample _R_<sup>2</sup> is obtained from stata’s crossfold command using five-fold cross-validation.

24
Table 7: Random Forest model Results in comparison with LASSO-selected Poisson Regression Results

||Ran|dom Fore|st|L|ASSO||
|---|---|---|---|---|---|---|
|Imagery Type|Out-of-<br>bag _R_<sup>2</sup>|RMSE|MAE|Out-of-<br>sample _R_<sup>2</sup>|RMSE|MAE|
|All sectors|||||||
|A. National sample|||||||
|1. Low resolution, public|0.767|1282|477|0.702|1463|563|
|2. All Public|0.783|1237|390|0.750|1323|459|
|B. Subsample of 55 sub-districts|||||||
|1. Low resolution, public|0.677|2088|948|0.677|1948|928|
|2. All Public|0.707|1984|784|0.710|1927|818|
|3. All Public + Propriety texture|0.719|1942|779|0.755|1845|808|
|4. All|0.792|1673|684|0.830|1506|698|
|Rural and Estate|||||||
|A. National sample|||||||
|1. Low resolution, public|0.776|696|329|0.726|788|375|
|2. All Public|0.832|601|246|0.808|653|289|
|B. Subsample of 55 sub-districts|||||||
|1. Low resolution, public|0.807|968|510|0.754|990|550|
|2. All Public|0.847|855|383|0.838|848|424|
|3. All Public + Propriety texture|0.849|850|380|0.882|788|415|
|4. All|0.872|781|352|0.869|747|355|
|Urban|||||||
|A. National sample|||||||
|1. Low resolution, public|0.598|3966|2285|0.540|4105|2344|
|2. All Public|0.612|3890|2079|0.569|3999|2343|
|B. Subsample of 55 sub-districts|||||||
|1. Low resolution, public|0.424|3680|2084|0.511|3277|1917|
|2. All Public|0.461|3559|1888|0.525|3357|1909|
|3. All Public + Propriety texture|0.491|3457|1850|0.668|2897|1676|
|4. All|0.626|2984|1620|0.804|2226|1446|

Note: Random Forest models are implemented using the randomForest() package in R with 500 trees, and using the census village population density as the dependent variable. The following independent variables were used in all models: Different types of satellite imagery sources as described in panel B in Table 1, district fixed effects, a binary indicator for urban villages, and log village-area. “Low-resolution public” indicators refer to the low-resolution and public indicators as defined in Table 1. “All public” indicators refers to all the public imagery based indicators, including the high-resolution imagery based indicators described in Table 1. “Proprietary texture” indicators include very high-resolution texture features based on Digital Globe from Engstrom et al. (2017) as defined in Table 1. “All” imagery refers to all public and proprietary texture indicators, and object identifiers, such as cars, roofs etc., as defined as in Table 1. The out-of-sample _R_<sup>2</sup> , Mean Absolute Error (MAE), and Root Mean Squared Error (RMSE) for the LASSO based Poisson regression are from tables 8-10.

25
Table 8: Accuracy of Population Density Predictions, by Training Sample Size

|||In-sample|Out-|of-samp|le|
|---|---|---|---|---|---|
|Sample Type|Number of Villages|Pseudo _R_<sup>2</sup>|Pseudo _R_<sup>2</sup>|MAE|RMSE|
|Medium sample|1178|0.918|0.842|1490|712|
|Small sample (50% of medium)|593|0.908|0.843|1447|698|
|Very small sample (50% of small)|292|0.920|0.832|1691|738|

Note: The results are based on a Poisson regression model whose dependent variable is census village population density, and the independent variables are selected based on LASSO regularization among the following: All indicators from public and proprietary imagery sources as described in panel B in Table 1, sectoral and district fixed effects, and log village-area. The out-of sample _R_<sup>2</sup> is obtained from stata’s crossfold command using five-fold cross-validation. MAE refers to mean absolute error and RMSE refers to Root Mean Squared Error.

26
Table 9: Out-of-sample Accuracy of HIES-based Village Population Density Estimates using Public and Proprietary satellite data, 55 Sub-Districts

||Popul|ation D|ensity|Popu|lation|Count|||
|---|---|---|---|---|---|---|---|---|
||_R_<sup>2</sup>|SRC|Mean<br>AE|_R_<sup>2</sup>|SRC|Mean<br>AE|Mean<br>RE|Median<br>RE|
||(1)|(2)|(3)|(4)|(5)|(6)|(7)|(8)|
|_A.Model estimates, Public & Propr_|_ietary _|_satellit_|_e image_|_ry_|||||
|HIES-based|0.79|0.91|664|0.60|0.67|607|37%|28%|
|HIES-based (balance correction)|0.78|0.92|665|0.58|0.67|607|35%|29%|
|Census-based|0.81|0.91|679|0.60|0.65|644|50%|28%|
|_B.Model estimates, public satellite _|_image_|_ry_|||||||
|HIES-based|0.75|0.92|663|0.56|0.69|595|34%|28%|
|HIES-based (balance correction)|0.77|0.92|657|0.55|0.69|606|34%|28%|
|Census-based|0.76|0.91|750|0.56|0.63|739|58%|36%|
|_C. Existing areal interpolation esti_|_mates_||||||||
|WorldPop 2015|0.99|1.00|312|0.99|0.99|237|13%|13%|
|Facebook HSRL 2015|0.91|0.97|417|0.90|0.93|309|20%|14%|
|GHSL Grid 2014|0.84|0.88|639|0.74|0.79|556|39%|26%|
|GPW 2015|0.68|0.90|1029|0.50|0.78|740|44%|27%|
|GPW 2010|0.29|0.28|1550|0.39|0.48|911|52%|35%|
|WorldPop 2010|0.31|0.81|1206|0.06|0.44|987|64%|34%|
|LandScan 2010|0.04|0.10|2562|0.01|-0.24|2012|135%|92%|

Note: HIES=the Household Income and Expenditure Survey; HRSL= High Resolution Settlement Layer; GHSL=Global Human Settlement Layer; GPW=Gridded Population of the World; SRC=Spearman Rank Correlation; AE= Absolute error; RE=Relative Error (with respect to the census). We retain only the villages in the 55 sub-districts for which the proprietary data are available. Models are estimated on subsample of 414 villages covered in 2012-2013 HIES survey. The satellite indicators used as covariates include all publicly available and all proprietary indicators. Out-of-sample prediction accuracy (with the census) are conducted on the 946 villages not covered in the HIES. The weights for the HIES-based model are _V illage P opulation_ <u>1</u> _v_<sup>.Balancecorrectionreferstoadjustmentof</sup> � weights based on the predicted probability of the village being sampled in HIES ( _INHIESv_ ) based on a probit model with village sampled in HIES as the dependent variable and all LASSO-selected variables as independent variables. <u>1</u> The balanced corrected weights are �<sup>Population count predictions were obtained by mul-</sup> _V illageP opulationv XINHIESv_<sup>.</sup> tiplying density predictions by village-level area. The accuracy parameters for population predictions are calculated based on comparisons with actual census population and reported. The mean and median RE are mathematically the same for population density and population count predictions.

27