KidSat: satellite imagery to map childhood poverty

## Abstract

Abstract Satellite imagery has emerged as an important tool to analyse demographic, health, and development indicators. While various deep learning models have been built for these tasks, each is specific to a particular problem, with few standard benchmarks available. We propose a new dataset pairing satellite imagery and high-quality survey data on child poverty to benchmark satellite feature representations. Our dataset consists of 33,608 images, each 10 km × \times × 10 km, from 19 countries in Eastern and Southern Africa in the time period 1997-2022. As defined by UNICEF, multidimensional child poverty covers six dimensions and it can be calculated from the face-to-face Demographic and Health Surveys (DHS) Program [ 25 ] . As part of the benchmark, we test spatial as well as temporal generalization, by testing on unseen locations, and on data after the training years.
Using our dataset we benchmark multiple models, from low-level satellite imagery models such as MOSAIKS [ 20 ] , to deep learning foundation models, which include both generic vision models such as Self-Distillation with no Labels (DINOv2) models [ 15 ] and specific satellite imagery models such as SatMAE [ 6 ] . We provide open source code for building the satellite dataset, obtaining ground truth data from DHS and running various models assessed in our work.

## 1 Introduction

Major satellites like the Landsat or Sentinel program regularly circle the globe, providing updated, publicly available, high-resolution imagery every 1-2 weeks. An emerging literature in remote sensing and machine learning points to the promise that these large datasets, combined with deep learning methods, hold to enable applications in agriculture, health, development, and disaster response. A cross disciplinary set of publications hint at the potential impact, showing how satellite imagery can be used to estimate the causal impact of electrification on livelihoods [], to measure income, overcrowding, and environmental deprivation in urban areas [] and to predict human population in the absence of census data []. Despite these successes, machine learning for satellite imagery is not yet a well-developed field [], with current approaches overlooking the unique features of satellite images such as variation in spatial resolution over logarithmic scales (from < 1 meter to > 1 km) [] and the heterogeneous nature of satellite imagery in terms of the number of bands available from 3 bands for RGB to multispectral to hyperspectral.

Many areas of machine learning have advanced through the development of standardized datasets and benchmarks. Given the wide set of possible use cases for satellite imagery, there is no doubt room for multiple benchmarks. However there are only a few sources of up-to-date, high-quality satellite imagery, especially Landsat and Sentinel, so it is natural to construct publicly available datasets using these satellite programs.

Given the proven effectiveness of remote sensing for tasks that are naturally visible from space, such as land usage prediction, crop yield forecasting, and deforestation, we instead choose to focus on a more difficult task: multidimensional child poverty.

Of the 8 billion people in the world, over 2 billion are children (aged $<$ 18 years old, as defined in the UN Convention on the Rights of the Child []). Child poverty is not the same as adult poverty; children are growing and developing so they have specific nutrition, health, and education needs—if these needs are not met, there can be lifelong negative consequences []. Poverty cannot simply be assessed by measuring overall household resources, as households may be very unequal and some of the needs of children, such as vaccines or education, may be neglected in households that are non-poor. Instead, child poverty must be measured at the level of the child and their experience [].

Child poverty is based on the “constitutive rights of poverty” []. What this concept means is that child poverty includes important dimensions for children that require material resources to realize them, like education, health and nutrition, but exclude non-material dimensions such as neglect, violence, and lack of privacy. Crucially for the purposes of establishing a useful dataset and benchmark, the internationally agreed definition of child poverty was designed to enable cross-country comparisons [].

While other benchmarks exist, most notably SUSTAIN-BENCH [] which covers a range of sustainable development indicators, our newly proposed benchmark has the following features:

- •
We demonstrate the importance of fine-tuning transformer-based foundation vision models to tackle a challenging prediction task.
- •
Child poverty is both a multidimensional outcome, appropriate for fine-tuning large models, and a univariate measure (percent of children experiencing severe deprivation ranging from 0% and 100%) meaning that model performance can be intuitively grasped by policymakers.
- •
The amount of both satellite and survey data appropriate for child poverty prediction will continue to increase in the future, as UNICEF is now releasing geocodes as part of their Multiple Indicator Cluster Survey (MICS) program.

Figure: (a) Kriging estimates using Kenya 2022 DHS
Refer to caption: x1.png

## 2 Related Work

### 2.1 Existing Satellite Imagery Datasets

With increased access to freely available high resolution satellite imagery through the Landsat and Sentinel programs, satellite image datasets have become very popular for training machine learning models. Models and datasets include
functional map of the world (fMoW) [], XView [], Spacenet [], and Floodnet [] where the tasks are object detection, instance segmentation, and semantic segmentation. These are computer vision-specific tasks, rather than applied health and economic prediction problems, meaning the use of these datasets and models may be inappropriate for applied health and development researchers and practitioners.

### 2.2 Satellite Imagery for Demographic and Health Indicators

Machine learning models applied to satellite images are becoming more commonplace for analysing demographic, health, and development indicators as they can increase coverage by allowing for interpolation and faster analysis in under-surveyed regions. In an early work, satellite images were used to track human development at increasing spatial and temporal granularity []. Since then satellite images have been used to track development indicators which are clearly visible from space such as agriculture and deforestation patterns [, , ] but also more abstract quantities such as poverty levels [], health indicators [], and the Human Development Index [].

### 2.3 Foundation Satellite Image Models

As increasing volumes of data become available, and with progress in self-supervised learning [, ], many foundation models are emerging. In computer vision, these large models are trained with self-supervised learning on hundreds of millions of images, serving as a “foundation” from which they can be fine-tuned for specific tasks. Popular examples of this are Vision transformers [], CLIP [], and DINO [].
Recently, foundation models have been trained for satellite imagery specifically on vast amounts of unlabelled satellite images. Examples of these are SatMAE [] based on masked autoencoders, SatCLIP [] based on CLIP [], and DiffusionSat [] which is a diffusion model [] for generating satellite images.
As it is not yet clear whether there is a benefit from training foundation models on more specific, but smaller datasets, we benchmark both generic foundation models for computer vision as well as satellite-specific foundation models.

## 3 Dataset

In this section, we introduce our unique dataset derived from the Demographic and Health Surveys (DHS) Program, combining high-resolution satellite imagery with detailed numerical survey data focused on demographic and health-related aspects in Eastern and Southern Africa. This dataset leverages the rigorous survey methodologies from DHS to offer high-quality data on health and demographic indicators, complemented by satellite images of the surveyed locations. The rich information embedded in the satellite images enables the application of advanced deep learning methods to estimate key poverty indicators in unsurveyed locations.

### 3.1 Satellite Images

This study utilizes high-resolution satellite imagery from two primary sources: Sentinel-2 and Landsat 5, 7, and 8. These satellite programs are chosen for their public accessibility, their specific advantages in computer vision applications, and their long history.

Landsat 5, 7, and 8: Part of a series managed by the United States Geological Survey (USGS), Landsat 5,7, and 8 together provide imagery with varying resolutions covering the time span from 1984 to the present (2024). Specifically, Landsat 8 captures data in 11 bands, including visible, near-infrared (NIR), and short-wave infrared (SWIR) at 30 meters resolution, panchromatic at 15 meters, and thermal infrared (TIRS) bands at 100 meters. This lower resolution for RGB bands contributes to space efficiency in data storage and processing, making it suitable for large-scale studies over extensive geographical areas. Additionally, the Landsat program, with missions dating back to 1972, provides an extensive historical archive of Earth imagery. This long timespan is particularly advantageous for our study as it allows the analysis of regions with survey data dating back to 1997.

Sentinel-2: Operated by the European Space Agency (ESA), Sentinel-2 features a multispectral imager with 13 spectral bands. The resolution varies by band: 10 meters for RGB and NIR, 20 meters for red edge and SWIR bands, and 60 meters for atmospheric bands. This high resolution in RGB bands provides richer information, which is valuable for large vision models requiring detailed visual data for accurate analysis. However, processing such high-resolution imagery can be computationally expensive, especially when dealing with large window sizes, posing challenges in terms of processing time and resource allocation. Additionally, Sentinel-2 only started collecting data in June 2015, which limits its use for analyzing events or changes that occurred before this date.

For each specified survey coordinate, we extract a 10 km $\times$ 10 km window of imagery using Google Earth Engine (GEE). Selection criteria for the imagery include the designation of a specific year and prioritization based on the least cloud cover within that year. This approach ensures that the images used are of the highest quality and most suitable for accurate analysis.

Both Sentinel-2 and Landsat series satellites include RGB bands, crucial for standard object recognition tasks in computer vision. Beyond the RGB spectrum, these satellites offer additional bands that are instrumental for advanced remote sensing analysis. This rich assortment of spectral data allows for sophisticated remote sensing techniques and predictive modeling, such as estimating vegetation density and water bodies, which are integral to our study on regional poverty estimation.

### 3.2 Demographic Health Surveys and Child Poverty

Dating back to 1984, the Demographic and Health Surveys (DHS) Program(^1^11http://www.dhsprogram.com) has conducted over 400 surveys in 90 countries, funded by the US Agency for International Development (USAID) and undertaken in partnership with country governments. These nationally representative cross-sectional household surveys, with very high response rates, provide up-to-date information on a wide range of demographic, health and nutrition monitoring indicators. Sample sizes range between 5,000 and 30,000 households, and are collected using a stratified, two-stage cluster design, with randomly chosen enumeration areas (EAs) called “clusters” forming the sampling unit for the first stage. In each EA, a random sample of households is drawn from an updated list of households. DHS routinely collects geographic information in all surveyed countries. Cluster locations are released, with random noise added to preserve anonymity with this ’jitter’ being different for rural and urban EAs.

The DHS data include both continuous and categorical variables, each requiring a different approach for aggregation to ensure accurate ecological analysis at the cluster level. For continuous variables, we calculated the mean of all responses associated with a particular spatial coordinate. Min-max scaling was applied after aggregation to normalise the data, ensuring that all values were on a scale from 0 to 1. Categorical variables were processed using one-hot encoding, which converts categories into binary indicator variables. Similarly, the mean of these binary representations was computed for each category at each cluster location.

Child poverty was assessed using a methodology formulated by UNICEF that evaluates child poverty across six dimensions: housing, water, sanitation, nutrition, health, and education. Each child was classified as moderately or severely deprived for each dimension based on a set of 17 variables in total []. An overall classification of moderate or severe deprivation is made if the child experiences moderate or severe deprivation on any of the six dimensions. Our target quantity of interest, severe_deprivation, was calculated as the percentage of children experiencing severe deprivation within a cluster. The detailed definition of moderate and severe deprivation and implementation of poverty calculation can be found in the supplementary material.

## 4 Benchmark

### 4.1 Spatial

We use 5-fold spatial crossvalidation at the cluster level across countries in Eastern and Southern Africa, spanning data collected from 1997 to 2022. We train our models on 80% of the clusters and evaluate its performance using the mean absolute error (MAE) of the severe_deprivation variable on the held-out 20% of clusters. This benchmark is designed to evaluate the model’s capability to estimate poverty or deprivation levels at any given location based solely on satellite imagery data, quantifying the model’s generalization capabilities to unsurveyed locations within surveyed countries.

### 4.2 Temporal

The temporal benchmark employs a historical data training approach, where we use data collected from 1997 to 2019 as the training set to develop our models. The objective is to predict poverty in 2020 to 2022. Model performance is evaluated using the MAE of the severe_deprivation variable. This benchmark tests the model’s ability to capture temporal trends and forecast poverty based on satellite imagery data, assessing its forecasting accuracy. This capability is crucial for, e.g. nowcasting poverty before survey data becomes available.

### 4.3 Models to be Compared

We consider both baseline models (Gaussian process regression, mean prediction) and a range of more advanced computer vision models, both unsupervised and semi-supervised, with and without fine-tuning.
Each model represents a distinct strategy in handling and processing satellite imagery:

MOSAIKS [] is a generalisable feature extraction framework developed for environmental and socio-economic applications. We obtain MOSAIKS features from IDinsight, an open-source package that utilizes the Microsoft Planetary Computer API. The framework leverages satellite imagery to extract meaningful features from the Earth’s surface. For our purposes, we used its Sentinel service, querying with specific coordinates, survey year, and a window size of 10 km $\times$ 10 km.

DINOv2 [] Initially designed for self-supervised learning from images, DINOv2 excels in generating effective vector representations from RGB bands alone. For our study, we selected the pre-trained base model with the vision transformer architecture as the backbone of our foundational model. We fine-tuned this foundational model with DHS variables to enhance its capability for predicting poverty. DINOv2 is evaluated in both its raw and fine-tuned forms using RGB imagery for both spatial and temporal benchmarks.

SatMAE [] was originally developed for landmark recognition from satellite imagery. We fine-tuned with DHS variables to enhance its performance for predicting poverty. SatMAE has 3 variants: RGB, RGB+temporal, and multi-spectral. For benchmarking, we use the RGB variant for the spatial benchmark, and RGB+temporal for the temporal benchmark. The RGB+temporal variant takes 3 images of different timestamps from the same location; however, to facilitate a direct comparison with the other methods which use only a single image, we provide SatMAE with the same image three times. Additionally, it takes in Year, Month, and Hour, but since a DHS survey spans up to years, we only provide the Year variable, with Month and Hour set to January 00:00.

#### 4.3.1 Evaluation and Fine-tuning

In our fine-tuning pipeline, we start from DINOv2’s and SatMAE’s original checkpoints with an uninitialised head and train it against 17 selected DHS variables to minimize mean absolute error (MAE). We then evaluate the model by replacing the head with a cross-validated ridge regression model mapping satellite features to the severe_deprivation variable and calculate the MAE loss on a test set that was neither seen by the fine-tuned model nor the Ridge Regression. For the spatial task, we perform a 5-fold cross-validation on the whole dataset, and for the temporal benchmark, we take the training set as the data before the year 2020 and evaluate on the data from 2020 to 2022.

For the spatial benchmark, we randomly split the data into five train-test splits using a reproducible script. For the temporal benchmark, we divided the data into a single fold, using data from before 2020 for training and data from 2020 onward for testing.

For DINOv2, we used a batch size of 8 for Landsat imagery and a batch size of 1 for Sentinel imagery, with L1 loss and an Adam optimiser of learning rate and weight decay both set to 1e-6. We trained the model for 20 epochs with Landsat imagery and 10 epochs with Sentinel imagery, selecting the model with the minimum validation loss on predicting the 17 DHS variables. Each task was trained on a single Nvidia V100 32GB GPU, with an average training time of 1 hour per epoch for Landsat and 2 hours per epoch for Sentinel imagery. For SatMAE, we resize the input to $224\times 224$ and use a batch size of 32 for the spatial task and 16 for the temporal task. Training is done with Adam optimiser with learning rate 1e-5 and weight decay 1e-6, for at most 20 epochs with the early stopping of patience 5 and delta 5e-4. Each task is trained on a single Nvidia L4 GPU, taking, for Landsat and Sentinel, 1 and 2 hours for the first epoch and 15 and 10 minutes for each subsequent ones with data caching.

## 5 Results

The performance of the child poverty prediction models is summarized in Table 1.

### 5.1 Spatial Benchmark

In the spatial benchmarking, Gaussian Process regression with geographic coordinates resulted in a mean absolute error (MAE) that is 0.04 lower than that achieved by the baseline mean prediction model. Notably, regressions using outputs from foundational vision models outperformed both the baseline and GP regression. The MOSAIKS features based on Sentinel-2 imagery achieved 0.2356 MAE on predicting the severe_deprivation variable. Utilising Landsat imagery, the DINOv2 and SatMAE achieved MAEs of 0.2260 and 0.2341 respectively. Further enhancements through fine-tuning with Demographic and Health Surveys (DHS) variables led to reduced prediction errors, with DINOv2 and SatMAE recording MAEs of 0.2042 and 0.2125 respectively. When using Sentinel-2 imagery, the SatMAE architecture achieved errors of 0.2347 and 0.2093 before and after the fine-tuning, while DINOv2 further lowered the errors to 0.2013 and 0.1836 respectively.

### 5.2 Temporal Benchmark

In the temporal benchmark, models faced greater challenges in forecasting poverty. Gaussian Process regression was substantially worse than the mean prediction. Using Sentinel-2 imagery, MOSAIKS recorded an MAE of 0.2588, with DINOv2 and SatMAE achieving MAEs of 0.2597 and 0.3067 respectively. Additional fine-tuning with DHS variables led to increased prediction errors, with DINOv2 and SatMAE resulting in MAEs of 0.2858 and 0.3139. Employing Landsat imagery, the pre-trained DINO v2 and SatMAE model achieved worse initial MAEs of 0.2704 and 0.3453; nevertheless, additional fine-tuning on DHS variables resulted in relative equal performance for both models, with MAEs of 0.2574 and 0.3376 respectively.

**Table 1: Comparison of MAE on severe_deprivation across Benchmarks and Imagery Sources. In the spatial task, random clusters are heldout, while the temporal task is a more difficult forecasting task, with the years 2020-2022 held out. Fine-tuning consistently gives better results. While SatMAE is a foundation model trained on satellite imagery, it is outperformed by the more generic DINOv2 foundation model.**
| Model | Benchmark Type | MAE $\pm$ SE (Spatial) | MAE (Temporal) |
| --- | --- | --- | --- |
| Mean Prediction | - | $0.2930\pm 0.0018$ | $0.3183$ |
| Gaussian Process Regression | - | $0.2436\pm 0.0002$ | $0.5656$ |
| MOSAIKS | Sentinel-2 | $0.2356\pm 0.0114$ | $0.2588$ |
| DINOv2 (Raw) | LandSat | $0.2260\pm 0.0005$ | $0.2704$ |
| DINOv2 (Raw) | Sentinel-2 | $0.2013\pm 0.0019$ | $0.2597$ |
| DINOv2 (Fine-tuned) | LandSat | $0.2042\pm 0.0015$ | $0.2574$ |
| DINOv2 (Fine-tuned) | Sentinel-2 | $0.1836\pm 0.0036$ | $0.2858$ |
| SatMAE (Raw) | LandSat | $0.2341\pm 0.0017$ | $0.3453$ |
| SatMAE (Raw) | Sentinel-2 | $0.2347\pm 0.0027$ | $0.3067$ |
| SatMAE (Fine-tuned) | LandSat | $0.2125\pm 0.0019$ | $0.3376$ |
| SatMAE (Fine-tuned) | Sentinel-2 | $0.2093\pm 0.0039$ | $0.3139$ |

### 5.3 Interpretation of Results

The performance of various poverty prediction models is shown in Table 1.
Our prediction task is the percentage of a location’s children who are experiencing severe deprivation, so a MAE on the order of 0.20 is equivalent to 20 percentage points of error, which policymakers may consider to be too high to be useful.
The spatial benchmark demonstrates the advantage of using foundational vision models over the baseline mean prediction model and GPR. Models like MOSAIKS, DINOv2, and SatMAE, particularly when improved through fine-tuning with DHS variables, show a further reduction in mean absolute error (MAE). This implies that spatial features extracted from satellite imagery are comparably more effective than GP modelling in estimating poverty indicators in regions where surveys have not been conducted.

The temporal benchmark, which evaluated a forecasting task (predict 2020-2022 using data from before 2020), was more difficult than the spatial benchmark. Satellite imagery is at best a proxy for multidimensional child poverty, and this finding suggests it is a better proxy for quantifying spatial as opposed to temporal variation. Satellite imagery models performed worse on the temporal as compared to spatial benchmark, and the fine-tuned models, particularly those using Sentinel-2 imagery as the source input, showed increased MAE compared to the raw output from both DINOv2 and SatMAE models. This suggests that the models overfit the historical data, and struggled to generalise to data collected after 2020. Gaussian process regression based on spatial coordinates had no way of predicting changes over time, explaining its very poor performance.

## 6 Discussion and Future Work

### 6.1 Satellite Imagery Sources

As compared to Landsat, models utilising Sentinel-2 imagery, such as the fine-tuned versions of DINOv2 and MOSAIKS, demonstrated improved performance in both spatial benchmarks. These models benefited from the rich spectral information provided by Sentinel-2, which enabled more precise predictions of deprivation levels across diverse geographical regions.

Additionally, the computational demands associated with processing high-resolution Sentinel-2 data present substantial challenges. For instance, large versions of vision transformers could not be accommodated within the memory constraints of a 32 GB GPU when processing the full Sentinel-2 data. In contrast, these larger models could be deployed with Landsat data, which offers lower resolution but requires less computational resources. Under the spatial setting, this scenario highlights a critical trade-off in model deployment: the choice between employing lightweight models to retain the high resolution of Sentinel-2 imagery or opting for more powerful models that necessitate a reduction in image resolution to ensure feasibility.

### 6.2 Modeling Choices

We considered a representative set of models: MOSAIKS is unsupervised, DINOv2 is a generic foundation model trained on images, and SatMAE is a foundation model trained on satellite imagery.

#### 6.2.1 MOSAIKS

MOSAIKS is designed to provide general-purpose satellite encodings and is notably accessible through Microsoft’s Planetary Computer service. This model generates a large output vector, typically around 4000 dimensions, which, while comprehensive, can lead to significant computational costs when methods beyond simple linear regression are employed. Furthermore, although MOSAIKS is well-suited for broad applications, integrating online feature acquisition into a fine-tuning process tailored specifically to poverty prediction presents challenges. This limitation can hinder its effectiveness when adapting to specific tasks where dynamic feature updates are crucial. We also note that MOSIAKS’ API at times returned no-features, even after implementing rate-limiting mechanisms. This random behaviour combined with unavailability of features before 2013 limits the use of MOSAIKS considerably.

#### 6.2.2 DINOv2

DINOv2 stands out as a state-of-the-art foundational model that excels in generating effective vector representations from RGB bands alone, achieving comparable performance to models that utilize additional spectral bands. Its flexibility in model sizing allows users to select the optimal model scale for specific training needs, enhancing its utility across various computational settings. The availability of pre-trained weights simplifies the process of fine-tuning for specialized tasks such as poverty prediction. However, DINOv2’s reliance solely on RGB bands means it does not leverage the broader spectral information available in other satellite imagery bands, which may limit its application scope to scenarios where such data could provide additional predictive insights.

#### 6.2.3 SatMAE

SatMAE demonstrates respectable results, surpassing baseline models even with only its raw, pre-trained configuration. Its architecture inherently supports the integration of multispectral and temporal analysis, making it well-suited for handling complex datasets typically encountered in satellite imagery analysis. Despite these strengths, the pre-trained SatMAE model is configured to process images of 224 $\times$ 224 pixels, constraining its ability to utilize higher-resolution imagery, such as the 1000 $\times$ 1000 pixel images from Sentinel-2. This limitation restricts its performance, particularly in comparison to models that can fully exploit high-resolution data, thereby failing to match the effectiveness of other advanced models in our analysis.
Another limitation is that with our simple benchmarking setup, we have not made full use of SatMAE’s temporal and multi-spectral capabilities. In the temporal setup, we are providing only one image-timestamp pair with only the Year variable, while the model is capable of taking up to 3 pairs, along with Month and Hour variables. We are also exclusively using the RGB bands, while the multi-spectral version of SatMAE is capable of taking in other bands of the satellite images in our dataset.

### 6.3 Further Discussion

The ability to accurately measure poverty across a vast number of geolocations is crucial for understanding and addressing the disparities that exist in different regions. The extensive and high-quality poverty measurement is valuable for researchers and policymakers. It allows for the analysis of poverty trends and the effectiveness of current policies, thereby facilitating more informed decision-making to reduce global poverty.

Traditional surveys, while rich in data, are limited by geographical and logistical constraints. Conducting extensive on-the-ground surveys is not only costly but also time-consuming—from data collection to processing and harmonisation. In regions lacking detailed survey data, traditional methods like GPR or nearest-neighbor approaches are typically used to estimate poverty levels. However, these methods can be unreliable, particularly when extrapolating data to locations far from surveyed areas or data with temporal dependencies, leading to high uncertainty.

On the other hand, satellite imagery, which was made widely available by organizations such as the European Space Agency and the United States Geological Survey, can be accessed from any geographic location. Recent advancements in the field of computer vision have made it possible to infer meaningful information from this imagery, which can effectively improve poverty prediction. By demonstrating the capabilities of large vision models and satellite imagery in this context, we aim to inspire and encourage others in the field to further develop and refine these methods, thus driving changes in sociology research and policy making.

### 6.4 Limitation and Future Directions

Our study had a number of limitations. While high-quality household survey data is expensive to acquire, it is an irreplaceable source of ground truth; machine learning can complement and enhance, but never replace, these datasets.
We highlighted the difficulty of the temporal benchmark, suggesting that future research could explore time series methods for forecasting or ways of better encoding temporal information into the foundation models.
Another limitation of our study is that, in our goal of learning general representations of satellite imagery, we fine-tuned the large vision models to predict DHS variables important to calculating the severe_deprivation variable rather than directly optimising the models on the severe_deprivation variable itself. While this approach could lead to better generalisation by leveraging a broader range of demographic and health indicators, it may not be the most effective strategy for minimising the MAE specifically for severe_deprivation. This indirect optimisation can result in sub-optimal performance for the target variable. Future work could explore the extent of benchmark improvement with direct optimisation techniques for severe_deprivation.
Additionally, future work could also evaluate the performance of various models on the six components of child poverty separately, on moderate as opposed to severe deprivation. While we evaluated spatial generalisation by leaving out clusters, a stricter evaluation would have considered a leave-one-country out evaluation.

## 7 Conclusion

In conclusion, our study demonstrates the potential of satellite imagery combined with large vision models to estimate child poverty across spatial and temporal settings. We introduced a new dataset pairing publicly accessible satellite images with detailed survey and child poverty data based on the Demographic and Health Surveys Program, covering 19 countries in Eastern and Southern Africa over the period 1997-2022. By benchmarking multiple models, including foundational vision models like MOSAIKS, DINOv2, and SatMAE, we assessed their performance in predicting child poverty. Our results show that advanced models with satellite imagery have the potential to outperform baseline methods, offering more accurate and generalisable poverty estimates. This work highlights the importance of integrating remote sensing data with machine learning techniques to address complex socio-economic issues, providing a scalable and cost-effective approach for poverty estimation and policy-making.

## 8 Acknowledgments

We thank Matthew Sutcliffe for his help during the early stages of this work, and Kaustubh Chakradeo, Enrique Delamónica, and Lucie Cluver for productive discussions. We acknowledge the support of Google Cloud for Researchers. S.F. and M.S. acknowledge the Alan Turing Institute. S.F. acknowledges the EPSRC (EP/V002910/2). S.M. acknowledges support from the National Research Foundation via The NRF Fellowship Class of 2023 Award (NRF-NRFF15-2023-0010). S.B. acknowledges funding from the MRC Centre for Global Infectious Disease Analysis (reference MR/X020258/1), funded by the UK Medical Research Council (MRC). This UK funded award is carried out in the frame of the Global Health EDCTP3 Joint Undertaking. S.B. is funded by the National Institute for Health and Care Research (NIHR) Health Protection Research Unit in Modelling and Health Economics, a partnership between UK Health Security Agency, Imperial College London and LSHTM (grant code NIHR200908). Disclaimer: “The views expressed are those of the author(s) and not necessarily those of the NIHR, UK Health Security Agency or the Department of Health and Social Care.”. S.B. acknowledges support from the Novo Nordisk Foundation via The Novo Nordisk Young Investigator Award (NNF20OC0059309). SB acknowledges the Danish National Research Foundation (DNRF160) through the chair grant. S.B. acknowledges support from The Eric and Wendy Schmidt Fund For Strategic Innovation via the Schmidt Polymath Award (G-22-63345).

## Appendix A Appendix / supplemental material

### A.1 Dataset description

The KidSat dataset we present in this work includes both cluster-wise child poverty derived from the DHS data and the satellite imagery corresponding to each cluster. Due to the confidentiality of the survey data, DHS requires registration prior to accessing the data.
We include detailed procedures for acquiring the satellite imagery and DHS data in our
[GitHub repository](https://github.com/MLGlobalHealth/KidSat).

#### A.1.1 Coding child poverty

The severe_deprivation variable is used in this work to represent severe child poverty for individual responses within the cluster. It is calculated by aggregating several indicators of severe deprivation across multiple dimensions such as housing, water, sanitation, nutrition, health, and education. A child is considered severely deprived if they experience at least one severe deprivation in any of the following areas:

- 1.
Housing: Severe deprivation if the number of persons per room is 5 or more.
- 2.
Water: Severe deprivation if the household uses unsafe water sources.
- 3.
Sanitation: Severe deprivation if the household lacks access to safe sanitation facilities.
- 4.
Nutrition: Severe deprivation if a child’s height-for-age z-score is below -3 (indicating severe stunting).
- 5.
Health: Severe deprivation if a child misses all essential vaccines or has untreated acute respiratory infections.
- 6.
Education: Severe deprivation if a school-aged child does not attend school and has not completed any level of education.

Among all DHS variables involved in child poverty calculation, we selected 17 variables, presented in Table 2, as the prediction objective during model fine-tuning. We scaled the continuous variables to the range from 0 to 1. We expanded the categorical columns using one-hot encoding, where each column is in the format of variable_value. In total, a vector of dimension 99 based on these 17 DHS variables was used to represent a cluster, where we mapped the satellite imagery to predict these vectors as the method of model fine-tuning.

**Table 2: DHS variables selected for model fine-tuning**
| Variable | Description | Type |
| --- | --- | --- |
| h10 | Whether the child ever received any vaccination to prevent diseases. | Categorical |
| h3 | DPT 1 vaccination. | Categorical |
| h31 | Whether the child had suffered from a cough in the last two weeks and whether the child had been ill with the cough in the last 24 hours. | Categorical |
| h5 | DPT 2 vaccination. | Categorical |
| h7 | DPT 3 vaccination. | Categorical |
| h9 | Measles 1 vaccination. | Categorical |
| hc70 | Height for age standard deviation (according to WHO). | Continuous |
| hv106 | Highest level of education the household member attended. | Categorical |
| hv109 | Educational attainment recoded. | Categorical |
| hv121 | Household member attended school during current school year. | Categorical |
| hv201 | Main source of drinking water for members of the household. | Categorical |
| hv204 | Time taken to get to the water source for drinking water | Continuous |
| hv205 | Type of toilet facility in the household. | Categorical |
| hv216 | Number of rooms used for sleeping in the household. | Continuous |
| hv225 | Whether the household shares a toilet with other households. | Categorical |
| hv271 | Wealth index factor score (5 decimals) | Continuous |
| v312 | Current contraceptive method. | Categorical |

### A.2 Compute

As one of the heavy-lifting parts is loading images, a multi-core CPU ($\geq 8$) is recommended to optimise the data loading using multiple workers with the data loader. The training was done using Nvidia Tesla V100 GPUs for DINO v2 experiments and Nvidia L4 GPUs for SatMAE experiments. In particular, for DINO v2 experiments with Sentinel imagery, 32 GB of GPU memory is a hard requirement.

### A.3 Experimental details

The dataset for our experiments was sourced from Sentinel-2 and Landsat-8 satellite imagery. Each image tile, corresponding to specific geographical centroids, was preprocessed by selecting and stacking RGB bands, normalising, and clipping pixel values to the 0-255 range. Data was organised into cross-validation folds with separate training and testing sets. Only centroids with available imagery were included, and rows with missing target values were excluded to maintain data integrity. (Note: As an unsupervised method, no fine-tuning was attempted for MOSAIKS; see main text for details.)

#### A.3.1 DINOv2

We used the base DINOv2 Vision Transformer (ViT) model which has an output of 768-dimension feature vector. We appended a regression head for mapping the feature vector to the 99-dimension target DHS variables. The Adam optimiser was used, with a learning rate and weight decay of 1e-6 for both the base model and the regression head. Training involved iterating over 20 epochs for Landsat imagery and 10 epochs for Sentinel imagery to minimise the L1 loss between predicted and actual target values. The batch size was determined by available GPU memory, 8 for Landsat imagery source and 1 for Sentinel. The average training time is 1 hour per epoch for Landsat experiments and 2 hours per epoch for Sentinel experiments.

#### A.3.2 SatMAE

The base SatMAE model is the decoder of a MAE, outputting a 1024-dimensional vector. Fine-tuning is done by appending a transformer head mapping to the 99-dimensional target DHS variables. We use the Adam optimiser, with a learning rate of 1e-5 and weight decay of 1e-6 for both the base model and the head. Training involved iterating over 20 epochs for both Landsat imagery and Sentinel imagery to minimise the L1 loss between predicted and actual target values, with early stopping of patience 5 and delta 5e-4. The batch size is 32 for spatial task and 16 for temporal task. Training takes 1 hour for the first epoch and 15-30 minutes for each subsequent one when data caching is used.

## NeurIPS Paper Checklist

- 1.
Claims
- Question: Do the main claims made in the abstract and introduction accurately reflect the paper’s contributions and scope?
- Answer: [Yes]
- Justification: We only make factual claims about the dataset and benchmark in the abstract.
- 2.
Limitations
- Question: Does the paper discuss the limitations of the work performed by the authors?
- Answer: [Yes]
- Justification: There is a section on limitations.
- 3.
Theory Assumptions and Proofs
- Question: For each theoretical result, does the paper provide the full set of assumptions and a complete (and correct) proof?
- Answer: [N/A]
- Justification: The paper does not contain theoretical results.
- 4.
Experimental Result Reproducibility
- Question: Does the paper fully disclose all the information needed to reproduce the main experimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
- Answer: [Yes] ,
- Justification: We are releasing all the code and full instructions for benchmarking and dataset creation.
- 5.
Open access to data and code
- Question: Does the paper provide open access to the data and code, with sufficient instructions to faithfully reproduce the main experimental results, as described in supplemental material?
- Answer: [Yes]
- Justification: We are releasing all the code and full instructions for benchmarking and dataset creation.
- 6.
Experimental Setting/Details
- Question: Does the paper specify all the training and test details (e.g., data splits, hyperparameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
- Answer: [Yes]
- Justification: Training details and experimental setting included in the paper. Full code also being released.
- 7.
Experiment Statistical Significance
- Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
- Answer: [Yes]
- Justification: we report standard errors from 5-fold cross validation, under the assumption of approximate normality.
- 8.
Experiments Compute Resources
- Question: For each experiment, does the paper provide sufficient information on the computer resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?
- Answer: [Yes]
- Justification: The appendix will include a section on compute resources.
- 9.
Code Of Ethics
- Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics [https://neurips.cc/public/EthicsGuidelines](https://neurips.cc/public/EthicsGuidelines)?
- Answer: [Yes]
- 10.
Broader Impacts
- Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
- Answer: [Yes]
- Justification: We discuss the potential impact our work has, as it can allow for finer grained and faster analysis of child poverty, a very important social issue.
- 11.
Safeguards
- Question: Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
- Answer: [N/A]
- Justification:
The dataset released does not fall in the above categories.
- 12.
Licenses for existing assets
- Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
- Answer: [Yes]
- Justification:
•
Landsat images are in the open domain.
•
Sentinel images have been released under the Open Access Creative Commons CC BY-SA 3.0 IGO licence.
•
The DHS data will be used by every user after signing a terms of use agreement with them.
- 13.
New Assets
- Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
- Answer: [Yes]
- Justification: We are releasing all the code and full instructions for benchmarking and dataset creation. Details will be included in appendix.
- 14.
Crowdsourcing and Research with Human Subjects
- Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well as details about compensation (if any)?
- Answer: [Yes]
- Justification: The data used in this paper is provided by the Demographic and Health Surveys Program. The methodology can be found on DHS’s website. The link is documented in the supplementary code repository.
- 15.
Institutional Review Board (IRB) Approvals or Equivalent for Research with Human Subjects
- Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
- Answer: [N/A]
- Justification: [N/A]

## Peer Reviews

confidence: 5
contribution: 1
initial_rating: 3
presentation: 1
questions: 1. Utilization of Multispectral Data
Could you provide insights into why the multispectral bands of Sentinel-2 were not fully leveraged in the models? Given that some indicators relevant to child poverty (e.g., vegetation indices for assessing agricultural productivity) could benefit from multispectral analysis, do you have plans to experiment with models that use the full spectral range? If so, which models do you anticipate could best handle this data?

2. Temporal Benchmarking and Forecasting Capability
The temporal benchmark indicates some difficulty in generalizing to future years. Could you clarify whether you attempted any temporal encoding techniques or time-series models in preliminary tests? Additionally, do you believe that training with time-stamped data would improve temporal generalization, and if so, what data sources might you consider?

3. Adaptive Image Resolution
In the current experiments, fixed input sizes (e.g., 224×224) are required for some models, which could result in a loss of spatial detail when processing high-resolution satellite images. Have you considered using models with patch-based processing or adaptive resolutions to maintain image fidelity? If so, could you share any initial observations or challenges you faced with these approaches?

4. Uncertainty Estimation
Since the predictions are deterministic, how do you envision integrating uncertainty estimation into the model outputs? For example, would probabilistic models or Bayesian approaches fit into your pipeline, and are there particular techniques you see as most promising for generating uncertainty estimates?

5. Geographic Generalizability
Given that the dataset focuses on Eastern and Southern Africa, have you considered testing on data from other regions, or could you discuss any plans to expand the dataset to other continents? Understanding how your models perform across diverse socioeconomic and environmental contexts would enhance the generalizability of this framework.

6. Validation with Alternative Ground Truth Sources
The paper uses DHS survey data as the primary ground truth. Could you elaborate on any efforts or future plans to validate predictions with additional poverty indicators or socioeconomic datasets? Are there other data sources you would consider integrating to broaden the validation framework?

7. Vision Transformer (ViT) Model Comparison
You’ve highlighted the performance of ViTs in this task. Could you clarify whether you have plans to compare ViTs with any recent transformer-based architectures optimized for spatial analysis? Additionally, did you consider using transformer architectures specifically pre-trained on remote sensing tasks, and if so, how did they perform?
soundness: 2
strengths: Originality
The "KidSat" paper presents a dataset that combines high-resolution satellite imagery with multidimensional child poverty data from Demographic and Health Surveys (DHS) across 16 countries in Eastern and Southern Africa. This dataset applies satellite imagery to estimate socioeconomic indicators related to child poverty, focusing on UNICEF’s six poverty dimensions—housing, water, sanitation, nutrition, health, and education. The dataset introduces a method for analyzing child-specific poverty indicators rather than general economic metrics.

Quality
The paper follows a methodical approach, providing reproducible steps for dataset construction and benchmark testing. The authors compare a range of models, including Gaussian Process Regression, MOSAIKS, DINOv2, and SatMAE, to assess performance on spatial and temporal generalization tasks. Fine-tuning, spatial and temporal benchmarks, and cross-validation are used to evaluate each model’s generalization capability.

Clarity
The paper is structured to clearly present the models, dataset, and evaluation metrics. The benchmark results, including error metrics for each model type, allow for straightforward comparisons of model performance in child poverty prediction. Visual aids, such as the heatmap of survey cluster locations and summary tables, provide concise representations of key findings.

Significance
This research provides a benchmark for child poverty estimation using satellite imagery, offering potential applications for data-informed policy and social science research in regions lacking detailed survey data. The dataset and benchmark may support further studies in poverty estimation using satellite data. Given the global availability of satellite imagery, this framework could be adapted to analyze other social or environmental indicators.
summary: The paper "KidSat: Satellite Imagery to Map Childhood Poverty" presents a dataset and benchmarking framework aimed at improving the analysis of child poverty indicators in Eastern and Southern Africa through satellite imagery. This research provides a dataset consisting of 33,608 images covering 16 countries from 1997 to 2022, paired with high-quality survey data on multidimensional child poverty factors such as housing, sanitation, water, nutrition, education, and health. By leveraging geocoded Demographic and Health Surveys data, the authors have created a resource for the social science and global health community, enabling detailed analyses across both spatial and temporal dimensions.
This work appears notable for its systematic comparison of feature representations, ranging from low-level satellite models like MOSAIKS to state-of-the-art deep learning foundation models, including DINOv2 and SatMAE. The authors assess each model's capacity for spatial generalization across different locations and temporal generalization beyond the training years, providing a framework to evaluate model efficacy across varied contexts.
weaknesses: 1. Limited Exploitation of Multispectral Data
While the "KidSat" dataset includes multispectral satellite imagery from sources like Sentinel-2, the study does not fully utilize the additional spectral bands beyond RGB. The paper mentions that DINOv2 relies solely on RGB bands, potentially limiting the model’s ability to capture information relevant to socioeconomic indicators, such as vegetation health or water bodies. Future work could include experiments with multispectral configurations or foundation models designed to leverage the entire spectral range available in Sentinel and Landsat imagery.

2. Constraints on Temporal Prediction Capability
The temporal benchmark reveals challenges in predicting poverty indicators in years beyond the training period (2020–2022). This suggests that the current models may be overfitting to historical data patterns and struggle to generalize to newer datasets. Incorporating temporal encoding or exploring time-series models may improve the framework’s capability to handle time-sensitive socioeconomic changes. Furthermore, training models with time-stamped data could aid in detecting trends, which would be valuable for forecasting.

3. Fixed Input Size and Resizing Effects on Model Performance
The study limits DINOv2 and SatMAE’s performance by requiring the imagery to be resized to a specific input size (224×224 for certain configurations). Resizing can lead to information loss, particularly when working with high-resolution satellite images. Evaluating models with variable or adaptive input sizes, or employing patch-based processing for high-resolution images, might preserve more spatial detail and enhance prediction accuracy.

4. Absence of Uncertainty Estimation
The current approach is deterministic and does not quantify uncertainty in model predictions, which is crucial for policymakers who may rely on this data to inform decisions. Including uncertainty metrics, such as confidence intervals or probabilistic models, would provide a more comprehensive assessment of model reliability, particularly in areas with sparse data coverage.

5. Limited Comparison with Non-ViT Architectures
Although the study highlights the use of Vision Transformers (ViTs), the experiments with alternative architectures, like CNN-based models, are limited and do not include any recent transformer alternatives. Comparing ViTs with other state-of-the-art architectures, particularly those optimized for spatial analysis, would help clarify the advantages and limitations of ViTs in the context of satellite imagery-based poverty estimation.

6. Dataset Coverage and Geographic Bias
The dataset, while extensive, is geographically focused on Eastern and Southern Africa. This focus may limit the applicability of the benchmark models to other regions with different socioeconomic and environmental conditions. An expanded dataset that includes diverse regions globally could help generalize the findings and provide a more comprehensive tool for poverty estimation.

7. Lack of Direct Validation with Ground Truth
The study’s reliance on DHS survey data as ground truth is practical, but there is limited discussion on validating model predictions with additional sources or cross-referencing with other poverty metrics. Incorporating data from other surveys, censuses, or economic indicators could enhance model robustness and provide a broader validation framework.

---

confidence: 4
contribution: 1
initial_rating: 1
presentation: 2
questions: * What implications/impact does this paper have to the machine learning community beyond poverty mapping?
* Why was the focus on severe_deprivation and not other poverty indicators in the poverty vector? Is severe_deprivation a particularly well-suited variables to test different foundation models?
soundness: 2
strengths: * Impacting problem and extensive dataset.
summary: The paper outlines a methodology to map poverty (i.e., severe deprivation) across southern Africa by encoding satellite images into feature representations with pre-trained models (Dino, SatMAE) , random kitchen sinks (MOSAIKS) and performing a linear regression on the target variable "severe deprivation". A baseline is the GP regression given the label data. 

While the problem is highly relevant and impacting, the implications for the Machine Learning community remain unclear. In particular, the discussion remains focused on poverty mapping and does not investigate an underlying methodological question of "which pre-training or feature representations are particularly useful for certain downstream tasks (like poverty mapping)". Currently, tested methods are used as is, and their feature vectors have not been investigated in depth. 

Hence, i believe that this paper would rather fit in an applied journal where reviewers can better evaluate the societal impact of poverty mapping with existing foundation models.
weaknesses: * methodological questions are not investigated in-depth. E.g., what impact does the pre-training algorithm have on downstream performance in general. Are (reconstruction-based) MAE features better than Dino (joint-embedding based) features (see for instance, Shekhar et al., 2023 for a comparison here) in general for poverty mapping? Do takeaways in that direction align with existing papers? 

Shekhar, S., Bordes, F., Vincent, P., & Morcos, A. (2023). Objectives matter: Understanding the impact of self-supervised objectives on vision transformer representations. arXiv preprint arXiv:2304.13089.

---

confidence: 4
contribution: 3
initial_rating: 3
presentation: 2
questions: Have the authors considered using building data as input instead of satellite imagery? A good dataset for this could be the recent Open Buildings 2.5D Temporal Dataset, which provides information on building presence, fractional building count, and building height.
soundness: 2
strengths: - The dataset focuses on an important task in the Global South, while most existing remote sensing datasets were developed for the Global North.

- The paper includes two simple baselines that help to understand the dataset.
summary: The paper proposes a dataset on child poverty featuring Sentinel-2 and Landsat imagery in Eastern and Southern Africa. Two dataset splits, a spatial and a temporal one, are proposed. The dataset is tested with several deep learning models such as MOSAIKS, Dinov2, and SatMAE, as well as two baseline approaches (mean regression and Gaussian process regression).

In general, the field benefits from challenging datasets. However, in this case, none of the factors constituting the target class, child poverty/prevalence of severe deprivation, are directly measurable from optical satellite imagery. Moreover, the dataset fails to incorporate key information such as nighttime light data that is typically used as a proxy for poverty. The paper also lacks visualizations that show examples of the satellite imagery, target value, and model outputs. Due to this and the fact that the spatial dataset split greatly simplifies the task, it is unclear whether models are capable of learning any information related to child poverty from this dataset.
weaknesses: - The description of the satellite data is incomplete (I am aware that the satellite specifications are listed in the supplementary material but I am referring to the preprocessed satellite imagery used as model input). For example, key information such as the processing level, the selected spectral bands, and the spatial resolution of the satellite imagery are missing. This also applies to the abstract which does not provide any information on the satellite imagery.

- A literature review on relevant remote sensing studies mapping poverty is missing. Instead, the related work section cites several less relevant datasets such as xView2 or SpaceNet 7.

- Since nighttime light data is a key modality for poverty mapping, it is surprising that the authors did not include it. Therefore, I strongly suggest enriching the dataset with nighttime light data from, for example, SDGSAT-1 (ideal due to its high spatial resolution) or VIIRS.  

- The visualizations are not useful for understanding the spatial patterns the models produce since they cover extensive areas. Therefore, I request some examples of model outputs over metropolitan areas of cities known to have deprived neighborhoods  (e.g., Maputo, Mozambique). This will lead to great insight into the capability of models trained on KidSat. 
 
- For the spatial benchmark, I assume the clusters were randomly split into five folds. However, this split largely overlooks spatial autocorrelations and does not sufficiently test generalization ability across space (see Rolf et al., 2024). This limitation is also indicated by the performance of the Gaussian process regression baseline. Therefore, I recommend adding experiments for a region-based split.

---

confidence: 5
contribution: 1
initial_rating: 1
presentation: 3
questions: - Should we be prioritizing the use of ML to predict variables based on satellite data when that data is *at best* a proxy (at worst, not informative at all?)? What are the risks of the conclusions that can be drawn from such analyses?

- How do simpler ML baselines that do use satellite imagery inputs perform compared to the foundation models?

- How do the models perform on disaggregated versions of the test data, e.g. by country or by the bins in Figure 4.A?

- The Sentinel-2 archive is only available since 2015/2016 but the survey data covers 1997-2022. Does this mean less data is used in the Sentinel-2 experiments?

- For Landsat, data was drawn from Landsats 5, 7, and 8 to cover the survey period. There are some differences between these three datasets that could increase the heterogeneity of observations in the survey period, potentially a source of error that was not discussed in the paper. Did the authors used a harmonized version of these products?

- Why was the SatMAE temporal model used if you did not include temporal information (replicated the same single-timestep input)? There is no temporal information being given to the model (except maybe the year of the data?).
soundness: 2
strengths: - The paper gives sufficient background about the definition of childhood poverty and how it is measured
- The paper contributes a benchmark dataset that covers an application area of health/development that is not well covered by existing benchmarks
- Two simple baselines that use no satellite imagery are reported to ground the lower limit of expected performance
summary: This paper presents a new benchmark dataset called KidSat for predicting childhood poverty severity using satellite imagery from the Sentinel-2 and Landsat satellites. The prediction targets are constructed from DHS survey data in Eastern and Southern Africa spanning 1997-2022. The authors compared three models that process satellite imagery - MOSAIKS, DINO, and SatMAE - as well as two no-imagery baselines that rely only on the survey data itself - Gaussian Process Regression and Mean Regression - on the proposed benchmark.
weaknesses: - The biggest limitation of this paper in my opinion is that it is unclear if this problem should be formulated as a satellite ML benchmark at all. Since the measures in the child poverty index are in most cases not directly observable from satellite data (especially at the resolution of Landsat and S2), it's unclear what features the authors expect to be learned. They even say on L385-386: "Satellite imagery is at best a proxy for multidimensional child poverty." Should we be prioritizing the use of ML to predict variables based on satellite data when that data is *at best* a proxy (at worst, not informative at all?)? What are the risks of the conclusions that can be drawn from such analyses?
 
- The results in Table 2 show that *some* useful information might be present in the satellite data because all of the imagery-based models do at least marginally better than the no-imagery baselines. However, it's not clear from the results if more sophisticated methods are needed to improve results, or if more sophisticated data are needed, since all of the models perform in a similar range that is not dramatically better than a no-imagery baseline.

- There is a lot of space in the text spent on redundant descriptions of the analysis, data, and discussion. Much of this space would have been better spent on additional experiments that would help better characterize the results and their utility, and evaluation with more modeling scenarios. For example: 
  - The authors only tested self-supervised foundation models that ingest images. I think simpler baselines that use the satellite data but do not learn spatial correlations, like a random forest or other pixel-based methods, would be useful baselines to include to test the hypothesis that spatial structure is actually being learned. 
  - The authors say that they only used RGB imagery to "directly compare" DINO with SatMAE. However, these methods are not directly comparable anyway - for example, different setups for the temporal benchmark. This does not seem like a sufficient reason not to test the multispectral versions of the data with SatMAE.
  - Results are reported as an average across the entire test dataset. Disaggregating these results by subgroup (e.g., country or the bins in Figure A.4) would give a more nuanced understanding of model performance and differences in performance. This would also be more useful for stakeholders to understand the utility (or lack thereof) of using the methods measured on this benchmark.  

- Why was the SatMAE temporal model used if you did not include temporal information (replicated the same single-timestep input)? There is no temporal information being given to the model (except maybe the year of the data?).

- Questions about the dataset
  - The Sentinel-2 archive is only available since 2015/2016 but the survey data covers 1997-2022. Does this mean less data is used in the Sentinel-2 experiments?
  - For Landsat, data was drawn from Landsats 5, 7, and 8 to cover the survey period. There are some differences between these three datasets that could increase the heterogeneity of observations in the survey period, potentially a source of error that was not discussed in the paper. Did the authors used a harmonized version of these products?

Meta-review: Dear authors,

Thank you for submitting the draft. The reviewers' rankings indicated that the draft is not ready for publication at this stage.

The draft proposes a new dataset to predict child poverty from satellite imagery.  Results are presented for both locations outside the training set, and "on data beyond the training years". Reviewers gave detailed comments, especially REgw, sp23, and rEvC. rEvC raised concern about missing relevant studies on mapping poverty. A concern raised by REgw was "Constraints on Temporal Prediction Capability" that is whether it can predict poverty beyond the training time of 2020-2022. sp23 raised a question should this problem be formulated as a satellite ML benchmark at all, this indicates that the draft lacks clarity to explain the need and promise of their methodology? The authors did not provide feedback. 

We hope comments by reviewers will help improve the draft.

regards

AC

Decision: Reject