reviews: 
Comments: This paper presents a new benchmark dataset called KidSat for predicting childhood poverty severity using satellite imagery from the Sentinel-2 and Landsat satellites. The prediction targets are constructed from DHS survey data in Eastern and Southern Africa spanning 1997-2022. The authors compared three models that process satellite imagery - MOSAIKS, DINO, and SatMAE - as well as two no-imagery baselines that rely only on the survey data itself - Gaussian Process Regression and Mean Regression - on the proposed benchmark.
Strengths: - The paper gives sufficient background about the definition of childhood poverty and how it is measured
- The paper contributes a benchmark dataset that covers an application area of health/development that is not well covered by existing benchmarks
- Two simple baselines that use no satellite imagery are reported to ground the lower limit of expected performance
Weaknesses: - The biggest limitation of this paper in my opinion is that it is unclear if this problem should be formulated as a satellite ML benchmark at all. Since the measures in the child poverty index are in most cases not directly observable from satellite data (especially at the resolution of Landsat and S2), it's unclear what features the authors expect to be learned. They even say on L385-386: "Satellite imagery is at best a proxy for multidimensional child poverty." Should we be prioritizing the use of ML to predict variables based on satellite data when that data is *at best* a proxy (at worst, not informative at all?)? What are the risks of the conclusions that can be drawn from such analyses?
 
- The results in Table 2 show that *some* useful information might be present in the satellite data because all of the imagery-based models do at least marginally better than the no-imagery baselines. However, it's not clear from the results if more sophisticated methods are needed to improve results, or if more sophisticated data are needed, since all of the models perform in a similar range that is not dramatically better than a no-imagery baseline.

- There is a lot of space in the text spent on redundant descriptions of the analysis, data, and discussion. Much of this space would have been better spent on additional experiments that would help better characterize the results and their utility, and evaluation with more modeling scenarios. For example: 
  - The authors only tested self-supervised foundation models that ingest images. I think simpler baselines that use the satellite data but do not learn spatial correlations, like a random forest or other pixel-based methods, would be useful baselines to include to test the hypothesis that spatial structure is actually being learned. 
  - The authors say that they only used RGB imagery to "directly compare" DINO with SatMAE. However, these methods are not directly comparable anyway - for example, different setups for the temporal benchmark. This does not seem like a sufficient reason not to test the multispectral versions of the data with SatMAE.
  - Results are reported as an average across the entire test dataset. Disaggregating these results by subgroup (e.g., country or the bins in Figure A.4) would give a more nuanced understanding of model performance and differences in performance. This would also be more useful for stakeholders to understand the utility (or lack thereof) of using the methods measured on this benchmark.  

- Why was the SatMAE temporal model used if you did not include temporal information (replicated the same single-timestep input)? There is no temporal information being given to the model (except maybe the year of the data?).

- Questions about the dataset
  - The Sentinel-2 archive is only available since 2015/2016 but the survey data covers 1997-2022. Does this mean less data is used in the Sentinel-2 experiments?
  - For Landsat, data was drawn from Landsats 5, 7, and 8 to cover the survey period. There are some differences between these three datasets that could increase the heterogeneity of observations in the survey period, potentially a source of error that was not discussed in the paper. Did the authors used a harmonized version of these products?
Recommendation: 1
Reviewer confidence: 5
Soundness/correctness: 2
Clarity: 3