From: NIPS.cc/2018/Workshop/Spatiotemporal/Paper28/Authors
title: Generalization
comment: Thank you for your question. It's a very good idea to see if a model trained on some country can help predict other countries. We haven't finished that part yet, but Jean et al. [1] does show very good generalization results.  We're currently concerned with the temporal aspect of our model, but we think our models can generalize spatially as well.

[1] Jean, Neal, et al. "Combining satellite imagery and machine learning to predict poverty." Science 353.6301 (2016): 790-794.

From: ~Konstantin_Klemmer1
comment: Thanks for your answer. Just as a last follow-up on this: Do you reckon there is a way to identify the geographic boundaries of your model validity? I.e. do you think the results could be interpolated to all of sub-Saharan Africa or is the vegetation structure not similar enough compared to your training data?
title: Some thoughts and comments

From: NIPS.cc/2018/Workshop/Spatiotemporal/Paper28/Authors
title: Larger Scale Experiments for Temporal Prediction
comment: Thank you for your feedback. We agree with your suggestion and certainly plan to perform temporal prediction at a larger scale. One caveat is that although temporal predictions can be made easily for any location at any time, we have very limited survey data to evaluate the predictions. That’s why we restrict our experiments to 209 sites that fortunately appear in both survey years. We are exploring more survey countries and response variables to evaluate our method.

From: NIPS.cc/2018/Workshop/Spatiotemporal/Paper28/Authors
title: Predictive Power in Different Areas
comment: Thank you for your thoughts and comments. You’re probably right that our method works best in areas that heavily depend on crop production. Unsurprisingly, agriculture accounts for the largest shares in the economies of many poor countries, such as those in sub-Saharan Africa.

We have rewritten the sentence in line 39. Thank you for pointing it out.

From: NIPS.cc/2018/Workshop/Spatiotemporal/Paper28/Authors
title: MODIS vs Landsat
comment: Thank you for your feedback. We agree that our method closely resembles Jean et al. in using CNNs as feature extractors. However, we emphasize that despite its popularity, NDVI hasn’t been combined with CNNs to predict poverty, at least to the best of our knowledge. Most existing methods [1, 2] use raw NDVI measurements as regressors, and we found that CNN feature maps corresponding to these raw values are often much more predictive.

As mentioned in the paper, our model takes as input annual NDVI images from MODIS for a survey year, one per sampled location, because (1) only annual nighttime light intensities are available for training with NDVI, and (2) our prediction pipeline with CNN extractors don’t take full advantage of frequent temporal signal yet. We are running experiments with Landsat and different network architectures such as ConvLSTM to capture both the fine-grained spatial and temporal signal.

We appreciate your suggestion about comparing the model performance in rural and urban areas. We will try to include it in our next revision.

[1] Sedda, Luigi et al. “Poverty, Health and Satellite-Derived Vegetation Indices: Their Inter-Spatial Relationship in West Africa.” International Health 7.2 (2015): 99–106.
[2] Dawson, Teddy, et al. "A Spatial Analysis of the Relationship between Vegetation and Poverty." ISPRS International Journal of Geo-Information 7.3 (2018): 83.

From: ~Ransalu_Senanayake1
comment: Dear authors,

You should now be able to see the reviews. We encourage you to engage in the discussion by commenting on the reviews. Please attempt to clarify the issues highlighted by the reviewers.  

Regards,
Ransalu, Neal, Fabio, and Girish
title: Rebuttal

From: NIPS.cc/2018/Workshop/Spatiotemporal/Paper28/AnonReviewer2
evaluation: This well-written paper demonstrates the possibility of using publicly-available, moderate-resolution data to predict poverty indicators in a number of African countries. This is important because previous work used high resolution images from DigitalGlobe (Google basemaps), and such imagery is proprietary and expensive to acquire.

Some questions, comments, and suggestions:

The methods are individually not novel as the network is VGG-16 and the prediction pipeline follows Jean et al. nearly completely. NDVI is also the most commonly used vegetation index and has been around for a long time. That said, demonstrating the possibility of poverty predictions using the combination of moderate resolution imagery and CNNs is potentially impactful for how socioeconomic assessments may be done going forward.

The current paper lacks a description of the remote sensing data obtained, especially temporal aspects. MODIS has a revisit rate of a couple days and there is no description of which images were used for a survey year (was it all of them?). Also, why was Terra/MODIS used instead of Landsat (which has higher resolution)? Is it for the higher temporal frequency?

The intro may benefit from an explanation of why Uganda and other countries were chosen for readers not familiar with LSMS surveys.

The success of the models at predicting income/wealth in poorer communities is explained by the authors as being due to such communities' reliance on crop production. Readers may be curious to hear more about the performance in urban areas, and how NDVI captures the presence of buildings.

From: NIPS.cc/2018/Workshop/Spatiotemporal/Paper28/AnonReviewer2
relevance: 4: high
impact: 4: high
novelty: 2: low

From: ~Konstantin_Klemmer1
comment: Thanks for this paper, it was an interesting read and the topic is certainly relevant.
Here are some thoughts and comments:
- In my opinion, you somewhat undersell the climate change / change through natural desasters argument, i.e. the opportunity for modelling temporal dynamics that your approach brings. Since your data is available openly and the model performs quite decent, I believe this is the strongest contribution of this paper.
- I am missing a bit of a discussion of how generalizable your results are. Since you are working with vegetation indices, I assume the predictive power of your model must be limited to certain areas that show a similar vegetation structure. I assume the model wouldn't work in desert areas or the arctic circle.
- The sentence starting in line 39 (regarding your contributions) was quite confusing to read. Consider rephrasing it.
Great work overall, I hope the comments help a bit!
title: Some thoughts and comments

From: NIPS.cc/2018/Workshop/Spatiotemporal/Paper28/AnonReviewer1
relevance: 5: very high
impact: 3: borderline
novelty: 2: low

From: NIPS.cc/2018/Workshop/Spatiotemporal/Paper28/AnonReviewer1
evaluation: Accurate and timely estimates of poverty are important for many applications, and recent work has shown that CNNs can be applied to high-resolution satellite imagery to produce accurate predictions of poverty. This paper shows that by applying CNNs to a moderate-resolution vegetation index (NDVI), they can achieve similar or better accuracies as the previous approaches that relied on having high-resolution daytime imagery. This NDVI-based approach allows for more frequently updated poverty predictions as the lower resolution imagery is collected more frequently. The authors also show that the NDVI-based approach has the potential to generalize across time, as a model in Uganda trained on 2011 data is able to make reasonably accurate predictions when applied to 2013 data.

Since NDVI measurements are available at a much higher cadence than high-resolution satellite imagery (at least the stuff that is publicly available), being able to use this type of data means that we can generate more up-to-date estimates of poverty — this could be very useful for real-time interventions and making policy decisions! Another interesting point that the paper made was that extremely poor communities depend heavily on crop production, and thus changes in NDVI over time could be predictive of changes in household consumption or wealth.

This paper attempts to answer the question of whether historical NDVI data can be used to train a model that can generalize to making predictions about the future, and shows that at least in Uganda for the period 2011-13 it can. I would be interested to see if this type of temporal generalization holds up in larger scale experiments — being able to predict future socioeconomic measurements would have broad impacts.

For the authors - here is a manuscript from the World Bank that uses NDVI as a feature for predicting poverty: https://dornsife.usc.edu/assets/sites/1003/docs/PovSpace_040517.pdf