Paper:
High intra-class diversity and inter-class similarity is a characteristic of remote sensing scene image data sets currently posing significant difficulty for deep learning algorithms on classification tasks. To improve accuracy, post-classification methods have been proposed for smoothing results of model predictions. However, those approaches require an additional neural network to perform the smoothing operation, which adds overhead to the task. We propose an approach that involves learning deep features directly over neighboring scene images without requiring use of a cleanup model. Our approach utilizes a siamese network to improve the discriminative power of convolutional neural networks on a pair of neighboring scene images. It then exploits semantic coherence between this pair to enrich the feature vector of the image for which we want to predict a label. Empirical results show that this approach provides a viable alternative to existing methods. For example, our model improved prediction accuracy by 1 percentage point and dropped the mean squared error value by 0.02 over the baseline, on a disease density estimation task. These performance gains are comparable with results from existing post-classification methods, moreover without implementation overheads.

Review:
The authors propose a method to extract features utilizing the adjacency between patches, for better classification/regression of satellite image patches. The proposed method achieves better results compared to a straightforward baseline method.
I have several significant concerns:
- In the abstract, the authors claim that existing approaches such as post-classification add computational overhead to the task, whereas the proposed method does not add significant overhead. However, to me, post-classification can be very simple and straightforward, whereas the proposed method adds a series of computations: the proposed method not only extracts features from the input image, but also for another neighboring image; then features are combined (if two images are similar), before feeding into the network. The authors need to validate the claim that their method is more efficient.
- The baseline the authors compare to is weak. There are existing works on satellite image classification/regression. Many of them also use semantic/contextual information, or aim to improve the robustness of features. For example:
[1] Derksen et al. Spatially Precise Contextual Features Based on Superpixel Neighborhoods for Land Cover Mapping with High Resolution Satellite Image Time Series. IGARSS 2018.
[2] Ghassemi et al. Learning and Adapting Robust Features for Satellite Image Segmentation on Heterogeneous Data Sets. Geoscience and Remote Sensing 2019.
I understand that the authors cannot compare to everything. But the authors should compare to representative baseline methods. Methods mentioned in the related work section (Section 2.1) can also be compared to.
- The proposed method is very application specific. The author only discussed the remote sensing application. Given the ICLR community's interest in general methods that can be applied to (or already been tested on) multiple applications, the paper would have been stronger if the methods applicabilityto other domains was discussed (and even better demonstrated).