Venue: ICLR 2021 Conference

Reviews:

final_score: rating: 5: Marginally below acceptance threshold
review_content: === Summary ====
This paper proposes a new method for image-to-image translation on multi-spectral imagery. The proposed method uses variational auto-encoders and generative adversarial networks to generate synthetic bands in satellite imagery.  Novelty of the proposed method is that the authors introduce a shared spectral reconstruction loss and skip connection to generate synthetic spectral bands. This allows to generate synthetic bands with higher accuracy that the original image-to image translation method.

=== Pros ==== 
Obtaining additional data for satellite imagery is notoriously difficult. Moreover, attempts to combine different sources of satellite imagery often fail due to the difference in resolution, band spectrum and the time, when the imagery was obtained. Therefore, all the work that aims to help solving these problems is extremely important. Authors here do not address the problem of time when the imagery was obtained (since they work with geostationary satellites), but they do address the other two: lack of data and band incompatibility. Therefore, the approach, presented in this paper, is very interesting and important application. 
To our knowledge, VAE-GAN approach to satellite imagery was used before for Sentinel imagery [1], however the method, proposed here, improves this approach by introducing a new type of loss. 
To summarise, the proposed method is technically correct and the application is interesting and important.

=== Cons === 
The authors claim three main contributions in the paper: “1) introduce a shared spectral reconstruction loss to a VAE-GAN architecture for synthetic band generation; 2) test our methodology on real-world scenarios; 3) present and release a test dataset of four hemispheric snapshots from three publicly available geostationary satellites for future research.”
1) The novelty of the method seems to be relatively limited and domain specific. Specifically, the original method was introduced before [2] and [3] in other applications. 
Authors introduce a new type of loss to improve the performance of the model on a particular type of the input data (multispectral). This new type of loss improves the performance of the model from [3] for this particular type of input. However, the other properties of shared spectrum reconstruction loss are unclear: is it domain-specific or it can be extended to other problems?
3) Sorry, I have not found the link to the dataset, which is mentioned as the third contribution of the paper. Could you please explicitly state where you present it or remove this contribution?

=== Basis for recommendation ===
I’m borderline on this paper since the paper is very interesting and the application is important, but technical novelty is slightly limited. 
Quality. The quality of the paper is very high and satisfy the standards of the ICLR publication.
Clarity. The material is presented clearly and correctly.
Originality. The paper introduces an original contribution, which was not published before (to my knowledge).  
Significance. The results are important to the domain of climate-change related applications, satellite imagery processing and EO/GIS . 

=== Questions to address in the rebuttal ===
1.	Please specify the novelty and importance of the proposed method. Is it possible to apply it elsewhere?
2.	Does the proposed method work with other types of imagery (other satellites/other types of sensors?) If yes, what other types of satellites/sensors can it be applied to?
3.	What are the limitations of the proposed approach?
4.	Could you please explicitly state where you present the dataset or remove this contribution?
5.	Could you explicitly state the steps for generation of new bands? 
6.	For the readers, unfamiliar with satellite imagery processing, it might be interesting to understand why generation of synthetic bands is of particular importance.

=== Minor comments and additional feedback (not necessary to address in rebuttal)===
1.	It’s not very clear how authors address the problem of different band resolution.
2.	How would performance change if the tiles (or parts of tiles)  with clouds are removed from original data and there is less noise in the Sensor condition?
3.	“Shortwave infrared band cannot be inferred from others”: this is an interesting observation. It would be interesting to see in the future the accuracy with which the missing bands can be synthesised from a (random) set of bands and how accuracy increases based on the initial number of bands that were used. 
4.	Can this method be applied to radar data?
References
1.	E. H. Sanchez, M. Serrurier, and M. Ortner, “Learning disentangled representations via mutual information estimation,” 2019. https://arxiv.org/abs/1912.03915 
2.	A. B. L. Larsen, S. K. Sønderby, and O. Winther, “Autoencoding beyond pixels using a learned similarity metric,”  (2015) http://arxiv.org/abs/1512.09300
3.	M.-Y. Liu, T. Breuel, and J. Kautz, “Unsupervised image-to-image translation networks,” in Advances in Neural Information Processing Systems 30, (2017) http://papers.nips.cc/paper/6672-unsupervised-image-to-image-translation-networks.pdf

review_title: Review of the paper "Spectral Synthesis for Satellite-to-Satellite Translation"

final_score: rating: 6: Marginally above acceptance threshold
review_content: This paper applied the unsupervised VAE-GAN model for satellite-to-satellite translation to generate synthetic spectral bands. My general evaluations of this paper are:
pros: The application is meaningful. This paper exploits the possible application of VAE-GAN based image-to-image translation model to the generation of synthetic spectral bands. Of course, it will bring new solutions for the subsequent application of satellite images. From the method perspective, the proposed Shared Spectral Reconstruction Loss (SSRS) has been testified to significantly improve the final spectral band simulation results.
cons: From the model perspective, the only shining point compared to the previous model is the introduction of SSRS loss.  The notations are not clear. In the experiments, the comparison methods are missing. It is hard to distinguish the performance of the proposed model.

More general comments are listed as the following.
There are so many notations in the proposed method introduction,  however, the meaning of notations to the satellite images are not clear. I think most notations are from ref[23], but the most contribution of this paper is the application to satellite images. It may be more meaningful to introduce the notations from the satellite images perspective.

The experimental dataset is complex, but the introduction is insufficient.  It is difficult to make the readers understand the task of the experiment. I suggest to enhance the introduction of the problem and formulate the mathematical model from satellite-to-satellite translation clearly.

In the experiments, it is hard to rate the performance of the proposed model. There are no comparisons to previous state-of-the-art, and I'm not so familiar with this dataset. The authors should pay more attention to make their datasets understood to the readers, including the performance judgment and possible further applications.
review_title: This manuscript is an application of unsupervised VAE-GAN model for satellite-to-satellite translation.  The application is meaningful, whereas the method is weak.

final_score: rating: 5: Marginally below acceptance threshold
review_content: This paper tries to generate synthetic unobserved spectral imagery from a set of existing spectral channels. This is an image-to-image translation task, for which VAEs and GANs are effective to address. For this specific satellite band-to-band translation task, authors adopt the VAE-GAN framework (adding a skip connection between the input and generator), with a new added spectral reconstruction loss.  Experiments show the effectiveness of the proposed method.

Pros:

1. As stated by the authors, this paper introduces a shared spectral reconstruction loss to a VAE-GAN architecture for synthetic band generation.
2. Simulated experiments show that the generated banded is possible and effective.
3. Using the generated band, the proposed model improves segmentation performance beyond baselines.
4. A test dataset of four hemispheric snapshots from three publicly available geostationary satellites is thankfully received.


Cons:

I would say this paper has limited contribution, in terms of the novelty of the method. For me, this paper is an application paper. Specifically:

1. I would say among the three contributions (listed before section 2. background), only the first contribution of proposing a new loss makes sense for me as we always need to validate methods on real datasets. For the proposed shared spectral reconstruction loss, it's a simple KL divergence loss (Equation 5) imposed on overlapping channels, specifically tailored for this band-to-band translation task under the VAE-GAN framework. In one word, the contribution is limited.

2. Throughout the experiments section, I cannot find a baseline method from a third-party. The reason could be that no one did this task before, which is good. However, including an off-the-shelf image-to-image translation method as a baseline is necessary. It will help readers to understand the difficulty of this band-to-band translation task.

3. To make the proposed shared spectral reconstruction loss stronger, I would like to see an ablation study concerning how it interacts with other losses in equation 6. Please use a different number of $\lambda_6$ to illustrate.

In summary, I would say this paper demonstrates the success of VAE-GAN on the band-to-band reconstruction task, which is good, but with limited technical novelty.









review_title: Good application of VAE-GAN, but the contribution is not big

Meta-review: This paper introduces an interesting application of VAE-GAN to the problem of Spectral Synthesis across satellite observations with some additional domain specific changes (new loss, ...). The introduction of a new dataset is also very interesting and can open the door for more methodological development in the community. 

While the application is original, the methodological contributions have been judged limited and domain specific by most of the reviewers. The responses from the authors was appreciated by the reviewers (especially the comparison to the UNIT baseline) but did not change their opinion about the limited methodological novelty of the approach. The AC recommends a reject but encourages the authors to resubmit it to a more applied remote sensing venue.

Decision: Reject