Venue: NeurIPS 2022 Conference

Reviews:

final_score: rating: 5: Borderline accept: Technically solid paper where reasons to accept outweigh reasons to reject, e.g., limited evaluation. Please use sparingly.
review_content: summary: There is not much to say here. The paper organizes a large precipitation dataset from both high res and low res sources and illustrates some baselines. The problem addressed is both interesting to ML (esp. to those interested in mixing physics and ml), and important. The question then, is; given that this data can be obtained directly from the original sources, is a new organization of it necessary and does it warrant a publication in an ML conference. Although I expect other reviewers will disagree, I have a positive view. Publishing this paper will probably increase interest in ML in this set of problems, and some in the community will find the dataset useful.
strengths_and_weaknesses: See above
questions: SPDnet is not a very good name…
limitations: It says in the checklist that the code and the data are proprietary. This needs to be clarified. What’s the point of publishing a dataset if it cannot be used by others?
review_title: null

final_score: rating: 6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.
review_content: summary: This paper presents a large scale dataset for spatial precipitation downscaling which contains more than 62, 400 pairs of high-quality low/high-resolution precipitation maps for over 17 years, ready to help the evolution of deep learning models in precipitation
downscaling. The precipitation maps  collected in the dataset  cover various meteorological phenomena such as hurricane and  squall. The data are organised in time series of maps, with 720 maps/month. Comprehensive metrics are also provided to evaluate the performances of models. 
strengths_and_weaknesses: This paper is well written and brings comprehensive dataset for spatial precipitation. However  the technical novelty is low.

questions: Eastern coast of US has been selected for data collections. What about other regions ?
limitations: The dataset lacks precipitation maps for several regions
review_title: null

final_score: rating: 6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.
review_content: summary: The paper proposed a large-scale spatial precipitation downscaling dataset named SPDNet. The dataset contains more than 62400 pairs of high-quality low/high-resolution precipitation maps for over 17 years, and covers more than 9 million squre kilometers of land area. The author also introduced 6 metrics that evaluate different aspects of the downscaling models, and 2 summary metrics that combine these 6 individual metrics. The author viewed the task as a video super-resolution problem and compared 14 methods. From the experimental results, the overall performance of the video super-resolution (VSR) models are better than Single-Image Super-Resolution (SISR) models.

strengths_and_weaknesses: Stengths:

1. The paper proposed the first large-scale precipitation downscaling dataset. This is an important scientific problem and a large-scale dataset can help move the area forward. In addition, the author pointed out the unique characteristics of the task such as temporal misalignment, temporal sparse, and fluid properties.
2. The paper proposed 6 metrics for evaluating the models, including 4 reconstruction metrics that focuses on evaluating if the predicted high-resolution precipitation map matches the ground-truth, and 2 dynamic metrics that evaluate the dynamics of the predicted precipitation (via first order dynamics).
3. The paper compared 14 models, including the Kriging method that has been widely used in the geospatial community, and other SISR and VSR methods. The author also extended SRGAN, EDSR, ESRGAN to be VSR methods.

Weaknesses
1. It seems that RMSE is itself a very good summary metric. Thus, it is not clear why we will still need PEM / PDEM.
2. Currently, the state-of-the-art image super-resolution model is based on vision Transformers (e.g., SwinIR) and the author need to reference the latest progress in this area.

questions: 1. The author mentioned that the dataset contains lots of different events such as hurricane, squall. Are the sequences in the dataset marked with the event type?

limitations: 1. The paper will be limited regarding the coverage of the baseline methods. However, it is difficult to cover all the latest image super-resolution methods so it is acceptable.
review_title: null

final_score: rating: 4: Borderline reject: Technically solid paper where reasons to reject, e.g., limited evaluation, outweigh reasons to accept, e.g., good evaluation. Please use sparingly.
review_content: summary: This paper proposed a dataset consists of precipitation image sequences named SPDNet for spatial precipitation downscaling as well as a novel implicit dynamics estimation driven model. The proposed model as well as baseline models are evaluated on SPDNet with task specific metrics. 
strengths_and_weaknesses: **Strengths**
1. **Valuable dataset**: The large scale dataset SPDNet is in high resolution and in sequence, which is a valuable contribution to data-driven meteorological research.
2. **Benchmark**: The authors have evaluate SOTA super resolution models on the proposed dataset ,and thus provide a good benchmark.

**Weaknesses**
1. **Necessity of low resolution data**: The authors claim in Introduction that the data obtained by simulated degradation is different from the real data collected by two different systems. However, there is no further discussion on it. It is insufficient to argue that their approach is better than obtaining low resolution data by simply downsampling the high resolution data. The author should compare these two approaches empirically, e.g., demonstrate that models trained with real data are able to reconstruct better high resolution data and hence boost the performance on downstream tasks.
2. **Unconvincing evaluation**: There are no clues about the effectiveness of proposed novel compound metrics PEM and PDEM. It would be much more convincing to conduct empirical studies to prove that models achieving better PEM/PDEM demonstrate better ability on addressing some concerned issues.
3. **Missing training details**: Selected baselines are not designed for precipitation data. It is necessary to (at least slightly) modify and tune the models for fair comparison. However, there is no information about these details except for a single statement "we also adjust the hype parameters of these models for better performance" in Supl. Sec.5.2.
4. **Formatting errors**: There are some language errors like "e.t.c." $\rightarrow$ "etc.", and misleading notations in mathematical expressions such as missing brackets in Eqn.1, missing description of "L", "H", "T" in line 250. Scores of 6 commonly used metrics in Table 1 should also be highlighted.
questions: 1. What are the advantages of using the real data collected by two different systems for training super resolution models than using simple downsampling algorithms? If the two domains need to be similar, directly downsampling the high resolution data is both cheap and effective. If the two domains need to differ a lot from each other, it would be better to categorize the task as domain transferring instead of "spatial precipitation downscaling".
2. The qualitative results shown in Figure 4 are hard to distinguish. Could the authors provide more evidence to demonstrate that the models achieving better PEM and PDEM generate better predictions? 
limitations: Listed in **Weaknesses**
review_title: null

Meta-review: metareview: This paper describes SPDNet, a dataset for spatial precipitation downscaling. 

Experiments are provided using a fairly wide set of alternative methods - 14 models (including Kriging which is a widely used standard method in the meteorological community) - as well as a novel architecture proposed by the authors. The authors also extended SRGAN, EDSR, ESRGAN from Single Image Super Resolution (SISR) methods to Video Super Resolution (VSR) methods. While the level of innovation on the neural architecture side of the work is not extreme, clear value is provided in terms of contributions to neural architecture development. Reviewers felt that the dataset itself, the wide variety of models examined and the large set of evaluation metrics offers value to the community and that this dataset could help bring more interest to the problem domain. 

During the discussion period it was made clear that "All relevant codes and datasets are open-source for research purposes" and that 
"The dataset and the code are not proprietary. We will build a dedicated github repository and website for users to easily use our datasets and codes." It is important that this is indeed is fully executed by the authors. 

Three of four reviewers recommended acceptance. 

For all these reasons the AC recommends acceptance.
confidence: Less certain
Decision: Accept