Venue: ICLR 2023 Conference

Reviews:

final_score: rating: 5: marginally below the acceptance threshold
review_content: summary_of_the_paper: 1) This paper propsed a different weather prediction model by inducing a new dataset [above russia] which is adopted to evaluate all current models such as ConvLSTM etc. 

2) The results looks promising on this new datasets. 
strength_and_weaknesses: Pro: 
1) The dataset is attractive, I wonder how the models performance across different datasets, are their performance rankings consistent or not ? 

Cons: 
2) The methods used are relatively low-cost, I wonder how heavy cost models, such as transformers like architectures, perform comparing with these baselines. 

How is the dataset usability ?  Is the model trained based on such dataset can be comparable with the most advanced weather forcast devices .


clarity,_quality,_novelty_and_reproducibility: 1) The paper is clear and easy to understand, novelty is limited but the dataset is attractive.  The paper should be easy to reproduce. 
summary_of_the_review: Good dataset, while limited domain relativity and novelty.
review_title: null

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: This work builds a new weather radar dataset to explore the precipitation nowcasting task, which additionally considers a wide variety of geographical and climatic conditions.  Moreover, the authors carefully describe the acquisition process and distributions of their dataset. Then, they perform several baselines for evaluation on their proposed dataset. 
strength_and_weaknesses: Strength: 
Their proposed weather radar dataset considers more dimensions in contrast to previously published works, which could be useful to achieve the precipitation nowcasting. This dataset covers multiple climate zones, large timestamps over a two-year period. 

Weaknesses:
The baseline methods seem too naïve and lacks more ablation study experiments.
1.	The authors claim their dataset could be applied into other ML task, e.g. data studying, anomaly detection. Wish to conduct related experiments to verify this statement.
2.	I wonder whether there exist additional data from other zones, which could be implement the cross evaluation of their baseline models.
3.	There seems lack more details related works about this topic.
4.	The evaluation metric just uses MSE in Table 2, which cannot reflect the accuracy of nowcasting.
5.	The results of the previous methods on the proposed dataset also should be listed.

clarity,_quality,_novelty_and_reproducibility: Could you provide an evaluation of the quality, clarity and originality of the work?
The proposed dataset is significantly important, which will be practical in precipitation nowcasting. 
The method could be easily implemented follow their descriptions. 

summary_of_the_review: I suggest the authors could submit to a more appropriate journal will have a higher impact in their community.
review_title: null

final_score: rating: 5: marginally below the acceptance threshold
review_content: summary_of_the_paper: The paper presents a database of precipitation forecasting from 30 Russian meteorological stations, and a benchmark of state-of-the-art deep learning methods for the forecast estimation. 
strength_and_weaknesses: Strengths:
- The dataset is rich, containing information at several altitude levels, during 2 years, and with additional fields that are not always present in such datasets.
- the comparison with other datasets is detailed, clear and very useful
- a large number of state-of-the-art deep learning methods are compared on this dataset
- a study on uncertainty estimation, performed by varying the seed of the learning phase, is performed

Weaknesses:
- from the table 1, it is not clear how this new dataset would make a difference, in particular, it looks to me that the fact that this dataset is coarse (the one with the smaller grid size yet on a large region) is probably very limiting for a task such as precipitation.
- there are no methodological novelties
- the uncertainty estimation is rather simple

clarity,_quality,_novelty_and_reproducibility: The paper is very well written and clear. 
While a very cutting-edge database release, revealing new insights and new model usages, can be worth publishing, it looks to me that this database is too similar to the others to be publishable in ICLR. The release of such dataset is important, and it is good for the community to have access to it, but it can't be the main contribution of an ICLR paper.
Both codes and data are released, even if it is not possible to access it now (the url is hidden due to blind review)
summary_of_the_review: The paper is very clear and presents an interesting database and benchmark of different precipitation forecasting databases and deep learning forecast estimation models. But I don't think it is enough for an ICLR publication as there is no particular novelty on either the database nor the methods.
review_title: null

Meta-review: metareview:_summary,_strengths_and_weaknesses: This paper is relevant to both the machine learning and meteorological communities with research interests in weather forecasting in general and precipitation nowcasting in particular. Unlike most of the ICLR submissions, the work proposes a new benchmark dataset on which multiple existing methods are evaluated and compared.

Admittedly, the general ICLR review criteria may not be the most suitable for this type of papers. In fact this is why NeurIPS has introduced the Datasets and Benchmarks Track (https://neurips.cc/Conferences/2022/CallForDatasetsBenchmarks). Their review criteria were specially designed for this type of papers and hence can be used here for reference as well.

Let me point to some factors that are considered when evaluating submissions for the NeurIPS special track: (1) Utility and quality of the submission: Impact, originality, novelty, relevance to the NeurIPS community will all be considered. (2) Completeness of the relevant documentation: For datasets, sufficient detail must be provided on how the data was collected and organized, what kind of information it contains, how it should be used ethically and responsibly, as well as how it will be made available and maintained. For benchmarks, best practices on reproducibility should be followed. (3) Accessibility and accountability: For datasets, there should be a convincing hosting, licensing, and maintenance plan.

I am afraid this paper could not be accepted if the same review criteria would be applied, particularly on the second and third points. The authors are recommended to consider them in revising their paper if they want it to be published in a suitable venue that can make high scientific impact.
justification_for_why_not_higher_score: This paper cannot satisfy the criteria for reviewing papers that focus on datasets and benchmarks.

justification_for_why_not_lower_score: N/A
Decision: Reject