PostCast: Generalizable Postprocessing for Precipitation Nowcasting via Unsupervised Blurriness Modeling

Authors: Junchao Gong, Siwei Tu, Weidong Yang, Ben Fei, Kun Chen, zhangwenlong, Xiaokang Yang, Wanli Ouyang, LEI BAI
Primary area: applications to physical sciences (physics, chemistry, biology, etc.)
Keywords: AI for Science; Precipitation Nowcasting; Diffusion Model; Zero-shot Blurriness Kernel; Auto-scale Denoise Guidance

Abstract:
Precipitation nowcasting plays a pivotal role in socioeconomic sectors, especially in severe convective weather warnings. Although notable progress has been achieved by approaches mining the spatiotemporal correlations with deep learning, these methods still suffer severe blurriness as the lead time increases, which hampers accurate predictions for extreme precipitation. To alleviate blurriness, researchers explore generative methods conditioned on blurry predictions. However, the pairs of blurry predictions and corresponding ground truth need to be given in advance, making the training pipeline cumbersome and limiting the generality of generative models within blurry modes that appear in training data. By rethinking the blurriness in precipitation nowcasting as a blur kernel acting on predictions, we propose an unsupervised postprocessing method to eliminate the blurriness without the requirement of training with the pairs of blurry predictions and corresponding ground truth. Specifically, we utilize blurry predictions to guide the generation process of a pre-trained unconditional denoising diffusion probabilistic model (DDPM) to obtain high-fidelity predictions with eliminated blurriness. A zero-shot blur kernel estimation mechanism and an auto-scale denoise guidance strategy are introduced to adapt the unconditional DDPM to any blurriness modes varying from datasets and lead times in precipitation nowcasting. Extensive experiments are conducted on 7 precipitation radar datasets, demonstrating the generality and superiority of our method.

Decision: accept (poster)
Reviewer scores: 3, 6, 8

Reviews and discussion:
<thread>
creation date:
1730506759539

thread_type:
official_review

<thread_object>
creator: reviewer_cfdx

<content>
summary:
The paper proposes an unsupervised postprocessing method for precipitation nowcasting outputs to remove blurriness from these predictions, especially at longer lead times. This can be important to obtain accurate extreme precipitation events. The blurriness is eliminated with an unconditioned denoising diffusion probabilistic model guided by blurry predictions. The authors introduce a zero-shot kernel estimation method and an auto-scale denoise guidance strategy and show that their proposed framework generalizes to many blurriness modes across different datasets and varying lead times.

strengths:
- The problem is well-motivated. An unsupervised method alleviates the need for labels and the model is no longer limited to the blur modes seen during training. Given that this framework is independent of the spatiotemporal prediction model and can be added on top of any prediction model, it makes this work very useful and widely applicable. 
- The results strongly demonstrate the ability of their proposed method. The experiments are extensive and prove PostCast’s effectiveness in recovering weather patterns across multiple prediction models, datasets, and lead times.

weaknesses:
- The "method" section could be more organized, creating subsections could help. Moreover, the “zero-shot kernel estimation mechanism” and “auto-scale denoise guidance strategy”, should be explained in more detail, as these are the main contributions this paper makes. 
- The authors should justify using "CSI" (Critical Success Index) scores as the only evaluation metric. Adding another metric such as “Recall” (True positive rate) can complement the current evaluation and make it more comprehensive.

questions:
The current metric/score mainly evaluates whether the small-scale features are recovered and extreme events are captured well after deblurring. However, isn’t it also important to assess how closely the nowcasting outputs, after deblurring with this methodology, match the ground truth, generally? If we don’t threshold the pixel values in this case and compute an error metric such as MSE and PSNR, that could also be informative. The authors can consider adding these scores.
</content>
</thread_object>

</thread>
<thread>
creation date:
1730517366195

thread_type:
official_review

<thread_object>
creator: reviewer_ipfr

<content>
summary:
This paper presents a novel postprocessing approach for precipitation nowcasting, named PostCast. The core innovation of PostCast lies in utilizing an unconditional diffusion model (DDPM) to remove blurriness from precipitation predictions without requiring paired data of real observations and blurry predictions. The method introduces a zero-shot blur kernel estimation mechanism and an auto-scale denoise guidance strategy, enabling the model to adapt to various datasets, prediction lead times, and blur modes to generate sharper predictions. The authors conducted experiments on multiple precipitation datasets and forecast models, demonstrating PostCast’s effectiveness in improving prediction accuracy for extreme precipitation events. The paper proposes a highly generalizable and adaptive postprocessing framework that shows strong potential for broad applications in precipitation nowcasting tasks.

strengths:
1. **Originality**: The primary innovation of PostCast lies in employing an unconditional diffusion model (DDPM) to remove blurriness from precipitation predictions without relying on paired data. By introducing zero-shot blur kernel estimation and an auto-scale denoise guidance strategy, PostCast adapts to various datasets, prediction lead times, and blur modes across different models. This unsupervised deblurring approach appears to be the first of its kind in the domain of precipitation nowcasting, demonstrating creative thinking.

2. **Completeness of Experiments**: The paper provides a well-designed experimental setup, covering multiple datasets and forecast models to showcase the generalizability and robustness of PostCast. The experiments include diverse evaluation metrics and extensive visual results, effectively validating the method’s performance in extreme precipitation event prediction.

weaknesses:
1. **Incomplete Evaluation Metrics**: The paper relies only on CSI, lacking POD, FAR, and similar precipitation metrics, as well as image quality metrics like PSNR and SSIM. Including these in the appendix would strengthen the evaluation’s comprehensiveness.

2. **Insufficient Related Work Discussion**: The discussion of GAN methods in precipitation nowcasting is limited, and the introduction to Transformer-based approaches is also lacking. Expanding these sections would better contextualize PostCast’s contributions.

3. **Lack of Comparison with GAN Methods**: There is no direct comparison with GAN-based approaches in precipitation nowcasting. Adding such comparisons would provide a more complete assessment of PostCast’s performance.

questions:
1. **Evaluation Metrics**: Could you add results for POD, FAR, and image quality metrics like PSNR and SSIM in the appendix to provide a more comprehensive evaluation of PostCast?

2. **Related Work**: Could you expand the related work section to discuss GAN and Transformer methods in precipitation nowcasting to better contextualize the background and contributions of PostCast?

3. **Comparison with GAN Methods**: Could you include comparisons with GAN-based methods in precipitation nowcasting to further validate PostCast's effectiveness?
</content>
</thread_object>

</thread>
<thread>
creation date:
1730606751600

thread_type:
official_review

<thread_object>
creator: reviewer_sepj

<content>
summary:
The paper proposed a universal post-processing method, PostCast, designed for precipitation nowcasting. This method achieves denoising of convolution based prediction models using unconditinal DDPM through two key innovative points: zero shot blur estimation mechanism and auto scale gradient guidance strategy. The model was trained on a combination of five different datasets and prediction results generated by different models, and outperforms other conditional denoising models in terms of CSI metrics.

strengths:
1. The paper proposes a novel approach to estimating blurriness by utilizing the gradient of metric respect to kernel parameters, which serves as a guide for the sampling of x_ {t-1}.
2. The paper introduces an auto-scale gradient guidance strategy that automatically calculates the guidance scale corresponding to different forecast time periods, models, and datasets. This strategy enables the model to adaptively denoise the prediction results.

weaknesses:
1. CasCast and DiffCast are both designed for **predicting precipitation**, but not **postprocessing tasks**. It is crucial for the authors to compare with other generative models such as GAN. This would provide a more comprehensive evaluation of PostCast's performance against a wider range of methodologies.
2. The authors have compared PostCast with CasCast and DiffCast only in **out-of-distribution datasets** but did not compare them on HKO7, SEVIR, etc.
3. Relying on CSI as the only metric may not sufficiently judge the quality of precipitation prediction. The average intensity of the precipitation predictions can significantly impact CSI. For instance, using simple histogram matching on predictions can also significantly improve CSI. The authors should clarify whether the CSI improvement is due to **increased intensity** from the model or other factors to provide a more accurate assessment of the model's performance.

questions:
The questions here are related to the three points described as weaknesses.
</content>
</thread_object>

</thread>
<thread>
creation date:
1732873314993

thread_type:
official_comment

<thread_object>
creator: authors

<content>
title:
General repsponse to all reviewers

comment:
We are very grateful to all the reviewers for their valuable comments and suggestions on this article.

---
We are glad to see the reviewers' recognition of our work.
- "The paper proposes a **novel** approach to estimate blurriness by utilizing the gradient of metric respect to kernel parameters. The paper introduces an auto-scale gradient guidance strategy that automatically calculates the guidance scale corresponding to different forecast periods, models, and datasets." (**Reviewer SepJ**)
- "**Originality**: The primary innovation of PostCast lies in employing an unconditional diffusion model (DDPM) to remove blurriness from precipitation predictions without relying on paired data...This unsupervised deblurring approach appears to be the first of its kind in precipitation nowcasting, demonstrating **creative thinking**. **Completeness of Experiments**: The paper provides a **well-designed experimental setup**, covering multiple datasets and forecast models to showcase the generalizability and robustness of PostCast." (**Reviewer iPfR**)
- "The problem is **well-motivated**. Given that this framework is independent of the spatiotemporal prediction model and can be added on top of any prediction model, it makes this work **very useful and widely applicable**. The results **strongly demonstrate the ability** of their proposed method. **The experiments are extensive** and prove PostCast’s effectiveness in recovering weather patterns across multiple prediction models, datasets, and lead times."  (**Reviewer CFdX**)

---
We have now addressed all weaknesses and questions raised by reviewers. Changes in the manuscript are highlighted in blue. Below, we summarise the major updates:
- For a comprehensive evaluation, we have included additional metrics such as CSI-avg, HSS-avg, POD-avg, SSIM, and PSNR. The details about how to calculate HSS and POD are added to the revised paper line 369-370. (**Reviewer SepJ, iPfR, CFdX**)
- To prove the effectiveness of our PostCast, we have added comparisons with other postprocessing methods including a diffusion-based and a GAN-based method. (**Reviewer SepJ**)
- We compare our method with other methods including postprocessing ones and precipitation nowcasting ones in both in-domain datasets and out-of-domain datasets. (**Reviewer SepJ**)
- To gain deep insight into the strength of our PostCast, we have provided a detailed analysis of the sources of CSI improvement. (**Reviewer SepJ**)
- To provide a more complete assessment of PostCast, we have conducted comparisons with two GAN-based precipitation nowcasting methods. (**Reviewer iPfR**)
- To better contextualize the background and contributions of PostCast, we have enriched the related work section to discuss GAN and Transformer methods in precipitation nowcasting. (**Reviewer iPfR**)
- To make a clearer clarification, we have reorganized the "method" section, creating subsections to explain the “zero-shot kernel estimation mechanism” and “auto-scale denoise guidance strategy” in more detail. (**Reviewer CFdX**)

---
Last but not least, thanks again to PCs, ACs, and all reviewers for their time and effort in reviewing.
</content>
</thread_object>

</thread>
<thread>
creation date:
1733224114415

thread_type:
official_comment

<thread_object>
creator: authors

<content>
title:
General Comments and Revision Summary

comment:
We sincerely thank all the reviewers for their insightful reviews and helpful suggestions, as well as for acknowledging the importance, comprehensiveness, and contributions of our work. 

---
We would like to re-emphasize the main contribution that our method aims to provide an independent framework that can be applied to a large range of spatiotemporal models, prediction steps, and datasets. Our PostCast improves the modeling of small-scale weather patterns in the blurry predictions and aligns them better with the distribution of ground truth in a generalizable way. Our contribution is also recognized by Reviewer CFdX that "it makes this work **very useful and widely applicable**; The results **strongly** demonstrate the ability of their proposed method. ", and by Reviewer iPfR "**Originality...the first of its kind in the domain of precipitation nowcasting**; a **well-designed** experimental setup, covering multiple datasets and forecast models to showcase the **generalizability and robustness**".

---
We have submitted a revised version of our paper and provide a summary of the revisions below:

---
1. Based on the suggestions from **Reviewers SepJ, iPfR, CFdX**, we have included additional metrics such as CSI-avg, HSS-avg, POD-avg, SSIM, and PSNR for a more comprehensive evaluation and demonstrated better results of predictions.
2. Following **Reviewer SepJ's** feedback, we have also included comparisons with other postprocessing methods including a diffusion-based and a GAN-based method. As **Reviewer iPfR** confirmed that in Strengths 2 "**the first of its kind in the domain of precipitation nowcasting**". We are only able to transfer diffusion-based and GAN-based postprocessing methods from computer vision for comparison.
3. As suggested by **Reviewer iPfR** and encouraged by his expert in precipitation nowcasting, we have included GAN and Transformer methods in the revised related work and introduction.
4. **Reviewer iPfR** requested more comparisons with two GAN-based precipitation nowcasting methods. We conducted these experiments and included them as well as the corresponding discussions in the paper.
5. We have compared our method with other methods including postprocessing ones and precipitation nowcasting ones in both in-domain datasets and out-of-domain datasets as recommended by **Reviewer SepJ**.
6. Thanks to **Reviewer CFdX's** suggestions, we have re-organized the "method" section to make our paper easier to follow.
7. We have provided a detailed analysis of the sources of CSI improvement to **Reviewer SepJ**. Our CSI improvement not only comes from the location accuracy (FSS score) but also the better alignments with the distributions of ground truth and our predictions.

---
Lastly, we would like to express our gratitude once again to all the reviewers for their constructive feedback, which has greatly improved our paper. We also hope that the reviewers will be available for further review discussions.
</content>
</thread_object>

</thread>


Metareview:
metareview:
The authors present a method to postprocess blurry precipitation future predictions by using a fine-tuned unconditional diffusion model on precipitation data (to have a reasonable generative prior on precipitation images) and guiding its generation process to the accurate prediction by using a blurry prediction (from any spatiotemporal predictive model) and a blur kernel estimation optimization algorithm. The authors state their primary motivation for this method (as opposed to existing conditional GAN/diffusion models to produce sharp forecasts) as to avoid repeated training for new OOD data and simplifying the training workflow of such conditional models that require blurry predictions as part of the training data.

The authors show results on applying their method (called PostCast) on spatiotemporal predictions from different models that show that PostCast is very effective in generating sharp predictions, reflected in several metrics. They further compare with other generative based approaches and show that their method is superior.

Strengths: Effective postprocessing method with positive results (clear visuals and many metrics) for precipitation nowcasting (an important research domain), experiments cover many datasets both in-domain and out-of-domain.
Weaknesses: The main weakness that seems to have been raised is the incomplete contextualization of the significance of the results/method. As a result, the contribution may be viewed as an interesting alternative to precipitation nowcasting through generative models, but incremental.

Points of improvement relating to the weakness:
* One issue raised by a reviewer is in incomplete comparisons with GAN-based postprocessing models. It is unclear why the CV models are chosen as baselines for this purpose. More in-depth investigations into GAN-based postprocessing models can greatly strengthen the paper.
* Relating to the above major issue, another issue was raised: The reviewer is also concerned about the applicability of the postprocessing method. One motivation is to reduce training complexity of requiring blurry predictions as data (for conditioning the generative model). More detailed discussions could have clarified this motivation better. Example:
  - it is unclear if all benchmarked GAN models have this issue (DGMR for example - does it require blurry prediction as data?). If not, this 
    needs to be discussed with proper context
  - PostCast is described as unsupervised - however, it seems that there is a finetuning stage with ground truth predictions from the  
    training dataset. This makes it difficult to judge how the training complexity is reduced because (1) both PostCast and DiffCast (etc) 
    require some blurry prediction model (same complexity) and (2) DiffCast requires blurry predictions as further input for the full dataset 
    (inference runs) and PostCast requires finetuning an unconditional diffusion model . Currently, the paper reads as though the 
    motivation is if the blurry prediction model is for some reason a black box. Further, models like DiffCast could also be trained with any 
    backbone (spatiotemporal model).

What could help: A clearer discussion on (1) the value of PostCast (some quantitative characterization of the cost that makes "the training pipeline cumbersome" (from the abstract)) with (2) more details on how existing methods work exactly (DGMR vs DiffCast vs PostCast vs CV-based GAN models) and why they are good baselines; (3) a discussion on other GAN-based postprocessing methods (downscaling models, bias-correction models) and (4) presenting PostCast as a valuable alternative in light of (1) rather than as a new SOTA model.