Perceptually Constrained Precipitation Nowcasting Model
Authors: Wenzhi Feng
Xutao Li
Zhe Wu
Kenghong Lin
Demin Yu
Yunming Ye
Yaowei Wang
Venue: ICML 2025 poster (2025)
Keywords: Precipitation Nowcasting
Spatio-Temporal Prediction
Generative model
Primary area: applications->chemistry_physics_and_earth_sciences

Abstract: Most current precipitation nowcasting methods aim to capture the underlying spatiotemporal dynamics of precipitation systems by minimizing the mean square error (MSE). However, these methods often neglect effective constraints on the data distribution, leading to unsatisfactory prediction accuracy and image quality, especially for long forecast sequences. To address this limitation, we propose a precipitation nowcasting model incorporating perceptual constraints. This model reformulates precipitation nowcasting as a posterior MSE problem under such constraints. Specifically, we first obtain the posteriori mean sequences of precipitation forecasts using a precipitation estimator. Subsequently, we construct the transmission between distributions using rectified flow. To enhance the focus on distant frames, we design a frame sampling strategy that gradually increases the corresponding weights. We theoretically demonstrate the reliability of our solution, and experimental results on two publicly available radar datasets demonstrate that our model is effective and outperforms current state-of-the-art models.

Perceptually Constrained Precipitation Nowcasting Model
Wenzhi Feng 123
Xutao Li
Zhe Wu? 2 Kenghong Lin" 13 Demin Yu" 13 Yunming Ye 123 Yaowei Wang" 12
Abstract
Most current precipitation nowcasting methods aim to capture the underlying spatiotemporal dy- namics of precipitation systems by minimizing the mean square error (MSE). However, these methods often neglect effective constraints on the data distribution, leading to unsatisfactory pre- diction accuracy and image quality, especially for long forecast sequences. To address this lim- itation, we propose a precipitation noweasting model incorporating perceptual constraints. This model reformulates precipitation nowcasting as a posterior MSE problem under such constraints. Specifically, we first obtain the posteriori mean se- quences of precipitation forecasts using precipi- tation estimator. Subsequently, we construct the transmission between distributions using rectified flow. To enhance the focus on distant frames, we design& a frame sampling strategy that gradually increases the corresponding weights. We theoret- ically demonstrate the reliability of our solution, and experimental results on two publicly avail- able radar datasets demonstrate that model is effective and outperforms current state-of-the-art models.
Tamkeer
High LPIPSIO 301
Low L71P5/0
Figure Illustration of the accuracy perception aceuracy-perception tradcoff in pre- cipitation prediction. Historical data (center) are input into two state-of-the-arc algorithms for forecasting precipitation. ter plot demonstrates the performance of current state-of-the-ar methods terms the cecuracy-pereepiie tradcoff, with CSI as the primary metric for precipitation prediction.
scat-
Precipitation nowcasting aims to predict rainfall within the next six hours, characterized by high spatiotemporal resolu- tion and uncertainty about the future. Most of the current precipitation prediction methods are dedicated to improv- ing the accuracy where the forecast image quality ne- glected. However, image quality can reflect to a certain ex- tent whether the model can simulate the real-woriks physical state, which is especially important in long-term prediction. As shown in the left part of Figure 1, the model- smoothed prediction is obtained on the left side, and the physically realistic prediction is obtained on the right side.
1. Introduction
Weather forecasting is important spatial and temporal prediction task that affects the comfort and safety of peo- ple's ' daily lives and profoundly affects a number of key industries, such as agriculture, aviation, navigation, and en- ergy supply. Accurate weather forecast plays a crucial role safeguarding people's lives and promoting economic and social development.
Existing precipitation nowcasting models can be broadly divided into two categories, deterministic models(Bai et al., 2022; Ning et al., 2023; Wu et al., 2021) and probabilistic models(Chang et al., 2022; Luo et al., 2022; Zhang et al., 2023). Deterministic models such as ConvLSTMM
et al., 2015), Transformer( Vaswani et al., 2017). Sim VP(Gao et al.,
2022b), are designed to predict future sequences by gener- alizing spatiotemporal patterns from historical sequences. These models reduce the MSE and tend produce an av- erage prediction over all possible future states in the case of uncertainty. However, this line of methods cannot match the expected distribution of ground-truth data and thus suf- fers from the problems of absent details and intensity de- cay, which is often shown as a better accuracy performance (lower MSE) but worse perceptual performance (higher
'School Computer Science and Technology, Harbin In- stitute of Technology, Shenzhen, China 'Pengcheng Laboratory, Shenzhen, China 'Shenzhen Key Laboratory of Internet Informa- tion Collaboration, School of Computer Science and Technology, Harbin Institute Technology, Shenzhen, China. Correspondence to: Xutao <iiuutao@e edd.en......
Proceedings of the ndnd International Conference Machine Learning. Vancouver, Canada. PMLR 267, 2025. Copyright 2025 by the author(s).

---

Perceptually Constrained Precipitation Nowcasting Model
LPIPS).On the other hand, probabilistic models, such as SChan al., 2022) and PreDiff(Gao al., 2024), predict future image sequences by learning their distribu- tion from current observations. While these methods can produce predictions with better detail and intensity (lower LPIPS), they often struggle with accuracy in positioning (higher MSE), as the space of samples following the target distribution is infinite.
We propose a temperature weighted schedul- ing that lets the model focus on frames at the tail of the sequence.
Our method achieves optimal performance on both Sevir and MetcoNet datasets.
2. Related Work
In Long-sequence precipitation noweasting, neither mean squared error nor perceived quality fully measures forecast accuracy, as shown on the right side of Figure 1. Accu- rate forecasts must satisfy both criteria, along with other key metrics (e.g., CST). Therefore, achieving high-quality predictions with low error is key challenge.
For precipitation nowcasting, deterministic models can effectively capture the overall trend of precipitation movement by modeling the posteriori mean sequence of future precipitation. The ConvLSTTM(Shi et al., 2015) model integrates convolution operators into the LSTM and uses extrapolation loops to maintain consis- tency of predicted motion. Farthformer(Gao et al., 2022a) builds encoder-decoder model with Transformer struc- tures, which replaces the original attention with elaborate cube attention. SimVP et al., 2022b) uses a simplified encoder and decoder structure based on convolutional neural networks, significantly improving computational efficiency. MAU(Chang et al., 2021) improves the precipitation fore- cast by mining the current and historical spatial states to extend the time horizon. PhyDNet(Guen & Thome, 2020) uses partial differential equations (PDEs) to disentangle PDE dynamics from unknown complementary information and performs PDE-comstrained prediction in the latent space. However, all these methods suffer from blurring and high echo fading issues. Probabilistic models make predictions by modeling the data distribution of future precipitation. DGMR(Ravuri et al., 2021) uses adversarial training con- strain prediction distribution close to the real precipitation by introducing spatial and temporal discriminators. PreD- (Gao et al., 2024) constructs a diffusion model of the potential space and designs a priori knowledge modules to adjust the predicted distributions. Diffcast(Yu et al., 2024) simulates local random variations by introducing a residual diffusion mechanism. This kind of model helps gener- ate realistic details in a predictive framework but performs poorly on predictive metrics.
Precipitation Nowcasting:
In this paper, we propose a novel perceptual gonstrained precipitation forecast model, named PerepCast, which effec- tively addresses the limitations of the existing methods by constraining the data distribution of the prediction sequence, thereby improving the accuracy of the resulting predictions. First, we use a precipitation estimator to obtain the approxi- mate posteriori mean sequence of the predicted target. Then, we train rectified flow model to predict the straight line between the corresponding posteriori mean sequence and the target sequence. Specifically, we use the autoregres- sive structure of ConvLSTM as precipitation estimator to capture the underlying spatiotemporal dynamics of the precipitation system and obtain a continuous sequence of posteriori mean values. However, affected by the cumula- tive error, the distribution difference between the frames forming the posteriori mean sequence becomes more signif- icant with time, so the path from each predicted frame to the target frame is different. To better model the transition from predicted frames to target frames, we design frame- sampled rectified flow model that models each predicted frame individually and provides weighted scheduling that lets the model focus on frames with longer prediction times. The entire model is trained in an end-to-end manner, and the set of posteriori mean sequences is fed into the flow model as an initial state to solve the ODE, which avoids falling into a local optimum.
The main contributions of this paper are summarized as follows:
Image Perception Quality: Image Perceptual Quality As- sessment is method of quantifying the quality of an image, which is widely used in image reconstruction, image com- pression, and image generation. In image enhancement, PCSGAN(He et al., 2024) develops a generative adversarial network with perceptual constraints, which imposes spe- cific positional and structural constraints on the image and achieves better sense of image realism. In image restora- tion, PiRN(Jaiswal et al., 2023) proposes physically inte- grated restoration networks that introduce physically based simulators and stochastic refiners to improve its perceptual quality. In the field of video generation, (TTRMM(Chang et al., 2022) introduces an additional perceptual loss in the
We propose an end-to-end precipitation prediction model based on perceptual constraints by constraining the data distribution while achieving better prediction error and image quality.
We propose novel video-rectified flow model that uses sampling training to simulate the path from the posteriori mean sequences to the real sequences and employs lpips loss for backward propagation to ensure perceptual consistency.
2

---

Perceptually Constrained Precipitation Nowcasting Model
the Mithh
Pesalittoo Moin Recuiion
Lomper appereturen stanio worightced
Aebedylle
Widdon Soyn
Frame
Precipitation mester
Frame
Input
Sampling
Output
Figure An overview the proposed PeropCast for precipitation nowcasting. The model first obtains posteriori mean sequences future precipitation employing precipitation estimator, then constrains the posteriori distribution mean sequences using the rectified flow. It designs samplig strategy to increase the attention paid to the long-term prediction frames.
generative adversarial network, which is used to constrain the feature representations of the generated image in the dis- criminator, thereby improving the perceptual quality of the prediction. In the field of super-resolution, TDPN(Cai al., 2022) defines information recovery, which helps to overcome the sub- optimums of MAE loss.
We assume that Y and Ý are independent of each other given X. In other words, the predicted values depend only on the historical observations, and there exists Y E X = æ). which the optimal or approximately optimal solution for the posteriori mean. Furthermore, inspired by (Freirich et al., 2021), give the optimal transport form of Eq.3. as
hybrid loss to enhance the boundary
B||lY - 1-1 / 11YY - Zo) - vu f)] dtj, (3)
3. PROBLEM STATEMENT
where Zo ~r 7, LY + (11-))(1) The detailed analysis is given in Appendix A.
Precipitation nowcasting can be formulated spatiotem- poral sequence prediction problem(Gao et al., 2022a; Shi al., 2015; Tan et al., 2023), which is defined as
4. The Proposed Model
min B 1. Py
(1)
address the accuracy and perception dilemma, we pro- pose perceptually constrained precipitation nowcasting model, as shown in Figure 2, which is composed of three main components:
Specifically, given an initial precipitation sequence X [wi]/nn Iz E HxHxxXCs, the model predicts that the future sequence Y = [ux]/err, yı E H HW xCout, where Lin and Lout denote the lengths of the input and predicted frames,
(1) precipitation estimator to capture the spatial and temporal dynamics of the precipitation sys- tem and to obtain a sequence of future posteriori mean val- ues; (2) a rectified flow network that aligns the distribution of the posteriori mean sequences to the target distribution, to simulate physical reality. (3) an end-to-end training scheme. Next, we elaborate on the proposed PerepCast method by introducing the three components, respectively.
respectively. H and W denote the heights and widths of each frame. Cin and Cout denote the number channels in the input and predicted frames each time point. In addition, we use the subscript to denote the i-th frame, e.g. V:-
The posteriori mean sequence of the model can be obtained from 1. However, as the prediction time increases, the difference between the distribution the predicted frame and the distribution of the target frame becomes larger and larger. To solve this problem, we introduce perceptual constraint align the predicted distribution with the target distribution. As a result, reformulate and define the precipitation task as follows:
4.1. Precipitation Estimator
To capture the evolving precipitation patterns from historical data and to obtain smoother, more continuous forecasts, utilize ConvLSTM as the precipitation estimator.
Specifically, first we apply the patch embedding to the his- tory sequence X = [x:] Lan embedding Xpatch [m:] E RRl.xH//AX///lla then Xpates is fed into the image encoder to obtain Xenc Jes) E Th x H/.xX///X//xx10 next we feed X enc into the
E R x H XxXXC to get the
and
min ElllY prix
2
S.L. ps = Py
3

---

Perceptually Constrained Precipitation Nowcasting Model
ConVLSTM and decode it through the decoder to get the estimated value y [ RRlo.x///xX//xll
only one frame is sampled for each training iteration, with the probability of selection increasing with the distance from zo, thereby assigning different training weights to the var- ious prediction frames. Specifically, we first compute the hase weights for each of the T prediction frames:
Finally, the estimated value is optimized using the Mean Square Error criterion:
Lpe - -
(4)
wi = exp(k= i)
(6)
Eq.4 illustrates that the estimated value Y represents the posteriori mean sequence of the future precipitation. Next, we detail the construction of perceptual constraints based on this posteriori mean sequence.
k is a fixed parameter that controls the rate of weight growth, E [0, 1] is the number of each of the frames, and then the weights are adjusted using a dynamic temperature T,
w = w
where, depending on the training process, the dynamic tem- perature can be linear:
4.2. Frame-sampled Rectified Flow Model
In Section 3, we demonstrated that the precipitation fore- casting problem under perceptual constraints can be solved by Eq.3. However, long-term spatiotemporal sequence pre- diction is highly susceptible to future uncertainty. This leads to variations in the distribution of each frame in the posterior mean sequence generated by the precipitation es- timator over time. For instance, as æ - 2o, the distribu- tion of becomes increasingly realistic as approaches Tg while as x + Jy, where T Lour. the gap between the distributions of x and x; widens, resulting in smoother predictions. We attempted transfer the distribution of the posterior mean sequence to the target sequence using rectified flow model. However, since the posteriori mean sequence consists of a mixture of distributions, modeling the entire target sequence becomes challenging due to the influence of mode overlap. To more effectively model the transition from the posteriori mean sequence the target sequence, we designed rectified flow model enhanced with frame sampling.
t
T r(t) = Tstart +
( (Tead
Tsstur)
(8)
tmax
or exponential:
Toad
Fmna
T(t) = Tstart
(9)
Tstart
then normalize the adjusted weights to probability distribution pt and obtain the cumulative probability distri- bution (CDF),
m Pt
(10)
Σ 1= Wy
CDF = Σ Pj. iE -
(1I)
j=0
For sampling, random number r is generated from a uni- form distribution r ~ U(0, 1) and then the random number r is mapped to the corresponding frame sequence number using cumulative probability distribution function (CDF).
Specifically, for the posteriori mean sequence obtained by the precipitation estimator, we first stop the gradient derivation to ensure that y* - Y follows Markov process. Next, we reshape the posteriori mean sequence Y' "'E BXBXTXXXX* into Y [BB then we construct the Frame Indexed Embedding (FIE) based the order of the frames in the posteriori mean sequence. Finally, the FIE and the reshaped posteriori mean sequence W are concatenated and fed into the U-Net
sampled.index
min CDF; 2r (12)
The final rectified flow model objective function with frame sampling can be written as
C,, EE [. sid 20dh
Y
vo( ,t, FIE 1|? dt
(13)
BT * C structure to learn the velocity field from the posteriori mean sequence the target sequence at all frame indexes, with the objective function follows:
Additionally, to ensure high-quality prediction results and prevent degradation of the generated model, we employ the Lpips loss to constrain the velocity field vo, enabling the model maintain strong perceptual consistency at each step of the reversal process.
E L
IIY - Zo) - to dt
(5)
Lipips
Σ Eailldn(75
Id + vo (Ziid FIE or (Y >|2
Where Y E BBT Zo E BBX FIE E
(14)
BTxd d is the embedding dimension.
where oz (P represent the image features extracted by the deep neural network (such as VGG or AlexNet) at the I-th layer, aq is the weighting coefficient for each layer's feature.
Furthermore, to encourage the model to place greater empha- sis on frames further away from Zo, we adopt a temperature- distance weighted frame sampling strategy. In this approach,
4

---

Perceptually Constrained Precipitation Nowcasting Model
Algorithm 1 Training Process
Algorithm 2 Inference Process
Input: data X fn xy R
Input: data = [xx]]=1 hin ds E HxxWxx
initial: K, repeat
h z0 for = -1 to Lin + Lout do h Patch Saliedimmed
h 20
for = 1 to Lin + Lout do h = Patch Embeinnng
8- Lin h 9 Linitii
(o)
8- Li ti = (x; h Lin end for
end for Reshape: Merge Channel Batch and Channel Time if RectFlow then
Compute loss Spe
2 = 0, (Y*,t)
Sampling a frame ÿ-sid form [8f Nion. by sam-
20 20 + else
pled index
detach gradient of Yrsid
while t in O-N do
Sampling flow step t [1 1]
= (20. t)
Forward Process: = ((Y + (1 s
20 20
Backward Process Ue ze; 02) = ysid
Sid
end while
Compute loss frf
end if
Update r
Y 20
Update Parameters 8,, 1,
Reshape: Split Channel Batch and Channel Time
until Loss convergence
Return: Y
RectFloo(optional): (Z8 ) = (Y*, r).
repeat
paths by reducing them through a vector field, and computes the loss, and finally chooses whether to rectify or not. In the inference phase, the process is similar. The precipita- tion estimator first generates the posterior mean sequence Y and then the time dimension of Y is converted to batch dimension. This input is fed into the rectified flow model, where predicted values with perceptual constraints are obtained via vector field iteration. The final output is then produced by separating the time steps from the batch dimension.
Reshape: Merge Channel Batch and Channel Time
Sampling flow step t ~ [4(0 1]
Forward Process: 2 = t(Zz) + (1 - +)
Backward Process (2;; 02) 4-
Compute loss Crf
until Loss convergence
4.3. End-to-End training
To avoid local optima, we implement end-to-end training framework while strategically blocking gradient propagation from the rectified flow model to the precipitation estimator, thereby preventing the flow model from interfering with the estimator's learning of physical motion dynamics. Since the precipitation estimator consistently converges before the rectified flow model during training, it can continuously provide the rectified flow model with augmented samples from the i-th frame, thereby enhancing the robustness of the rectified flow model.
5. Experiments
To validate the effectiveness of the proposed model, we conduct experiments on two real precipitation datasets and perform an ablation study to analyze the contributions of the individual components within the model. The experi- mental results lead the following key conclusions: The proposed model effectively improves both prediction accu- racy and perceptual quality; 2) The distance-bases frame sampling strategy enhances model performance; 3) The LPIPS loss contributes to improved visual quality; and 4) End-to-end training proves to be effective.
The training and inference processes of the model are out- lined in Algorithm and Algorithm In the training phase, the precipitation estimator first generates the posteriori se- quence Y and computes the MSE loss, then stops the gradi- ent of 4 and inputs Y as the initial set to the rectified flow model, which randomly selects posteriori frame y in each round according to the frame sampling strategy, and then performs scheduling as scheduled, computes the forward paths between the posteriori frame 3" and the real frame ac- cording to the acceptance step f, and computes the forward
Table
Statistics of the datasets used in the experiments.
Datasel
Size
Sey Len
Spatial Resolution
Train
Valid
Test
In
Out
H W
SEVIR
13020
1000
2000
13
36
128 x 128
MeteoNet
8640
500
1500
13
36
128 x 128
5

---

Perceptually Constrained Precipitation Nowcasting Model
Table Quantitative evaluation of state-of-the-art spatiotemporal prediction algorithms Sevir and MetcoNet benchmarks.
SEVIR
MeteoNet
Method
CSI
HSS
SSIM
MSE
LPIPS
CSI
HSS
SSIM
MSE
LPIPS
MAU
0.241
0.312
0.705
0.0093
0.348
0.197
0.281
0.819
0.0029
0.250
ConvLSTM
0.240
0.306
0.711
0.0084
0.304
0.192
0.272
0.811
0.0031
0.256
SimVP
0.241
0.306
0.717
0.0083
0.334
0.165
0.239
0.774
0.0037
0.294
Earthformer
0.214
0.277
0.685
0.0100
0.374
0.158
0.224
0.765
0.0035
0.303
Earthfarseer
0.209
0.262
0.675
0.0077
0.385
0.161
0.232
0.789
0.0030
0.306
STRPM
0.213
0.281
0.621
0.0130
0.322
0.154
0.223
0.701
0.0064
0.322
CasCast
0.238
0.301
0.709
0.0120
0.285
0.183
0.274
0.810
0.0062
0.252
DiffCast
0.244
0.318
0.692
0.0100
0.286
0.199
0.287
0.816
0.0054
0.241
PerepCast 0.267
0.360
0.722
0.0092
0.268
0.209
0.305
0.820
0.0049
0.237
Table Quantitative evaluation sate-of-the-ass art spatiotemporal prediction algorithms on Sevir and MeteoNet benchmarks with different thresholds for CSI.
SEVIR
MeteoNet
Method
CS174
CSI133
CSI160
CS1181
CS1219
CSI16
CSI24
CSI32
CS136
CSI40
MAU
0.484
0.206
0.096
0.064
0.020
0.348
0.236
0.092
0.039
0.023
ConvLSTM
0.497
0.203
0.085
0.052
0.013
0.349
0.218
0.079
0.031
0.021
SimVP
0.490
0.177
0.090
0.064
0.026
0.307
0.181
0.053
0.022
0.011
Earthformer
0.453
0.165
0.066
0.040
0.003
0.308
0.173
0.031
0.010
0.002
Earthfarseer
0.469
0.110
0.048
0.029
0.012
0.300
0.176
0.041
0.018
0.009
STRPM
0.424
0.164
0.084
0.059
0.025
0.289
0.183
0.084
0.049
0.022
CasCast
0.440
0.193
0.105
0.067
0.023
0.315
0.228
0.108
0.043
0.020
DiffCast
0.449
0.203
0.113
0.084
0.032
0.323
0.242
0.127
0.053
0.024
PeropCast
0.496
0.251
0.134
0.099
0.037
0.354
0.276
0.132
0.068
0.027
5.1. Experimental Setting
MSE. CSI quantifies the proportion of correct predictions, reflecting the model's event identification ability. HSS com- pares model's predictions to random guessing, measur- ing its forecasting skill. MSE calculates the mean squared error between the predicted and the real value. Their formu- las are follows:
The SEVIR dataset (Veillette et al., 2020) includes satellite images, NEXRAD VIL radar echograms, and lightning data from 20,393 weather events. Each event spans 384 km 384 km, with 49 images taken every 5 minutes over 4-hour sequence. The first 13 images are used to predict the subsequent 36.
Datasets. SEVIR:
TP
CSI TP+FN+FP
(15)
The MeteoNet dataset (Larvor et al., 2020) con- sists of radar and satellite images, ground-based observa- tions, and meteorological data. Each event covers an area of 550 km x 550 km, with 49 images captured every minutes in 4-hour sequence. As with SEVIR, the first 13 images used to predict the next 36.
MeteoNet:
where TP is the number of true positives, FP is the number of false positives, and FN is the number of false negatives.
TP - FN FP
HSS
Both datasets are divided into training, validation, and test sets. The statistical data for all datasets used in the experi- ments are summarized Table 1. The images are rescaled to the range [0, 1] and normalized.
(16)
where TN is the number of true negatives.
MSE 1 Σ (8) 0z
Evaluation The accuracy metrics include CSI, HSS and
(17)
i=
6

---

Perceptually Constrained Precipitation Nowcasting Model
Simne Aatifaane
Heut
204in
428in
E38in
238in
1288
1508
Perssdast
STRRM
5TM
0.30
Sendy Ca fefeer
Creal 3TM
617
Perpcart
Stal?
0.0
180
Tinder
Figure 3. Performance changes of CSI, HSS, SSIM and LPIPS at different moments time on SEVIR.
Figure 4, A visual comparison example on a precipitation event from SEVIR. More cases are in the Appendix D.
where yi is the true value and i is the predicted value.
Image perception metrics include SSIM and LPIPS. SSIM evaluates perceptual similarity based on brightness, contrast, and structure, and is related to the overall structure of the image. LPIPS uses deep learning models to evaluate percep- tual similarity, focuses on human visual perception, and is related to image smoothing.
et al., 2024), STRPM (Chang et al., 2022) and Earthfarseer (Wu et al., 2024).
5.2. Experimental Results
From the results presented in Table 2, Table 3, Table4, and Figure 3, the following key observations can be made: (I) Compared to state-of-the-art models, our framework demon- strates marked enhancements in both prediction accuracy and perceptual quality, achieving average increases of 5% in CSI and 10% in LPIPS metrics, with pooled CSI showing 2-9% improvements across test scenarios. This result under- lines the merits of the proposed approach, in particular its balanced performance in terms of accuracy and perceived quality. As for mse, our method performs optimally in the generative model (STRPM, less than twice mse of any method. (2) The accuracy improvement of our model especially notable in high echo regions, where the prediction outcomes are enhanced by 155% to 22% compared to other models for CSI thresholds greater than 74 and greater than 24. This suggests that our model exhibits superior robust- ness in handling high echo regions. (3) As the prediction time increases, our model continues to maintain the high- est perceptual quality and prediction accuracy compared to other models, demonstrating its stability and advantage in long-term forecasting.
(2palyy + (20 - C2)
SSIM(x,
(18)
(n2 + p3 C) + o C2)
where H4r and My are the mean values of the image patches, oz and of are the variances, and Zzy is the covariance.
LPIPS(,,8) N Σ (1(()) - (())l
(19)
where fi(x) and f(ly) are the features extracted from the i-th layer of pre-trained neural network.
Implementation Details We employ cosine learning rate schedule to train the model, with maximum learning rate of le-4 and a minimum learning rate of 1e-7. The warm-up ratio set to 20%, with the warm-up learning rate set to 3e-4. The model is trained for 100 epochs using the AdamW optimizer. detailed analysis of the model's hyperparame- ters is presented in the next section.
Baselines To validate the effectiveness of the proposed model, we selected several notable works as comparison baselines, including SP (Gao et al., 2022b), Earthformer (Gao et al., 2022a), MAU (Chang et al., 2021). ConvLSTM (Shi al., 2015), DiffCast (Yu et al., 2024), CasCast(Gong
In general, the deterministic models (Sim ConvLSTM) of the compared methods tend to output a sequence of poste- riori mean values, which are susceptible to high echo value decay with increasing prediction time, leading to perfor-

---

Perceptually Constrained Precipitation Nowcasting Model
Table Quantitative evaluation of state-of-the-ar spatiotemporal prediction algorithms on Sevir and MeteoNet benchmarks with different Pooled CSI.
and
Method
SEVIR
Metconet
P1
P4
P16
P1
P4
P16
MAU
0,241
0.268
0.285
0.197
0.231
0.260
COnVLSTM
0.240
0.266
0.292
0.192
0.236
0.264
Si
0.241
0.263
0.283
0.165
0.196
0.214
Earthformer
0.214
0.254
0.265
0.158
0.189
0.207
(a)
(b)
Earthfarseer
0.209
0.252
0.267
0.161
0.193
0.212
STRPM
0.213
0.236
0.271
0.154
0.190
0.203
Figure 5. Sampling probability: (a) probability over time dif-
CasCast
0.238
0.262
0.289
0.183
0.207
0.231
ferent values. (b) probability per frame in the 60th epoch at different k values
DiffCast
0.244
0.270
0.294
0.199
0.235
0.265
PerepCast
0.267
0.287
0.299
0.209
0.240
0.268
box. DiffCast's residual nature leads to focus mainly on local distributions, limiting its ability capture global infor- mation and resulting in poor performance for longer-term forecasts (red box).
Table Analysis of different training strategies SEVIR.
METHOD
CSI
HSS
SSIM
LPIPS
WITH GRADINET
0.258
0.338
0.694
0.292
TWO STAGE
0.249
0.321
0.694
0.289
5.3. Analysis and Discussions
LP.PS(0.0)
0.256
0.328
0.701
0.324
Is end-to-end training preferable to two-stage training? The present model has been trained in an end-to-end manner, aiming to optimize both the precipitation estimator and the rectified flow module simultaneously. In theory, the model could also be trained in a two-stage manner, where the opti- mal precipitation estimator first determined, then frozen, followed by modeling the posteriori mean sequence using a rectified flow model. To validate the performance of these two training methods, experimented with the two-stage training strategy. The results, shown in Table 5, reveal that our end-to-end approach outperforms the two-stage method. During end-to-end training, the gradient transfer between the precipitation estimator and the rectified flow model is stopped, ensuring that the performance of the precipitation estimator remains stable, regardless of the training approach. By treating the posteriori mean sequence conditional, the rectified flow model can learn mapping from the data dis- tribution (i.e., the posteriori frame distribution) to the target frame distribution at each index. As training progresses, the precipitation estimator gradually converges, and the pos- teriori frame each position in each iteration becomes a sample of the data distribution at that position, enhancing the robustness of the rectified flow model. Additionally, experiments with end-to-end training without gradient stop- ping also outperformed the two-stage method, despite not meeting the theoretical requirements.
LPIPS(0.5)
0.267
0.360
0.722
0.268
LPIPS(L.0)
0.265
0.358
0.711
0.272
k=0.00
0.262
0.348
0.703
0.278
K=0.02
0.263
0.343
0.709
0.276
k=0.05
0.267
0.360
0.722
0.268
K=0.1
0.266
0.346
0.705
0.280
mance degradation. On the other hand, while probabilis- tic models account for the modelled data distribution, the STRPM model suffers from an excessive number of degrees of freedom, preventing it from effectively capturing future changes. The residual nature of DiffCast limits its focus to local distributions, while its ability to capture global infor- mation is insufficient.
In Figure 4, we present and compare the results of different methods for the precipitation forecasting task. The cases represent three typical precipitation processes: precipitation attenuation and separation (highlighted in the red box), pre- cipitation dissipation (in the black box), and precipitation formation (between the red and black boxes). Overall, Con- vLSTM suffers from cumulative errors, leading to a rapid decay in intensity (red box). SP captures the attenua- tion and dissipation processes well but struggles to model precipitation formation (between the red and black boxes). Earthformer effectively models the formation of the inter- mediate rainbands as transfer of precipitation within the black box. but its prediction is too widespread in the red box. STRPM and DiffCast both preserve good precipitation details, but STRPM exhibits overly broad precipitation ex- tent and a rapid intensity decay (120-180 min) in the red
Is the loss of Lpips necessary? We used Lpips loss to constrain the perceptual ability of the flow model at each time step during the transport process. We conducted ex- periments with different weights for the LPIPS loss, setting them 0, 0.5, and 1. The experimental results are shown
8

---

Perceptually Constrained Precipitation Nowcasting Model
in Table 5, where the perceptual quality of the model de- creases by 22% and the prediction performance drops by 5% when the LPIPS loss weight is set to O. When the LPIPS loss weights are set to 0.5 and 1, the metrics remain essen- tially unchanged. Additional experiments with varying loss weights can be found in Appendix C.
Acknowledgements
This work was supported in part by NSFC under Grants 62376072, 62272130, and in part by Shenzhen Science and Technology Program No. XCXF22002409055033006000
Analysis of different hyperparameters of frame sampling strategies. We employ the weighted sampling strategy in attempt to focus the rectified flow model longer predicted frames. This strategy involves two hyper- parameters: the weight growth factor k and the temperature parameter t. The temperature parameter is empirically set with a maximum value of fmx = 50 and a minimum value of <min = 0.5. We then investigate the impact of the weight growth factor on training performance. Notably, k is typically sampled uniformly as it approaches O; the greater the deviation of & from O, the higher the probability that distant samples will be selected. To evaluate this, we conducted experiments with sampling strategies for k = O, = 0.02, i = 0.05, and & = 0.1, respectively. The prob- ability density curves for these values of & are shown in Figure 5. As illustrated in Table 5, appropriate distance weighting improves the prediction accuracy and perceptual quality of long-distance frames. However, excessive empha- sis on distant frames reduces attention to other frames with distributional differences (e.g., intermediate frames), lead- ing to a degradation in performance. Further experiments with hyperparameters are provided in Appendix C.
Impact Statement
The primary focus of our work is technological advance- ments and improving machine learning algorithms. As such, our work does not directly involve ethical considerations or has immediate societal consequences.
References
Bai, C., Sun, F., Zhang, J., Song, Y., and Chen, S. Rain- former: Features extraction balanced network for radar- based precipitation nowcasting. IEEE Geoscience and Remote Sensing Letters, 19:1-5, 2022.
Blau, Y. and Michaeli, T. The perception distortion trade- off. In Proceedings of the IEEE conference on compuler vision and pattern recognition, pp. 2228-,237,2 2018.
prrcetion-ddistottinn
Cai, Q., Li, J., Li, H., Yang, Y.-H., Wu, F., and Zhang, D. Tdpn: Texture and ppeeerriresr network for single image super-resolution. IEEE Transactions on Image Processing, 31:2375--288, 2022. doi: 10. 10.1109/TIP:022.. 3154614.
6. Conclusion
Chang, Z., Zhang, X., Wang, S., Ma, S., Ye, Y., Xinguang, X., and Gao, W. Mau: A motion-aware unit for video prediction and beyond. Advances in Neural Information Processing Systems, 34:26950-2666 26962, 2021.
This paper presents a precipitation nowcasting method based on perceptual constraints, addressing the challenge that tra- ditional approaches struggle to effectively capture future uncertainty in long-term sequences. The method transforms posteriori mean sequences into real distribution using rectified flow and introduces frame sampling strategy to enhance focus frames further into the future. Experimen- tal results demonstrate that the proposed model outperforms existing techniques on publicly available radar datasets, val- idating its effectiveness and reliability.
Chang, 7.,, Zhang, X., Wang, S., Ma, S., and Gao, W. Strpm: A spatiotemporal residual predictive model for high-resolution video prediction. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 13946-13955, 2022.
Freirich, D., Michaeli, T., and Meir, R. A theory of the pestotionnperrcettion tradeoff in wasserstein space. Ad- vances Neural Information Processing Systems, 34: 25661-25672, 2021.
Limitations and Future work
PerepCast achieves competitive performance on different datasets, but it may not be able to predict sudden convec- tive storms, that is, storms that suddenly form precipitation without storm signals in the initial stage. Improving such predictions requires incorporating atmospheric variables such as temperature, humidity, and wind direction during the precipitation formation process. Our future work aims explore how integrate atmospheric variables build unified prediction precipitation model.
Gao, Z., Shi, X., Wang, H., Zhu, Y., Wang, Y. B., Li, M., and Yeung, D.-Y. Earthformer: Exploring space-time trans- formers for earth system forecasting. Advances in Neu- ral Information Processing Systems, 35:25390-25403, 2022a.
Gao, Z., Tan, C., Wu, L., and Li, S. Z. Simvp: Simpler yet better video prediction. In Proceedings of the IEEF/CVF conference on computer vision and pattern recognition, pp. 3170-3180, 2022b.
9

---

Perceptually Constrained Precipitation Nowcasting Model
Gao, Z., Shi, X., Han, B., Wang, H., Jin, X., Maddix, D., Zhu, Y., Li, M., and Wang, Y. B. Prediff: Precipitation nowcasting with latent diffusion models. Advances in Neural Information Processing Systems, 36, 2024.
Shi, X., Chen, Z., Wang. H., Yeung, D.-Y., Wong, W.-K., and Woo, W.-c. Convolutional Istm network: A machine learning approach for precipitation nowcasting. Advances in neural information processing systems, 28, 2015.
Gong, J., Bai, L., Ye, P., Xu, W., Liu, N., Dai, J., Yang. X., and Ouyang, W. Cascast: Skillful regguresolution precipitation nowcasting via cascaded modelling. In In- ternational Conference on Machine Learning, pp. 15809- 15822. PMLR, 2024.
Tan, C., Li, S., Gao, Z., Guan, W. Wang. Z., Liu, Z.. Wu, L.,
and Li, S. Z. Openstl: A comprehensive benchmark of spatio-temporal predictive learning. Advances in Neural Information Processing Systems, 01669989969811,,, 2023.
Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, E., and Polosukhin, I. At- tention is all you need. Advances in neural information processing systems, 30, 2017.
Guen, V. L. and Thome, N. Disentangling physical dy- namics from unknown factors for unsupervised video prediction. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 11444- 11484, 2020.
Veillette, M., Samsi, S., and Mattioli, C. Sevir: A storm event imagery dataset for deep learning applications in radar and satellite meteorology. Advances in Neural In- formation Processing Systems, 53:22009-22011 2020.
He, S., Jian, Z., Liu, S., Liu, J., Fang, Y., and Hu, J. Pesgan: A perceptual constrained generative model for railway defect sample expansion from single image. IEEE Transactions on Intelligent Transportation Systems, 25 (8):8796-8806, 2024, doi: 10.1
Wu, H., Yao, Z., Wang. J., and Long, M. Motionrnn: flex- ible model for video prediction with spacetime-varying motions. In Proceedings of the IEEE/CVF conference computer vision and pattern recognition, pp. 15435- 15444, 2021.
Jaiswal, A., Zhang, X., Chan, S. H., and Wang, Z. Physics- driven turbulence image restoration with stochastic refine- ment. In Proceedings of the IEEE/CVF International Con- ference on Computer Vision (ICCV), pp. 12170 12170-12181, October 2023.
Wu, H., Liang, Y., Xiong. W., Zhou, Z., Huang, W., Wang, S., and Wang, K. Earthfarsser: Versatile spatio-temporal dynamical systems modeling in one model. In Proceed- ings of the AAAI Conference on Artificial Intelligence, volume 38, pp. 15906-15914, 2024.
Langley, P. Crafting papers on machine learning. In Langley, P. (ed.), Proceedings of the 17th International Conference on Machine Learning 2000), pp. 1207-1216, Stan- ford, CA, 2000. Morgan Kaufmann.
Yu, D., Li, X., Ye, Y., Zhang, B., Luo, C., Dai, K., Wang, R., and Chen, X. Diffcast: A unified framework via residual diffusion for precipitation nowcasting. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 27758-27761 2024.
Larvor, G., Berthomier, L.., Chabot, V., Le Pape, B., Pradel, B., and Perez, L. Meteonet, an open reference weather dataset by meteo france. France: MeteoFrance, 2020.
Liu, X., Gong, C., and Liu, Q. Flow straight and fast: Learning to generate and transfer data with rectified flow. arXiv preprint arXiv:2209.03003 2022.
Zhang, Y., Long, M., Chen, K., Xing, L., Jin, R., Jordan, M. I., and Wang, J. Skilful nowcasting of extreme pre- cipitation with nowcastnet. Nature, 6196797))-5665533 2023.
Luo, C., Li, X., Ye, Y., Feng. S., and Ng. M. K. Experimen- tal study on generative adversarial network for precipita- tion nowcasting. IEEE Transactions on Geoscience and Remote Sensing, 60: 1-01- 2022.
Ning, S., Lan, M., Li, Y., Chen, C., Chen, Q., Chen, X., Han, X., and Cui, S. Mimo is all you need: A strong multi-in- multi-out baseline for video prediction. In Proceedings of the AAAI conference on artificial intelligence, volume 37, pp. 1975-1983, 2023.
Ravuri, S., Lene, K., Willson, M., Kangin, D., Lam, R., Mirowski, P., Fitzsimons, M., Athanassiadou, M., Kashem, S., Madge, S., et al. Skilful precipitation now- casting using deep generative models of radar. Nature, 597(7878):672-677, 2021.
10

---

Perceptually Constrained Precipitation Nowcasting Model
A. Analysis and Proof
(b) From (Freirich et al., 2021). F - 911 can be decom-
posed as E[IlY 1-1 + F11 - 119] Since ppr,8-
According to (Freirich et al., 2021: Blau & Michaeli, 2018), Eq. 2 can be obtained by solving the optimal transport problem.
Pr, it follows that F[1| - 1
FlllY - 13)
it means Elll - 101 = 2E|||Y - =
2MMSE.
Following arguments similar to those used in the proof of Theorem 3.5 in (Liu et al., 2022), it holds that
PU.V E arg
min Per, per,y E
BIIU" V1 (20)
[[/ uRr (2,, t) dt
i.e., given an observation X, first predict the posteriori mean Y = -[ = z]. and then get the optimal transport plan from Py- to py, which get the twice minimal mean square e1ror(MMME).
2112, - 8-1 = E
(23)
27
E
URF VIF((Z,t)) (Zz,t) dt
(24)
Similar to (Freirich et al., 2021), in precipitation forecast- ing,
L. | (z, (2,,1)112 di
assume that given the historical precipitation X, the real precipitation Y and the estimated precipita- tion Y are independent of each other, and y is the opti- mal solution for the posteriori mean, and then we can get Y + Y + X + Y. which can satisfy the above process for the two-stage training, but in the end-to-end training, the y can affect y*, and although this may not affect our ability to construct the optimal transfer from Y Y. it does impact the analysis of property b. Therefore, to make the above process satisfy the independence principle, we stop the gradient backpropagation from Y to Y
S E
(25)
[ B[(Y -/-)) dt
E
(26)
< R
BIIYY - 8.119 di
(27)
E
E - 1111 dt
(28)
{ FEILY - -p- 9 |13] de
(29)
For this optimal transport problem, (Liu et al., 2022) pro- posed a flow matching algorithm based on straight lines transportation. The algorithm composed of two processes: the forward process is given by: Z, 121 + (1 (1- Zo and the backward process is: va va(Ze, (Z,,t) t) = 2 - Zo Zr), (Liu et al., 2022) noted that solving the ODE with Uy provides an approximation of the optimal transport map from the source distribution to the target distribution, especially when the process is repeated multiple times (i.e., reflow). To learn dg, the model is trained by minimizing the following loss:
= EiilY - -
(30)
Equation (23) follows from the definition of 2, and Y
Equation (24) follows from the fact that pz, = pz, Equa-
tion (25) follows from Jensen's 's inequality. Equation (26)
follows from the definition of vo (Z,, Equation (27) follows from Jensen's inequality. Equation (28) follows from the linearity of the integral operator. Equation (29)
follows from the law of total expectation, then we get 2[| 1+1[3] S BiIlY - y Since in is the final output under the independence assumptions, we can get the conclusion by using the following equation:
J.' E [IICK Yo) vo (Y,, di,
(21)
where Y1 = Y. = y According to (Freirich et al., 2021) and Eq. 21, we get Eq. 3. From Eq. 3, we can get these properties as follows:
E Y
21 |]] =
Y - y-
+ E 21 --1
2E
Y Y
||]
2MMSE,
(a) P2, = PY
(31)
(b) The MSE of 2, is smaller than the MSE of the solu- tion in Eq. 20.
which means that the MSE of our method is less than 2 MMSE.
Proof. (a) We assume Eq. 22 exists a unique solution. From Theorem 3.3 in (Liu et al., 2022), we have Pž. = pz, for every t E [0. 1]. This implies that p2, = =pz = py, i.e. our method satisfies the constraints of Eq. 2.
B. Datasets
SEVIR: The SEVIR (Storm EVvent ImagRy) dataset, as described by Veillette et al., 2020), includes vis- ible and infrared satellite images, NEXRAD VIL radar echograms, and surface lightning events. This study fo- (22) cuses on VII. radar echograms from 20,393 weather events
d², = vo(22, dt. with 2. = Zy = V..

---

There are there losses in our method, Lpes Lrf and Lipips. in Table 6 and Figure 6, We analysed the weights of the three losses. In our experiments, firstly, because the gradi- ents of the previous and subsequent parts of our model are Perceptually Constrained Precipitation Nowcasting Model
recorded between 2017 and 2020. Each event spans 384 km 384 km and 1 km spatial resolution, with 4- hour sequence consisting of 49 images taken at S-minute intervals.
separated, the weights of Le and Ers do not have much effect on the training, secondly, we set the weight of Le to O and do not use supervised signals for the precipitation estimator, which results in the training not being able to converge, and then, we set the weight of Lipips to O, which can lead to a tessellated artefacts problem, and lastly, the rectified flow part will be not be used, the Although lower lpips are obtained the lack of effective estimation of the data distribution leads to poor precipitation prediction, image quality assessment is a com- plex task, simply using I.PISS does not accurately reflect the structural information of the image, but Ipips is the most important factor in it, who is directly related to whether the image is smooth or not, which the main problem solved by our model. Since diversity metrics such as FID are not applicable this problem, in order to get better image quality assessment suggest combining metrics such as SSIM at the same time, on the sevir dataset, our experience suggests that relatively good visual quality can be obtained with LPIPS below 0.29 and SSIM above 0.7.
For the analysis, 49 consecutive images from each event are used to predict the next 36 VIL images (180 min- utes) based on the first 13 observed images (65 min- utes). The data is split into training (October 2018-Au- gust 2019), validation (September- October 2019), and test (Oetober-November 2019) sets. The images are rescaled to the range of 0-255 and binarized using specific thresh- 1,,,,,,,,,,33,16000 111111, Performance is evaluated us- ing the Critical Success Index (CSI). Heidke Skill Score (HSS), Structural Similarity Index (SSIM), Learned Percep- tual Image Patch Similarity (LPIPS), and Mean square error (MSE), with mean values reported.
and accuracy. In fact, in
The MeteoNet dataset, as described by Lar- vor(Larvor et al., 2020), is multimodal dataset that includes satellite and radar imagery, ground-base -bund observations, and metcorological data. covers a 550 km x 550 km area and km spatial resolution in the northwest of France, with each 4-hour sequence consisting of 49 images taken every 5 minutes.
MeteoNet:
Scale factor K
Table Results of different K on SEVIR.
For the analysis, 49 consecutive images from each event are used to predict the next 36 VIL images (180 minutes) based on the first 13 observed images (65 minutes). The data is di- vided into training (October 2016-August 2019), validation (September -ettobr 2019), and test CettembbrrOcttbbee 2019) sets. The images are rescaled to the range of 0-70 and binarized using specific thresholds[8, threshoodd(8.11..4.3...... Performance is evaluated using the CSI, HSS, SSIM, LPIPS, and MSE, with mean values reported.
K
CSI
HSS
SSIM
LPIPS
0.00
0.262
0.348
0.703
0.278
0.02
0.263
0.343
0.709
0.276
0.05
0.267
0.360
0.722
0.268
0.07
0.266
0.352
0.716
0.265
0.1
0.266
0.346
0.705
0.280
0.2
0.250
0.327
0.682
0.292
C. More Experiment Analysis
In our experiments k represents the level of focus dif- ferently located frames, and we analysed the magnitude of different. Table 7, Figure 7 and Figure 9 shows that overfo- cusing on distant frames leads to performance degradation.
Table 6. Results of different weights of the losses on SEVIR.
(Cuer Lyts Lipsps)
CSI
HSS
SSIM
LPIPS
MSE
(0, 0.5)
0.044
0.312
0.311
0.369
0.0217
(I, 0, 0.5)
0.240
0.307
0.663
0.233
0.0085
Table 8. Results of different scheduler SEVIR.
(1, 0.00)
0.256
0.328
0.701
0.324
0.0102
(2, 0,5)
0,266
0,360
0.717
0.269
0.0091
(I, (1,2,0.5) 0.5)
0.264
0.355
0.712
0.270
0.0093
(SCHEDULER)
CSI
HSS
SSIM
LPIPS
MSE
(1, 0.5))
0.267
0.360
0.722
0.268
0.0092
(1, 1.0)
0.265
0.358
0.711
0.272
0.0094
LINEAR EXPONENTIAL
0.262
0.354
0.701
0.269
0.0089
0.267
0.360
0.722
0.268
0.0092
Weighted of Losses
Temperature We experimented with both linear and ex- ponential temperature scheduling, and the results of the experiments are shown in Table 8 and Figure 8
Rectflow We performed 1 1-rectflow according to (Liu et al.,
12

---

Perceptually Constrained Precipitation Nowcasting Model
-60M in
-40M in
-20M
OM in
Input
59
100
133
160
219
20Min
40Min
60M
80M
100Min
120M
140M
160Min
180Min
Target
Ape 1
4r
lpips 0.0
lpe
4f
lpips 0.5
lpe
Upips = 1.0
Ipe 1
lef
lepips 1.0
Figure Cases of different loss weights for precipitation event from SEVIR.
PercpCastik- PercpCast.k-0.0 PercpCast:k=0.1
scheduler
exponential scheduler
2.00
1.50-
0
8 1.25
0.3
0.75
20
40
50
80
100
120
140
100
180
Time(minute)
30
60
Epoch
Figure 7. Performance changes of CSI at different moments in time on SEVIR.
Figure 8, scheduler loss on SEVIR
2022) and the results of the experiment shown in Table 9 and Figure 10.
frame to frame transition and be consistent with the physical motion, and secondly, ease of training, we evalu- ated a variety of models as the precipitation estimator, and found that some of the models are unstable in training and rely heavily on the selection of hyperparameters, overall, we chose the ConvL as the precipitation estimator. in
About the Precipitation Estimator
For the choice of the precipitation estimator, we considered two aspects: firstly, stability, which ensures a continuous
13

---

Perceptually Constrained Precipitation Nowcasting Model
Table Results of 1-rectified flow SEVIR.
HSS
S5IM
1.PIPS
MSE
I-RECTIFIED
0.263
0.356
0.717
0.265
0.0089
WITHOUT RECTIFIED FLOW
0.267
0,360
0.722
0.268
0.0092
Table 10 and Figure 11, We give the forecast performance and case of SimVP a precipitation estimator.
Table 10. Results of the precipitation estimation with Si SEVIR.
PRECIPITATION ESTIMATOR
CSI
HSS
SSIM
LPIPS
MSE
0.246
0.321
0.701
0.297
0.0091
CONVLSTM
0.267
0.360
0.722
0.268
0.0092
D. More Precipitation Case
14

---

Perceptually Constrained Precipitation Nowcasting Model
-60Min in
-40M
-20M
OM in
Input
16
31
59
74
100
133
160
181
219
255
20Min in
40M in
60M in
8OM in
100Min in
120M in
140M
160Min
180M in
Target
k=
k= 0.05
k= 0.1
Figure 9. Cases of different k for precipitation event from SEVIR.
-60Min n
-40M in
-20Min
OM
Input
59
100
133
160
219
20Min
40M
60Min
80M
100M
120Min
140Min
160M in
180Min
Target
loreflow
no-reflow
Figure 10. Cases of I-reflow for precipitation from SEVIR.
15

---

Perceptually Constrained Precipitation Nowcasting Model
in00in
-40M
in
OM in
Input
16
31
59
74
100
133
160
181
219
255
20Min
40Min
60Min in
80Min
100M
120M i
140Min
160M in
180M
Target
Sim/P
ConvLSTM
Figure I.. Cases of precipitation estimation with Sim for a precipitation event from SEVIR.
16

---

Perceptually Constrained Precipitation Nowcasting Model
-60Mi n
-40Min n
-20M in
OM in
Input
2
4
60M
80M
100M
120M
140M
160Min
180Min
Target
PercpCast
STRPM
DifffCast
ConvLSTM
S Simvp
Earthformer
Figure 12, Cases of different methods for precipitation event from MetcoNet.
17

---

Perceptually Constrained Precipitation Nowcasting Model
-60Mi n
-40M in
-20M in
OM in
Input
16
31
59
100
133
160
181
219
255
2
4
80M
100M
120M
140Min
160M
180M
Target
PeropCast
STRPM
Di iffCast
ConvLSTM
imvp
thfrrfrrrer
Figure 13, Cases of different methods for precipitation event from SEVIR.
18

---

Perceptually Constrained Precipitation Nowcasting Model
-60Min in
-40Min n
-00Mi
OM in
Input
12
16
20
48
56
n0Min
40Min
60Min
80M
100Min
120M in
140Mi n
160M in
180M in
Target
PercpCast
STRPM
ffCast
ConvLSTM
S imvp
Earthformer
Figure 14, Cases of different methods for precipitation event from MeteoNet.
19


Reviews:
[{"review_id": "Sf3hARB0hH", "reviewer": "Reviewer_s3wb", "summary": "This paper proposes a precipitation forecast model based on perceptual constraints. Its main contributions include: proposing a new perspective on the precipitation forecast problem, that is, converting it into a posterior mean square error problem under specific constraints; designing a model architecture based on precipitation estimator and correction flow to predict precipitation series while maintaining its authenticity and continuity; and proposing a weighted sampling strategy for long-distance frames to improve the model's prediction ability for long-term series. Experimental results show that the model has better prediction accuracy than the existing optimal model.", "questions": "", "limitations": "", "rating": 4, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for recognizing our ideas and theory. \n\n###  **W1**\nThank you for your question. Due to the introduction of perceptual constraint, our model has the advantage of accurately preserving high-value part in prediction image, which indicates extreme weather storms. As shown in Figures 4 and 13, our model accurately predicts the evolution of the heavy precipitation band (above 160) and gives reliable intensity estimates.In spite of the advantage, our model may also fail to predict sudden convective storms that develop precipitation abruptly where no storm signals appear at the beginning. Improving such predictions requires incorporating atmospheric variables including temperature, humidity, and wind patterns during precipitation formation, which is a key objective for our subsequent research. We will add necessary discussions in our final version.\n###  **W2**\nWe have elaborated on the impact of weight configurations for loss functions and distance sampling (Tables 4-6) in both the experiments and appendices. For other hyperparameters (e.g., learning rates), we identified appropriate values within the range of 1e-3 to 1e-5 and documented them in the main text(Section 5.1 Implementation Details). All hyperparameters in the model have been thoroughly specified, and the experimental code will be made publicly available on a community platform shortly as well.", "claims_and_evidence": "The precipitation forecast model proposed in this paper is based on a new perspective and its effectiveness is demonstrated through experiments.", "methods_and_evaluation_criteria": "The method and evaluation criteria proposed in this paper are meaningful for solving the current precipitation forecast problem. This paper proposes a new perspective to solve the problems of existing methods in predicting long series, and adopts appropriate evaluation indicators to measure the accuracy and perceived quality of the model.", "theoretical_claims": "This paper argues that the introduction of perceptual constraints can improve the performance of the current precipitation forecast model. Specifically, the model transforms the current precipitation forecast problem into a posterior mean square error problem and implements perceptual constraints by constructing a transmission between distributions. The experimental results of the model show that its performance is better than the current state-of-the-art model.", "experimental_designs_or_analyses": "From the paper, it is evident that the authors have considered multiple factors in their experimental design and analysis, conducting detailed comparisons and evaluations. They selected several representative baseline models for comparison and tested the model performance under different parameter settings. Additionally, the authors provided a thorough explanation of the hyperparameter selection process and presented concrete experimental results, including specific data and figures. Therefore, I believe the experimental design and analysis in this paper are sound.", "relation_to_broader_scientific_literature": "The main contribution of this paper is the proposal of a perceptually constrained precipitation prediction model, which improves prediction accuracy and image quality by introducing perceptual constraints. This is different from the current precipitation prediction methods that only focus on minimizing the mean square error (MSE). This model addresses the limitations of existing methods by reconstructing the precipitation prediction problem and using perceptual constraints. The model also uses a sparse sampling strategy based on the attention mechanism and a residual flow structure to enhance the ability to focus on distant frames and capture future changes. These methods have better performance and stability than existing precipitation prediction methods. Therefore, the research results of this paper are meaningful for improving related research in the field of precipitation prediction.", "essential_references_not_discussed": "N/A", "other_strengths_and_weaknesses": "Strengths:\n\nA new perceptually constrained precipitation prediction model is proposed, which can effectively improve the prediction accuracy and image quality.\n\nThe residual flow structure and sparse sampling strategy are used to enhance the ability to focus on distant frames and capture future changes.\n\nExperimental verification is carried out on two public datasets, and better performance and stability are achieved than existing methods.\n\nWeaknesses:\n\nThe prediction effect in some extreme cases has not been analyzed in detail and needs further discussion.\n\nThe experimental results do not provide detailed parameter settings and hyperparameter adjustment processes, making it difficult to reproduce the experimental results.", "other_comments_or_suggestions": "N/A", "questions_for_authors": "Please refer to the weaknesses.", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "WVf2GSggVN", "reviewer": "Reviewer_PwA2", "summary": "This article proposes a new precipitation forecasting model PercpCast, which introduces perceptual constraints into precipitation forecasting tasks. This method first uses ConvLSTM as a precipitation estimator to obtain the posterior mean sequence of future frames. Then, a module based on \"rectified flow\" is used to adjust the distribution of the posterior mean sequence to the distribution of the real target frame. Finally, a distance weighted frame sampling strategy is used to further enhance the attention to future frames. The experimental part was thoroughly validated on two public datasets, SEVIR and MeteoNet, and the results showed that the method exhibited certain advantages in perceptual quality (LPIPS, SSIM) and event detection metrics (CSI, HSS) while maintaining a low mean square error (MSE).", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thanks for the reviewer's valuable suggestions. We will try to address the reviewer's concerns and are eager to engage in a more detailed discussion with the reviewer.\n###  **Theoretical Claims**.\nThank you for pointing out this issue. We perform ​normalization (not binarization) to rescale images to [0, 1]: SEVIR (0–255) is divided by 255, and MeteoNet (0–70) by 70. We will correct the 'binarized' with 'normalized' in the revised manuscript.\n### **Experimental Designs Or Analyses 1**\nWe would like to clarify that unlike diffusion models reconstructing precipitation predictions via conditional integration, our method employs end-to-end learning to directly optimize the posterior mean sequence distribution from the precipitation estimator. To enhance this framework, we introduce two key components: ​temperature-weighted scaling and ​LPIPS perceptual loss. Comprehensive ablation studies demonstrate: (1) LPIPS regularization successfully suppresses checkerboard artifacts in rectified flow, enhancing visual coherence(Tables 4 & 5); (2) Temperature weighting significantly improves long-term frame prediction accuracy(Tables 4 & 6); (3) The rectified flow module achieves exceptional modeling of data distributions, generating meteorologically plausible precipitation patterns that effectively address issues such as high echo attenuation and missing details(Tables 5, Figure 4 & 6).\n\nTo further compare the performance of diffusion models and Rectified Flow in precipitation prediction, we conducted experiments by replacing the Rectified Flow module with a diffusion model. Specifically, due to the instability caused by adapting end-to-end training to diffusion models, we first constructed a pre-trained precipitation estimator. While keeping other configurations unchanged, we then utilized noise and predicted frame as inputs for diffusion modeling during the frame sampling process. Additionally, we employed CasCast as a baseline comparison. CasCast is a non-end-to-end precipitation prediction framework where its first stage originally uses a Vision Transformer (ViT) for precipitation estimation, followed by a second stage that applies diffusion models for distribution refinement. In our implementation, we replaced CasCast's ViT-based precipitation estimator with a ConvLSTM model. The experimental results, presented in the following Table, further validate the effectiveness of the rectified flow model. The results will be supplemented in the revised manuscript.\n\n| Method    |       |       | SEVIR |       |       |       |       | MeteoNet |      |        |\n|-----------|------:|------:|------:|------:|------:|------:|------:|---------:|-----:|-------:|\n|                  | CSI   | HSS   | SSIM  | MSE   | LPIPS | CSI   | HSS   | SSIM     | MSE  | LPIPS  |\n| with Diffusion        | 0.223 | 0.288 | 0.697 | 0.0135| 0.297 | 0.177 | 0.269 | 0.797    | 0.0065| 0.268  |\n| CasCast          | 0.238 | 0.301 | 0.709 | 0.0120| 0.285 | 0.183 | 0.274 | 0.810    | 0.0062| 0.252  |\n| Proposed Model   | 0.267 | 0.360 | 0.722 | 0.0092| 0.268 | 0.209 | 0.305 | 0.820    | 0.0049| 0.237  |\n\n| Method    |       |       | SEVIR |       |       |       |       | MeteoNet |      |        |\n|-----------|------:|------:|------:|------:|------:|------:|------:|---------:|-----:|-------:|\n|                  |CSI74| CSI133| CSI160| CSI181| CSI219| CSI16| CSI24| CSI32| CSI36| CSI40|\n| with Diffusion   |0.437 |0.185| 0.075| 0.054 |0.021|0.299| 0.215| 0.098 |0.035| 0.022|\n| CasCast          |0.440 |0.193| 0.105| 0.067 |0.023|0.315| 0.228| 0.108 |0.043| 0.020|\n| Proposed Model   | 0.496| 0.251| 0.134| 0.099| 0.037| 0.354| 0.276| 0.132| 0.068| 0.027\n\n### **Experimental Designs Or Analyses 2**\n\nThese materials will be supplemented in the revised manuscript.\n\n###  **W**\n\nOur experiments have included SOTA methods DiffCast (CVPR 2024), Earthfarseer (AAAI 2024). Following your suggestion, we also compare our method with CasCast (ICML 2024) as shown in above tables and the results will be updated in the revised manuscript. This ensures necessary comparisons with 2024 conference benchmarks.\n\n[1] Yu, D., Li, X., Ye, Y., Zhang, B., Luo, C., Dai, K., Wang, R.,and Chen, X. Diffcast: A unified framework via residual diffusion for precipitation nowcasting. In The IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2024\n\n[2] Wu, H., Liang, Y., Xiong, W., Zhou, Z., Huang, W., Wang,S., and Wang, K. Earthfarsser: Versatile spatio-temporal dynamical systems modeling in one model. In Proceed-ings of the AAAI Conference on Artificial Intelligence, volume 38, pp. 15906–15914, 2024.\n\n[3] Gong, J., Bai, L., Ye, P., Xu, W., Liu, N., Dai, J., Yang,X., and Ouyang, W. Cascast: Skillful high-resolutionprecipitation nowcasting via cascaded modelling. In International Conference on Machine Learning, pp. 15809–15822. PMLR, 2024 \n### **S** \n\n Thank you for identifying this inconsistency. We will correct the description from \"blue box\" to \"black box\" in the revised manuscript.", "claims_and_evidence": "Yes, the problem being solved is the inaccurate landing point of gan and diffusion in precipitation, that is, the inability to balance CSI and image quality index LIPIS.", "methods_and_evaluation_criteria": "Yes, it does.", "theoretical_claims": "The images are rescaled to the range [0, 1] and binarized. \"Are you sure about binarized? Because the forecast is based on values in the range of 0-255, Binarized doesn't look right.", "experimental_designs_or_analyses": "1. In the innovation of the paper, it is mentioned that the current refined Gan and Diffusion have random sampling, which cannot balance CSI and image quality LIPIS (poor CSI, good LIPIS). Under the perceptual constraints of LIPIS, the second stage of flow matching can move towards a determined path towards the target distribution, reducing the accuracy of high echo landing points. The obvious difference between this method and Diffusion is that in the second stage, flow matching is used instead of Diffusion to refine the model, lacking the ability to ablate Diffusion and your Rectified Flow Model when using CnovLSTM as a precipitation estimator. This makes it difficult to verify the advantages of flow matching in balancing CSI and LPIPS, and it is unclear whether it is temperature weight weighting, lip loss, or flow matching performance that leads to the advantages of balancing CSI and LPIPS.\n2. The comparison method in quantitative indicators uses MAU and Earthfarseer, There is no visual comparison between these two models in the visualization (Figure 3 and Figure 4, as well as the visualization in the supplementary materials)", "relation_to_broader_scientific_literature": "The problem being solved is the inaccurate landing point of gan and diffusion in precipitation, that is, the inability to balance CSI and image quality index LIPIS.", "essential_references_not_discussed": "No.", "other_strengths_and_weaknesses": "Strength:\nThe innovation lies in the use of stream matching in the second stage, which assigns greater weights to frames with longer lead times as the forecast progresses, resulting in better forecast performance after 1 hour compared to other models. The motivation is clear.\n\nWeakness:\nSome comparison methods are relatively old and lack comparison with some updated typical SOTA methods.", "other_comments_or_suggestions": "Fig. 4 with 'preparation study (in the blue box)' appears to be a black box, not a blue box.", "questions_for_authors": "See the above questions.", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "PXrlB5VM2F", "reviewer": "Reviewer_xKo8", "summary": "This work proposes PercpCast, integrating both Precipitation Estimator (Video prediction model) and the Rectified Flow module. Rectified Flow module learns the transmission from the distribution of the posterior mean predicted by Precipitation Estimator to the distribution of ground truth. Further, LPIPS regularization is introduced in addition to the two typical loss terms for the Precipitation Estimator and Rectified Flow Modules. Besides, temperature-distance weighted scheduling is implemented to ensure the model focuses on the later frames. With all these techniques, PercpCast showcases its effectiveness in the outlined evaluation setting across two radar echoes datasets: SEVIR and MeteoNet.", "questions": "", "limitations": "", "rating": 4, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We are grateful for the reviewer's acknowledgment of our work ​and their detailed feedback, which will help us refine our research.\n\n### **Theoretical Claims.**\nEquation (2) can be solved through either Equation (20) or our proposed method, which has different error bounds. Freirich et al. established and showed that the theoretical optimal solution for Equation (20) is 2MMSE. Here we show the error bound of our method is smaller than 2MMSE by proving $\\\\mathbb{E}\\\\left[\\\\left\\\\|\\hat{Z}_1-\\\\hat{Y}^*\\\\right\\\\|^2\\right] \\\\leq \\\\mathbb{E}\\\\left[\\\\left\\\\|Y-\\\\hat{Y}^*\\\\right\\|^2\\\\right]$ in Appendix A. Since $\\\\hat{Z}_1$ is the final output ​under the independence assumptions, we can get the conclusion by using the following equation:\n\n$\\\\begin{aligned} \\\\mathbb{E}\\\\left[\\\\left\\\\|Y-\\\\hat{Z}_1\\\\right\\\\|^2\\\\right] & =\\\\mathbb{E}\\\\left[\\\\left\\\\|Y-\\\\hat{Y}^*\\\\right\\\\|^2\\\\right]+\\\\mathbb{E}\\\\left[\\\\left\\|\\\\hat{Z}_1-\\\\hat{Y}^*\\\\right\\|^2\\\\right] \\\\ & \\\\leq 2 \\\\mathbb{E}\\\\left[\\\\left\\\\|Y-\\\\hat{Y}^*\\\\right\\\\|^2\\\\right]=2MMSE\\\\end{aligned}$\n\n### **W1 & S3.**\n\nThank you for the advice. We incorporate additional experiments with CasCast and reported the pooled CSI prediction results as shown in the following table. The experimental results further validate the effectiveness of our method. The results will be updated in the revised manuscript.\n\n\n| Method         | SEVIR              |                     |                     | Meteonet            |                     |                     |  \n|----------------|-------------------:|-------------------:|-------------------:|-------------------:|-------------------:|-------------------:|  \n|                | Pool1              | Pool4              | Pool16             | Pool1              | Pool4              | Pool16             |  \n| MAU            |              0.241 |              0.268 |              0.285 |              0.197 |              0.231 |              0.260 |  \n| ConvLSTM       |              0.240 |              0.266 |              0.292 |              0.192 |              0.236 |              0.264 |  \n| SimVP          |              0.241 |              0.263 |              0.283 |              0.165 |              0.196 |              0.214 |  \n| Earthformer    |              0.214 |              0.254 |              0.265 |              0.158 |              0.189 |              0.207 |  \n| Earthfarseer   |              0.209 |              0.252 |              0.267 |              0.161 |              0.193 |              0.212 |  \n| STRPM          |              0.213 |              0.236 |              0.271 |              0.154 |              0.190 |              0.203 |  \n| CasCast        |              0.238 |              0.262 |              0.289 |              0.183 |              0.207 |              0.231 |  \n| DiffCast       |              0.244 |              0.270 |              0.294 |              0.199 |              0.235 |              0.265 |  \n| PercpCast      |          ​**0.267** |          ​**0.287** |          ​**0.299** |          ​**0.209** |          ​**0.240** |          ​**0.268** |  \n\n\n\n \n### **W2 & S2.**\n\nThanks for your careful review. We have reviewed Appendix C and corrected the experimental results in Tables 5–6, which will be updated as follows: \n\n\n\n| (Lpe, Lrf, Llpips) | CSI   | HSS   | SSIM  | LPIPS | MSE    |  \n|---------------------|------:|------:|------:|------:|-------:|  \n| (0, 1, 0.5)         | 0.044 | 0.312 | 0.311 | 0.369 | 0.0217 |  \n| (1, 0, 0.5)         | 0.240 | 0.307 | 0.663 | 0.233 | 0.0085 |  \n| (1, 1, 0.0)         | 0.256 | 0.328 | 0.701 | 0.324 | 0.0102 |  \n| (2, 1, 0.5)         | 0.266 | 0.360 | 0.717 | 0.269 | 0.0091 |  \n| (1, 2, 0.5)         | 0.264 | 0.355 | 0.712 | 0.270 | 0.0093 |  \n| (1, 1, 0.5)         | 0.267 | 0.360 | 0.722 | 0.268 | 0.0092 |  \n| (1, 1, 1.0)         | 0.265 | 0.358 | 0.711 | 0.272 | 0.0094 |  \n\n\n\n| $K$  | CSI   | HSS   | SSIM  | LPIPS  |  \n|-------|------:|------:|------:|-------:|  \n| 0.00  | 0.262 | 0.348 | 0.703 | 0.278  |  \n| 0.02  | 0.263 | 0.343 | 0.709 | 0.276  |  \n| 0.05  | 0.267 | 0.360 | 0.722 | 0.268  |  \n| 0.07  | 0.266 | 0.352 | 0.716 | 0.265  |  \n| 0.1   | 0.266 | 0.346 | 0.705 | 0.280  |  \n| 0.2   | 0.250 | 0.327 | 0.682 | 0.292  | \n### **S1.**\n\nThank you for identifying this issue. The precipitation estimator ​reconstructs the input 13 frames and ​predicts 36 future frames, resulting in a total sequence length of 49. To eliminate ambiguity, we will revise Table 1 as follows:  \n\n\n| Dataset   |       |Size |      |    Seq Len|    | Spatial Resolution |  \n|-----------|----------|--------|--------|-------|-----------|---------|  \n|           | Train | Valid | Test | In | Out | H × W      |  \n| SEVIR     | 13020 |  1000 | 2000 | 13 |  36 | 128 × 128 |  \n| MeteoNet  |  8640 |   500 | 1500 | 13 |  36 | 128 × 128 |", "claims_and_evidence": "Most claims are supported by the quantitative result in Table 2 and 3.", "methods_and_evaluation_criteria": "The proposed methods and evaluation criteria makes sense for this problem. To verify the generalisation of PercpCast, its performance is evaluated across two datasets and compared with several SOTA.", "theoretical_claims": "The use of Rectify Flow is well grounded by ample previous works. There is no concern on the attempt here. However, I am quite confused on what do author attempt to show in Appendix A by proving $\\mathbb{E}[ || \\hat{Z}_1 - \\hat{Y}^{\\*} ||^2] \\leq \\mathbb{E}[ || Y - \\hat{Y}^{*} ||^2]$. The said MSE is between the final prediction and **precipitation estimation’s output**, not the ground truth. Do the authors intend to use this to show that equation 20 is small enough so that equation 2 can also be satisfied?", "experimental_designs_or_analyses": "The experimental design is generally fine, except a minor problem:\n\n- LPIPS is chosen to be one of the evaluation metrics. Meanwhile, it is introduced as a regularization term during the training of PercpCast. This might have fairness issues compared with other baselines. The authors can consider using FVD or pooled CSI as an alternative to LPIPS like the PreDiff paper.", "relation_to_broader_scientific_literature": "This work adopts a common strategy of using a precipitation estimator and fine-tuning module for precipitation nowcasting like DiffCast and CasCast. Different from previous works like DiffCast and PreDiff which utilize diffusion models in the second stage, the use of the Rectified Flow Module presents a similar but novel idea to model the difference in distribution. I believe this will very much benefit future studies.", "essential_references_not_discussed": "Most essential references are discussed. It will be better to compare a few more diffusion-based models with PercpCast, such as PreDiff and CasCast (Gong et al., ICML 2024) since they have a closer structure with PercpCast.", "other_strengths_and_weaknesses": "This section summarizes the strengths and weaknesses discussed above.\n\n**Strengths:**\n- Using Rectified Flow to learn the distribution difference between the posterior mean and the ground truth is quite a new idea in this task.\n- It showcases its remarkable performance compared with SOTA in the perspectives of perceptuality and accuracy.\n\n**Weaknesses:**\n- The current evaluation scheme (LPIPS) might be unfair.\n- A few minor but confusing parts in the appendix.\n\nOverall, this paper delivers an interesting solution to precipitation nowcasting. Judging from the good performance results, I am inclined to accept the paper.\n\n---\n### Update after rebuttal\nThe authors mostly addressed my concerns and adopted my suggestions. I will keep the recommendation.", "other_comments_or_suggestions": "- Some information is not very consistent. In Table 1, the output sequence length is shown to be 49, but in the main text it is described to be 36. Does that mean the PercpCast model also reconstructs the input beside forecasting the future?\n- The writing in the appendix is quite messy, especially in Appendix C. Please proofread and fix.\n- A lot of previous works (PreDiff, DiffCast, CasCast, etc.) also report a pooled CSI with different thresholds to evaluate the “skillfulness” of the forecasts. Observing the tables in the papers, realistic and clear forecasts tend to have higher pooled CSI. This will also be a good indicator to replace LPIPS.", "questions_for_authors": "Questions are asked in the above sections.", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "ZHjq5qv4UY", "reviewer": "Reviewer_XBn6", "summary": "This paper proposes a model called PercpCast for precipitation nowcasting, aiming to predict future rainfall patterns more accurately while also improving how realistic those predictions appear.  The authors use a two-stage approach in a single end-to-end framework: first, they generate a \"posteriori mean\" sequence of future precipitation using a ConvLSTM-based estimator, then refine those estimates through a \"rectified flow\" module that better aligns the predicted distributions with the ground truth. To address the challenge of longer-horizon forecasting, the authors introduce a frame-sampling strategy that assigns more weight to frames further in the future. The model incorporates an LPIPS-based loss function to enforce perceptual consistency. Experiments on radar datasets (SEVIR and MeteoNet) show that proposed method outperforms various existing approaches in terms of accuracy metrics and visual quality metrics.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We appreciate the reviewer’s detailed feedback. We will address their concerns and eager to engage in a more detailed discussion with the reviewer.\n\n\n### **Q1.** \n\nThank you for the comment. In precipitation nowcasting, the high uncertainty in short-term evolution means a single historical observation can correspond to multiple future scenarios. ConvLSTM minimizes the mean squared error loss$\\mathcal{L}_{pe}=\\mathbb{E}\\left[\\|Y-\\hat{Y}^*\\|^2\\right]$, causing predictions  $\\hat{Y}^*$  to converge to the conditional expectation $\\mathbb{E}[Y|X]$. This produces a posterior mean sequence – a statistical average of all possible future precipitation outcomes under given input conditions.\n\n### **Q2 & W1** \n\nIn Section 3 and Appendix A, we outlined the gradient-stopping operation. To clarify: Due to diverging learning objectives between the precipitation estimator and the rectified flow model, gradient stopping is applied to prevent the rectified flow model from interfering with the estimator's acquisition of physical motion dynamics. This constraint forces the rectified flow model to actively learn physical consistency (e.g., motion continuity and distribution alignment) directly from input data. Meanwhile, end-to-end training improves model robustness and alleviates suboptimal solutions inherent in two-stage frameworks.\n\nRegarding independence assumptions, our approach aligns with Freirich et al. (2021) in theoretical framework but diverges in the training methodology. Under the condition of stopping gradient propagation from $\\hat{Y}^*$ to $\\hat{Y}$, the generation of $\\hat{Y}$ does not influence $\\hat{Y}^*$. This establishes a Markov process: $\\hat{Y} \\leftarrow \\hat{Y}^* \\leftarrow X \\rightarrow Y$. Consequently, given historical data $X$, $Y$ remains independent of both $\\hat{Y}$ and $\\hat{Y}^*$, while $\\hat{Y}$ depends solely on $\\hat{Y}^*$. This ensures compliance with the predictive independence assumption with theoretical justification, achieving causal decoupling between variables.\n\n### **Q3 & W2.**   \n\nThank you for the comment. Our model employs mixed-precision training (FP16) on a single NVIDIA A100 80GB GPU, with supporting hardware including an Intel(R) Xeon(R) Platinum 8350C CPU @ 2.60GHz. Resource utilization metrics of our method compared to the ConvLSTM on the SEVIR dataset are summarized in the following Table.\n\n\n| Method              | Parameters (M) | GPU Memory (Batch Size=1,GB) | Training Time(hour) |  Inference Time (Batch Size=1,Second) |  \n|---------------------|---------------:|----------------------:|------------------------:|------------------------:|  \n| Proposed Model      |          55.87 |                   4.1 |                     16  |                       2.47  |  \n| ConvLSTM            |          17.81 |                   3.4 |                     11  |                     1.92  |  \n\n\n### **Q4.**  \n\nTo understand the issue, we can see Figure 9 that short-term predictions are relatively easier due to closer alignment between predicted and real frame distributions, while long-term predictions suffer significant distribution drift - a discrepancy amplified by frame sampling strategies that prioritize learning long-term variations, thereby compromising short-term prediction accuracy. As shown in the Figure 9,  the issue can be alleviated by using a moderate small $k$. However, a smaller $k$ may reduce long-term prediction accuracy. Hence, it is a trade-off.\n\n\n### **Q5.**  \n\nWe present the experimental results of different LPIPS loss weight configurations in Table 4, Table 5, and Figure 6. These demonstrate that incorporating the LPIPS loss effectively suppresses checkerboard artifacts in the rectified flow model. Notably, within a reasonable range(0.5~1), the specific weighting configurations do not significantly affect the experimental outcomes. Detailed results for additional weight configurations will be supplemented as the table below.\n\n| lpips weight | CSI   | HSS   | SSIM  | LPIPS | MSE    |  \n|------|------:|------:|------:|------:|-------:|  \n| 0.0         | 0.256 | 0.328 | 0.701 | 0.324 | 0.0102 | \n| 0.2         | 0.260 | 0.342 | 0.714 | 0.287 | 0.0098 |  \n| 0.5         | 0.267 | 0.360 | 0.722 | 0.268 | 0.0092 |  \n| 0.7         | 0.265 | 0.357 | 0.718 | 0.265 | 0.0095 |  \n| 1.0         | 0.265 | 0.358 | 0.711 | 0.272 | 0.0094 | \n---\n\n\n\n  \n### **Q6.**   \nThe experimental results and visualisations (Tables 4 and 6; Figures 5 and 7-9) show our exponential sampling strategy, where K determines the range of variation of the sampling probability. Figure 5 shows the k-probability relationship, while Table 4 evaluates different k settings. Figure 7 confirms larger k improves distant frame prediction accuracy. Figure 8 compares linear and exponential sampling. Experiments show that with more iterations, moderate increases in distant frame sampling probability improve long-term prediction. However, over-amplification undermines learning of other frames, causing performance drops.", "claims_and_evidence": "The paper's main claims about improved accuracy and perceptual quality are largely supported by experiments on two types of datasets. \nHowever, some points could benefit from clearer evidence or explanation. Specifically, it is not entirely clear why the authors stop the gradient transfer between the precipitation estimator and the rectified flow model. Additionally, it is unclear how the authors decided on the specific weight for the LPIPS loss.", "methods_and_evaluation_criteria": "The authors use recognized datasets (SEVIR, MeteoNet) and metrics (CSI, HSS, MSE, SSIM, LPIPS) that directly relate to precipitation forecasting and capture both accuracy and perceptual realism.", "theoretical_claims": "In appendix, the authors includes a theoretical derivation that connects the precipitation nowcasting objective to an optimal transport framework. The derivation appears logically consistent with prior work on rectified flows.", "experimental_designs_or_analyses": "The experiments use known radar datasets and standard precipitation metrics, aligning with typical nowcasting  research. The train/validation/test splits are standard, and the comparisons with multiple baselines are appropriate. However, ablation studies on the LPIPS loss weight or frame sampling would further validate the design choices.", "relation_to_broader_scientific_literature": "The paper extends ongoing research in precipitation nowcasting, which traditionally separates into deterministic approaches that focus on reducing mean-squared error, and probabilistic approaches that aim for more realistic detail. By combining a precipitation estimator with a rectified flow model, this work bridges both views: it maintains the long-term accuracy of deterministic models while incorporating the realistic detail of generative methods. The introduction of a frame-sampling strategy also connects to broader ideas in temporal modeling tasks.", "essential_references_not_discussed": "No additional foundational works appear to be missing.", "other_strengths_and_weaknesses": "Strength: PercpCast integrates a precipitation estimator with a rectified flow model to achieve forecasts that are both accurate and visually realistic. The model's frame-sampling strategy, which emphasizes distant frames, makes it particularly effective for long-horizon predictions. The proposed work is supported by thorough evaluations on established public datasets and comparisons with multiple baselines. \n\nAdditionally, the paper combines theoretical explanations with visual demonstrations, providing a well-rounded view of its strengths.\nWeakness:\n\nOne notable weakness is the lack of clarity regarding why gradient transfer is stopped between the precipitation estimator and the rectified flow model. The paper does not clearly explain how this decision aligns with the assumption that the predicted and true frames are independent, which leaves an important theoretical justification underexplored. \n\nAdditionally, the paper omits details about the hardware specifications used during training, making it challenging to assess the method's computational requirements compared to simpler baselines.", "other_comments_or_suggestions": "Suggestions are addressed in the “Questions for Authors” section", "questions_for_authors": "1. The author often labels the ConvLSTM’s output as a \"posteriori mean sequence\", but does not clearly explain why it represents the average of future rainfall. A brief note on how minimizing MSE leads the model to predict an \"average\" outcome would make this point clearer.\n\n2. In Section 5.3, the authors states \"During end-to-end training, the gradient transfer between the precipitation estimator and the rectified flow model is  stopped....\". Could you clarify why this is done and how it aligns with the assumption that the predicted and true frames are independent, given the historical data?\n\n3. Please include details about the hardware specifications on which the model was trained. It would help readers compare the resource requirements of the proposed method with simpler baselines.\n\n4. In Figure 4, PercpCast appears to overestimate precipitation in certain areas (indicated by the pink colors), compared to the ground-truth maps. Could you clarify what might cause these overestimations.\n\n5. The paper mentions using LPIPS loss to make the predictions look more realistic, but it's not clear why a specific weight was chosen for it in the loss function. Did the authors run any experiments to figure out the best value, or was it selected based on intuition?\n\n6. The frame sampling strategy gives different importance to frames depending on how far they are in time. Could the authors share any experimental results or tests that show how this choice affects the model's accuracy, especially for longer predictions?", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Meta-review: All reviewers support the acceptance of the paper, with two Accepts and two Weak Accepts. The main contribution is an end-to-end precipitation prediction model based on perceptual constraints using a RectifiedFlow framework. I agree with the reviewers that the paper has merits and therefore recommend acceptance. However, the authors are encouraged to incorporate the points discussed in the rebuttal into the final version.
Decision: Accept (poster)