SlowGuess/NeurIPS_2023_OCR huggingface dataset:
The results from Table 14 demonstrate that deterministic spatiotemporal forecasting models, such as UNet [55], ConvLSTM [47], PredRNN [61], PhyDNet [11], E3D-LSTM [60], and Earthformer [8], tend to produce predictions with lower intensity. These models prioritize avoiding high-intensity predictions that have a higher chance of being incorrect due to their limited ability to handle such uncertainty effectively. On the other hand, probabilistic spatiotemporal forecasting baselines, including DGMR [41], VideoGPT [65] and LDM [42], demonstrate a more daring approach by predicting possible high-intensity signals, even if it results in lower CSI scores, as depicted in Table 2. Among these baselines, PreDiff achieves the best performance in BIAS. It consistently achieves BIAS scores closest to 1, irrespective of the chosen threshold. These results demonstrate that PreDiff has effectively learned to unbiasedly capture the distribution of intensity.
Table 14: Quantitative Analysis of BIAS on SEVIR. The BIAS is calculated at different precipitation thresholds and denoted as BIAS-thresh. BIAS-m reports the mean of BIAS-[16, 74, 133, 160, 181, 219]. A BIAS score closer to 1 indicates that the model is less biased to either F.Alarms or Misses. The best BIAS score is in boldface while the second best is underscored.
Model	Metrics
BIAS-m	BIAS-219	BIAS-181	BIAS-160	BIAS-133	BIAS-74	BIAS-16
Persistence	1.0177	1.0391	1.0323	1.0258	1.0099	1.0016	0.9983
UNet [55]	0.6658	0.2503	0.4013	0.5428	0.7665	0.9551	1.0781
ConvLSTM [47]	0.8341	0.5344	0.6811	0.7626	0.9643	0.9957	1.0663
PredRNN [61]	0.6605	0.2565	0.4377	0.4909	0.6806	0.9419	1.1554
PhyDNet [11]	0.6798	0.3970	0.6593	0.7312	1.0543	1.0553	1.2238
E3D-LSTM [60]	0.6925	0.2696	0.4861	0.5686	0.8352	0.9887	1.0070
Earthformer [8]	0.7043	0.2423	0.4605	0.5734	0.8623	0.9733	1.1140
DGMR [41]	0.7302	0.3704	0.5254	0.6495	0.8312	0.9594	1.0456
VideoGPT [65]	0.8594	0.6106	0.7738	0.8629	0.9606	0.9681	0.9805
LDM [42]	1.2951	1.4534	1.3525	1.3827	1.3154	1.1817	1.0847
PreDiff	0.9769	0.9647	0.9268	0.9617	0.9978	1.0047	1.0058
D.2 CSI at Varying Thresholds on SEVIR
We include representative deterministic methods ConvLSTM and Earthformer, and all studied probabilistic methods to compare CSI, CSI, CSI-poo14 and CSI-poo116 at varying thresholds. It is important to note that CSI tends to favor conservative predictions, especially in situations with high levels of uncertainty. To ensure a fair comparison, we calculated the CSI scores by averaging the samples for each model, while scores in other metrics are averaged over the scores of each sample. The results presented in Table 15, 16, 17 demonstrate that our PreDiff achieves competitive CSI scores and outperforms baselines in CSI scores at pooling scale  $4 \times 4$  and  $16 \times 16$ , particularly at higher thresholds.
Table 15: CSI at thresholds [16, 74, 133, 160, 181, 219] on SEVIR.
Model	Metrics
CSI-m ↑	CSI-219 ↑	CSI-181 ↑	CSI-160 ↑	CSI-133 ↑	CSI-74 ↑	CSI-16 ↑
ConvLSTM [47]	0.4185	0.1220	0.2381	0.2905	0.4135	0.6846	0.7510
Earthformer [8]	0.4419	0.1791	0.2848	0.3232	0.4271	0.6860	0.7513
DGMR [41]	0.2675	0.0151	0.0537	0.0970	0.2184	0.5500	0.6710
VideoGPT [65]	0.3653	0.1029	0.1997	0.2352	0.3432	0.6062	0.7045
LDM [42]	0.3580	0.1019	0.1894	0.2340	0.3537	0.5848	0.6841
PreDiff	0.4100	0.1154	0.2357	0.2848	0.4119	0.6740	0.7386
Table 16: CSI-pool4 at thresholds [16, 74, 133, 160, 181, 219] on SEVIR.
Model	Metrics
CSI-pool14-m ↑	CSI-pool14-219 ↑	CSI-pool14-181 ↑	CSI-pool14-160 ↑	CSI-pool14-133 ↑	CSI-pool14-74 ↑	CSI-pool14-16 ↑
ConvLSTM [47]	0.4452	0.1850	0.2864	0.3245	0.4502	0.6694	0.7556
Earthformer [8]	0.4567	0.1484	0.2772	0.3341	0.4911	0.7006	0.7892
DGMR [41]	0.3431	0.0414	0.1194	0.1950	0.3452	0.6302	0.7273
VideoGPT [65]	0.4349	0.1691	0.2825	0.3268	0.4482	0.6529	0.7300
LDM [42]	0.4022	0.1439	0.2420	0.2964	0.4171	0.6139	0.6998
PreDiff	0.4624	0.2065	0.3130	0.3613	0.4807	0.6691	0.7438
Table 17: CSI-pool116 at thresholds [16, 74, 133, 160, 181, 219] on SEVIR.
Model	Metrics
CSI-pool16-m ↑	CSI-pool16-219 ↑	CSI-pool16-181 ↑	CSI-pool16-160 ↑	CSI-pool16-133 ↑	CSI-pool16-74 ↑	CSI-pool16-16 ↑
ConvLSTM [47]	0.5135	0.2651	0.3679	0.4153	0.5408	0.7039	0.7883
Earthformer [8]	0.5005	0.1798	0.3207	0.3918	0.5448	0.7304	0.8353
DGMR [41]	0.4832	0.1218	0.2804	0.3924	0.5364	0.7465	0.8216
VideoGPT [65]	0.5798	0.3101	0.4543	0.5211	0.6285	0.7583	0.8065
LDM [42]	0.5522	0.2896	0.4247	0.4987	0.5895	0.7229	0.7876
PreDiff	0.6244	0.3865	0.5127	0.5757	0.6638	0.7789	0.8289
E More Qualitative Results on  $N$ -body MNIST
Fig. 6 to Fig. 13 show several sets of example predictions on the  $N$ -body MNIST test set. In each figure, visualizations from top to bottom are context sequence  $y$ , target sequence  $x$ , predictions by ConvLSTM [47], Earthformer [8], VideoGPT [65], LDM [42], PreDiff, PreDiff-KA. E.MSE denotes the average error between the total energy (the sum of kinetic energy and potential energy) of the predictions  $E(\hat{x}^j)$  and the total energy of the last step context  $E(y^{L_{\mathrm{in}}})$ .
Figure 6: A set of example predictions on the  $N$ -body MNIST test set. The red dashed line is to help the reader to judge the position of the digit "1" in the last frame.
Figure 7: A set of example predictions on the  $N$ -body MNIST test set. The red dashed line is to help the reader to judge the position of the digit "0" in the last frame.
Figure 8: A set of example predictions on the  $N$ -body MNIST test set. The red dashed line is to help the reader to judge the position of the digit "0" in the last frame.
Figure 9: A set of example predictions on the  $N$ -body MNIST test set. The red dashed line is to help the reader to judge the position of the digit "8" in the last frame.
Figure 10: A set of example predictions on the  $N$ -body MNIST test set. The red dashed line is to help the reader to judge the position of the digit "4" in the last frame.
Figure 11: A set of example predictions on the  $N$ -body MNIST test set. The red dashed line is to help the reader to judge the position of the digit "1" in the last frame.
Figure 12: A set of example predictions on the  $N$ -body MNIST test set. The red dashed line is to help the reader to judge the position of the digit "7" in the last frame.
Figure 13: A set of example predictions on the  $N$ -body MNIST test set. The red dashed line is to help the reader to judge the position of the digit "7" in the last frame.
F More Qualitative Results on SEVIR
Fig. 14 to Fig. 19 show several sets of example predictions on the SEVIR test set. In subfigure (a) of each figure, visualizations from top to bottom are context sequence  $y$ , target sequence  $x$ , predictions by ConvLSTM [47], Earthformer [8], VideoGPT [65], LDM [42], PreDiff, PreDiff-KA. In subfigure (b) of each figure, visualizations from top to bottom are context sequence  $y$ , target sequence  $x$ , predictions by PreDiff-KA with anticipated average future intensity  $\mu_{\tau} + n\sigma_{\tau}$ ,  $n = 4, 2, 0 - 2, -4$ .
Figure 14: A set of example predictions on the SEVIR test set. (a) Comparison of PreDiff with baselines. (b) Predictions by PreDiff-KA under the guidance of anticipated average intensity.
Figure 15: A set of example predictions on the SEVIR test set. (a) Comparison of PreDiff with baselines. (b) Predictions by PreDiff-KA under the guidance of anticipated average intensity.
Figure 16: A set of example predictions on the SEVIR test set. (a) Comparison of PreDiff with baselines. (b) Predictions by PreDiff-KA under the guidance of anticipated average intensity.
Figure 17: A set of example predictions on the SEVIR test set. (a) Comparison of PreDiff with baselines. (b) Predictions by PreDiff-KA under the guidance of anticipated average intensity.
Figure 18: A set of example predictions on the SEVIR test set. (a) Comparison of PreDiff with baselines. (b) Predictions by PreDiff-KA under the guidance of anticipated average intensity.
Figure 19: A set of example predictions on the SEVIR test set. (a) Comparison of PreDiff with baselines. (b) Predictions by PreDiff-KA under the guidance of anticipated average intensity.