DAWP: A framework for global observation forecasting via Data Assimilation and Weather Prediction in satellite observation space
Authors: Junchao Gong
Jingyi Xu
Ben Fei
Fenghua Ling
Wenlong Zhang
Kun Chen
Wanghan Xu
Weidong Yang
Xiaokang Yang
LEI BAI
Venue: NeurIPS 2025 poster ()
Keywords: satellite forecasting
ai for earth science
spatial temporal learning
data assimilation
Primary area: machine_learning_for_sciences

Abstract: Weather prediction is a critical task for human society, where impressive progress has been made by training artificial intelligence weather prediction (AIWP) methods with reanalysis data.
 However, reliance on reanalysis data limits the AIWPs with shortcomings, including data assimilation biases and temporal discrepancies.
 To liberate AIWPs from the reanalysis data, observation forecasting emerges as a transformative paradigm for weather prediction.
 One of the key challenges in observation forecasting is learning spatiotemporal dynamics across disparate measurement systems with irregular high-resolution observation data, which constrains the design and prediction of AIWPs.
 To this end, we propose our DAWP as an innovative framework to enable AIWPs to operate in a complete observation space by initialization with an artificial intelligence data assimilation (AIDA) module.
 Specifically, our AIDA module applies a mask multi-modality autoencoder (MMAE) for assimilating irregular satellite observation tokens encoded by mask ViT-VAEs. 
  For AIWP, we introduce a spatiotemporal decoupling transformer with cross-regional boundary conditioning (CBC), learning the dynamics in observation space, to enable sub-image-based global observation forecasting.
 Comprehensive experiments demonstrate that AIDA initialization significantly improves the roll-out and efficiency of AIWP. 
 Additionally, we show that DAWP holds promising potential to be applied in global precipitation forecasting.

DAWP: A framework for global observation forecasting via Data Assimilation and Weather Prediction in satellite observation space
Junchao Gong Shanghai Jiao Tong University g] chimself@sjtu. edu. cn
Jingyi Xu Fudan University jyxu220m. fudan edu cn
Ben Fei
Fenghua Ling Shanghai Al Laboratory engg org.
The Chinese University of Hong Kong
Shanghai AI Laboratory benfei @cuhk. edu, hk
Wenlong Zhang
Kun Chen
Wanghan Xu
Shanghai Al Laboratory zhangvenlong@ojien jabe org. on
Shanghai AI Laboratory thenkun@pjlate org. cn
Shanghai Al Laboratory xuwanghan@pjles jab org. cn
Weidong Yang Fudan University wdyang@fudan. edu. cn
Xiaokang Yang
LEI BAI
Shanghai Jiao Tong University xkyangos jtu. edu. cn
Shanghai Al Laboratory Mailei@pjlal org. cn
Abstract
Weather prediction is a critical task for human society, where impressive progress has been made by training artificial intelligence weather prediction (AIWP) meth- ods with reanalysis data. However, reliance on reanalysis data limits the AIWPs with shortcomings, including data assimilation biases and temporal discrepancies. To liberate AIWPs from the reanalysis data, observation forecasting emerges as transformative paradigm for weather prediction. One of the key challenges in observation forecasting is learning spatiotemporal dynamics across disparate measurement systems with irregular high-resolution observation data, which strains the design and prediction of AIWPs. To this end, we propose our DAWP as innovative framework to enable AIWPs operate in a complete observation space by initialization with an artificial intelligence data assimilation (AIDA) mod- ule. Specifically, our AIDA module applies mask multi-modality autoencoder (MMAE) for assimilating irregular satellite observation tokens encoded by mask ViT-VAES. For AIWP, we introduce spatiotemporal decoupling transformer with cross-regional boundary conditioning (CBC), learning the dynamics in observation space, to enable satb-image-baane based global observation forecasting. Comprehensive experiments demonstrate that AIDA initialization significantly improves the roll- out and efficiency of AIWP. Additionally, we show that DAWP holds promising potential to be applied in global precipitation forecasting. Code will be available at this github repo
'Equal Contribution
1 Corresppnding Authors: Ben Fei (benfei@cuhk.edd and Lei Bai (bailei @illei@ppiln pilab.org.cn)
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

Introduction
Weather prediction is a critical task that significantly impacts various socioeconomic aspects, including transportation, agriculture, and public safety. Traditional numerical weather prediction (NWP) systems rely on intricate human-designed workflows [ 2). such as numerical assimilation systems and physical solvers, generate global precipitation predictions.
Recently, transformative progress has been made by artificial intelligence weather prediction (AIWP) models. These AIWP models now achieve forecast skill scores comparable to even surpassing those of leading physics-based NWP systems
To learn the atmospheric dynamics, reanalysis
products widely used 77 8 10].
However, reanalysis data, generated by numerical data assimilation, introduce intrinsic limitations in AIWP models built upon them. (I) Data Assimilation Biases: (Re)analysis products are synthesized by numerical data assimilation (DA), where direct observations are blended with physics- based forecast. During DA, information loss of direct observation occurs due to the limited utilization of raw observational data and the preprocessing that resamples observations to regular grids of reanalysis data format with finite resolution [ process modeling and uncertainty parameterizations physics- based forecast systems also introduce biases, which could hinder learning the actual dynamics of the atmosphere [1.) (II) Temporal The temporal lag between direct observation acquisition (nearly real-tin real-time) and analysis data generation (up to six hours) severely degrades the quick response ability of AIWP models [2 [6]. These limitations may be further exacerbated by the discrepancy between real-works observation space and physical forecasting space which is required by NWP systems to implement dynamic equations of atmospheric [7]. As AIWP models do not require physical solvers to predict the evolution of the atmosphere, there is potential for them to directly predict atmosphere states in real-world observation space.
1 I3 [4]. Additionally, the incomplete physical
Discrepancies:
Artificial Intelligence Direct Observation Predic- tion (AI-DOP) is emerging as transformative, data-driven approach with the potential to over- come the limitations in reanalysis- drivevn AIWP methods. The key challenge of AI-DOP is learn- ing the spatiotemporal dynamics not only within single observation source but also across dis- parate measurement systems, given the irreg- ular and high-resolution observation data [17]. The spatiotemporal modeling approaches are restricted learn the relationships between dif- ferent observations with irregular data. [17 applies transformer with mask tokens re- format multiple observations into regular ones. Further. [18] uses graph encoder to flexibly encode different measurements into a uniform latent representation for latent forecasting with naive transformer backbone. In addition, as AI-DOP requires being generalized to any loca- tion grids given observations with variable miss- ing values [7], the dense forecasts and sparse observation inputs result in input-output distri- bution shift in rollout as shown in (b) and (c) of Figure Motivated by these questions, argue that training AIWP models in a uniform observation space where input and output are both regular grid data.
Rowaadyiis
Physical Space
Rolling Out Prediction
Temporsi lag Limited use of observation NWP dependency
ATWP
(a)
Obwevvaiin
O Oservaaion Space
Rolling Out Prediction
Imsular
Restricted spatio-
AIWP
tempural modelling
Distrtbution
Satelitt Olsrrvation
Analysts Olservation
Predicted
Rolling Out
Observation
AIDA
ATWP
Initialization
Forecasting
(c)
Figure 1: A paradigm shift from physical space to observation space. Our DAWP is illustrated in (c).
We propose our DAWP, an AI-DOP system composed of an observation space data assimilation (AIDA) module and an AIWP module, to learn the spatiotemporal dynamics between various satellite observations with irregular and high-resolution characteristics. encoder/decoder is designed with observation masks to regionally encode high-resolution irregular direct earth observations for efficient I/O and computation. Then, we implement a sub-image-bant AIDA module for observation space data assimilation by multi-modal masked autoencoder with en-
begin with, a transformer VAE

---

coded observation tokens. By learning the spatiotemporal correlations between various observations the AIDA initialization, irregular observations are transformed into a uniform completed observa- tion space. In this imputed space, we train our sub-image-basee AIWP module with Cross-regional Boundary Conditioning (CBC), which could forecast observations with global state cache providing atmospheric states of neighbours. Finally, a combination of mapping operators is used to obtain global observation predictions or precipitation variables. We conduct comprehensive experiments to demonstrate the effectiveness of our DAWP framework for global direct observation predictions. We summarize the contributions of this paper as follows:
Innovative framework integrating AIDA and ATWP methods for direct observation pre- dictions: We propose brand-new framework that leverages an observation space AIDA module, transforming irregular observations into uniform observation space, with AIWP modules to achieve skillful direct observation predictions, bypassing the limitations of reanalysis data.
High-resolution global forecasting for observation and precipitation: We introduce mask ViT- VAE and spatiotemporal transformer with eross cross-regional boundary information for encoding high-resolution irregular observations, and implement global forecasting efficiently on sub-images of global observations.
Comprehensive experiments: We organize composite satellite observation dataset with a size of over 35TB, which has spatiotemporal resolution of 12x 1 x524x304. Compre- hensive experiments and reanalyses presented, demonstrating the effectiveness of our DAWP framework and the potential of direct observation predictions.
2
Related work
Weather prediction with deep learning. Recent studies have demonstrated that machine learning systems can produce accurate medium-range forecasts, comparable to physics-based models, for key weather parameters B 4
20 21 [22] FourcastNet was the first to propose using deep neural networks to learn global atmospheric dynamics 233. By scaling up the training stage, Pangu- Weather (3] and GraphCast 4 simultaneously achieved accuracy levels comparable to those obtained by the operational IFS systems at ECMWF. Other works extend the AIWP from aspects including forecasting skill [5] 6). resolution (19), probabilistic modelling (24). and physics informed (25). Although impressive progress has been made, previous AIWP methods still rely reanalysis data, which introduces inherent limitations, including temporal lag, limited observation use, and
6,
5
19
dependency on NWP systems.
Direct observation prediction. Direct observation prediction holds transformative potential for overcoming the dependency reanalysis, enabling the forecasting of weather using direct observations. Although the concept of direct observation prediction has been widely applied in fields such precipitation nowcasting, where radar echoes utilized for short-term forecasting [26] 27] 28
its application in global weather prediction remains limited. In contrast to gridded radar observations, direct observations of the global atmosphere are irregular and non-gridded, making global weather DOP challenging. Transformer-Di DOP proposes using a transformer with mask tokens handle the irregular global Earth observations [17]. Simultaneously, Graph-DOP employs graph neural networks to flexibly encode direct observations [18. EarthNet proposes pretraining the backbone with an observation assimilation task and then finetuning with prediction task (31). Although they successfully apply irregular global observation for prediction, these designs suffer from rollout distribution and ineffective spatiotemporal learning as the input space discrete while the output space is dense. We propose applying AIDA to transform the input space into dense one, thereby solving the misalignment between input and output.
29 30
3 Method
Our DAWP designed for global observation forecasting in a uniform satellite observation space. The key components of DAWP are an observation space data assimilation module and a cross-regional boundary conditioning weather prediction module. Additionally, we introduce the mask VIT to encode observations and produce precipitation variables. Our DAWP is illustrated in Figure 2
3

---

inn-ingge Paichhes
Completed Sub-inage Patches
Subeaquent hours predotioon MSSUA
AMSU-A
Frooddrs
Globol Whole Image Cache
SEED BESS
144 x 146
Quey dureent Sub-tmages Condion
MHS
T+
NHS
BBB NO0
128 Sab inaqes
Update Cochhe for I--lsi
HIRS
14A
MAE
T:l tubriagm
Subsequent 12 hours prediction
HIRS
BBC
Seb-
144
Global State Cache
Rolling 2 Map Combine
ATMS
ATMS
BBC
sxaque
Spatio-temporal Transformer
Thent
Inpoi inlieeeess from n--B44
1.38 Sateihegee
1200 Sobo
AIDA Initialization Observation Space
AIWP Forecating Observation Space
Figure 2: The framework of our DAWP. There are two stages in our DAWP: (1) Initialization and (2) Forecasting.
3.1 Initialization: observation space assimilation by multi-modal masked autoencoder
The missing and sparse observations, attributed the inherent characteristics of orbital motion, present an irregular input observation space, as shown in the left column of Figure Directly taking these satellite observations as inputs not only restricts the network design for spatiotemporal modeling but also leads to distribution shift when implementing rollout forecasting, as the output space is required to regular one. To meet the gap. we propose using observation space assimilation with a Multi-modal Masked Autoencoder
32 33 (MMAF) as the initialization stage for direct observation predictions. The MMAE fills missing areas by leveraging contextual information from different sensors and spatiotemporally nearby observations.
The core of our assimilation module is naive MAE that imputes masked multi-modal satellite tokens following (34 32 31]. Since imputation mainly relies on space-time nearby observations from multiple satellite observations, our MMAE sub-regionally processes data from multiple sources within fixed time window of 12 and sub-image of 144x 144. Observations in the time window are tokenized frame by frame by pretrained sppeciite-seecific mask ViT encoders, where missing patches are ignored. Remaining spatiotemporal tokens from each satellite are concatenated and passed through MAE complete missing information the observation space. For MAE training, we randomly mask given number of tokens from the whole concatenated remaining tokens and reconstruct the left observed but masked tokens. As the number of observed tokens from different time windows can vary, we flexibly pad [EOS] tokens to maintain uniform sequence length for efficient attention computation. In the inference stage, masks for MAE are released utilize as many available observations as possible.
3.2 Forecasting: cross-regional boundary conditioning direct observation prediction
After AIDA, we implement an efficient weather prediction module for direct observation prediction through cross-regional boundary conditioning in the imputed observation space, initialized by our assimilation module. For efficiently integrating with the assimilation module pretrained by sub- images, weather prediction module is also applied to sub-images generated by the assimilation module. Since sub-images only contain local atmospheric states, predictions on sub-images require cross-regional information interaction for continuous spatiotemporal modeling. We introduce a global state cache to store observation states for cross-regional boundary conditioning during forecasting.
With the global state cache, our weather prediction module achieves efficient cross-regional boundary conditioning observation forecasting by applying spatiotemporal decoupling attention structure. In the forecasting stage, the assimilated observations simplify the prediction task into standardized spatiotemporal forecasting problem, which widely explored [35] 36 37]. We follow the concept of spatiotemporal decoupling in spatiotemporal forecasting [35] 30). Specifically, our weather prediction

---

Table 1: Dataset overview.
Modality/cnnoor
Satellite
Channels
Lovel
Period
Advanced Microwave Sounding Unit (AMSU-N) 40
NOAA18 & 19
Microwave radiance 15 bands
1B
2007 2007-2023 2023
Advanced Technology Microwave Sounder (ATMS)
Brightness lemperature
2012-2023
High Resolution Infrared Radiation Sounder (HIRS)
40
NOAAI8 & 19
Infrared rodiance 20 bands
1B
2007-2023
Microwave Humidity Sounder (MHS) (8)
NOAA18 19
Microwave radiance hands
1B
2007-2023
ATMS MUMS-Peeciiitation 3 14
NPP & NOAA20
Precipitation product channels
2A
2012-2023
module is composed of N Temporal-Spatial (TS) attention blocks 37 36 Attention blocks have the advantages of simplicity and scalability, while the TS spatiotemporal decoupling reduces the sequence length of spatiotemporal tokens for efficient computation. Utilizing the efficiency of 'TS decoupling, simply pad tokens from neighbouring areas to the tokens of the prediction region as SD3 [38] and Flux [39] do. In this way, cross-regional boundary information of the border area is passed the center forecasting region. To forecast multistep global observations, we maintain a global state cache during the inference phase, ensuring consistent updates for subsequent steps. Specifically, when forecasting observations of center sub-images, current adjacent sub-images are queried as conditions from the state cache according to the relative coordinates. After one step of global prediction has been completed, the global state cache is updated with the subsequent 12 hours prediction to support rollout forecasting, More details can be found in the Appendix
3.3 Encoding and precipitation mapping via mask VIT-VAE
In our DAWP, we introduce a mask VAT both for encoding and mapping multiple-channel satellite observations with missing values. The mask ViT-VAE consists of a vision transformer (ViT) encoder/decoder with masks enabling the model to ignore patches without sufficient observa- tions. The VIT encoder/decoder provides better compression capability, detailed in Appendix [D] compared to VAE for satellite observations, which typically have more channels than natural images. Moreover, it explicitly maintains spatial consistency between tokens and pixels by position embeddings and mitigates the influence of missing values through mask attention. With our mask Vit-VAE, encoding/decoddng is pretrained reconstruction task, while the precipitation mapping is trained with ATMS inputs and ATMS paee--ecpppitaiionn outputs.
4 Experiments
In the experiment part, comprehensive analysis of our DAWP is presented. First, we introduce the composition of our data and training details. Based on the observation data, a comparison of our DAWP with other AI-DOP methods implemented. Further, we evaluate the precipitation forecasting skill of these AI-DOP methods by applying a precipitation mapping network. In addition to evaluating DAWP's capabilities of forecasting capabilities, we also conducted ablation studies to validate the effectiveness of our modular designs. Finally, the importance of each satellite for DOP is tested by ablating the input modalities of the assimilation module.
4.1 Experimental Setups
Data. The top four datasets listed in Table are used for training our DAWP, while ATMS- precipitation is used to train the precipitation mapping with ATMS. We generate hourly, 0. resolution composites by interpolating and reprojecting the raw data detailed in Appendix B
Additional information for dataset split and introduction is presented in Appendix
Training details. The training details are presented in Appendix G Training our mask VIT-VAE. MMAE AIDA, and decoupling AIWP takes about day, days, and days, respectively. Besides, our DAWP is compared with the persistence model, using our AIDA module for completion of
missing values, as mentioned in [35]. We replicate the EarthNet (3I and Transformer-DOO (6
models on our composite dataset, their codes and data are closed source.
5

---

Table 2: MAE error of forecasting during 3 lead time periods (0-12h, 12-24h, and 24-36h) for different channels of the satellite data. We use the unit of le-5 for AMSU-A, le-4 for MHS, and le-0 for both ATMS and HIRS.
AMSU-A
ATMS
HIRS
MHS
Methods
Lead time
chO
ch1
ch0
ch1
ch9
ch10
ch0
ch1
Persistence
35
5.86
9.15
14.37
11.69
12.43
2.21
7.01
14.74
EarthNct 31
2.93
4.89
6.96
6.14
9.15
1.39
3.96
9.65
0-12h
Transformer- DOP
17
2.67
4.48
6.40
5.61
9.22
1.42
3.91
9.48
Ours
1.92
3.39
3.36
3.27
7.70
1.12
3.07
7.91
Persistence 35
4.35
6.94
10.40
8,86
13.58
2.30
6.07
15.09
EarthNet
31
4.12
6.65
11.25
9.00
11.14
1.98
5,46
13.11
12-24h
Transformer- DOP 1F
3,84
6.14
10.04
8.04
11.08
1.95
5.19
12.65
Ours
3.11
5.12
7.35
6.35
9.57
1.54
4.51
10.54
Persistence 3
6.39
9.84
15.37
12.52
14.61
2.61
8.00
17.86
EarthNet 31
5.17
12.52
10.08
12.37
2.36
6.41
15.00
24-36h
Transformer DOP 17
4.91
7.54
11.35
9.07
12.39
2.27
6.22
14.70
Ours
3.66
5.80
7.84
6.81
10.71
1.79
5.15
12.22
ATMS channelD
ATMS channell
AMSU- channelo
AMSU-A channell
1.6-
1.25
7.5
1.0-
MAE (e+01)
1.2
MAE (e.000
1.00
MAE (c (c.05)
MAL (e-04)
0.8-
0.75
4.5
0.6-
0.50
3.0
0
20
40
60
0.25
20
40
60
1.5
20
60
20
d0
60
MHS channelo
MHS channell
HIRS channel9
HIRS channel10
9.0
3.0-
1.50
MME (e-00)
MAE (e-03)
MAE (e+01)
1.25
MAt (e400)
2.4
6.0
1.2
2.00
1.0-
3.0
0.8
0.75
1.2
40
20
20
60
20
60
Lead time (hours)
Lead time (hours) EarthNet
Lead time (hours)
Lead time thours)
DAWP(ours)
Persistence
Transformer-DOP
Figure 3: Curves of MAE for the prediction of different modalities. The max leadtime is 72h with 1h temporal resolution.
4.2 Direct observation prediction
We compare our DAWP with other AI-DOP methods to evaluate the ability to predict direct ob- servations. The metric for evaluation is the mean absolute error (MAE) between the predictions and observed values. The baselines are also trained with sub-images our DAWP. Among them, persistence a naive method that uses the last observation as the prediction of the next one.
The time-averaged MAES within fixed time windows of O-12h, 12-24h, and 24-36h are presented in Table We select two channels with meaningful patterns of each modality observation to calculate the MAE. The results show that our DAWP significantly outperforms other methods. Specifically, our DAPW's MAE on AMSU-A among 24-36h is 3. 3.66 and 5.80, which not only outperforms those of EarthNet and Transformer-De DOP, but also surpasses EarthNet's (3.84 and 6. 14) and Transformer-
DOP's (4.12 and 6.65) MAES on AMSU-A among 12-24h. is the same when comparing our DAWP with baselines on other modalities. These results indicate our DAWP has 12-hour lead time advantage direct observation prediction. To provide a more detailed analysis of the observation prediction, we present the MAE figure in time range of 0-72h in Figure It can be observed that the 1h temporal resolution of the figure depicts the periodicity of the prediction errors. This periodicity attributed to the strip-scanning characteristics of polar-orbiting satellites, which generate periodic

---

780 ATMS choglead time:12hour
ATMS chê@ end time:24hour
ATMS choglead tin 36horr
ATMS cho time:48hour
Dowth
DAYP
305
180
200
200
240
20c
300
Figure 4: A visualization of rollout predictions for global satellite observation forecasting.
Table 3: Forecasting skills in 12 hours on precipitaiion-rllteed variables Total Column Water Vapor
(TCWV) and Surface Precipitation (SP). CSI and FAR scores are calculated on different thresholds.
Method
CSE-10
TCWV
SP (mm/h)
FAR-2.0
Persistence
0.853
0.702
0.635
0.121
0.266
0.332
0.110
0.073
0.031
0.841
0.861
0.655
EartaNet
0.909
0.822
0.786
0.047
0.130
0.172
0.127
0.050
0.008
0.666
0.692
0.180
Transformer- DOP
17
0.905
0.822
0.789
0.053
0.130
0.169
0.136
0.057
0.010
0.655
0.685
0.214
0.088
0.577
observations. Although there is periodicity in the MAE, our DAWP consistently outperforms other methods, establishing lead time advantage from the beginning. Specifically, in subfigure (a), at the beginning of the prediction, our DAWP's MAE about 3.5 while the MAE of EarthNet is about 6.8. which is almost times larger than ours. In addition, EarthNet is surpassed by the naive persistent baseline at the lead time of 16h, but our DAWP doesn't t meet the persistent baseline until about 64h.
In Figure 4 we visualize the prediction results of these methods at different lead times. At the lead time of 12 hours (the first prediction step), our DAWP exhibits slightly better prediction than others. When the lead time increases, a significant distortion appears in the predictions of EarthNet and Transformer- -OOP while our DAWP maintains relatively stable structure. It has the same trend as the MAE curve in Figure indicating that our DAWP is more robust in rollout predictions.
The results of direct observation prediction indicate that DAWP outperforms other methods in the 0-72h time range and selected channels, demonstrating the effectiveness of our method both initial prediction and rollout prediction. More evaluations other channels shown in the Appendix
4.3 Global precipitation forecasts
In this section, evaluate the potential of AI-DOP methods for global precipitation forecasting, precipitation mapping network is trained to transform observation predictions into precipitation products. It is worth noting that in this way, our DAWP does not require satellite precipitation products as input, making the forecasting a quick response to observations.
The precipitation skill is evaluated by the Critical Success Index (CSI) 45 and False Alarm Ratio (FAR) [46] metrics on observed points time range of 12h. CSI measures the ability of the model to correctly identify precipitation events, while FAR assesses the reliability of the model's predictions. The combined application CSI and FAR enables comprehensive precipitation forecasting skill assessment. We present detailed definition of CSI and FAR in Appendix H

---

Table 3 presents presents the quantitative results of the precipitation forecasting skill of AI-DOP methods on total column water vapor index (TCWV) and surface precipitation (SP). The thresholds for TCWV and SP are set to [10mm, 20mm, 30mm] and [0.5mm/h, 1.0mm/h, 2.0mm/h]. respectively. On variable TCWV, our DAWP achieves a slight advantage in CSI over other methods. Specifically, our DAWP's achieves 0.807 on CSI-30, while Earthnet and transformer-dop achieve 0.786 and O. 7,78, respectively. This advantage is maintained across all thresholds. In terms of FAR, our DAWP is significantly lower than other methods. It indicates that our DAWP predicts more accurate TCWV with lower false alarm rates than baseline methods, demonstrating the reliability of our DAWP. We also analyze the forecasting skill SP variable. The CSI of our DAWP is significantly higher than that of other methods on CSI-0.5 and CSI-1.0, achieving 0. 197 and 0.1. Compared with transformer-dos -op. our DAWP increases CSI-0.5 and CSI-1.0 by 44.8% and 78.9%. Another observation about CSI is decreasing trend with increasing thresholds. Especially, when the threshold increases to 2.0, the CSI of EarthNet and transformer-dop are even lower than that of the naive persistent model. Only our DAWP maintains a slightly higher forecast skill on CSI 2.0. The decrease of forecasting skill is caused by the temporal decay of prediction intensity (26] 29] which could further hamper the mapping network's performance. When evaluating FAR on threshold 2.0mm/h, our DAWP uncommonly surpasses Earthnet and transformer-dop slightly. It could be explained by considering the CSI-2.0 results, as is difficult for EarthNet and transformer-dop to produce predictions greater than 2.0mm/h, resulting in few false alarms. Besides, compared to the persistent baseline with comparable CSI-2.0 skill, our DAWP has FAR score which 57.8% lower than persistent's s, showing stronger reliability. The evaluation of CSI and FAR scores demonstrates the potential of our DAWP for global precipitation forecasting, simultaneously increasing the accuracy and reliability for precipitation forecasting skill of AI-DOP methods.
4.4 Ablation study
Loss Curve of training
Effect of AIDA initialization. An ablation study is conducted to validate the effectiveness of our AIDA module on our DAWP and other AI-DOP methods. The results demonstrate that AIDA enables efficient spatiotemporal modeling and enhances the rollout prediction ability of AI-DOP methods by imputing the observation space.
1.00
0.75
AMMP without AIDA
$0.00
0.50
AMP with AIDA
0.0
100 125 173 200 Training Steps steps)
500
Figure 5: Training loss curve.
The training is unstable after cancelling the AIDA stage in our DAWP. We show the curve of training loss in Figure 5 It can be observed that the training loss of our DAWP with AIDA decreasing steadily, while the loss curve with- out the initialization of AIDA dramatically in- creases about 20k training steps and maintains a MSE loss of 1.0. In contrast, the loss of our DAWP with AIDA could converge to less than 0.1 after 200k training steps. This phenomenon indicates that the classical spatiotemporal learn- ing methods are difficult to learn the dynamics in an observation space with variable missing values. By imputing the observation space into
ATMS
AMSU-A
10
Ang. MAE (e+00)
Aug, MAE (c-04)
1.6
0.8
0
20
40
60
0
20
40
60
MHS
HIRS
4.8
1.8-
Nug MAE (e-03)
Aug. MAE (e: (00)
4.0-
completed spatiotemporal space, AIDA benefits efficient spatiotemporal learning modeling.
1.2
Applying AIDA initialization enhances the roll- out prediction ability of AI-DOP methods. The quantitative results of transformer-dop are de- picted in Figure 6 We exhibit the averaged MAE of 0-72h prediction on each sensor with- /without AIDA initialization.
20
60
70
60
Lead time (hours)
Lead time thours)
Transformer-DDR
our AIDA
Figure 6: The MAE curves of Transformer-DOP with(w/o) our AIDA initialization.
Ponnffomer-DOP with AIDA initialization significantly outperforms the original Transformer-DOP when the lead time increases. It reveals the potential of AIDA initialization for boosting the AI-DOP model for multi-step rollout prediction.
8

---

Drop one modality
Keep one modallty
500
AMSU-
AMSU-A
Enror
HIRS
HIRS
100
AMSU-A
HIRS
AMSU-A
HIRS
MHS
Figure 8: Matrix of relative under the setting of dropping modality and keeping one modality. The three columns in a black rectangle represent the relative MAE error ratios of 0-12h, 12-24h, and 24-36h lead times, respectively.
Gains of cross-regional boundary conditioning. We compare the observation predictions with and without cross-regional conditions by training DAWP without neighbour regions as inputs. This ablation study is conducted to verify the effect on the accuracy and spatial continuity of the prediction.
The convergence loss of our DAWP with and without cross-regional condition is presented Table 4. showing conditioning on neighbour regions improves the prediction accuracy. In this table, we evaluate the convergence loss center areas and neighbor areas. For DAWP without Cross-regional Boundary Conditioning (CBC), we take the center area the neighbour area. The average convergence loss the center area is O. 106, which is significantly than that of the neighbour area. Besides, also significantly outperforms the DAWP without CBC the centre area by about 15.6%. This result validates the necessity of cross-regional conditions for weather prediction, the boundary information of atmospheric physical motion is crucial for accurate weather forecasting.
Ours v/oCaa
w/o CBC
Figure A visualization of forecasting results with(w/o) Cross-regional Boundary Conditioning.
Another benefit of cross-regional conditions is the improvement of prediction continuity, as shown Figure 7 The first and second rows show the predictions of ATMS channel at a lead time of hours without and with CBC, respectively. It can be observed in the black box that the continuity of adjacent regions is improved, which helpful for keeping the atmospheric structure.
Modality sensitivity analysis. We introduce the experiment of using different modality com- binations for multistep observation prediction to gain insight into the importance of each modal- ity's data. As shown in Figure & drop one and keep one combination are explored. The set- ting of drop one means removing one modality's observation before AIDA initialization and con- ducting multi-step prediction, while in the keep one setting, only keep one modality's observation before AIDA initialization for forecasting. We evaluate the influence of each modality by calculating the relative MAE error ratios between the MAE of DAWP with completed modality inputs.
Table 4: Converged loss of our DAWP with and without CBC module. The average loss is caleu- lated by averaging the loss of all four modalities.
Modalitiy
Module
Area
AMSU-A
ATMS
HIRS
MHS
Avg.
w/o CBC
center
0.063
0.101
0.236
0.111
0.128
border
0.324
0.148
CRC
center
0.054
0.074
0.214
0.084
0.106
The result of drop one is shown in the left Figure Row names of the figure indicate the modality that is dropped, while columns sequentially included in modality name represent the relative MAE error ratio of this modality in time windows 0-12h, 12-24h, and 24-36h. The overall trend of the drop one setting demonstrates that dropping any modality's 's data leads to an increase of MAE error. Besides, it's observed that for the rollout predictions, the MAE error ratio is gradually decreasing, indicating the insufficient usage of observations for multistep predictions. When focusing on single modalities, we find that HIRS and MHS can still achieve a relatively low MAE error ratio when their own data is dropped, indicating that they have redundant information for the prediction.

---

The eиror ratio matrix of keep one setting is also shown in Figure 8 When only keeping ATMS observation, other modalities' MAE enror ratios of predictions are the lowest, which are even lower than keeping the satellites themselves. This indicates that ATMS has substantial amount of information for prediction. In contrast, for predicting one modality itself, AMSU-A exhibits the highest MAE enror ratio, showing that it has the least spatiotemporal dynamics information.
Conclusion
In this paper, we propose DAWP, a novel framework using AIWP for observation prediction with an AIDA module as initialization. Comprehensive experiments are conducted to validate the efficiency and potential of our DAWP framework for observation forecasting and downstream applications such precipitation forecasting. Broader Impacts&Futury Work: First, our framework readily integrates variable observations, demonstrating its potential as an implicit Earth system modeling framework. Second, our framework has broad application prospects. It can seamlessly adapt diverse downstream tasks-such as surface parameter estimation, wildfire monitoring, and sea ice mapping- whenever observations retrieval operators our framework holds a promising potential for directly predicting physical variables by integrating observations of weather variables such station data. Linitations: Although our DAWP framework improves the observation forecasting, the sources of observation are still homogeneous in satellite observations. More observation sources will be integrated with DAWP in the future.
available, similar to precipitation forecasting. Third,
Acknowledgements
This work is supported by Shanghai Artificial Intelligence Laboratory and the JC STEM Lab of AI for Science and Engineering, funded by The Hong Kong Jockey Club Charities Trust, the Research Grants Council of Hong Kong (Project No. WCHKK4213324) This work was done during Junchao Gong's 's internship Shanghai Artificial Intelligence Laboratory.
References
[1] Peter D Dueben and Peter Bauer. Challenges and design choices for global weather and climate models based on machine learning. Geoscientific Model Development, 11(10):39994440 -4099 2018.
[2] Anna Allen, Stratis Markou, Will Tebbutt, James Requeima, Wessel P Bruinsma, Tom R Andersson, Michael Herzog, Nicholas D Lane, Matthew Chantry, Scott Hosking, et al. End- to-end data-driven weather prediction. Nature, pages 1-3, 2025.
[3] Kaifeng Bi, Lingxi Xie, Hengheng Zhang, Xin Chen, Xiaotao Gu, and Qi Tian, Accurate medium-range global weather forecasting with 3d neural networks. Nature, 61957970):533-538, 2023.
[4] Remi Lam, Alvaro Sanchez-Gonzalez, Matthew Willson, Peter Wirnsberger, Meire Fortunato, Ferran Alet, Suman Ravuri, Timo Ewalds, Zach Raton-Rosen, Weihua Hu, al. Learning skillful medium-range global weather forecasting. Science, 382(6677):1416- 1421, 2023.
[5] Lei Chen, Xiaohui Zhong, Feng Zhang, Yuan Cheng, Yinghui Xu, Yuan Qi, and Hao Li. Fuxi: A cascade machine learning forecasting system for 15-day global weather forecast. npj climate and atmospheric science, 6(1):190, 2023.
[6] Kang Chen, Tao Han, Junchao Gong, Lei Bai, Fenghua Ling, Jing-Jia Luo, Xi Chen, Leiming Ma, Tianning Zhang. Rui Su, al. Fengwu: Pushing the skillful global medium-range rrage weather forecast beyond 10 days lead. arXiv preprint DeXXi:2304.0294: 2023.
[7] Hans Hersbach, Bill Bell, Paul Berrisford, Shoji Hirahara, András Horányi, Joaquín Muñoz- Sabater, Julien Nicolas, Carole Peubey, Raluca Radu, Dinand Schepers, et al. The era5 global reanalysis. Quarterly journal of the royal meteorological society, 1999730):199--204 2020.
10

---

[8] Michele M Rienecker, Max J Suarez, Ronald Gelaro, Ricardo Todling, Julio Bacmeister, Emily Liu, Michael Bosilovich, Siegfried D Schubert, Lawrence Takacs, Gi-Kong Kim, et al. Merra: NasaåAZs modern-era retrospective analysis for research and applications. Journal of climate, 24(14):3624-3648 2011.
[9] Ronald Gelaro, Will McCarty, Max J Suárez, Ricardo Todling, Andrea Molod, Lawrence Takacs, Cynthia A Randles, Anton Darmenov, Michael Bosilovich, Rolf Reichle, et The modem-era retrospective analysis for research and applications, version 2 (merra-2). Journal of climate, 30(14) 4094454549- 2017.
[10] Eugenia Kalnay, Masao Kanamitsu, Robert Kistler, William Collins, Dennis Deaven, Lev Gandin, Mark Iredell, Suranjana Saha, Glenn White, John Woollen, et al. The ncep/ncar 40-year reanalysis project. In Renewable energy, pages Vol1_146-Von VoI1_194. Routledge, 2018.
[11] Peter Bauer, Alan Thorpe, and Gilbert Brunet. The quiet revolution of numerical weather prediction. Nature, 525(7567):47-55, 2015.
[12] Arianna Valmassoi, Jan D Keller, Daryl Kleist, Stephen English, Bodo Ahrens, Ivan Bašták Durán, Elisabeth Bauernschubert, Michael G Bosilovich, Masatomo Fujiwara, Hans Hersbach. et al. Current challenges and future directions in data assimilation and reanalysis. Bulletin of the American Meteorological Society, 104(4):E55-EE76) 2023.
[13] Guannan Hu, Sarah L Dance, Ross N Bannister, Hristo G Chipilski, Oliver Guillet, Bruce Macpherson, Martin Weissmann, and Nusrat Yussouf. Progress, challenges, and future steps in data assimilation for convection permitting numerical weather prediction: Report on the virtual meeting held on 10 and 12 november 2021. Amospheric Science Letters, 24( 1130, 2023.
convection-pemmttigg
[14] Jing-An Sun, Hang Fan, Junchao Gong, Ben Fei, Kun Chen, Fenghua Ling, Wenlong Zhang, Wanghan Xu, Li Yan, Pierre Gentine, and Lei Bai. Align-da: Align score-based atmospheric data assimilation with multiple preferences, 2025.
[15] Patrick Laloyaux, Thorsten Kurth, Peter Dominik Dueben, and David Hall. Deep learning to estimate model biases in operational nwp assimilation system. Journal of Advances in Modeling Earth Systems, 14(6):22022M000016,, 2022.
[16] Haiyu Dong. Omg-hd: A high-resolution ai weather model for end-to-end forecasts from observations. In A14X 2025 International Conference, 2025.
[17] Anthony McNally, Christian Lessig, Peter Lean, Eulalie Boucher, Mihai Alexe, Ewan Pinning- ton, Matthew Chantry, Simon Lang, Chris Burrows, Marcin Chrust, et al. Data driven weather forecasts trained and initialised directly from observations. arXiv preprint arXiv: 2024.
[18] Mihai Alexe, Eulalie Boucher, Peter Lean, Ewan Pinnington, Patrick Laloyaux, Anthony McNally, Simon Lang, Matthew Chantry, Chris Burrows, Marcin Chrust, al. Graphdop: Towards skilful data-driven medium-range weather forecasts learnt and initialised directly from observations.
arXiv preprint arXiv:2412.1566 2024.
[19] Tao Han, Song Guo, Fenghua Ling, Kang Chen, Junchao Gong, Jingjia Luo, Junxia Gu, Kan Dai, Wanli Ouyang, and Lei Bai. Fengwu-ghr: Learning the kilometer-scale medium-range global weather forecasting. arXiv preprint N0V:22422000599, 2024.
[20] Sijie Zhao, Feng Liu, Xueliang Zhang, Hao Chen, Tao Han, Junchao Gong, Ran Tao, Pengfeng Xiao, Lei Bai, and Wanli Ouyang. Transforming weather data from pixel to latent space. arXiv preprint arXiv:2503.06623 2025.
[21] Xiangyu Zhao, Zhiwang Zhou, Wenlong Zhang, Yihao Liu, Xiangyu Chen, Junchao Gong, Hao Chen, Ben Fei, Shiqi Chen, Wanli Ouyang, al. Weathergfm: Learning a weather generalist foundation model via in-context learning. arXiv preprint arXiv:2411.055. 1.05420, 2024.
[22] Wanghan Xu, Fenghua Ling, Tao Han, Hao Chen, Wanli Ouyang, and LEI BAI. Generalizing weather forecast to fine-grained temporal scales via physics-ai hybrid modeling. Advances in Neural Information Processing Systems, 01123325-2235/1 2024.

---

[23] Jaideep Pathak, Shashank Subramanian, Peter Harrington, Sanjeev Raja, Ashesh Chattopadhyay. Morteza Mardani, Thorsten Kurth, David Hall, Zongyi Li, Kamyar Azizzadenesheli, al. Fourcastnet: A global data-driven high-resolution weather model using adaptive fourier neural operators. arXiv preprint arXiv:2202. 1214, 2022.
[24] Ilan Price, Alvaro Sanchez-Gonzaler Ferran Alet, Tom R Andersson, Andrew El- El-Kadi, Dominic Masters, Timo Ewalds, Jacklynn Stott, Shakir Mohamed, Peter Battaglia, et al. Probabilistic weather forecasting with machine learning. Nature, 637(8044):84 90, 2025.
[25] Dmitrii Kochkov, Janni Yuval, Ian Langmore, Peter Norgaard, Jamie Smith, Griffin Mooers, Milan Klöwer, James Lottes, Stephan Rasp, Peter Düben, et al. Neural general circulation models for weather and climate. Nature, 632(8002):1000- 2024.
[26] Suman Ravuri, Karel Lenc, Matthew Willson, Dmitry Kangin, Remi Lam, Piotr Mirowski, Megan Fitzsimons, Maria Athanassiadou, Sheleem Kashem, Sam Madge, et al. Skilful pre- cipitation nowcasting using deep generative models of radar. Nature, 597(7878):622-777, 2021.
[27] Yuchen Zhang. Mingsheng Long. Kaiyuan Chen, Lanxiang Xing, Ronghua Jin, Michael I Jordan, and Jianmin Wang. Skilful nowcasting of extreme precipitation with nowcastnet. Nature, 619(7970)5526-533 2023.
[28] Zhihan Gao, Xingjian Shi, Boran Han, Hao Wang, Xiaoyong Jin, Danielle Maddix, Yi Zhu, Mu Li, and Yuyang Bernie Wang. Prediff: Precipitation nowcasting with latent diffuuson models. Advances in Neural Information Processing Systems, 36:78621-78656, 2023.
[29] Junchao Gong, Siwei Tu, Weidong Yang, Ben Fei, Kun Chen, Wenlong Zhang, Xiaokang Yang, Wanli Ouyang, and Lei Bai. Postcast: Generalizable postprocessing for precipitation nowcasting via unsupervised blurriness modeling. arXiv preprint arXiv:2410.0558 055000, 2024.
[30] Junchao Gong, Lei Bai, Peng Ye, Wanghan Xu, Na Liu, Jianhua Dai, Xiaokang Yang, and Wanli Ouyang. Cascast: Skillful high-resolution precipitation nowcasting via cascaded modelling. arXiv preprint 2:Xx::2402.04290 2024.
[31] Thomas J Vandal, Kate Duffy, Daniel McDuff, Yoni Nachmany, and Chris Hartshom. Global atmospheric data assimilation with multi-modal masked autoencoders. arXiv preprini arXiv:2407. 1696, 2024.
[32] Roman Bachmann, David Mizrahi, Andrei Atanov, and Amir Zamir. Multimae: Multi-modal multi-task masked autoencoders. In European Conference on Computer Vision, pages 348-367. Springer, 2022.
[33] David Mizrahi, Roman Bachmann. Oguzhan Kar, Teresa Yeo, Mingfei Gao, Afshin Dehghan, and Amir Zamir. 4m: Massively multimodal masked modeling. Advances in Neural Information Processing Systems, 66:58363-58408 2023.
[34] Kaiming He, Xinlei Chen, Saining Xie, Yanghao Li, Piotr Dollár, and Ross Girshick. Masked autoencoders are scalable vision learners. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 16000-16009, 2022.
[35] Zhihan Gao, Xingjian Shi, Hao Wang, Yi Zhu, Yuyang Bernie Wang, Mu Li, and Dit-Yan Yeung. Earthformer: Exploring space-time transformers for earth system forecasting. Advances in Neural Information Processing Systems, 35:25390- 224490,,,,4,,,..- 2022.
[36] Yujin Tang, Lu Qi, Fei Xie, Xiangtai Li, Chao Ma, and Ming-Hsuan Yang. Predformer: Transformers are effective spatial-temporal predictive learners. arXiv preprint arXiv:2410.04733, 2024.
[37] Anurag Amab, Mostafa Dehghani, Georg Heigold, Chen Sun, Mario Lučic, and Cordelia Schmid. Vivit: video vision transformer. In Proceedings of the IEEF/CVE international conference computer vision, pages 6836-6846, 2021.
12

---

[38] Patrick Esser, Sumith Kulal, Andreas Blattmann, Rahim Entezari, Jonas Müllrr, Harry Saini, Yam Levi, Dominik Lorenz, Axel Sauer, Frederic Boesel, al. Scaling rectified flow trans- formers for high-resolution image synthesis. In Forty-first international conference on machine learning, 2024.
[39] Black Forest Labs. Flux. httpp///gtthubb..co/aooooooooooe
2024.
[40]
EUMETSAT. User services
client.
https:///rccive.eeum eumetsat. int int/usc/
EseršervicesCliene htmm
[41]
GES DISC. Gpm atms on suomi-npp common calibrated brightness temperature.
https
//disc. gafc. nasa. (atpaaaaaeetttaaaaPMAAAAAAAAAA
[42] GES DISC. Gpm atms noaa-20 common calibrated brightness temperatures.
https
7/disc. gsfc. nasa.
summary
[43]
GES DISC. Gpm atms on suomi-npp radiometer precipitation nasa. casa.gov/daas wtw.ggoooddaattteee /PM_ /WTP://AAAAPKPPPPPPPTSSSSSS
prappppccicccccccccciiccittitiii profiling, https: /tops://disc.. gafc
[44]
GES DISC. Gpm atms on noaa-20 climate-based radiometer precipitation profiling. hhtpss //disc. nasa. / SAAAS///PPPPAPPPPOAAAAAAAAAAASS CWWCAATTSSSSLMMMMMMMAAEA 07/sumarry
[45] Joseph T Schaefer. The critical success index as an indicator of warning skill. Weather and forecasting, 5(4):570-577) 1990.
[46] Lindsey R Barnes, David M Schultz, Eve C Gruntfest, Mary H Hayden, and Charles C Benight, Corrigendum: False alarm rate or false alarm ratio? Weather and Forecasting, 2455):1452-145, 2009.
[47] Yi Xiao, Lei Bai, Wei Xue, Hao Chen, Kun Chen, Tao Han, Wanli Ouyang, et al. Towards a self-contained data-driven global weather forecasting framework. In Forty-first International Conference on Machine Learning. 2024.
[48] Kun Chen, Peng Ye, Hao Chen, Tao Han, Wanli Ouyang, Tao Chen, LEI BAI, et al. Fnp: Fourier neural processes for arbitrary-resolution data assimilation. Advances Neural Information Processing Systems, 37:137847-13787. 2024.
[49] Langwen Huang, Lukas Gianinazzi, Yuejiang Yu, Peter D Dueben, and Torsten Hoefler. Diffda: a diffusion model for weather-seale data assimilation. In Proceedings of the 41st International Conference on Machine Learning, pages 19798-19815, 2024.
[50] Xingjian Shi, Zhourong Chen, Hao Wang, Dan Yeung, Wai-Kin Wong, and Wang-chun Woo. Convolutional Istm network: A machine learning approach for precipitation nowcasting. Advances in neural information processing systems, 28, 2015.
[51] Yunbo Wang, Haixu Wu, Jianjin Zhang, Zhifeng Gao, Jianmin Wang. Philip S Yu, and Ming- sheng Long. Predmın: A recurrent neural network for spatiotemporal predictive learning. IEEE Transactions on Pattern Analysis and Machine Intelligence, 25(2):2208-22) 2022.
[52] Cong Bai, Feng Sun, Jinglin Zhang, Yi Song, and Shengyong Chen. Rainformer: Features extraction balanced network for Remote Sensing Letters, 19:1-5, 2022
precipitation nowcasting. IEEE Geoscience and
[53] Zhangyang Gao, Cheng Tan, Lirong Wu, and Stan Z Li. Simvp: Simpler yet better video prediction. In Proceedings of the IEEE/CVF conference computer vision and pattern recognition, pages 3170-3180, 2022.
[54] Cheng Tan, Zhangyang Gao, Lirong Wu, Yongjie Xu, Jun Xia, Siyuan Li, and Stan Z. Li. Temporal attention unit: Towards efficient spatiotemporal predictive learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 18770-18782,
2023.
[55] Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Björn Ommer. High- resolution image synthesis with latent diffusion models. In Proceedings of the IEEF/CV conference computer vision and pattern recognition, pages 10684-10695, 2022.
13

---

[56] Tao Han, Zhenghao Chen. Song Guo, Wanghan Xu, and Lei Bai. Cra5: Extreme compression of era5 for portable global climate and weather research via an efficient variational transformer. arXiv preprini arXiv:2405.03370 03336, 2024.
[57] Dan Hendrycks and Kevin Gimpel. Gaussian error linear units (gelus), arXiv preprint 1e0.:66660.815 2016.
[58] Noam Shazeer. Glu variants improve transformer. arXiv preprint 20Xi:22002.5222,20020
14

---

NeurIPS Paper Checklist
The checklist is designed encourage best practices for responsible machine learning research, addressing issues of reproducibility, transparency, research ethics, and societal impact. Do not remove the checklist: The papers not including the checklist will be desk rejected. The checklist should follow the references and follow the (optional) supplemental material. The checklist does NOT count towards the page limit.
Please read the checklist guidelines carefully for information on how to answer these questions. For each question in the checklist:
You should answer [Yes], [No]. or [NA].
[NA] means either that the question Not Applicable for that particular paper or the relevant information is Not Available.
Please provide short (1ãAŞ2 sentence) justification right after your answer (even for NA).
The checklist answers are an integral part of your paper submission. They are visible to the reviewers, area chairs, senior area chairs, and ethics reviewers. You will be asked to also include (after eventual revisions) with the final version of your paper, and its final version will published with the paper.
The reviewers of your paper will be asked use the checklist as one of the factors their evaluation. While "[Yes] is generally preferable to "[No]" ", it is perfectly acceptable to answer "[No] providedd proper justification is given (e.g.. "error bars are not reported because it would be too computationally expensive" or "we were unable to find the license for the dataset we used"). In general, answering binary way, we
"[No] or "[NA]" is not grounds for rejection. While the questions are phrased acknowledge that the true answer is often more nuanced, please just use your best judgment and write justification to elaborate. All supporting evidence can appear either in the main paper or the supplemental material, provided in appendix. If you answer [Yes] to a question, in the justification please point to the section(s) where related material for the question can be found.
IMPORTANT, please:
Delete this instruction block, but keep the section heading "NeurIPS Paper Checklist",
Keep the checklist subsection headings, questions/answers and guidelines below.
Do not modify the questions and only use the provided macros for your answers.
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification:
We clarify our contributions in the abstract and introduction.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made in the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or
NA answer to this question will not perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results be expected to generalize other settings.
is fine include aspirational goals as motivation long is clear that these goals are not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
Justification:
We discuss the limitations of our work in the conclusion section.
15

---

Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create a separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, c.g., if the approach was only tested few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution is low or images are taken in low lighting. Or speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails handle technical jargon.
The authors should discuss the computational efficiency the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory assumptions and proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [NA].
Justification:
This is paper for applications.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced in the statement of any theorems.
The proofs can either appear in the main paper the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide a short proof sketch provide intuition.
Inversely, any informal proof provided in the core of the paper should complemented by formal proofs provided in appendix supplemental material.
Theorems and Lemmas that the proof relies upon should properly referenced.
4. Experimental result reproducibility
Question: Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that it affects the main claims and/or conclusions the paper (regardless whether the code and data are provided or not)?
Answer: [Yes]
Justification:
We provide all experimental settings for reproductivity.
Guidelines:
The answer NA means that the paper does not include experiments.
16

---

If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution a specific model and empirical evaluation, may be necessary to either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g., in the case of large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) the contribution is primarily a new algorithm, the paper should make it clear how reproduce that algorithm. (b) If the contribution is primarily new model architecture, the paper should describe the architecture clearly and fully.
(c) If the contribution is a new model (e.g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g.. with an open-source dataset instructions for how construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions faithfully reproduce the main experimental results, as described in supplemental material?
Answer: [Yes]
Justification: We train our model ATMS, AMSU-A, MHS, and HIRS, which could be obtained publicly on their websites. The detailed settings for data resampling are provided in supplemental material. We will release our code on github.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurlPS code and data submission guidelines (https://nips.cm cc/
for more details.
While we encourage the release of code and data, we understand that this might not be possible, so âÄIJNoãλ is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g.. for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurlPS code and data submission guidelines (https
/ //nips. cc/ wwwpcccccc////c..ccc.cccc.oeelee cubbiccgguide.. httic//gguid////eeeuubbiiiaa www.ssubbmssiioPoo..
for more details.
The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
17

---

Providing as much information as possible in supplemental material (appended to the paper) recommended, but including URLs to data and code permitted.
6. Experimental setting/details
Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Question:
Answer: [Yes]
Justification: The complete experimental settings, including the dataset, hyperparameters. type of optimizer, etc, are provided in the Experiments section and supplemental materials.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper to level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, as supplemental material.
7. Experiment statistical significance
Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Question:
Answer: [No]
Justification:
The paper does not report error bars, like other related work.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results accompanied by error bars, confi- dence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper.
The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split. initialization, random drawing of some parameter, or overall run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to library function, bootstrap, etc.)
The assumptions made should be given (e.g.. Normally distributed errors).
should be clear whether the error bar is the standard deviation or the standard error of the mean.
OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out range (e.g. negative еттог rates).
If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables the text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information on com- puter resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?
Question:
Answer: [Yes]
Justification: The sufficient information on the computer resources is provided in the Experimets section
Guidelines:
The answer NA means that the paper does not include experiments.
18

---

The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount compute required for each of the individual experimental runs well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make into the paper).
9. Code of ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurlPS Code of Ethics httpp////aaaarsss.c////b///aaaaa
Answer:
[Yes]
Justification:
We perform this work following the NeurIPS Code of Ethics exactly.
Guidelines:
The answer NA means that the authors have reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g.. if there is special consid- cration due to laws or regulations in their jurisdiction).
Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
Justification: We discuss both potential positive societal impacts and negative societal impacts of the work performed in the conclusion section.
Guidelines:
The answer NA means that there is no societal impact of the work performed.
If the authors answer NA or No, they should explain why their work has societal impact or why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied partieular applications, let alone deployments. However, if there is direct path any negative applications, the authors should point it out. For example, it is legitimate point that improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition attacks, mechanisms for monitoring misuse, mechanisms to monitor how system learns from feedback over time, improving the efficiency and accessibility of ML).
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (c.g., pretrained language models, image generators, scraped datasets)?
Question:
Answer: [NA]
19

---

Justification:
This work does not involve such risks.
Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
recognize that providing effective safeguards challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
Licenses for existing assets
Question: Are the creators or original owners of assets (e.g.. code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Answer: [Yes]
Justification: The original papers websites that produced the code dataset are properly cited and we use an open-source dataset for our experiments.
Guidelines:
The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package or dataset.
The authors should state which version of the asset is used and, if possible, include URL.
The name of the license (e.g., CC-BY 4.0) should be included for each asset. For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.
If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, wwwpssppppsssseeeeeeeeeeee thcode wwt.coo///aaaaa comddatasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
If this information is available online, the authors are encouraged to reach out to the asset's 's creators.
13. New assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [Yes]
Justification: The code in the paper is well documented and the documentation is provided alongside the code.
Guidelines:
The answer NA means that the paper does release new assets.
Researchers should communicate the details the dataset/code/moodl as part their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset used.
At submission time, remember to anonymize your assets (if applicable). You can either create anonymized URL or include an anonymized zip file.
14. Crowdsourcing and research with human subjects
20

---

Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well as details about compensation (if any)?
Answer: [No]
Justification: The paper has no crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included in the main paper.
According to the NeurIPS Code of Ethies, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals or equivalent for research with human subjects
Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or equivalent approval/review based the requirements of your country or institution) were obtained?
Question:
Answer: [NA]
Justification: The paper does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing research with human subjects.
Depending on the country in which research is conducted, IRB approval (or equivalent)
may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
16. Declaration of LLM usage
Question: Does the paper describe the usage of LLMs if it is an important, original, or non-standard component of the core methods in this research? Note that if the LLM is used only for writing, editing, formatting purposes and does not impact the core methodology, scientific rigorousness, or originality of the research, declaration is not required.
Answer: [NA]
Justification: The core method development in this research does not involve LLMs as any important, original, or non-standard components.
Guidelines:
The answer NA means that the core method development in this research does not involve LLMs any important, original, non-standard components.
Please refer to our L.LM policy hato:s//merrrps.cc//ooffeeeeeeee for what should or should not be described.
21

---

A Global state cache
In this section, we clarify the design of the global state cache. It is composed of current cache and a previous cache. The current is used to store the predicted sub-images of 122412((+1)) and the previous one, storing sub-images of T 1):12: used to provide neighbour information for the prediction.
class GlobalStateCache:
def -_init_.... (self, domains, Comain_Chs, time time_windov, aelf domains domains tettttimeiiiiieeee = tindww domain_Chs
culf.cur_cachie self init
self. pref.pree_cccee self imit wetffinitt_caacic cache ()
def init_ (self)
domain_cache 0
for domain in self.domaing.
domain_Ch self domain Chs [domain]
original image size is 1152x2304. 8x16 subimages with 9x9 is the number of tokens in a subimage. ammain_cccia [domain]
size of 144x144.
torch zeros ( 16, self. time window, domain_Ch, 9, 9)) Cacain_cachhe
The main operators of the global state cache are query neighbors and update cache. Query neighbors used to get the adjacent sub-images conditions for the prediction of central areas. Update cache is used to update the previous and current cache with the subsequent 12 hours predictions.
(self, def query.neighbours (self, rel_coordinate): neighbour_coore neighbours coords self -get_ neighbour.coord wwt.:/oooooooooooooooeeeeeeeeee
for domain in self domains:
neighbours [domain] = [] for coord in neighbour neighbours [domain] append
self prev_ [domain] [coord [O], coord [1]]
return neighbours
def update_ cache rel. rel_coordinate, pred_subimg)
for domain in self.domains.
coords rel rel_ cel_doordinate [0], rel_coordinate [1] self cur_cache [domain] [coords] pred_subing [domain] have pave_pped__hooe__ngg
self prch_ self. cur_ cache return None
To get the neighbour coordinates from a Plane Rectangular Coordinate System, we utilized get___________ S_nnennbouurreeeee ccord function that incorporates Earth's spherical geometry, specifically han- dling the left-right and top-bottom boundaries of the image.
def
get _ fet___neightt www.neighourrrcco (esoorddsllll (self, rel.coord,,ate, h-8, w=16):
h, is the height and width of the image r, is the coordinate of the pixel case 1: the pixel in the center the image, return
8 neighbours
case 2: if the pixel in a border, treat the image as h border is connected
22

---

case 3: if the pixel in h border, symmetrically get the neighour
case 4: the pixel in corner, use the rule of both # border and h border
ret:
8 neighbours ordered [up_left, up, up_right, left, right, dewt_left, down, right
rel_coordinate [0], rel_coordinate [1]
assert >= and h
assert с >= c w
w_border_flag (c 0
or c w 1)
h L.too_border.flee flag = (r == 0)
h_bottom_border.u flag (r (h 1))
neighbours
if not (h_top_border___ flag or h_ bottomtommmt www.bttmmooddder______
up-left ((r 1) % h, (c 1) % u)
up ((r 1) % h, c)
up-right ((r - h, (c 1) w)
left (r, (c-1) k)
right = (r, (c + 1)
down_left ((r 1) 11.%% hhh,h (c
1) % w)
down ((r 1) %,,, c)
down_right ((r +1) h,
1) % x)
elif h ..top_border_flag
up-left (r, (c + + % m)
up (r, (c
2)% w)
up_right (r, (c-11 w//2) %
left (r, (c w)
right = (r, +) [.w.w
down_left (1)+1)) 1),,,, h, (c 1) % w)
down ((r 1) 1,,
downrright ((r () h,
1) % w)
elif h_bottom_it booettom_bordrr_flggg
up_left = ((r 1) h, (c
1) % w) up_right ( h, (c
up ((r 1) h, (c)% w)
1) w)
left (r, k)
right (r, (c+ {))
dovn_left (r, (c+11 w//2) % w)
down (r, (c + 4//2)% w)
down_right (r, (c w//2) w)
else:
raise NotImplementedBrro
neighbours.append (up_left)
neighbours. (pp)
beighbours.appen (up_right)
neighbours.appen (left)
neighbours
(ppoonrrr.pp)) (right)
apigh (dosm_
neighbours. (do (down)
neighbours (down_right) neighbours
B Artificial intelligence data assimilation
The development of data assimilation has also been revolutionized by artificial intelligence. Xiao al. [47] were the first to apply the popular traditional numerical data assimilation method, Four- Dimensional Variational, to the AIWP model FungWu. Furthermore, researchers have explored the
23

---

development of artificial intelligence assimilation methods, such as FNP 48 and DiffDA [49,, which could be applied both NWP and AIWP models. Although impressive progress has been made, these methods remain limited by reanalysis data and NWP models, which require transforming the observations into physical space. Unlike previous methods, EarthNet [31] proposes implementing ob- servation space data assimilation with masked reconstruction. We are motivated to use observation AIDA model for formulating a complete observation space.
C Comparisons with spatiotemporal learning methods
Table 5: More results on spatiotemporal methods. MAE error of forecasting during 3 lead time periods (0-12h, 12-24h, and 24-36h) for different channels of the satellite data. We use the unit of le-5 for AMSU-A, le-4 for MHS, and le-0 for both ATMS and HIRS.
AMSU-A
ATMS
HIRS
MHS
Methods
Lead time
chO
ch1
ch0
ch1
ch9
ch10
chO
ch1
Persistence
35
5.86
9.15
14.37
11.69
12.43
2.21
7.01
14.74
ConvLSTM
50
127.82
208.90
73.21
78.44
77.40
11.05
62.27
158.57
PredRNN 51
2.95
4.96
6.99
6.21
9.26
1.42
4.11
10.20
RainFormer 52
3.95
6.52
9.08
8.05
10.41
1.63
4.98
11.69
Forthrrrmer
35
18.97
33.94
37.33
38.75
22.49
3.62
18.00
55.04
0-12h
SimVP (53
3.61
6.02
7.29
6.61
9.84
1.53
4.68
11.35
TAU [54]
3.84
6.31
7.70
9.94
1.58
4.87
11.42
EarthNet
31
2.93
4.89
6.96
6.14
9.15
1.39
3.96
9.65
Transformer- DOP
2.67
4.48
6.40
5.61
9.22
1.42
3.91
9.48
Ours
1.92
3.39
3.36
3.27
7.70
1.12
3.07
7.91
Persistence
35
4.35
6.94
10.40
8.86
13.58
2.30
6.07
15.09
ConvLSTM
50
128.
211.14
81.22
82.60
71.64
9,98
59.26
145.44
PredRNN
3.85
6.26
11.14
8.82
10.92
1.84
5.48
13.16
RainFormer
11.46
15.72
19.43
17.76
18.75
3.28
11.74
32.10
EarthFormer 335
18.98
33.96
37.33
38.76
22.50
3.62
18.01
55.03
12-24h
SimVP 533
4.83
7.74
11.48
9.10
11.49
2.02
6.43
14.20
TAU [54
4.46
7.21
11.46
9.06
11.65
2.04
6.24
13.94
NarthNet H1
4.12
6.65
11.25
9.00
11.14
1.98
5.46
13.11
Transformer- DOP 17
3.84
6.14
10.04
8.04
11.08
1.95
5.19
12.65
Ours 35
3.11
5.12
7.35
6.35
9.57
1.54
4.51
10.54
Persistence
6.39
9.84
15.37
12.52
14.61
2.61
8.00
17.86
ConvLSTM
50
128.42
211.83
82.03
85.17
72.62
10.15
60.44
148.62
PredRNN 51
4.58
7.23
12.18
9.69
12.03
2.08
6.24
15.04
RainFormer 52
24.89
32.34
35.49
36.54
30.78
4.92
21.51
69.68
EarthFormer 35
18.98
33.96
37.34
38.77
22.51
3.63
18.02
55.04
24-36h
SimVP 533
5.72
8.91
12.93
10.33
12.71
2.32
73.95
15.99
TAU [54
5.61
8.76
13.23
10.53
12.98
2.34
7.15
15.85
EarthNet 31
5.17
8.14
12.52
10.08
12.37
2.36
6.41
15.08
Transformer-Do
17
4.91
7.54
11.35
9.07
12.39
2.27
6.22
14.70
Ours
3.66
5.80
7.84
6.81
10.71
1.79
5.15
12.22
As shown in Table 5 we compare DAWP with more spatiotemporal methods including RNN-based
( (50). (5D), CNN-based (53 (54) ) and transformer-based [52). (35)). Our DAWP maintains a
significant advantage over these methods, demonstrating the effectiveness of our AIDA module in improving the roll-out and efficiency of AIWP
We present implementation details of EarthNet and Transformer-DOP here. There is no open-sourced code for EarthNet [3T] or Transformer-DOP [17]. For EarthNet [3I], it follows the implementation of MultiMAE (32] as detailed in EarthNet's Appendix C and D. Therefore, we reproduce it on our datasets following MultiMAE. As for Transformer-DOP, since the original paper presents only sketch without details, we implemented it according to our best available understanding. Specifically. EarthNet is reproduced as a 12-layer encoder (hidden dimension 768) paired with an 8-layer decoder (hidden dimension 512), and Transformer-DOP is implemented
transformer consisting of 18 layers (hidden dimension 1024). We employ sub-images because the full 12-hour global observation sequence would result in I 124k-token sequence, which is computationally infeasible.

---

Table 6: Reconstruction enrrr of various VAEs on different modalities. The column of Improvement represents relative average improvement over SD-VAE DAE.
Modality reconstruction error
VAEs
Improvement
AMSU-A (Ic-3)
ATMS (Ic-2)
HIRS (Ie-3)
MHS (Ic-2)
SD- VAE [55]
1.07
1.26
5.84
2.28
Mask-SD VAE
1.21(-13.1%)
1.31(-4.0%)
5.60(+4.11)
2.35(-3.1%)
-4.1%
ViT- [56]
0.92(+14.0%)
1.36(-7.9%)
4.28(+26.7%)
2.45(-7.4%%
+6.3%
0.78(427.1%)
1.29(-2.3%)
4.11(429.6%)
2.41(-5.7%)
+12.2%
D VAE comparison
We explore the ability of our mask VIT VAE compress satellite data with multiple channels and missing values by comparing it with other VAEs.
The results are shown in Table 6 where we compare the reconstruction loss of different VA First, VAES with VIT structure are more effective for reconstructing modality data with multiple channels, such as HIRS and AMSU-A. while modalities with fewer channels, there is only slight increase in reconstruction loss. Another observation is that the application increases VAESs reconstruction ability HIRS. For ViT-VAE, it is beneficial to use the mask for the computation of attention between patch tokens, as it could directly weaken the influence of missing tokens.
mask consistently
E Satellite data preprocessing
The original satellite observation data points are extremely sparse and irregular. To spatially align different observation sources and channels for model training, remapping procedure is performed beforehand. The pseudocode for the preprocessing algorithm is given in algorithm []
Preprocessing:
Algorithm 1: Remapping Satellite Observation
Input: target resolution R, observation D., corresponding latitudes Clat and longitudes Cton Output: remapped observation data points D Dorid desired global grid
Generate global grid CR of desired resolution that follows Equirectangular projection:
Assign latitudes and longitudes (Coas,Ctom) the nearest coordinates (Clat- grid Cr:
3 D Darid t- NaN with the shape of Cri
D Doonnt Zeros with the shape of Cn:
5 for do.Ctat-Coon in D,, Clatt Ci do Locate corresponding data point dyrid of D Dorid according to coordinates (Gia Gon) Locate corresponding point counter doount of D Dount according to coordinates (c) (oors dyrid equals NaN then dgrid +- do:
else 10
11
dgrid +- (dgrid + do);
12
end
end
14 Average each dyrid where doouns 2
We normalize each modality for efficient convergence. The direct observation modalities are normalized by:
Normalization:
J.M Mean(a
T norm
Sld(x:3)
where M represents the modality M. For ATMS- we first implement the log- transformation as:
Tprec log (æ/a b),
(2)
alleviate long-tail distribution, and then normalize these variables like other modalities. Specifically, select le 7, 6 1e2 for SP channel, and choose = 1, b 1 for TCWV channel. The
25

---

SP
SP(log)
TCWV
TCWVilog)
0.7
0.00
0.6-
0.48
0.05
0.48
Reeuan Seg
0.55
0.40
0.40
0.4-
0.32
0.24
0.24-
0.16
0.02
0.01
0.08
0.0
60
60
0.00
10
15
20
0.00
20
40
0.00
mmnh
mm
Figure 9: Distribution of ATMS precipitation productions. SP(log) indicates applying log-transform on SP. is the same for TCWV(
distribution shift is shown Figure 9 The original distributions also motivate us to choose a threshold list of [0.5 mm/h, mm/h, 2 mm/h] for SP.
F Dataset introduction
The four sensors below are organized into satellite observation dataset for direct observation forecasting. This composite dataset has a training split with data from January of 2012 to June of 2022 for training and a testing split composed of data from May of 2023 to July of 2023, present detailed introductions these sensors.
The Advanced Microwave Sounding Unit-A (AMSU-A) 15-channel microwave radiometer used for measuring global atmospheric temperature profiles and will provide information on atmospheric water in all of its phases (with the exception of small ice particles, which are transparent at microwave frequencies). AMSU-A will provide information even in cloudy conditions. AMSU-A measures Earth radiance at frequencies (in GHz) as listed under the instrument channel information. Level 1B data was collected from EUMETSAT htttttpss//acchiiieeeee eumetsat. int/ isc/UserServicesClies html
AMSU-A:
ATMS: The Advanced Technology Microwave Sounder (ATMS) and the Cross-track Infrared Sounder (CrIS) work together provide global high-resolution profiles of temperature and moisture. These advanced atmospheric sensors create cross-sections of storms and other weather conditions, helping with both short-term nowcasting and long-tern forecasting. Level IC data was collected from GES DISC at /attps://diss gsf nasa. gov/
The High Resolution Infrared Sounder (HIRS) operates at 20 channels (19 channels in the infrared and one in the visible). Its main purpose is to provide input for the vertical temperature and humidity profile retrievals. In addition, the HIRS pixel resolution serves the standard grid resolution for all ATOVS level 2 products. Level IB data was collected from EUMETSAT at cettp://arrchiin eumetsat... hatt://ss/DuerrSrreeeellls
HIRS:
The Microwave Humidity Sounder (MHS) is 5 channel instrument used to provide input to the retrieval of surface temperatures, emissivities. and atmospheric humidity. In combination with AMSU-A information it can also be used to process precipitation rates and related cloud properties, well as to detect sea ice and snow coverage. Level IB data was collected from EUMETSAT at https://archive eumetsat....... wttp/usc/Usserrervvcco
MHS:
httl
ATMSSPreecipitation: The ATMS- ATMSS-reecpiien Precipitatio is one of the products of the Global Precipitation Measurement (GPM) mission. is based on the LIC-level calibrated brightness temperature data of the ATMS sensor and extracts information such as precipitation rate and precipitation type through a physical inversion algorithm. Level data was collected from GES DISC at [https //disc. gsfc nasa. gov/ pttp//aatssesssppageettt
G Training details
Our DAWP framework is trained in 3 stages on A100 80G GPUs, including training mask ViT- VAEs for encoding and mapping, an MMAE for data assimilation in observation space, and spatiotemporal transformer for direct observation prediction. Specifically, these modules are all trained within 144x 144 sub-images. The encoder and decoder of the mask VAE use the same
26

---

Table 7: Hyperparameters for training the mask ViT-VAE of DAWP on the composite dataset.
Hyper-parameter
Value
Learning rate
0.0001
31
0.9
32
0.999
Weight decay
0.00001
Batch size
200
Training steps
200000
Warm up percentage
10%
Warmup learning rate
0.000001
Learning rate decay
Cosine
Min learning rate
0.000001
KL-loss weight
0.000001
Table 8: Hyperparameters for training the AIDA module of DAWP on the composite dataset.
Hyper-parameter
Value
Learning rate
0,0001
91
0.9
32
0.999
Weight decay
0.00001
Batch size
48
Training steps
200000
Warm up percentage
10%
Warmup learning rate
0.000001
Learning rate decay
Cosine
Min learning rate 0.000001
Table 9: Hyperparameter for training the AIWP module of DAWP on the composite dataset.
Hyper-paramen
Value
Learning rate
0,0001
31
0.9
B3
0.999
Weight decay
0.00001
Batch size
8
Training steps
200000
Warm up percentage
10%
Warmup learning rate
0.000001
Learning rate decay
Cosine
Min learning rate
0.000001
transformer structure with a patch size of 16 and a hidden dimension of 768. It is trained with reconstruction MAE loss and KI. loss weight of 0.000001 for robust representation. We freeze the pretrained mask ViT-VAE as the encoders for each modality in our MMAE, Each modality observation with 12h time window in 144x 144 sub-image is tokenized into 972 spatiotemporal tokens. Thus, our MMAE totally received 3888 tokens. We randomly select 128 observed tokens of them ( (333%) to reconstruct the remaining observed tokens via MAE training. Given the 144x 144 sub-images assimilated by MMAE, our spatiotemporal transformer is trained. is structured with 12 TS spatiotemporal decoupling blocks, whose hidden dimension is 768. The hyperparameters for optimizing these modules are similar. All of them use the Adam optimizer with 30 0.9, 31 0.999, and learning rate of 0.0001. The learning rate is scheduled by cosine scheduler, warming up 10k steps, step by step.
In the table
13 present the computation cost.
27

---

Table 10: The details of the mask VIT-VAE model on different satellite datasets. Conv16 16 is the 2D convolutional layer with 16 kernel. The FFN consists of two Linear layers separated by GeLU activation layer [57]. The operator SamplePosterior samples a latent representation from μ and SD did [55]
Module
Layer
Resolution
Channels
Input
144 x 144
c
Conv16 x 16
9 x 9
- 768
PatchEmbed
Flatten
- 81
768
PosEmbed
81
768
LayerNorm
81
768
MaskAttention
81
768
Trnasformer Block x 10
LayerNorm
81
768
FFN
768
TransformerBlock
768
TransformerBlock
81
768
Concat
81
768 -> 1536
Qauntify
Linear
81
1536 - 8c
SamplePosterior
+ 4c
Linear
81
- 768
LayerNorm
81
768
MaskAttention
81
768
Traasformer Block x 12
LayerNorm
81
768
FFN
81
768
Rearrange
81 + 9
768
Conv1
9 9
768 - 256c
Out
Rearrange Conv3 x 3
9 x 9 144 144
256c
144 X 144
C
H
Metrics defination
H.1
CSI and FAR
For the evaluation of global precipitation variables, the metrics include the Critical Success Index (CSI) and False Alarm Ratio (FAR). They are core binary classification evaluation metrics that quantify the detection accuracy and reliability of precipitation events. In the field of meteorology, these metrics assess the consistency and accuracy between precipitation predictions and observed results, quantitatively evaluating the performance of models. To measure the accuracy prediction for precipitation with different intensities. Before calculating these metrics, we transform the predicted pixel values and ground truth into binary values or using given threshold т. The value is to 0 if it is less than π; otherwise, it is set to 1. These binary values enable us to determine the true positive (TP), false positive (FP), false negative (FN), and negative (TN) counts. CSI, HSS, and FSS are calculated by these counts as follows:
Critical Success Index. CSI is metric that evaluates the proportion of correctly predicted events of hits among conditions, including hits (TP), false alarms (FN), and misses (FP). The formulation of CSI is:
TP
CST TP+
The value of CSI ranges from 0 to Higher values indicate better prediction accuracy.
2) False Alarm Ratio. The FAR metric quantifies the proportion of predicted positive events that were actually negative in metcorological verification, emphasizing the reliability of alarm triggers. It is
28

---

Table 11: The details of the MMAE on encoded multimodal tokens within a sub-image in a time window 12. Convi X is the 2D convolutional layer with 1 x kernel. (c1, C2, C3 C4) means multimodal input list with input channels C1, C22 C3, and Ca. The MaskTokens is similar to the function of random mask in 34 while adding the [EOS] tokens to keep the sequences from different samples the same length. The operator of Padding Tokens fills the feature map as [34] did.
The FFN consists two Linear layers separated by a GeLU activation layer
Module
Layer
Resolution
Channels
Multimodal Input
x x 12
(C1. C3 Ca)
Conv1
9 X 12
(c1. 02, C4) + (768, 768, 768, 768)
Flatten
9 x 9 12 - 81 12
(768, 768, 768, 768)
PosEmbed
81 12
(768, 768, 768, 768)
Multimodal PatchEmbed
TemporalEmbed
81 12
(768, 768, 768, 768)
Rearrange
81 12 -> 3888
(768, 768, 768, 768) - 768
Random Masking
MaskTokens
3888 + 128
768
LayerNorm
128
768
XaskAttention
128
768
Trnasformer Block x
LayerNorm
128
768
FFR
128
768
Linear
128
768 - 512
PaddingTokens
128 3888
512
Feature Map Filling
Rearrange
3888 81 12
512
PosEmbed
81 x 12 x
512
CmmporalEmbed
81 x 12 x 3888
512
Rearrange
81 x 12 3888
512
LayerNorr
512
Attention
3888
512
Trnasformer Block x
LayerNorm
3888
512
FFR
3888
512
Rearrange
3888 972
(512, 512, 512, 512)
LayerNorm
972
(512,512, 512,512)
Multimodal Out
Linear
972
(c1. C2 C3. eq)
Rearrange
972 9
12
(3, ca)
Table 12:
The details of our AIWP module training in the assimilated space. The inputs of this module are sub-images with neighbours in a multimodal way. (CI, C2 C3, C4) means a multimodal input list with input channels Cı, C2 ca, and c4. Triew and Sview indicate the temporal dimension and the spatial dimension as the sequence, respectively. The FFNwithSwiGH. consists of two Linear layers separated by SwiGLD activation layer (58)
Module
Layer
Resolution
Channels
Input with Conditions
27 x 27 x 12
(G), O2 Ca) +
Concat
27 x 27 x 12
(4,, C2 Cg. C4) + C1
02 + + C4
PatchEmbed
Linear
27 x 27 x 12
+ C3
- 768
Rearrange
27 27 12 - 729 x 12
768
PosEmbed
729 12
768
Tview
729
12
(729x)12
768
Attention
(729 12
768
LayerNorm
(729 12
768
FFNwithSwiCLU
(729 12
768
LayerNorm
(729 12
768
TS Block x 12
Sview
(729x) (12x)729
768
Attention
(12x)729
768
LayerKorm
(12x)
768
PENwithSwiCLU
(12x 729
768
LayerNorm
[12x 7720
768
Rearrange
(12x - x x 12
768
Multimodal Out
LayerNorm
7 x 27 x 12
768
Linear
972
(C), Oz C3 Ca)
29

---

Table 13: Computation cost during inference.
Inference time(ms)
ParametersMMB)
Memory(MB)
Batch size(per GPU)
Mask-ViT-VAE
53
96
7262
50
AIDA
310
105
18242
12
AIWP
491
216
47134
2
defined as:
FP
FAR FP+TP where FP denotes false positive predictions (e.g.. forecasted rainfall with no ground observation) and TP represents true positives (correctly predicted rainfall events). FAR ranges from 0 (perfect reliability) to (all alarms are false), with lower values indicating better prediction specificity.
(4)
H.2 MAE
To evaluate the accuracy of direct observation predictions, we use pointwise Mcan Absolute Error (MAF) as the metric to calculate errors on the ground truth with variable missing values. It is worth noting that the MAE is calculated with the raw observation point by point to ignore the influence of missing values. It is defined as:
MAE 1
Σ M 9e
(S)
is the total number of points with observation, is the ground truth at the jth location, and 9J is the prediction.
I More results of direct observation predictions
30

---

AMSU A channel0
AMSU A thannell
AMSU A channel2
AMSU A channel3
2.5
2.0
MAB (c+01)
6.0-
MAE (e-04)
NAE (e-04)
MAE (e-04)
0.8-
1.5
1.5)
4.5
1.0
0.4
1.5
40
0.5
thours)
(hours) AMSU-A channel5
(hours)
(hours)
AMSU-A channel4
AMSU-A channel66
AMSU-A channel7
10-
2.0
MAE (e-04)
1.5
2.8
(e
1.0
MAE
1.2-
0.8
0.6
0.5
00
00
20
Lead time (hours)
time (hours)
Lead time (hours)
Lead time (hours)
AMSU-A channell
channelg
AMSU-A Channel10
AMSU-A channer11
2.5
12.5
10.0
MAA (e-05)
20.0
MME (e+-01)
MAE (e-04)
1.2
7.5
1.5
7.5-
0.8
5.0-
5.0-
2.5
2.5
0.5
40
60
60
20
Lead time thours)
Lead time (hours)
Lead time Thours]
Lead time (hours)
AMSU-A
3.0
3.0
2.4
MAL (0-00)
HAC je 04)
1.0
1.2
2.
0.6
20
40
60
20
40
60
20
40
60
Lead time (haurs)
Lead time (hours)
Load time (hours)
DAWP(ours)
EarthNet
Persistence
Transfomer:DOP
Figure 10: Curves of MAE for the prediction of different channels in sensor AMSU-A. The max leadtime 72h with 1h temporal resolution.
31

---

ATMS channelO
ATMS channell
ATMS channel2
ATMS channel3
1.6-
12.5
12.5
9.0-
MAE (++01)
1.2
MAE (++00)
20.0
MAE (e+000)
10.0
MAE (e+00)
7.5-
6.0
0.8-
7.5-
7.5-
5.0-
5.0-
2.5
0
20
40
60
5
20
40
20
40
20
40
60
Lead time (hours)
Lead time (hours)
Lead time (hours)
Lead time (hours)
ATMS channel4
ATMS channel5
ATMS channel6
ATMS channel7
(e+00)
MIE (e+00)
MAE (e+00)
MAE (c+00)
1.5
20
40
60
0
20
40
60
0
20
40
60
0
40
60
Lead time (hours)
Lead time (hours)
Lead time (hours)
Lead time (hours)
ATMS channel8
MAE (e+00)
c-
40
62
04
40
Lead time (hours)
Transformer-DOP Figure 11: Curves of MAE for the prediction of different channels in sensor ATMS. The max leadtime 72h with a temporal resolution.
DAWP(ours)
EarthNet
Persistence
32

---

HIRS channelO
HIRS channell
HIRS channel2
HIRS channel3
4.0-
0.7
3.2-
MAE (e+01)
(e+00)
3.2
(c+00)
2.4
0.6
2.4
0.3
2.6-
1.0-
1.0
0.4
0.8
(hours)
(hours) HIRS channelS
HIRS channel4
HIRS channel6
HIRS channel7
10-
12
15.0-
MAE (c+00)
MAE (c+00))
MAE (c+00)
10-
12.5
4.5
10.0
7.5-
(hours)
(hours) HIRS channel9
HIRS channel8
HIRS channel10
HIRS channell1
9.0
0.30
0.75
MAE (e+00)
(c+01)
MAE (e=00)
0.24
12.5
0.60-
6.0-
20.0
0.18
0.45-
3.0
(hours)
(hours) HIRS channei13
HIRS channel12
HIRS channell4
HIRS channel15
0.40
0.10
0.12
0.35
0.10
(e400)
0.04
0.30
0.12
0.08-
MAE
8.03-
0.25
0.09
0.02
0.20
0.86
(hours)
(hours) HIRS channel17
HIRS channel16
HIRS channel18
HIRS channel19
0.21
0.24
0.18
MAE (e400)
0.20
0.15
(e+00)
0.15
0.16
0.12
4.5-
0.12
0.12
0.09
0.09
d0
1.5
60
Leed (hours)
(hours)
(hours)
Transfomer:DOP Figure 12: Curves of MAE for the prediction of different channels in sensor HIRS. The max leadtime 72h with lh temporal resolution.
DAWP(ours)
EarthNet
Persistence
33

---

MHS channel0
MHS channell
MHS channel2
MHS channel3
9.0-
2.0
2.4-
MAB (e+01)
7.5-
MAE (e-03)
MAE (e-03)
1.8-
2.0
MAE
1.2
1.6-
1.2
3.0
0.8
1.2
0.6
20
60
40
40
20
Load time thours!
Load time thours]
Lead time thours!
Lead time thours]
2.4
MAE (e-03)
1.8
1.2
0.6
thours!
DAWP(ours)
EarthNet
Persistence
Tranformer-DO0
Figure 13: Curves of MAE for the prediction of different channels in sensor MHS. The max leadtime 72h with a temporal resolution.
AMSU-A chogleead time=12hour
AMSU-A chogglead time:24hour
AMSU- chogllaad time:3éhour
AMSL-A chOgglead time:48hour
OC20J 0.2002 0E003 D0111 90112 9caas occas ocoLs D0lLs
Figure 14: A visualization of rollout predictions for channel O of AMSU-A. From top bottom the results of ground truth, Transformer-DOP. EarthNet, and DAWP (ours).
34

---

chogglaad time 2hour
chogglead time:24hour
HIRS choiglead time:36nour
HIRS chogrlead time 48hour
120
Figure 15: A visualization of rollout predictions for channel of HIRS. From top to bottom the results of ground truth, Transformer DOP, EarthNet, and DAWP (ours).
MHS chogleed time:12hour
MHS chogleed time:24hour
MHS chogglead time:3ehour
MH5 chomllea time:48hour
0200
cole 0.17
001
0.72
Figure 16: A visualization of rollout predictions for channel 0 of MHS. From top bottom the results of ground truth, Transformer-DOO EarthNet, and DAWP (ours).
35

---

chogllead time:12hour
ATMS ch0 ead time:24hour
ATMS chogleea tEhorr
ATMS chogelead time:48hour
200
300
Figure 17: A visualization of rollout predictions for channel 1 of ATMS. From top to bottom the results of ground truth, Transformer DOP, EarthNet, and DAWP (ours).
AMSU-A choglead Lime11hhurr
AMSU-A chDglead time:24hour
AMSU- chogleed time:3ehour
AMSL chogllead times48hour
OCMI4 0501 ocona 0102 3002 00524 ousie
Figure 18: A visualization of rollout predictions for channel I of AMSU-A. From top bottom the results of ground truth, Transformer-DOP. EarthNet, and DAWP (ours).
36

---

chogglaad time :1 2hour
HIRS chogglead time:24hour
HIRS choiglead time:36nour
HIRS chogrlead time 43hour
50 101
M75
Figure 19: A visualization of rollout predictions for channel 10 of HIRS. From top to bottom the results of ground truth, Transformer DDOP EarthNet, and DAWP (ours).
MHS chogleed time:12hour
MHS chogleed time:24hour
MHS chogglead time:3ehour
MH5 chomllea time:48hour
0530 C0SS 0003 Gos cose cass sise 0res
Figure 20: A visualization of rollout predictions for channel I of MHS. From top bottom the results of ground truth, Transformer-DOP EarthNet, and DAWP (ours).
37


Reviews:
[{"review_id": "rOh9Rt5NGf", "reviewer": "Reviewer_HaE5", "summary": "The proposed data assimilation module based on a mask multi-modality autoencoder (MMAE) integrates multiple sources of observation data and transforms them into a uniform observation space. After data assimilation, it is treated as a standard spatiotemporal forecasting problem. In particular, in the forecasting stage, tokens from neighboring areas are padded to the tokens of the predicted region. In this way, the cross-regional boundary information of the border area is passed to the central forecasting region to make the forecasting smooth. Experiments are performed to demonstrate the good performance of the proposed method.", "questions": "1. As the data assimilation modular applies the multi-modal masked autoencoder, it can process different types of data, such as the observation data from weather stations. I would suggest the authors to expand it to weather station data.\n2. Compare with more forecasting baselines.", "limitations": "Yes.", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 3, "significance": 3, "originality": 2, "strengths_and_weaknesses": "Strengths:\n1. The paper addresses an interesting problem and proposes a practical approach. By leveraging the observation data directly, the proposed method achieves good forecasting performance. \n2. The data assimilation module based on multi-modal masked autoencoder sounds good. As the key contribution of this paper is the data assimilation modular, I would recommend highlight the general role of the data assimilation for different weather forecasting settings. \n3. Experiments performed are well designed, such as visual comparison of cross-region boundary. \n\nWeaknesses:\n1. Technically, the forecasting stage is treated as a standard spatio-temporal forecasting problem. Except the cross-region boundary conditioning, the forecasting algorithm looks mediocre. \n2. The compared baseline in forecasting is limited. Except Persistence (the naïve method which uses the last observation as the prediction), only EarthNet and Transformer-DOP are compared. It is encouraged to compare more baselines such as EarthFormer.", "was_revised": true, "final_justification": "The extensive experiments in the rebuttal include several baselines I have requested. And the experimental results clearly show the advange of the of the proposed method.", "rebuttal": "We sincerely appreciate your constructive and detailed feedback. Thank you for your positive evaluation of our work’s novelty, methodological rigor, performance, and overall contribution. Your recognition that our paper addresses an interesting problem and proposes a practical approach is highly encouraging.\n\n___\n\n> **Response to Q1: More Types of Data**\n\nThank you for your helpful suggestion. We are currently exploring the possibility of extending DAWP to incorporate point-cloud MAE pretraining[1] combined with multi-modal MAE (MMAE)[2] for integrating weather station data. However, due to constraints in data collection, preprocessing, and ttraining ime, we were unable to include any experimental results in our rebuttal. We will add these experiments in the future version of our DAWP and include discussions into the revised manuscript.\n\n>  **Response to W2&Q2: Comparisons with More Baselines**\n\nThank you for your suggestion. Due to length constraints in the main manuscript, we have moved the comparisons with additional baselines to the **Supplementary Material**. There, we include quantitative results comparing DAWP against six classical spatio-temporal forecasting models including RNN‑based, CNN‑based and Transformer‑based ones. We will ensure that future versions of our paper prominently present this baseline comparison in the main text, to better contextualize the strengths of our DAWP framework.\n\n| Lead time: 0-12h | AMSU-A-ch0 | AMSU-A-ch1 | ATMS-ch0 | ATMS-ch1 | HIRS-ch9 | HIRS-ch10 | MHS-ch0 | MHS-ch1 |\n|------------------|------------|------------|----------|----------|----------|-----------|---------|---------|\n| Persistence[3]      | 5.86       | 9.15       | 14.37    | 11.69    | 12.43    | 2.21      | 7.01    | 14.74   |\n| ConvLSTM[4]         | 127.82     | 208.9      | 73.21    | 78.44    | 77.4     | 11.05     | 62.27   | 158.57  |\n| PredRNN[5]          | 2.95       | 4.96       | 6.99     | 6.21     | 9.26     | 1.42      | 4.11    | 10.2    |\n| RainFormer[6]       | 3.95       | 6.52       | 9.08     | 8.05     | 10.41    | 1.63      | 4.98    | 11.69   |\n| EarthFormer[3]      | 18.97      | 33.94      | 37.33    | 38.75    | 22.49    | 3.62      | 18.00      | 55.04   |\n| SimVP[7]            | 3.61       | 6.02       | 7.29     | 6.61     | 9.84     | 1.53      | 4.68    | 11.35   |\n| TAU[8]              | 3.84       | 6.31       | 7.70      | 6.86     | 9.94     | 1.58      | 4.87    | 11.42   |\n| EarthNet[9]         | 2.93       | 4.89       | 6.96     | 6.14     | 9.15     | 1.39      | 3.96    | 9.65    |\n| Transformer-DOP[10]  | 2.67       | 4.48       | 6.40      | 5.61     | 9.22     | 1.42      | 3.91    | 9.48    |\n| **Ours**             | 1.92       | 3.39       | 3.36     | 3.27     | 7.7      | 1.12      | 3.07    | 7.91    |\n\n| Lead time: 12-24h | AMSU-A-ch0 | AMSU-A-ch1 | ATMS-ch0 | ATMS-ch1 | HIRS-ch9 | HIRS-ch10 | MHS-ch0 | MHS-ch1 |\n|-------------------|------------|------------|----------|----------|----------|-----------|---------|---------|\n| Persistence[3]       | 4.35       | 6.94       | 10.40     | 8.86     | 13.58    | 2.30       | 6.07    | 15.09   |\n| ConvLSTM[4]          | 128.13     | 211.14     | 81.22    | 82.6     | 71.64    | 9.98      | 59.26   | 145.44  |\n| PredRNN[5]           | 3.85       | 6.26       | 11.14    | 8.82     | 10.92    | 1.84      | 5.48    | 13.16   |\n| RainFormer[6]        | 11.46      | 15.72      | 19.43    | 17.76    | 18.75    | 3.28      | 11.74   | 32.10    |\n| EarthFormer[3]       | 18.98      | 33.96      | 37.33    | 38.76    | 22.50     | 3.62      | 18.01   | 55.03   |\n| SimVP[7]             | 4.83       | 7.74       | 11.48    | 9.10      | 11.49    | 2.02      | 6.43    | 14.2    |\n| TAU[8]               | 4.46       | 7.21       | 11.46    | 9.06     | 11.65    | 2.04      | 6.24    | 13.94   |\n| EarthNet[9]          | 4.12       | 6.65       | 11.25    | 9.00        | 11.14    | 1.98      | 5.46    | 13.11   |\n| Transformer-DOP[10]   | 3.84       | 6.14       | 10.04    | 8.04     | 11.08    | 1.95      | 5.19    | 12.65   |\n| **Ours**              | 3.11       | 5.12       | 7.35     | 6.35     | 9.57     | 1.54      | 4.51    | 10.54   |\n\n| Lead time: 24-36h | AMSU-A-ch0 | AMSU-A-ch1 | ATMS-ch0 | ATMS-ch1 | HIRS-ch9 | HIRS-ch10 | MHS-ch0 | MHS-ch1 |\n|-------------------|------------|------------|----------|----------|----------|-----------|---------|---------|\n| Persistence[3]       | 6.39       | 9.84       | 15.37    | 12.52    | 14.61    | 2.61      | 8.00       | 17.86   |\n| ConvLSTM[4]          | 128.42     | 211.83     | 82.03    | 85.17    | 72.62    | 10.15     | 60.44   | 148.62  |\n| PredRNN[5]           | 4.58       | 7.23       | 12.18    | 9.69     | 12.03    | 2.08      | 6.24    | 15.04   |\n| RainFormer[6]        | 24.89      | 32.34      | 35.49    | 36.54    | 30.78    | 4.92      | 21.51   | 69.68   |\n| EarthFormer[3]       | 18.98      | 33.96      | 37.34    | 38.77    | 22.51    | 3.63      | 18.02   | 55.04   |\n| SimVP[7]             | 5.72       | 8.91       | 12.93    | 10.33    | 12.71    | 2.32      | 73.95   | 15.99   |\n| TAU[8]               | 5.61       | 8.76       | 13.23    | 10.53    | 12.98    | 2.34      | 7.15    | 15.85   |\n| EarthNet[9]          | 5.17       | 8.14       | 12.52    | 10.08    | 12.37    | 2.36      | 6.41    | 15.08   |\n| Transformer-DOP[10]   | 4.91       | 7.54       | 11.35    | 9.07     | 12.39    | 2.27      | 6.22    | 14.70    |\n| **Ours**              | 3.66       | 5.80        | 7.84     | 6.81     | 10.71    | 1.79      | 5.15    | 12.22   |\n\n> **Response to W1: Design Choice of Forecasting Model**\n\nThank you for recognizing our key contributions of a data assimilation stage and a forecasting stage for direct observation–based prediction, as highlighted in the review strengths. Although the forecasting model is simple, it still effectively supports the point that data assimilation initialization can aid direct observation forecasting. Moreover, our forecasting model efficiently resolve conflicts of input areas between regional data assimilation and global forecasting through the cross-region boundary conditioning.  \n\nIn addition, within the DAWP framework, the forecasting algorithm is flexible and has the potential to incorporate with more specialized forecasting methods in future work. \n\n\n> **Response to S2: Highlight DA's Role**\n\nThank you for your valuable suggestion. The general role of Data Assimilation (DA) is to integrate observational data. Specifically, in physics-based numerical weather prediction systems, DA incorporates real-time observations into the model state to provide more accurate initialization for forecasting.\nIn the context of Direct Observation-based Prediction, we leverage DA to fuse sparse observational inputs into dense, forecast-compatible representations. This helps mitigate key challenges such as the input–output mismatch, which often leads to inefficient roll-outs and unstable training dynamics. Following your suggestion, we will highlight the general role of DA more clearly in revised versions of the paper to better clarify its key contribution to our framework.\n\n___\n\nWe appreciate your feedback once again, which has helped us strengthen the overall presentation and completeness of the paper. If further clarification is needed, please do not hesitate to mention it, and we will promptly address your inquiries. We look forward to receiving your feedback.\n\nReferences:\n\n[1] Hess, Georg, et al. \"Masked autoencoder for self-supervised pre-training on lidar point clouds.\" Proceedings of the IEEE/CVF winter conference on applications of computer vision. 2023.\n\n[2] Mizrahi, David, et al. \"4m: Massively multimodal masked modeling.\" Advances in Neural Information Processing Systems 36 (2023): 58363-58408.\n\n[3] Gao, Zhihan, et al. \"Earthformer: Exploring space-time transformers for earth system forecasting.\" Advances in Neural Information Processing Systems 35 (2022): 25390-25403.\n\n[4] Shi, Xingjian, et al. \"Convolutional LSTM network: A machine learning approach for precipitation nowcasting.\" Advances in neural information processing systems 28 (2015).\n\n[5] Wang, Yunbo, et al. \"Predrnn: A recurrent neural network for spatiotemporal predictive learning.\" IEEE Transactions on Pattern Analysis and Machine Intelligence 45.2 (2022): 2208-2225.\n\n[6] Bai, Cong, et al. \"Rainformer: Features extraction balanced network for radar-based precipitation nowcasting.\" IEEE Geoscience and Remote Sensing Letters 19 (2022): 1-5.\n\n[7] Gao, Zhangyang, et al. \"Simvp: Simpler yet better video prediction.\" Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 2022.\n\n[8] Tan, Cheng, et al. \"Temporal attention unit: Towards efficient spatiotemporal predictive learning.\" Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 2023.\n\n[9] Vandal, Thomas J., et al. \"Global atmospheric data assimilation with multi-modal masked autoencoders.\" arXiv preprint arXiv:2407.11696 (2024).\n\n[10] McNally, Anthony, et al. \"Data driven weather forecasts trained and initialised directly from observations.\" arXiv preprint arXiv:2407.15586 (2024)."}, {"review_id": "90JRA8EJsM", "reviewer": "Reviewer_Ht96", "summary": "This work proposes a novel framework to do the weather forecast directly from the observation space. The framework consists of two steps: (a) Initializing reanalysis data and data assimilation are substituted by an artificial intelligence assimilation module via multi-modality masking, (b) Then a regional boundary conditioning is used to learn the spatio-temporal correlations between irregular and high-resolution satellite observations, resulting in sub-image global observation forecasting.", "questions": "Questions:\n\n- Don’t we also lose valuable information if you don’t use reanalysis? Is there any experiments to support the main claim of the manuscript?\n- What happens if you remove the analysis observation or the AIDA and predicts directly the output without griding?\n- L138: How were these values defined? The fixed time window of 12 and a sub-image of 144×144?\n- L175-176: “The ViT encoder/decoder provides better compression capability, as detailed in Appendix C, compared to SD-VAE for satellite observation”, is there any experiments to support this claim?\n- How about Graphdop I think it is an important baseline?\n- Table 5 looks like the MAE is lower for SD-VAE for higher values while small fluctuation for smaller values like AMSU-A and HIRS. What is the reason and are these results reported with different random seeds?\n- L656-687: I don't think this mimics a real operational setting. Are these satellite measurements available at near-real time?\n\nMinor:\n\n- L93: Do we need 35TB, for a 12×1152×2304 resolution?\n- Describing the method is very short.\n- The manuscript uses a lot of uncommon abbreviation like AIDA, AIWP, and transformer-dop.\n- L197: transformer-dop? The reference what model is used here?\n- The qualitative results don’t show anything, it is better to have a difference to ground truth as a comparison rather than plotting the absolute value over the full domain.\n- Figure 5 looks weird the model doesn’t learn at all if you remove AIDA.", "limitations": "Would the framework work for very sparse observations data? This should be mentioned in the limitation.", "rating": 3, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 2, "significance": 2, "originality": 2, "strengths_and_weaknesses": "Strengths:\n\n- This work addresses a critical problem in the weather forecast especially suited for precipitation forecast.\n- The idea of the proposed framework is clear for the reader.\n\nWeaknesses:\n- The manuscript argues that using reanalysis and data assimilation as initial condition has shortcomings and the main idea of this work is to overcome this issue. However, there was no comparison to baselines trained directly on reanalysis data.\n- The paper does not have a validation set but rather optimized directly on the test set. I assume that ablation studies are done on the test set as well instead on the validation one. In addition, the test set is very short and covers only 3 months. In my opinion this split is inadequate and the test set does not ensure a representative evaluation.\n- The improvement is not statistically significant (i.e., the results are generated by one random seed).\n- The baselines were trained with sub-image and not with their own original implementations. The artifacts on the grid are obvious from Fig. 4.\n- Experimental result reproducibility: The manuscript does not provide details about the baselines.\n- The manuscript does not provide sufficient information on the computer resources (type of compute workers, memory, time of inference) needed to reproduce the experiments, parameters and model size.", "was_revised": true, "final_justification": "The paper presents a new deep learning model and a new protocol to learn directly from the observation space. \n\nFirst, the idea of learning directly from observations has been introduced by Vandal et al. 2024 (EarthNet) and McNally et al. 2024 (Transformer-DOP) and followed by GraphDOP by Alexe et al. 2024. I do not think the novelty is high enough regarding this aspect. \n\nSecond, in my opinion, the adaptation of the baselines is not suitable and the evaluation can be misleading. The paper gives a false impression that the baselines are the ones from Vandal et al. 2024 and McNally et al. 2024 . However, they were re-implemented and tested in a different setting than their own original protocols (see weaknesses 4 and 5). There is no information about the baselines other than the number of layers which the authors provided during the discussion. There are no details about the baselines in the supplementary too. What I want to say is that whatever baseline is being tested here, it will probably fail and generate artifacts because of the sub-images approach and the new protocol that the baselines were not designed for in the first place. I believe that the baselines should have been evaluated in a fair setting e.g., following their original protocols to generate results.\n\nGiven these remaining issues I tend toward a reject.", "rebuttal": "We sincerely appreciate your thoughtful and constructive feedback. Thank you for recognizing the novelty and clarity of our proposed framework.\n\n**OUR CORE CONTRIBUTION:**\n\nWe **identify a key challenge** in observation-based forecasting: the **mismatch between sparse observations and dense prediction**. DAWP mitigates this mismatch via **a two-stage solution**: AIDA for initialization and AIWP for forecasting. Figure 6 provides empirical evidence.\n\n---\n\n> **Response to Q1: Main claim of DAWP and Relationship between Reanalysis or Observation**\n\nThank you for the thoughtful question. Our main claim is that DOPs trained in the space initialized by AIDA improve prediction, as stated in Figure 1, lines L54–56 and **OUR CORE CONTRIBUTION**. Table 2 and Figure 6 support this claim. \n\nWe agree that Reanalysis models benefit from human priors in NWP (physical laws, bias correction, assimilation). However, exploring data‑driven forecasting remains scientifically valuable and timely. DOP extends the traditional reanalysis-based forecasting paradigm into a heterogeneous, spatiotemporal observational space, encompassing multiple data modalities—an area that remains underexplored to date. \n\n> **Response to Q2: Results without AIDA.**\n\nThank you for raising this important point. Results without AIDA are shown in Figures 5 and 6.\n\n**Learning dynamics**: When AIDA is not applied, learning dynamics are unstable for our spatial-temporal prediction model, producing severe spikes in learning phase (Fig.5) due to the irregular and sparse nature of input. AIDA stabilizes learning by mapping sparse inputs into a dense, structured latent space.\n\n**Roll-out**: Without AIDA, roll-out degrades because sparse training inputs mismatch dense inference inputs in roll-out (Fig. 6).\n\n> **Response to Q3: Window Length and Sub-image Size**\n\nA 144×144 sub-image with patch size 16 over 12 steps and 4 domains yields 3884 tokens. This length fits our transformer and computational budget, ensuring efficient attention. \n\nThe 12-step window matches the satellite revisit cycle, ensuring global spatial coverage.\n\n> **Response to Q4: Experiments to Support ViT-VAE's Capability**\n\nThank you for the suggestion. The comparisons are presented in Appendix C. When compared against SD‑VAE, our MaskViTVAE model achieves approximately 12 % relative improvement in reconstruction loss.\n\n> **Response to Q5: GraphDOP**\n\nGraphDOP is an excellent project. With ECMWF's data infrastructure and preprocessing capabilities, it was trained on 32 observation types, broader than  DAWP’s 4 types. Additionally, pecific implementation details are not clearly disclosed. Without access to the codes, the direct comparison is currently infeasible. We are willing to evaluate it using data available to us when GraphDOP is open-sourced. \n\n> **Response to Q6: Reasons for Different Improvments and VAEs with Random Seeds**\n\nIn Table 5, performance variation is due to channel count differences, not data scale. All datasets are normalized, so value ranges do not affect results. Gains follow channel counts (HIRS 20 > AMSU-A 15 > ATMS 9 > MHS 5). Our method excels on high-channel data, while SD-VAE is more suited to low-channel cases. Thus, DAWP provides stronger compression for high-dimensional satellite observations.\n\nWe also tested four seeds (0–3), retraining models and reporting mean ± std. The results show our method’s gains are robust to random seed choice.\n\n\n||AMSU-A(1e-3)|ATMS(1e-2)|HIRS(1e-3)|MHS(1e-2)|\n|-|-|-|-|-|\n|SD-VAE|1.08±0.00|1.37±0.02|5.94±0.08|2.40±0.02|\n|Ours|0.76±0.01|1.16±0.05|3.88±0.11|2.48±0.05|\n\n\n> **Response to Q7:  Near-realtime Availability**\n\nWe use satellite Level‑1 products, available immediately after transmission; for instance, EUMETSAT[1] provides MHS with <60 min latency.\n\n\n> **Response to W1: Main Idea and Baselines on Reanalysis**\n\nWe thank the reviewers and clarify our main points. The core contribution of DAWP is the design of a satellite‑observation prediction framework—comprising AIDA and AIWP—to directly predict satellite observational quantities, rather than comparing with reanalysis-based forecasts.\n\n**Data**: DAWP predicts satellite radiances (microwave, brightness temperature, infrared). In contrast, the reanalysis‑based baselines forecast physical variables like wind and temperature. As these predictions involve different modalities, direct comparison is infeasible.\n\n**Architecture**: Reanalysis-based models lack designs to process sparse, irregular satellite inputs, hence not directly applicable for predicting satellite observables.\n\n\n> **Response to W2&W3: Dataset Split and Results with Random Seeds**\n\nThank you for your suggestions. Originally: Training: 2012.1 – 2022.6, Validation: 2022.7 – 2023.4, Testing: 2023.5 – 2023.7. All experiments strictly followed this training–validation–testing partitioning protocol. This 3-month test follows GraphDOP (which used only 1 month in Section 5.2) and EarthNet (which used 2 months in Section E.2). Following your suggestion, we revised to: Train 2012.1–2021.7, Val 2021.8–2022.7, Test 2022.8–2023.7.\n\nWe retrained all models on the revised split with seeds {0,1,2,3}; results (mean±std) confirm statistical significance. We thank you again and will include this extended evaluation.\n\n|Lead time: 0-12h |ATMS0|ATMS1|AMSU-A0(e-5)|AMSU-A1(e-5)|MHS0(e-4)|MHS1(e-4)|HIRS9|HIRS10|\n|-|-|-|-|-|-|-|-|-|\n|EarthNet|7.09±0.18|6.13±0.15|2.52±0.04|4.35±0.07|3.91±0.04|9.86±0.06|8.60±0.02|1.31±0.01|\n|Transformer-DOP|6.19±0.19|5.46±0.12|2.37±0.01|4.10±0.02|3.78±0.17|9.56±0.01|8.55±0.02|1.31±0.00|\n|Ours|**3.25±0.01**|**3.17±0.00**|**1.76±0.00**|**3.22±0.02**|**3.07±0.02**|**8.06±0.04**|**7.29±0.02**|**1.05±0.00**|\n\n|Lead time: 12-24h|ATMS0|ATMS1|AMSU-A0(e-5)|AMSU-A1(e-5)|MHS0(e-4)|MHS1(e-4)|HIRS9|HIRS10|\n|-|-|-|-|-|-|-|-|-|\n|EarthNet|11.29±0.43|9.06±0.09|4.07±0.09|6.56±0.08|5.35±0.02|13.42±0.06|10.31±0.05|1.85±0.03|\n|Transformer-DOP|10.02±0.37|7.99±0.19|3.65±0.02|5.86±0.05|5.23±0.06|12.94±0.08|10.20±0.05|1.78±0.01|\n|Ours|**7.97±0.34**|**6.59±0.19**|**3.08±0.07**|**5.04±0.09**|**4.48±0.08**|**10.94±0.06**|**9.07±0.04**|**1.48±0.01**|\n\n|Lead time: 24-36h|ATMS0|ATMS1|AMSU-A0(e-5)|AMSU-A1(e-5)|MHS0(e-4)|MHS1(e-4)|HIRS9|HIRS10|\n|-|-|-|-|-|-|-|-|-|\n|EarthNet|12.55±0.49|10.18±0.18|5.13±0.13|8.04±0.11|6.42±0.08|16.05±0.16|11.65±0.11|2.23±0.06|\n|Transformer-DOP|11.48±0.43|8.98±0.25|4.57±0.05|7.06±0.07|6.32±0.07|15.31±0.12|11.36±0.08|2.08±0.09|\n|Ours|**8.73±0.11**|**7.16±0.06**|**3.64±0.07**|**5.73±0.10**|**5.20±0.08**|**13.02±0.13**|**10.26±0.03**|**1.75±0.01**|\n\n\n> **Response to W4&W5: Baselines Implementations**\n\nThank you for your constructive feedback. There is no open‑sourced code for EarthNet[2] or Transformer‑DOP[3]. For EarthNet, it follows the implementation of MultiMAE[4] as detailed in EarthNet’s Appendix C and D. Therefore, we reproduce it on our datasets following MultiMAE. As for Transformer‑DOP, since the original paper presents only a sketch without details, we implemented it according to our best available understanding. Specifically, EarthNet is reproduced as a 12‑layer encoder (hidden dimension 768) paired with an 8‑layer decoder (hidden dimension 512), and Transformer‑DOP is implemented as a transformer consisting of 18 layers (hidden dimension 1024). We employs sub‑images because the full 12‑hour global observation sequence would result in 124k-token sequence, which is computationally infeasible.\n\nArtifacts (Fig. 4) arise from (1) mismatch between dense autoregressive inference and sparse training and (2) lack of cross‑region info in sub‑images. Our AIDA initialization and CBC module resolve these problems.\n\n> **Response to W6: Computer resources**\n\nThank you for your suggestion. In our experiments, we used 4 A100 80G GPUs for parallel training. In the table below, we present other details, which would be added in the future version.\n||Inferennce time(ms)|Parameters(MB)|Memory(MB)|Batch size(per GPU)|\n|-|-|-|-|-|\n|Mask-ViT-VAE|53|96|7262|50|\n|AIDA|310|105|18242|12|\n|AIWP|491|216|47134|2|\n\n> **Response to M1: 35TB Observations**\n\nEach observation is a 12×1152×2304 grid with spectral channels (ATMS‑9, AMSU‑A‑15, MHS‑5, HIRS‑20). With two observations per day over 11 years, the dataset totals ~35 TB.\n\n> **Response to M2: Method Details**\n\nIn the Methods section, we emphasize the conceptual pipeline combining AIDA and AIWP to address prediction based on irregular and sparse satellite observations. Model architecture details (layers, dimensions, attention, tokenization) are in Tables 9–11. We will expand methodological descriptions and design rationale in future work.\n\n> **Response to M3: Uncommon Abbreviation** \n\nWe would like to clarify that Artificial Intelligence Data Assimilation (AIDA) and Artificial Intelligence Weather Prediction (AIWP) are both expanded and carefully defined in L2 and L11, respectively. Transformer‑DOP denotes a Transformer backbone for the DOP task (Ref. 16). We are open to clearer alternative naming suggestions\n> **Response to M4: Explanations about Transformer–DOP**\n\nTransformer‑DOP refers to Ref. 16. As the paper provides only a high-level description, we reproduced it based on its conceptual design.\n> **Response to M5: Visualization** \n\nThank you for the suggestion. Our aim is to emphasize the complementation capability of DAWP for handling irregular and sparse satellite observations, especially in comparison to other methods that suffer from artifacts arising from sub-image tiling and mismatched input-output density. In future versions, we will incorporate visualizations of prediction–truth differences.\n\n___\nThank you for your thorough review, which improved our paper. We believe that some minor points can be better addressed during the discussion phase. We welcome any further questions and will respond promptly.\n\nReferences:\n\n[1] https://data.eumetsat.int/extended?query=MHS \n\n[2] arXiv:2407.11696\n\n[3] arXiv:2407.15586\n\n[4] MultiMAE (ECCV'2023)"}, {"review_id": "kMTAxXKkJu", "reviewer": "Reviewer_RPch", "summary": "Recent data-driven weather prediction methods often rely on reanalysis data, which suffer from issues such as data assimilation biases and temporal discrepancies. As an alternative, observation-based forecasting is gaining attention due to its potential to overcome these limitations. This study focuses on learning the spatiotemporal dynamics of observational data. To this end, the paper proposes DAWP, which leverages the AIDA module to enable AI-based weather prediction to learn within a uniform feature space. The AIDA module specifically addresses the issue of missing data commonly encountered in real-world observations by employing a Masked Autoencoder architecture. Additionally, the paper introduces cross-regional boundary conditioning (CBC) to resolve the boundary issues that typically arise in methods based on reanalysis or grid-structured datasets, which are common in the Climate AI community.", "questions": "- I agree with the claim that “Temporal Discrepancies: The temporal lag between direct observation acquisition (nearly real-time) and analysis data generation (up to six hours) severely degrades the quick response ability of AIWP models.” However, further investigation into research that recognizes this as a problem and aims to address it is necessary. For example, studies such as \"LT3P\" [1] have explored this issue.\n\n- The sentence, “As AIWP models do not require physical solvers to predict the evolution of the atmosphere, there is potential for them to directly predict atmosphere states in real-world observation space,” raises some concerns. Are AIWP models truly independent of physical solvers? Isn’t there potential for them to be used in a complementary manner? This could be considered an overclaim. You can refer to \"ClimODE\" [2].\n\n- In the “Direct Observation Prediction” subsection in Related work, it would be beneficial to include further discussion on studies that specifically address the challenge of “overcoming the dependency” on reanalysis data. While benchmark datasets like WeatherBench1 [3] and WeatherBench2 [4] have long been established within the Climate AI community, they rely on reanalysis data, which introduces latency in data acquisition—making real-world applications difficult. Including references to studies that raise this concern would strengthen the discussion and provide a more comprehensive perspective.\n\n- In Subsection 3.1, the use of an MAE-based model to handle imputation in observation data is a reasonable choice. However, as the authors emphasize the importance of learning spatio-temporal patterns, it might have been more appropriate to adopt a baseline designed for such data—such as VideoMAE [5], which is tailored for video (i.e., spatio-temporal) inputs rather than static images. Why was VideoMAE not considered or used in this context?\n\n- Precipitation nowcasting is typically evaluated using metrics such as CSI (Critical Success Index) and FAR (False Alarm Ratio). However, it would also be beneficial to include the Probability of Detection (POD) metric in the evaluation.\n\n\n[1] Park, Young-Jae, et al. \"Long-Term Typhoon Trajectory Prediction: A Physics-Conditioned Approach Without Reanalysis Data.\" International Conference on Learning Representations (ICLR), 2024.\n\n[2] Verma, Yogesh, et al \"ClimODE: Climate and Weather Forecasting with Physics-informed Neural ODEs.\" International Conference on Learning Representations (ICLR), 2024.\n\n[3] Rasp, Stephan, et al. \"WeatherBench: a benchmark data set for data‐driven weather forecasting.\" Journal of Advances in Modeling Earth Systems 12.11 (2020): e2020MS002203.\n\n[4] Rasp, Stephan, et al. \"Weatherbench 2: A benchmark for the next generation of data‐driven global weather models.\" Journal of Advances in Modeling Earth Systems 16.6 (2024): e2023MS004019.\n\n[5] Wang, Limin, et al. \"Videomae v2: Scaling video masked autoencoders with dual masking.\" Proceedings of the IEEE/CVF conference on computer vision and pattern recognition (CVPR), 2023.", "limitations": "- It is encouraging that this study collects a dataset, constructs it thoughtfully, and establishes baseline models to conduct diverse and robust experiments. However, it would be beneficial if the specific problem the study aims to solve were more clearly discussed in both the Introduction and Conclusion, and explicitly connected to the baselines.", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 2, "originality": 2, "strengths_and_weaknesses": "## Strengths\n- The dataset and experimental setup are robust. In particular, the experiments effectively highlight the individual importance of each component within the dataset and baseline models. Moreover, as shown in Section 1 of the Supplementary Material, the study includes comparisons with existing video prediction models from the computer vision (CV) community, thereby addressing the interests and questions of both the Climate AI and the ML/CV communities.\n- The \"Ablation Study: Gains of Cross-Regional Boundary Conditioning\" section, including Figure 7 and the Supplementary Material, represents the strongest aspect of this study. Most existing data-driven methods for weather forecasting rely on gridded datasets, which often produce unnatural results near boundaries. In contrast, this work effectively mitigates that issue to a significant extent.\n- The study effectively leverages multiple modalities or sensors to address the problem. In particular, the use of a Masked Autoencoder (MAE) to tackle the common issue of dataset imputation in real-world scenarios is both timely and appropriate.\n\n## Weaknessess\n- The limitations of prior studies and the motivation presented in the Introduction are framed in a way that aligns with general themes commonly discussed in the Climate AI community, but they lack specificity. The mentioned issues, such as data assimilation biases and temporal discrepancies, are also rather generic. Notably, the discussion on temporal discrepancies is not addressed beyond the Abstract and Introduction, which weakens the clarity and depth of the problem statement.", "was_revised": true, "final_justification": "The rebuttal provides clear and convincing clarifications that fully resolve my earlier concerns. The authors have effectively restated their core contribution, provided rigorous comparisons with relevant prior work, and offered well-justified explanations for key design choices such as the AIDA–AIWP framework and their approach to handling sparse observational inputs. The inclusion of additional references, expanded evaluation metrics (including POD), and thoughtful discussion on both methodological and real-world aspects further enhance the manuscript. The experiments are solid and demonstrate a genuine effort to address recognized challenges within the climate community. Based on these improvements and clarifications, I am increasing my score from 3 to 4.", "rebuttal": "We sincerely thank you for your thoughtful and encouraging feedback. We greatly appreciate your recognition of the importance of **observation-based forecasting as a promising direction** beyond reanalysis data, as well as your positive assessment of our AIDA module and cross-regional boundary conditioning (CBC) approach. Your comments on the robustness of our experimental design, especially the detailed ablation studies and cross-community comparisons, are highly motivating. We are particularly grateful for your acknowledgment of our efforts to address real-world challenges such as missing data and boundary artifacts through well-justified architectural choices.\n\n**OUR CORE CONTRIBUTION:**\n\n\nWe **identify a key challenge** in observation-based forecasting (L55): the **mismatch between sparse observations and dense prediction**. Our proposed DAWP framework effectively mitigates the input–output mismatch by introducing a **two-stage solution**—AIDA for assimilation-based initialization, followed by an AIWP forecasting model specifically adapted to AIDA outputs. As shown in Figure 6, our results provide strong empirical evidence supporting this claim.\n\n----\n\n\n> **Response to Q1: Discussion about LT3P**\n\nThank you for drawing our attention to LT3P as a relevant real‑time reference. We will include LT3P in our “Related Work” section. As you point out, LT3P leverages reanalysis data for pre‑training and combines NWP forecasts with historical typhoon trajectories to make real‑time predictions. \n\nIn contrast, we would like to clarify that our DAWP framework intentionally avoids reanalysis data and NWP systems, instead focusing exclusively on observation-driven methods. The primary contribution of DAWP is to demonstrate that the combined AIDA–AIWP framework offers an effective solution to the mismatch between observational input space and the forecasting objective.\n\nWe still believe that the approach adopted in LT3P—pretraining on ERA5 reanalysis data followed by fine-tuning on real-time data—also provides valuable inspiration for future extensions of our framework.\n\n\n> **Response to Q2: Relationship of AIWP and Physical Solver**\n\nThank you for bringing ClimODE to our attention and enabling a more focused discussion.\n\n**Network**: ClimODE leverages the advection equation to design a prediction network trained on reanalysis data. While it does incorporate physical priors via a physics‑informed Neural ODE, its forecasting performance remains well below that of the numerical weather prediction system IFS (see Figure 4 of ClimODE). By contrast, methods such as Pangu‑Weather[1], GraphCast[2], and GenCast[3] are also trained on reanalysis datasets but do not incorporate any explicit physical solver. Despite the lack of physical constraints, they consistently outperform IFS across a wide range of performance metrics and lead times. This suggests that including a physics-based solver in network design is not strictly necessary for achieving superior forecast skill. \n\n**Data**: Moreover, the use of reanalysis data itself implicitly introduces physical solver dependence, since such data are generated using NWP systems. In fact, purely observation‑driven models also show remarkable performance in some domains—for example, precipitation nowcasting[4]. Thus, from both network design and data selection perspectives, AIWP models operate independently of physical solvers.\n\nNevertheless, we fully agree with you that complementary integration of physical priors could be beneficial especially in scenes without enough data. Such priors can be incorporated either by training with reanalysis data or by applying Physics‑Informed Neural Networks (PINNs) to observational physical variables. In future versions of our manuscript, we will expand on these points and provide clearer justification, thereby avoiding any potential overclaim.\n\n\n> **Response to Q3: More References**\n\nThank you very much for your kind suggestion regarding WeatherBench. Since the ERA5 reanalysis dataset serves as the foundation for WeatherBench, it is an excellent reference to raise the concern of reanalysis. Additionally, beyond the references already included namely EarthNet[5], Transformer-DOP[6], and GraphDOP[7]—we will also incorporate references of OMG-HD[8] and LT3P, which further support the concerns raised.\n\n\n> **Response to Q4: VideoMAE**\n\nThank you for suggesting VideoMAE—this enables a more precise discussion. \n\nVideoMAE’s tube masking strategy creates spatiotemporal tokens by masking the same spatial location across multiple frames to exploit temporal redundancy. However, due to the polar-orbiting satellite motion characteristic, consecutive frames often lack overlapping observations at the same location. If we employed VideoMAE’s tube masking, the resulting tokens would contain empty values, forcing the model to learn from missing data—an undesirable artifact in our context. Consequently, we did not adopt this type of MAE design that relies on temporal redundancy between neighboring frames.\n\n> **Response to Q5: Evaluating POD**\n\nThank you for your suggestion. We have evaluated the POD metric, and the conclusions are consistent with those presented in Table 3—our proposed DAWP framework significantly outperforms the other methods.\n\n|                 | TCWV (mm)   |             |             | SP (mm/h)   |             |             |\n|-----------------|-------------|-------------|-------------|-------------|-------------|-------------|\n|                 | POD-10      | POD-20      | POD-30      | POD-0.5     | POD-1.0     | POD-2.0     |\n| EarthNet        | 0.948 | 0.923 | 0.915 | 0.171 | 0.053 | 0.009 |\n| Transformer-DOP | 0.948 | 0.924 | 0.913 | 0.175 | 0.062 | 0.011 |\n| Ours            |     **0.951**        |      **0.926**       |      **0.917**       |     **0.257**        |    **0.121**         |      **0.039**       |\n\n\n>**Response to W1: Motivations and In-depth Analysis about Challenges**\n\nWe would like to re-clarify **OUR CORE CONTRIBUTION**, as presented at the beginning of the rebuttal.\n\nWe agree with you that the limitations of reanalysis data are a consensus within the community. As a result, Direct Observation-based Prediction have recently gained significant attention, given their potential to overcome those limitations.\n\nOur main contribution lies in first identifying the mismatch between sparse inputs and dense predictions as a key challenge within DOPs. To mitigate such mismatch, we propose a novel AIDA–AIWP framework. Our experimental results provide strong support for the effectiveness of this framework in resolving the mismatch issue as shown in figure 5, highlighting its value as a step forward in DOP research.\n\n___\n\nWe sincerely appreciate your feedback, which has significantly improved the paper’s clarity and completeness. Should further clarification be required, please do not hesitate to reach out, and we will address your concerns promptly. We look forward to your continued input.\n\nReferences:\n\n[1] Bi, Kaifeng, et al. \"Accurate medium-range global weather forecasting with 3D neural networks.\" Nature 619.7970 (2023): 533-538.\n\n[2] Lam, Remi, et al. \"Learning skillful medium-range global weather forecasting.\" Science 382.6677 (2023): 1416-1421.\n\n[3] Price, Ilan, et al. \"Probabilistic weather forecasting with machine learning.\" Nature 637.8044 (2025): 84-90.\n\n[4] Gao, Zhihan, et al. \"Earthformer: Exploring space-time transformers for earth system forecasting.\" Advances in Neural Information Processing Systems 35 (2022): 25390-25403.\n\n[5] Vandal, Thomas J., et al. \"Global atmospheric data assimilation with multi-modal masked autoencoders.\" arXiv preprint arXiv:2407.11696 (2024).\n\n[6] McNally, Anthony, et al. \"Data driven weather forecasts trained and initialised directly from observations.\" arXiv preprint arXiv:2407.15586 (2024).\n\n[7] Alexe, Mihai, et al. \"GraphDOP: Towards skilful data-driven medium-range weather forecasts learnt and initialised directly from observations.\" arXiv preprint arXiv:2412.15687 (2024).\n\n[8] Zhao, Pengcheng, et al. \"OMG-HD: A high-resolution AI weather model for end-to-end forecasts from observations.\" arXiv preprint arXiv:2412.18239 (2024)."}, {"review_id": "U6OWqrfnxb", "reviewer": "Reviewer_bPKH", "summary": "This paper presents DAWP, a framework for global observation forecasting via Data Assimilation and Weather Prediction in satellite observation space. The authors propose to integrate the AIDA module with an AIWP module to enable AIWPs to operate in a complete observation space. Experiments demonstrate that DAWP significantly improves the rollout and efficiency of AIWP and holds promising potential for global precipitation forecasting.", "questions": "Have you considered incorporating explicit physical constraints or laws into your data-driven framework to enhance the physical plausibility of the predictions?\nHow do you address the issue of model explainability in your data-driven approach? Given the complexity of weather systems, how can you ensure that the model’s predictions are not only accurate but also interpretable and trustworthy?", "limitations": "The author should expand and explain the potential negative impacts of the risk of misuse of the technology or the potential for creating misleading information.", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 2, "significance": 4, "originality": 3, "strengths_and_weaknesses": "The paper proposes a novel framework that combines data assimilation and weather prediction in a unified observation space, addressing the limitations of existing AIWP methods that rely on reanalysis data. The proposed modules are effective. The paper's contribution to the field of weather prediction is not incremental but rather innovative.\nThe authors claim that \"AIWP models do not require physical solvers to predict the evolution of the atmosphere,\" which highlights the data-driven nature of their approach. However, this does not preclude the incorporation of physical information into the model. In fact, integrating physical insights could enhance the model's explainability and reliability, potentially preventing it from overfitting to spurious patterns in the data.", "was_revised": true, "final_justification": "My concerns are well addressed.", "rebuttal": "We sincerely thank you for your insightful and encouraging feedback. We greatly appreciate your recognition of the novelty and significance of our proposed DAWP framework, particularly the integration of the AIDA and AIWP modules to enable forecasting directly in satellite observation space. We are particularly encouraged by your assessment that our contribution is **not merely incremental but genuinely innovative**, as this reinforces the impact of our work.\n\n----\n\n> **Response to Q1: Incorporation with Physical Constraints**\n\nWe sincerely appreciate your valuable suggestion and fully agree that incorporating physical information into the system could enhance performance—particularly in data-limited scenarios—by helping mitigate overfitting.\n\nHowever, in our current experimental framework, DAWP is trained exclusively on satellite-observed variables—such as microwave radiance, brightness temperature, and infrared radiance—rather than on derived physical quantities like temperature or wind. Because no well‑established physical forward model adequately describes the dynamics of these observed variables, this limitation underscores the need for purely data‑driven approaches in satellite‑observation forecasting.\n\nMoving forward, we see promising opportunities to introduce physical insights gradually. For example, we can incorporate directly measured physical quantities (e.g. wind, atmospheric temperature profiles) into our pipeline and apply Physics-Informed Neural Networks (PINNs) or similar techniques to learn physical relationships, linking these profiles with satellite-observed radiance.\n\nWe will make these future directions explicit in the revised manuscript to clarify both the current limitations and the potential for integrating physical priors into DAWP.\n\n> **Response to Q2: Trustworthiness and Explainability**\n\nThank you for your valuable feedback regarding the credibility and interpretability of data-driven forecasting methods.\n\nWhile interpretability and trustworthiness remain fundamental challenges for data-driven approaches, DAWP provides an additional perspective on model trustworthiness compared to existing methods such as [1–3], where the completion of sparse observations cannot be supervised. Specifically, during the AIDA initialization stage, DAWP evaluates input completion quality by reconstruction loss over the masked regions, thus enabling a supplementary mechanism for evaluating the credibility of model predictions.\n\nWe have also conducted an initial investigation into explainability of forecasting results. In our Modality Sensitivity Analysis, we examine how different input modalities influence the model’s outputs. We observe that using only ATMS data retains most of the model’s predictive power across other modalities, indicating that Brightness Temperature inherently contains substantial information from Microwave and Infrared radiance.\n\nThese experiments demonstrate that Brightness Temperature carries rich cross-modal signals, supporting the model’s robustness even when trained with limited modalities.\n\nWe believe that interpretability and credibility in direct-observation forecasting remain rich areas for further exploration.\n____\n\nReferences:\n\n[1] McNally, Anthony, et al. \"Data driven weather forecasts trained and initialised directly from observations.\" arXiv preprint arXiv:2407.15586 (2024).\n\n[2] Alexe, Mihai, et al. \"GraphDOP: Towards skilful data-driven medium-range weather forecasts learnt and initialised directly from observations.\" arXiv preprint arXiv:2412.15687 (2024).\n\n[3] Vandal, Thomas J., et al. \"Global atmospheric data assimilation with multi-modal masked autoencoders.\" arXiv preprint arXiv:2407.11696 (2024)."}]

Author rebuttal:

Meta-review: This paper introduces DAWP, a two-stage framework (AIDA initialization followed by AIWP forecasting) for global observation forecasting in satellite observation space. The approach moves beyond reliance on reanalysis data, addressing the challenge of learning spatiotemporal dynamics from irregular, high-resolution satellite observations. The core contribution is the mitigation of the mismatch between sparse observational inputs and dense prediction outputs. Extensive experiments benchmark against multiple classical spatio-temporal prediction baselines and demonstrate improved performance, including in global precipitation forecasting tasks.

The work tackles an important and timely problem by enabling direct forecasting in observation space without analysis data dependency. The experiments are decently strong and there is broad spectrum of baselines. The authors addressed specific requests for more baseline comparisons, more detail on experimental protocols, full-year test splits etc. There are still concerns w.r.t. how the baselines have been adapted to this setting as the protocol and settings are different. Would be good to further emphasize differences w.r.t. EarthNet and Transformer-DOP lines of work in the observation based forecasting setting. 

Overall the paper is an ambitious attempt to advance direct observation forecasting. Despite some concerns w.r.t baselines, the work is likely to stimulate further research. I encourage the authors make sure the results are reproducible by the community, including the baselines.
Decision: Accept (poster)