FNP: Fourier Neural Processes for Arbitrary-Resolution Data Assimilation
Authors: Kun Chen
Peng Ye
Hao Chen
kang chen
Tao Han
Wanli Ouyang
Tao Chen
LEI BAI
Venue: NeurIPS 2024 poster (2024)
Keywords: data assimilation
neural processes
deep learning
end-to-end weather forecasting
Primary area: machine_learning_for_physical_sciences

Abstract: Data assimilation is a vital component in modern global medium-range weather forecasting systems to obtain the best estimation of the atmospheric state by combining the short-term forecast and observations. Recently, AI-based data assimilation approaches have attracted increasing attention for their significant advantages over traditional techniques in terms of computational consumption. However, existing AI-based data assimilation methods can only handle observations with a specific resolution, lacking the compatibility and generalization ability to assimilate observations with other resolutions. Considering that complex real-world observations often have different resolutions, we propose the Fourier Neural Processes (FNP) for arbitrary-resolution data assimilation in this paper. Leveraging the efficiency of the designed modules and flexible structure of neural processes, FNP achieves state-of-the-art results in assimilating observations with varying resolutions, and also exhibits increasing advantages over the counterparts as the resolution and the amount of observations increase. Moreover, our FNP trained on a fixed resolution can directly handle the assimilation of observations with out-of-distribution resolutions and the observational information reconstruction task without additional fine-tuning, demonstrating its excellent generalization ability across data resolutions as well as across tasks. Code is available at https://github.com/OpenEarthLab/FNP.

FNP: Fourier Neural Processes for Arbitrary-Resolution Data Assimilation
Kun Chen
Peng Ye 1,2
Hao Chen
Kang Chen
Tao Han
Wanli Ouyang
Tao Chen 1,*
Lei Bai 2,*
Fudan University
Shanghai Artificial Intelligence Laboratory
kunchen220m. fudan. edu. cn
eetchen@fudan.co edu. en
baisanshi @estanssi@@mai...oomm com
Abstract
Data assimilation is vital component in modern global medium-range weather forecasting systems to obtain the best estimation of the atmospheric state by combining the short-term forecast and observations. Recently, Al-based data assimilation approaches have attracted increasing attention for their significant advantages
traditional techniques in terms of computational consumption. However, existing Al-based data assimilation methods can only handle obser- vations with specific resolution, lacking the compatibility and generalization ability to assimilate observations with other resolutions. Considering that complex real-world observations often have different resolutions, propose the Fourier Neural Processes (FNP) for Leveraging the efficiency of the designed modules and flexible structure of neu- ral processes, FNP achieves state-of-the- with varying resolutions, and also exhibits increasing advantages over the coun- terparts as the resolution and the amount of observations increase. Moreover, our FNP trained on a fixed resolution can directly handle the assimilation of ob- servations with out-of- offdissribbutin resolutions and the observational information reconstruction task without additional fine-tuning, demonstrating its excellent gen- eralization ability across data resolutions as well as across tasks. Code is available at hhttts://gitiee wwtsstuu..commpppee conm/pennaatthaab///
data assimilation this paper.
results in assimilating observations
Introduction
Accurately estimating the true state of complex and chaotic Earth systems is an important and challenging task, which can contribute to better understanding of nature and improve forecasting by reducing the error of initial conditions. The most accurate human knowledge of the Earth's 's state comes from observations, which are inherently limited in their scopes due to practical constraints. Data assimilation, based limited observational information and short-term forecasts (referred to as the background), serves as the primary approach for state estimation [38 21 32]. Traditional data assimilation methods employed in operational systems include Kalman filters based on minimum variance estimation and variational methods based on maximum likelihood estimation 4 g 49) Taking 3D variational (3D-Var) data assimilation an optimization problem under given conditions, aiming to find the analysis v. that minimizes the
example, data assimilation is regarded as
Conresponding authors
38th Conference on Neural Information Processing Systems (NcurIPS 2024).

---

objective function [zx). It can be formulated as
Ta = x² = arg min (xx
(I)
1 x1))) ((x) = (1- -
mb) 20
Hx)
where B and R correspond to the error covariance matrix of the background T4, and observation respectively, and H is observation operator that maps state variables observational space, aligning the background and observations with different modalities (for example, satellites do not directly observe state variables such as wind speed) and resolutions.
With the significant achievements of machine learning in medium- weather prediction 9 35| [1], data assimilation has gained increasing attention as of the core components in building end-to-end global weather forecasting systems. Compared to traditional methods, machine learning-based data assimilation models offer the potential for competitive results with significantly reduced resource consumption and execution time 10 28 62), making it a promising research
direction with practical applications. Chen al. [T0] proposed data assimilation model for weather variables based on the idea of gated masks, and combined it with Feng Wu [9]. an advanced AI- based weather prediction model, to build the first end-to-end Al-based global weather forecasting system. Subsequently, data assimilation models integrated with other Al-based weather prediction models were propsed 28 62]. These methods have demonstrated performance and efficiency improvements through various experiments. but all of them can only assimilate observations with the same resolution as the forecasting model. Therefore, they need to interpolate the observations onto grids of corresponding resolution through pre-processing in advance, and the pre-trained models do not have the flexibility and out-of-domain generalization to assimilate observations with other resolutions. The pre-processing step implements part of the function of the observation operator H to perform resolution alignment, and it will introduce additional errors inevitably, thereby affecting the performance and generality of data assimilation methods.
offer a promising and universal data assimilation framework for addressing the aforementioned challenges. Neural processes are series of conditional generative models that continuously model the distribution of functions and fields based on paired coordinate- value conditions, and generate values at arbitrary target locations based on coordinate indices. Their flexible features that allow for grid or off-grid data are well-suited for assimilating observational data with diverse forms, without requiring any prior interpolation or mapping (55] 2
Neural processes
20
53 16 54). In
this context, data assimilation defined as the process of generating the analysis given both back- ground conditions and observational information conditions. The network models the comprehensive functional representation based on the two conditional inputs and decodes it to obtain the posterior distribution of the target. Compared to deterministic data assimilation, the modeling of distribution by neural processes can provide uncertainty estimates and further be used for ensemble data assimi- lation 23 Moreover, data assimilation task degrades to observational information reconstruction when the background condition is missing. These two tasks can be broadly categorized as conditional generation. enabling their straightforward integration into unified framework for direct application through simple fine-tuning.
In this paper, we propose the Fourier Neural Processes (FNP) for data assimilation with arbitrary- resolution observations. FNP is flexible to adapt to varying resolutions and can be extended to any conditional generation task. Leveraging the efficiency of the designed modules and flexible structure of neural processes, FNP achieves state-of-the-ar (SOTA) results in data assimilation experiments with different resolutions, and demonstrates increasing advantages over other models as the resolution and amount of observational information increase. The visualization of the analysis showcases the promising performance of FNP in capturing high-free -egg-fregeency information. Importantly, the FNP trained at a fixed resolution can be dircctly applied to data assimilation with other resolutions and ohservational information reconstruction task without funennuning, highlighting its excellent out-of-domain generalization. Additionally, ablation study for different modules and experimental settings validate the effectiveness and robustness of approach.
2 Related Work
Machine learning for data assimilation. There exist strong mathematical similarities between machine lcarning and data assimilation, enabling their integration within unified Bayesian frame-

---

With their powerful nonlinear fitting capabilities and low computational cost, machine learning techniques can both enhance traditional data assimilation methods and provide alternative algorithms [3 i 22 56] Convolutional neural networks (CNNs) and recurrent neural networks (RNNs) are often employed as surrogate models to replace computationally expensive components in data assimilation, such as tangent-linear and adjoint models in 4D variational (4D-Var) data assimilation [25]. localization functions in ensemble Kalman filters (EnKF) [58), and error co- variance matrices 22 45]. Implicit neural representations (INRs) [33 and various autoencoders (AEs) 47 I can offer efficient order reduction frameworks for latent assimilation to address the challenges of high-dinn high-dimensional data. More recently, algorithms based on diffusion models have also provided new solutions for data assimilation driven by the advancements and maturity of AIGC [28 52 48). However, all these studies are aimed assimilating fixed- fixeuresoluttin observations, and we are the first to focus on arbitrury-resolution data assimilation.
work 19 3].
Machine learning for observational information reconstruction. Observational information reconstruction is the process of recovering missing values and obtaining complete field information from limited sparse observations. Traditional reconstruction methods primarily rely on kriging
interpolation and principal component analysis- based infilling [29]. As Kadow al. [29] successfully
applied the image inpainting techniques in computer vision to reconstruct the global temperature data,
deep learning has been widely used various reconstruction tasks [40 59
15
57
We associate
the reconstruction with the data assimilation task here, and the flexibility of our method allows the
FNP pre-trained on data assimilation task to be directly applied to the observational information reconstruction without fine-tuning and achieve promising performance.
Neural processes family and its application in geoscience. Neural processes combine the advan- lages of neural networks and Gaussian processes, and have demonstrated excellent performance in function regression, image completion and classification tasks [17 18]. Attentive neural processes (ANP) 30| enable the network to learn location-releve relevann representations by introducing the atten- tion mechanism, which improves the accuracy of predictions and broadens the scope of modeling. Convolutional conditional neural processes (ConvCNP) (20 model translation equivariance in the data, adding an important inductive bias to the model and enabling zero-shot generalization to out-of- domain tasks. Evidential conditional neural processes (ECNP) 43) replace the standard Gaussian distribution with a hierarchical Bayesian structure through evidence learning to achieve the decompo- sition of epistemic-aleatoric uncertainty. Neural processes and their variants, leveraging their unique advantages, have been successfully applied in geoscience tasks such as climate downscaling (55), sensor placement [2] and observational information reconstruction (53, and have shown promising performance. Here we apply the neural processes to urbirrary-resolation data assimilation for the first time and achieve SOTA performance, further demonstrating its huge application potential.
3 Methods
3.1
Model Overview
The overall framework and model details of FNP are depicted in Figure I Initially, the background and observations undergo unified coordinate transformation to obtain the coordinates x and values y of the conditional points when they are input into the network. This ensures spatial alignment of them, even in the presence of disparate resolutions, modalities, and data formats. Subsequently, FNP models the two components of the conditional information globally to get their respective spatial- variable functional representation. The dynamic alignment and merge (DAM) module integrates and aligns these functional representation into the target domain, resulting in a comprehensive functional representation over the target space. Finally, multi-layer perceptrons (MLPs) are employed to decode the functional representation and output the mean and variance of the analysis based on the coordinates æf of the target points. In the following subsections, we provide detailed description of the process for modeling the functional representation and the internal structure of the DAM module.
3.2 Spatial- Functional Representation
Modeling the functional representation involves two main steps: embedding the data sets into an infinite-dimensional function space and performing deep feature extraction. The former accom- plished through the SerComv layer (20), which is generalized form of the standard convolutional
3

---

Buekgroodd
Unifed Cooditate Trunfortation
Variale Pinctions Reprosotttion
Dysanic Algsoeet and Merge
Quary
MiL's Beeader
Arolydk
Aralysi
Puxctoral Ropeescrattion
Dynamic Aliguuoe and Merge
Nearal Fozrkr Layer X
Sieslarit
Sieilarry
SenCver
Goondroaze
Denilty
Corry
yd
Rostra
Valae
Sigel
liettly
Sientarry
Maay
Figure I: Overview of the network architecture of FNP. Unified coordinate transformation ensures spatial alignment of the background and observations, and extracts the coordinates and values of the conditional points, FNP models the two components of the conditional information globally to get their respective spatial-variable functional representation through SerConv for data embedding and stacking of neural Fourier layers for deep feature extraction. The dynamic alignment and merge module integrates these functional representation based on similarity shared features and aligns them into the target domain, resulting in comprehensive functional representation over the target space. MLPs are finally employed decode the functional representation and output the mean and variance of the analysis based on the coordinates of the target points.
layer extended to operate sets. It takes set of continuous coordinate-valus pairs input and outputs a function that can be queried at continuous positions. The SerConv operation is permutation- invariant and includes an additional channel to estimate the density of the conditional points. When the input coordinates are discrete, SerConv essentially degenerates into a standard convolutional layer, simplifying the model into the an-the-grid version [14]. We strongly recommend readers to refer to the theoretical proofs and derivations of ConvCNP (20), and the project homepage of neural processes family [14] for more detailed explanation.
The original deep feature extraction module implemented using standard CNN with residual structures. We choose to replace the basic convolutional layer with more efficient neural Fourier layer (NFL) FNP. Additionally, to address the multi-variable optimization problem in weather modeling tasks, we decouple the representations in spatial and variable dimensions reduce the difficulty of network training. Below we provide further explanations on the motivations and implementation details of these design choices.
Spatial-variable decoupled representation. Data for different weather variables are usually stacked in the channel dimension, and direct data embedding will mix the spatial auto-correlation within variables with the inter-correlation among variables. An intuitive understanding is that explicitly separating the information in the spatial and variable dimensions allows for clearer learning objective each block, thereby reducing the difficulty of network training and fully unleashing the network's potential. In terms of implementation, we model spatial functional representation separately for each meteorological variable, such as geopotential and temperature (surface variables are treated together one variable), and model variable functional representation that encompasses all variables, which are then concatenated together. The benefits of this approach have been confirmed in experiments. We found that the varialev decoupled (SVD) representation achieves better performance with fewer parameters and faster convergence speed. The detailed comparison of performance can be seen in Table
Neural Fourier layer. The smoothness of neural network outputs poses a challenging drawback in weather modeling tasks, and the background generated by Al-based forecasting models also tends to be smoother. our experiments, we find that neural processes also struggle to overcome the issue of smoothness. To address the desire for high- figh-freuueccy information, we choose to introduce the

---

Fourier neural operator [3 Besides, operations in the frequency domain can also bring additional advantages in terms of global receptive fields for models based on CNN. Therefore, in addition to the convolutional operation, each neural Fourier layer consists of a branch for linear operation in the frequency domain and a branch for identity mapping to preserve high-frrquency details as much as possible [26].
3.3 Dynamic Alignment and Merge
The DAM module aligns functional representations from two conditional domains to the target domain for obtaining outputs at the target locations. In data assimilation tasks, the analysis typically shares the same resolution and modalities as the background, and the previous data embedding has already mapped inputs that may have different modalities into the same feature space. Therefore, it is only necessary to align the functional representation of the observation in the spatial resolution. We choose use interpolation adjust the spatial dimensions' size it accommodate inputs of arbitrary resolutions, thereby enhancing the dynamics and generality of the model. Interpolation in the feature space differs fundamentally from that the original observational space because the former has already extracted helpful information and contains redundancy to support dimensionality reduction, while the latler compresses valid information and missing values the same extent. The performance of data assimilation with different resolutions in Table [ provides proof for this. As the amount of observational information increases, our model achieves significant improvements. while other models do not. A linear layer extracts shared features ÿ from both parts after alignment, which are then used to calculate similarities with their respective feature components. The similarity calculation is performed in the channel dimension as the spatial distribution of information differs significantly between them, with the background having more uniform spatial distribution while the observation exhibits greater spatial variability. In our implementation, the feature similarity is represented by the Euclidean distance between the two features, i.e.,
Simba
(Whaon Wh,w. )2
where h and ws denote the indices for each grid point along the longitudinal and latitudinal directions, respectively, and k is the dimension of data embedding. The relative values of the similarity map then determines the selection of features. Specifically, features that are more similar to shared features will be retained, while features that are less similar will be discarded, that is,
Sim Simg,
(4)
yh
Simb < Simb,
The dynamically filtered features will be spliced together with the shared features and sent to a convolutional layer for spatial smoothing, and the result will be used as the functional representation in the target domain for decoding and output.
4 Experiments
4.1 Experimental Settings and Implementation
Data preparation. We demonstrate the effectiveness of our methodology the ERA5 dataset 27 a global atmospheric reanalysis archive containing hourly weather variables such as geopotential, temperature, wind speed, humidity, etc. We choose conduct experiments on total of 69 variables. including five upper-air variables with 13 pressure levels (i.c., 50hPa. 100hPa, 150hPa, 200hPa, 250hPa, 300hPa, 400hPa, 500hPa, 600hPa, 700hPa, 850hPa, 925hPa, and 1000hPa), and four surface variables. Specifically, the upper-air variables are geopotential (z), temperature (t), specific humidity (q), zonal component of wind (u) and meridional component of wind (v), whose 13 sub-variables at different vertical level are presented by abbreviating their short name and pressure levels (e.g., z500 denotes the geopotential at a pressure level of 500 hPa), and the surface variables are 10-meter zonal component of wind (u10), 10-meter meridional component of wind (v10), 2-meter temperature (12m) and mean sea level pressure (msl). subset of ERA5 dataset for 40 years, from 1979 to 2018, chosen to train and evaluate the model.
5

---

Experimental settings, The advanced Al-based weather forecasting model, FengWu [9] is used as the surrogate model to generate the background. The observations are simulated by adding a proportional mask to ERAS, and the default setting corresponds to 24-hour forecast lead time and 10% observations. In other words, the background used for data assimilation is produced by Feng Wu (with 6-hour interval) through four auto outo-regressive iterative predictions based on ERAS data from one day ago. The observational space usually has higher spatial resolution than the state space in actual operation systems. Therefore, the resolution of the forecasting model and background is set to 1.40625 (128 x 256 grid points) so that we can conduct experiments using observations with different resolutions such as 0.25 (721 1440 grid points) to verify the assimilation performance with arbitrary resolution.
Model training and evaluation. The FNP model is implemented based on the open-source code of the neural processes family project [14, and trained for 20 epochs using the Adam optimizer [39] with learning rate of le-4. We divide the ERA5 data from 1979-2015 as the training set, 2016-2017 as validation set, and 2018 as test set. The training is run on 4 NVIDIA Tesla A100 GPUs with a global batch size of 16, and takes approximately 2.5 days. The inference only needs a few minutes to perform data assimilation for a whole year on single A100 GPU. The dimension of data embedding for default setting is 128 and the number N of NFLs is 4, and Gaussian likelihood is used with negative log- likeliilloood (NLL) loss. We evaluate the performance of models by calculating the overall mean square error (MSH), mean absolute error (MAE), and the latitude- error (WRMSE) which is a statistical metric widely used in geospatial analysis and atmospheric science 50 51. Given the estimate Bs,,.c and its ground truth Fh,w.c for the c-th channel, the WRMSE is defined as
root mean square
WRMSE(() ~V H.W En.. H cos( Ex cos(or ..)
(J
where H and W represent the number of grid points in the longitudinal and latitudinal directions, respectively, and 0h, w is the latitude of point (h, w).
4.2 Arbitrary-Resolution Data Assimilation
We validate the performance of models by assimilating 10% observations with resolutions of 1.40625. 0.703125", and 0.25%, respectively, onto a 24-hour forecast background with 1.40625 resolution. Table 1 provides a quantitative comparison of the analysis errors between FNP and other models. The first row corresponds to the error level of the background. When assimilating observations with the same resolution as the forecasting model, FNP achieves SOTA results (indicated in bold) in terms of overall MSE, MAE, and WRMSE metrics for the majority of variables. Since Adas [10] is not flexible enough to support inputs with different resolutions, we follow its common practice to interpolate the observations and average the observations falling within the corresponding grid range when assimilating higher-resolution observations. Despite this, Adas still produces results with significantly high errors, so we only present the performance after fine-tuning to the interpolated observations. This indicates that Adas lacks the ability of oft-of-domain generalization. In contrast, FNP and ConvCNP (20). with their flexible structures, can assimilate observations with different resolutions directly without interpolation. Therefore, the table presents the results for both cases with and without fine-tuning for FNP and ConvCNP.
In order better understand and explain the performance of different models, add different colors to represent the variations in results compared to that with 1.40625 resolution (blue indicating worse results, i.e., increased errors, and red indicating improved results). For the q700 variable, we additionally annotate the percentage of error increase or decrease. It is worth noting that with increasing resolution, the same ratio of observations implies a greater number of absolute observations and a larger amount of information. However, during the interpolation process, averaging the observation values within region does not guarantee reflection of the overall conditions unless there are observations at all points within the region. Therefore, interpolation inevitably leads to information loss, and the amount of lost information is negatively correlated with the number of observations within the region. Based on the balance between these two factors, the fine-tuned Adas exhibits different trends with two different resolutions: the results generally improve when assimilating observations with 0.703125 resolution, while all the results worsen when assimilating observations with 0.25 resolution. This indicates that as the resolution increases gradually, the impact of information loss due to interpolation becomes more significant and surpasses the positive effect of

---

Table 1: Quantitative performance comparison for arbiraay-resolution data assimilation. The best performance are shown in bold while the second best is underscored. Red color indicates the improved assimilation results compared to that with 1. 1.40625" resolution, and blue color indicates worse results.
WRMSE v10
Model
Resolution
MSE
MAE
2500
1850
12m
010
0500
+500
9700((100))
Background
L:16255
0.0288
0.0861
45,455
0,7200
0.7790
0,9336
0.9645
1,7278
1.7535
6.7220
Adas T0
1.40625'
0.0221
0.0705
22,930
0.6323
0,7198
0.8198
0,8369
1,4673
1,4499
6,4955
ComCCNP
8D
1.40625
0.0252
0.0795
31,253
0.6831
0. 0,7662
0.8334
0,8553
1,5770
1.5876
6.5717
FNP (ours)
1. 1.40625
0.0202
0.0664
26,910
0,6600
0.7049
0.7558
0.7665
1.4263
1.4450
6.4691
Adas 10
0.703125
0.0144
0.0614
26.423
0.5981
0.6995
0.6631
0.6552
1,2035
197955
1.3798
117.28
ComCNN A0
w/o tunintnnng
0,703125
0.0378
0.1134
142.38
0.9106
0.8471
1,2264
1.1115
2,9765
2.6264
5,7900
.11.9%
ComCNN
0,703125=
0.0202
0.0712
25.452
0.5923
0.7861
0.5812
0.6339
1.3795
1.4331
5.7562
.12.4%
(ours) w/o fine-tuning
0,703125'
0,0150
0.0618
52.075
0.6021
0.7385
0.7052
0.6833
1.3723
1.3036
5/4333
116.0%
FNP (ours)
0.703125
0.0085
0.0419
20,710
0,5805
0.5310
0.4457
0.4406
0.9033
0.9180
4.5296
130.0%
Adas
10
0.25
0.0231
0.0766
34.694
0.6979
0.7728
0.8455
0.8537
1,5861
1.5265
6.5019
10.
ComCCNP 20
w/o fine-tuning
0.25
0.0681
0.1514
234.50
1,3208
0,8705
1,7134
1.5122
4.1316
3.6655
6.6619
11337%
Con/CNP
201
0.25
0,0280
0.0831
26.437
0.6326
0. 74799
0.9288
0.96400
1.7265
1,7533
6. 6.3612 L1,30%
FNP cours) wio fine- tuning
0.25
0.0196
0.0729
84.516
0.6992
0.6912
0.9264
0.8687
1.8315
1.6723
6.1761
(4.53%
FNP (ours)
0.25
0.0058
0.0339
19.996
0,4549
0.4999
0.3839
0.3753
0.6308
0.5320
2.9614 154.2%
increased observation quantity, In practice, the observational data used in operational systems usually have resolutions of 0.1º or even higher, while the resolution of the most commonly used forecasting models 0.25%. Therefore, the information loss caused by interpolation in existing methods is very common and urgently addressed issue.
In contrast, the fine-tuned FNP not only achieves SOTA performance in all metrics (including the z500 and 1850 variables, in which FNP does not reach the optimum results with 1.40625 resolution). but also improves all assimilation results with the largest magnitude of error reduction. The performance differences between FNP and other models increases significantly with increasing resolution, and the WRMSE decreases some variables such as v10, u500, v500 and q700 are even more than 50% when assimilating observations with 0.25 resolution. Furthermore, as the absolute number of observations increases, providing more information, FNP is the only model that consistently improves the performance of data assimilation. This plays crucial role in practical applications, as it means that all deployed observation instruments can be fully utilized, reducing the waste of human, material and financial resources as much as possible.
The results without fine-tuning reflect the out-of generalization capability of models, as they have not encountered observational data with other resolutions during training. As the resolution increases, the discrepancy between the samples used for testing and the visible samples during training becomes larger, leading to a gradual decline in performance. This regular pattern can be observed in the assimilation results of both FNP and ConvCNP. However, the increased quantity of observations and information will bring the benefits, resulting in improved performance for some variables in out-of-domain settings, although the number of such variables decreases as the resolution increases. FNP consistently outperforms ConvCNP in terms of the number of variables showing improvement and even exhibits superior performance to fine-tuned versions of other models in some variables. This demonstrates the excellent out-of-dee domain generalization capability of our method in adapting to changes in resolution, and this capability is also applicable to changes in background resolution theoretically.
Figure 2 presents the visualization of assimilation results by different models for q700, with the visualization date-time randomly selected at 201 8-04-02 06:00 UTC. The first row displays the ERAS (ground truth), background, background error (background minus ERA5), and raw observations with a resolution of 0.25%. Other rows show the analysis, analysis increment (analysis minus background), and analysis error (analysis minus ERA5) obtained through data assimilation by different models, as well as the interpolated observations for Adas and analysis variances for ConvCNP and FNP. The background is smoother compared with ERA5, and the background error shows high spatial variability. It can be observed that FNP accurately captures the distribution pattern of the background error, leading to analysis with rich high-frequency information and significantly reduced analysis error. The comparison with ConvCNP, which also assimilates raw observations directly, confirms that FNP's excellent ability to capture high-frequency features does not sorely rely on higher-resolution observations. Furthermore, the smaller analysis variance also indicates a lower uncertainty in state
7

---

estimation achieved by FNP. More visualizations with different variables and resolutions are shown in Appendix B
ERAS
Background
Bockground Error
Caservations
COGE ESE2 0.04 0006 0065 6350
GON GEED
C004 0009 0006 CODE
-0,04
0,00 1999
000
EI34
1099
COCE 1530 0oup tylu Interpolated Obsevrvations
0006
Pghg Analyss
1gng
Analysis increment
Analysia of AdoS
EOGE E207 039 D000 DICE 5310
+0.00% H0.000
0.022
2225
-5.01 +E.0T
0L000 G000
2.01
1353 GOX
IEDE DOCE 2528 DND
ysis Mean ConvENU
Analysis increment COnVENP
Analysis Enor COnVON
Analysis Variance ConnChP
EoEE B502 030 0000 dacs 5313
-0000
2031
-3.54 -GA0
0.000
2.5
Agi
Anelysis
Analysi increment
Analysts
Analyss Varance
8350
2034
12:66
Figure 2: Visualization of assimilation results by different models for 9000 The visualization date-time is randomly selected at 2018-04-02 06:00 UTC. The first row shows the ERA5 (ground truth), background, background error and observations with 0.25 resolution. Other rows show the assimilation results of different models.
Data assimilation aims to improve weather forecasting results by reducing initial errors. Therefore, is crucial to explore the impact of different methods on forecast error reduction. Figure 3 provides results on the forecast WRMSE improvement of z500 variable over the next ten days through data assimilation, where lead time 0 corresponds to the reduction of initial errors. Darker colors indicate stronger improvements, meaning a greater reduction in forecast errors compared to not using data assimilation. Similar to the results of data assimilation, FNP consistently achieves state-of-the-are results in most cases, with its advantage becoming more pronounced as the resolution increases. Moreover, FNP is the only model that strictly enhances forecast improvement with increasing resolution and observational information. Additionally, apart from the accuracy of initial values affecting forecast errors, the physical characteristics of the initial states (e.g., physical balance) also influence the rate of forecast error growth. FNP demonstrates greater improvements in forecast errors at all lead times compared to improvements in initial errors, indicating that FNP not only reduces forecast errors but also slows down the growth rate of forecast errors. Other models do not exhibit the same trend, further highlighting the superior characteristics of the initial states produced by FNP.
4.3 Generalization to Observational Information Reconstruction
Theoretically, the functional representation learned based on observational conditions can be directly decoded through MLPs and output reconstruction results for the observational information without fine-tuning. Therefore, we evaluated the reconstruction performance of different models in the absence of the background conditions, as shown in Table 2 Similarly, Adas pre-trained on data assimilation task, cannot be directly used for information reconstruction. Hence, the table only

---

improvement
26.5
11.03
34.28
1755
2232
2325
33.24
2632
40.00
40.75
44.66
44.00
46.3
42.02
4333
2643
3247
23.40
26.85
23.25
24.50
23:44
PAP 60625/1
18.35
2432
26.15
2521
10.70
45.78
$2.06
54.34
$1.00
54.25
12
66
51.11
5282
46.75
6523
13.11
41.03
24/66
26.45
12.65
27.33
28.08
2400
28.74
2385
30.32
43.3
77
50.47
21.33
20.70
48.02
42.44
20.10
20.02
23:21
26.30
22:17
FAP 2.10022271
24.25
29.88
1611
4131
10.99
16.9
12
K3
77 76
12.14
$1.0
59.75
68.9
01.00
41.93
11.0
29:51
Ada 10.2111
COTVONP 12 2071
13-72
3.57
1037
1228
14.54
16.00
5T
20.33
27557
15.72 17 77 35.59 1553
34.0
13.44
11.72
31.25
13.59
11.00
959
25.46
36.55
4427
32.00
50.76
50
24.85
72.72
30 70.23 05.00
64.39
55.33
50.14
45.64
28.05
Say
Figure 3: Quantitative comparison of different data assimilation methods on the improvement of forecast errors for the next ten days.
presents the performance of retrained Adas, while both FNP and ConvCNP show results with and without fine-tuning. The fine-tuned FNP achieves SOTA performance across all metrics, while FNP without fine-tuning also demonstrates good generalization.
Table 2: Performance comparison in observational information reconstruction with 10% observations.
WRMSE
Model
MSE
MAE
z500
1850
12m
u10
u500
Y300
4700 (10
Adas
10%
0,0370
0.1006
46.592
1.0482
1,3715
1.1539
1.1711
2.0015
1,9932
9.5399
CNP
20
w/o fine-tuning
10%
0.0376
0.10442
67.098
1.0630
1.4864
1.2482
1,2756
1,9896
2,0312
9,3894
CNP
20
0.0350
0.0933
59,864
0.8789
0.8219
1.2168
1,2145
1,9715
2,0069
6,8643
TNP (ours) w/o fine tuning
10%
0,0355
0.1014
45.835
1.0378
1,3382
1.1723
1.2024
1.9559
1,9075
9.1517
TNP (ours)
10%
0,0291
0.0876
35.491
0.7298
0,8027
0.9411
0,9716
1.7464
1.7564
6.7258
4.4 Ablation Study
We conduct ablation experiments on both the designed modules employed in FNP and experimental settings. Tablee3]resent the quantitative performance comparison of FNP when different components are replaced. The overall MSE, MAE, and WRMSE metrics of all variables exhibit varying degrees of performance degradation when specific module in FNP is replaced. FNP achieves the best performance when these designed components work in synergy and mutually reinforce each other.
Table 3: Ablation study of different components in FNP for data assimilation with 1.40625 resolution.
WRMSE
Model
Resulation
MSE
MAE
z500
1850
12m
u10
v10
u500
v500
(10-F)
Background
1.40625
0.0288
0.0861
45.455
0.7200
0.7790
0.9336
0.9645
1.7278
1.7535
6.7220
FNP w/o NFL
1.40625
0.0230
0.0749
30.040
0.6733
0.7504
0.8212
0.8254
1.5459
1.5566
6.4941
ENP w/o DAM
1.40625
0.0214
0.0702
30.189
0.6816
0.7138
0.8248
0.8090
1.4674
1.4831
6.5631
FNP w/o SVD
1.40625*
0.0239
0.0757
27.588
0.6683
0.7555
0.7805
0.8430
1.5743
1.5789
6.5538
FNP
1.40625*
0.0202
0.0664
26.910
0.6600
0.7049
0.7558
0.7665
1.4263
1.4500
6.4691
Ablation study on the experimental settings is conducted by changing the forecast lead time of the
background and the ratio of observations while keeping a fixed resolution of 1.40625". Table 4 provides a quantitative performance comparison of different models when the observation proportion is reduced to 1% and when the forecast lead time of the background is extended to 48 hours. In these scenarios, all the models exhibit robustness and consistently improve the background. When the number of observations decreases or the background error increases, the amount of conditional

---

information they can provide becomes less, so it is reasonable to observe an increase in analysis error compared [ Similarly, FNP achieves SOTA performance terms of overall MSE, MAE, and WRMSE for the majority of variables.
Table 4: Ablation study the forecast lead time of the background and the ratio of observations.
WRMSE
Model
Ratio
MSE
MAE
2500
(2m
u10
u500
4700 (10-1)
24h Background
0.0288
0.0861
45.455
0,7200
0.7790
0,9336
0.9645
1.7278
1.7535
6.7220
0,0272
30.595
0.9115
1.6881
6.7074
Con-CNP 20
1%
0.0283
0.0856
43.272
0.7186
0.7785
0,9286
0.9574
1.7203
1.7428
6.7173
FNP (ours)
1%
0.0269
0.0812
30.678
0.6971
0.7708
0,9069
0.9351
1.6586
1.6778
6,7084
48h Background
0.0467
0.1148
85.017
0,9103
0,9283
1,2525
1,2882
2.3583
2.3856
8.2664
Adas [0
10%
0.0305
0.0840
30.473
0.7355
0.8134
1.0047
1.0176
1.7747
1.7217
7,8209
ConvCNP
10%
0.0334
0.0918
40.378
0.7774
0.8619
0.9251
0.9422
1.8112
1.8160
7.7774
FNP (ours)
10%
0.0252
0.0743
28.883
0,7459
0.7810
0.8307
0.8517
1.6285
1.6348
7,6078
Conclusions
In summary, present FNP that can assimilate observations with arbitry resolution. The outstanding performance and out-of-domain generalization of FNP in data assimilation and observational infor- mation reconstruction demonstrate its significant potential and broad application prospects. It not only contributes to the field of data assimilation but also makes meaningful explorations for Al-based end-to-end weather forecasting systems. Technically and theoretically, this methodology can also be applied to more tasks and scenarios, such as downscaling [37] station-scale state estimation and
weather prediction (36 60 24
Our work has certain limitations. Firstly, the observational data used our experiments are generated through simulations rather than real-world observations. This may lead to differences in model performance when applied actual scenarios, thus discounting its value for practical application. In fact, due to the complex and diverse nature of real observational data, the data assimilation community lacks relevant benchmarks and large-scale datasets. The establishment of such benchmarks and datasets would be highly meaningful endeavor, enabling fair comparisons among different models and fostering rapid udvancements in the field. Secondly, FNP inherently performs 3D data assimilation without the temporal dimension. Considering the flexibility of the FNP architecture, incorporating the temporal dimension is not challenging and is expected to produce additional benefits. In the future, will further explore the broader possibilities of data assimilation and end-to-end weather forecasting.
Acknowledgments and Disclosure of Funding
This work is supported by National Natural Science Foundation of China (No. 62071127 and 62101137), National Key Research and Development Program of China (No. 2022ZD0160101, Shanghai Natural Science Foundation (No. 23ZR 1402900), Shanghai Municipal Science and Tech- nology Major Project (No.2021SH2DDXX0000) and Shanghai Artificial Intelligence Laboratory.
References
[1] Maddalena Amendola, Rossella Arcucci, Lactitia Mottet, César Quilodrán Casas, Shiwei Fan, Christopher Pain, Paul Linden, and Yi-Ke Guo. Data assimilation in the latent space of convolutional autoencoder. In International Conference on Computational Science, 2021.
[2] Tom Andersson, Wessel Bruinsma, Stratis Markou, James Requeima, Alejandro Coca- Castro, Anna Vaughan, Anna-Louise Ellis, Matthew A Lazzara, Dani Jones, Scott Hosking, et al. Environmental sensor placement with convolutional gaussian neural processes. Environmental Data Science, 2023.
[3] Rossella Areucci, Jiangcheng Zhu, Shuang Hu, and Yi-Ke Guo. Deep data assimilation: integrating deep learning with data assimilation. Applied Sciences. 2021.
10

---

[4] Mark Asch, Mare Bocquet, and Maëlle Nodet. Data assimilation: methods. algorithms, and applications. 2016.
[5] Zied Ben Bouallègue, Mariana CA Clare, Linus Magnusson, Estibaliz Gascon, Michael Maier- Gerber, Martin Janoušek, Mark Rodwell, Florian Pinault, Jesper S Dramsch, Simon TK Lang, al. The rise of data-driven weather forecasting: A first statistical assessment of machine lcarning-based weather forecasts in an operational-like context. Bulletin of the American Meteorological Society, 2024.
[6] Kaifeng Bi, Lingxi Xie, Hengheng Zhang, Xin Chen, Xiaotao Gu, and Qi Tian. Accurate medium-range global weather forecasting with 3d neural networks. Nature, 2023.
[7] Caterina Buizza, César Quilodrán Casas, Philip Nadler, Julian Mack, Stefano Marrone, Zainab Titus, Clémence Le Cornec, Evelyn Heylen, Tolga Dur, Luis Baca Ruiz, et al. Data learning: Integrating data assimilation and machine learning. Journal of Computational Science, 2022.
[8] Alberto Carrassi, Mare Bocquet, Laurent Bertino, and Geir Evensen. Data assimilation in the geosciences: An overview of methods, issues, and perspectives. Wiley Interdisciplinary Reviews: Climate Change, 2018.
[9] Kang Chen, Tao Han, Junchao Gong, Lei Bai, Fenghua Ling, Jing-Jia Luo, Xi Chen, Leiming Ma, Tianning Zhang, Rui Su, et al. Fengwu: Pushing the skillful global medium-range weather forecast beyond 10 days lead, arXiv preprint urXiv:2304.00944 2023.
[10] Kun Chen, Lei Bai, Fenghua Ling, Peng Ye, Tao Chen, Kang Chen, Tao Han, and Wanli Ouyang. Towards an end-to-end artificial intelligence driven global weather forecasting system. arXiv preprint arXiv:2312. /2462, 2023.
[11] Lei Chen, Xiaohui Zhong, Feng Zhang, Yuan Cheng, Yinghui Xu, Yuan Qi, and Hao Li. Fuxi: a cascade machine learning forecasting system for 15-day global weather forecast. npj Climate and Atmospheric Science, 2023.
[12] Sibo Cheng and Mingming Qiu. Observation error covariance specification in dynamical systems for data assimilation using recurrent neural networks. Neural Computing and Applications, 2022.
[13] Sibo Cheng, César Quilodrán- Said Ouala, Alban Farchi, Che Liu, Pierre Tandeo, Ronan Fablet, Didier Lucor, Bertrand Iooss, Julien Brajard, et al. Machine learning with data assimi- lation and uncertainty quantification for dynamical systems: review. IEÉE/CAA Journal of Automatica Sinica, 2023.
[14] Yann Dubois, Jonathan Gordon, and Andrew YK Foong. Neural process family. http:
//yanndubs Bat.aa/.ooo/euuaa-ooeeeeee Family/ 2020.
[15) Marius Egli, Sebastian Sippel, Angeline G Pendergrass, Iris de Vries, and Reto Knutti. Re- construction of zonal precipitation from sparse historical observations using climate model information and statistical learning. Geophysical Research Letters, 2022.
[16] Andrew Foong, Wessel Bruinsma, Jonathan Gordon, Yann Dubois, James Requeima, and Richard Turner. Meta-learning stationary stochastic process prediction with convolutional neural processes. Advances in Neural Information Processing Systems, 33:8284-8299 8295, 2020.
[17] Marta Garnelo, Dan Rosenbaum, Christopher Maddison, Tiago Ramalho, David Saxton, Murray Shanahan, Yee Whye Teh, Danilo Rezende, and SM Ali Eslami. Conditional neural processes. In International conference on machine learning, 2018.
[18] Marta Garnelo, Jonathan Schwarz, Dan Rosenbaum, Fabio Viola, Danilo J Rezende, SM Eslami, and Yee Whye Teh. Neural processes. arXiv preprint arXiv:1807.01622 2018.
[19] Alan Geer. Learning earth system models from observations: machine learning data assimilation? Philosophical Transactions of the Royal Society A, 2021.
[20] Jonathan Gordon, Wessel Bruinsma, Andrew YK Foong, James Requeima, Yann Dubois, and Richard E Turner. Convolutional conditional neural processes. In International Conference Learning Representattonss 2019.
[21] Nils Gustafsson, Tijana Janjić, Christoph Schraff, Daniel Leuenberger, Martin Weissmann, Hendrik Reich, Pierre Brousseau, Thibaut Montmerle, Eric Wattrelot, Antonín Bučánek, et al. Survey of data assimilation methods for convective-scale numerical weather prediction at operational centres. Quarterly Journal of the Royal Meteorological Society, 2018.
11

---

[22] Yoo-Geun Ham, Yong-Sir Sik Joo, Jeong-Hwan Kim, Kang-Min Min Kim, and Jeong- -il Lee. Partial- convolution convolution-mmlleeentee generative adversarial network (gan) for global oceanic data assimila- tion. 2022.
[23] Thomas M Hamill. Ensemble- bbasd atmospheric data assimilation. Predictability of weather and climate, 2006.
[24] Tao Han, Song Guo, Zhenghao Chen, Wanghan Xu, and Lei Bai. Weather-5B 5k: A large-scale global station weather dataset towards comprehensive time-series forecasting benchmark. arXiv preprint arXiv:2406. 14399, 2024.
[25] Sam Hatfield, Matthew Chantry, Peter Dueben, Philippee Lopez, Alan Geer, and Tim Palmer. Building tangent- linear and adjoint models for data assimilation with neural networks. Journal of Advances in Modeling Farth Systems, 2021.
[26] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition. 2016.
[27] Hans Hersbach, Bill Bell, Paul Berrisford, Shoji Hirahara, András Horányi, Joaquín Muñoz- Sabater, Julien Nicolas, Carole Peubey, Raluca Radu, Dinand Schepers, et al. The era5 global reanalysis. Quarterly Journal of the Royal Meteorological Society, 2020.
[28] Langwen Huang, Lukas Gianinazzi, Yuejiang Yu, Peter D Dueben, and Torsten Hoefler. Diffda: a diffusion model for weather-scale data assimilation. arXiv preprint PrXiv:2401.09933 2024.
[29] Christopher Kadow, David Matthew Hall, and Uwe Ulbrich. Artificial intelligence reconstructs missing climate information. Nature Geoscience, 2020.
[30] Hyunjik Kim, Andriy Mnih, Jonathan Schwarz, Marta Garnelo, Ali Eslami, Dan Rosenbaum, Oriol Vinyals, and Yee Whye Tch. Attentive neural processes. In International Conference on Learning Representations, 2018
[31] Remi Lam, Alvaro
Matthew Willson, Peter Wirnsberger, Meire Fortunato, Ferran Alet, Suman Ravuri, Timo Ewalds, Zach Eaton-Rosen, Weihua Hu, et al. Learning skillful medium-range global weather forecasting. Science, 2023.
[32] François-Xave Xavier Le Dimet and Olivier Talagrand. Variational algorithms for analysis and assimilation of meteorological observations: theoretical aspects. Tellus A: Dynamic Meteorology and Oceanography, 1986.
[33] Zhuoyuan Li, Bin Dong, and Pingwen Zhang. Latent assimilation with implicit neural represen- tations for unknown dynamics. Journal of Computational Physics, 2024.
[34] Zongyi Li, Nikola Kovachki, Kamyar Azizzadenesheli, Burigede Liu, Kaushik Bhattacharya, Andrew Stuart, and Anima Anandkumar. Fourier neural operator for parametric partial differen- tial equations. arXiv preprini arXiv:2010.08899,, 2020.
[35] Fenghua Ling, Lin Ouyang, Boufeniza Redouane Larbi, Jing-Jia Luo, Tao Han, Xiaohui Zhong, and Lei Bai. Is artificial intelligence providing the second revolution for weather forecasting? arXiv preprint Xrxiv:2401. 16669, 2024.
[36] Zili Liu, Hao Chen, Lei Bai, Wenyuan Li, Keyan Chen. Zhengyi Wang, Wanli Ouyang, Zhengxia Zou, and Zhenwei Shi. Deriving accurate surface metcorological states at arbitrary locations via cerrvaationguuddee continous neural field modeling. IEFE Transactions on Geoscience and Remote Sensing, 2024.
[37] Zili Liu, Hao Chen, Lei Bai, Wenyuan Li, Wanli Ouyang, Zhengxia Zou, and Zhenwei Shi. Mambads: Near-surface meteorological field downscaling with topography constrained selective state space modeling. arXiv preprint arXiv:2408. 10854, 2024.
[38]
Andrew C Lorenc. Analysis methods for numerical weather prediction. Quarterly Journal of the Royal Meteorological Society, 1986.
[39] Ilya Loshchilov and Frank Hutter. Fixing weight decay regularization in adam. 2018
[40] Ziqi Ma, Jianbin Huang, Xiangdong Zhang, Yong Luo, Minghu Ding, Jun Wen, Weixin Jin, Chen Qiao, and Yifu Yin. Newly reconstructed arctic surface air temperatures for 1979-2021 with deep learning method. Scientific Data, 2023.
[41] Boštjan Meline and Ziga Zaplotnik. keural-network data assimilation using variational autoen- coder. arXiv preprint 23Xi:2308.16077 2023.
12

---

[42] Peiman Mohseni and Nick Duffield. Spectral convolutional conditional neural processes. arXiv preprint arXiv:2404.13188 13182, 2024.
[43] Deep Shankar Pandey and Qi Yu. Evidential conditional neural processes. In Proceedings of the AAAI Conference on Artificial Intelligence, 2023.
[44] Jaideep Pathak, Shashank Subramanian, Peter Harrington, Sanjeev Raja, Ashesh Chattopadhyay, Morteza Mardani, Thorsten Kurth, David Hall, Zongyi Li, Kamyar Azizzadenesheli, al. Fourcastnet: A global data-driven high-resolution weather model using adaptive fourier neural operators. arXiv preprint arXiv:2202. 1214, 2022.
[45] Stephen G Penny, Timothy A Smith, T-C Chen, Jason A Platt, H-Y Lin, Michael Goodliff, and Henry DI Abarbanel. Integrating recurrent neural networks with data assimilation for scalable data-driven state estimation. Journal of Advances in Modeling Earth Systems, 2022.
[46] Jens Petersen, Gregor Köhler, David Zimmerer, Fabian Isensee, Paul F Jäger, and Klaus H Maier-Hein. Gp-convenp: Better generalization for convolutional conditional neural processes on time series data. arXiv preprint orXiv:2106.04966 2021.
[47] Mathis Peyron, Anthony Fillion, Selime Gürol, Victor Marchais, Serge Gratton, Pierre Boudier, and Gael Goret. Latent space data assimilation by using deep learning. Quarterly Journal of the Royal Meteorological Society, 2021.
[48] Yongquan Qu, Juan Nathaniel, Shuolin Li, and Pierre Gentine. Deep generative data assimilation in multimodal setting. arXiv preprint arXiv:2404.0660
2024.
[49] Florence Rabier and Zhiquan Liu. Variational data assimilation: theory and overview. In Proc. ECMWF Seminar on Recent Developments in Data Assimilation for Amooshere and Ocean, Reading, UK, September 8-12, 2003.
[50] Stephan Rasp. Peter D Dueben, Scbastian Scher, Jonathan A Weyn, Soukayna Mouatadid, and Nils Thuerey. Weatherbench: benchmark data set for data-driven weather forecasting. Journal of Advances in Modeling Earth Systems, 2020.
[51] Stephan Rasp, Stephan Hoyer, Alexander Merose, Ian Langmore, Peter Battaglia, Tyler Russel, Alvaro Maachez-GGnzalezz. Vivian Yang, Rob Carver, Shreya Agrawal, et al. Weatherbench 2: A benchmark for the next generation of data-driven global weather models. arXiv preprint arXiv:2308. 15560, 2023.
[52] François Rozet and Gilles Louppe. Score-based data assimilation. Advances in Neural Informa- tion Processing Systems, 2023.
[53] Jonas Scholz, Tom R Andersson, Anna Vaughan, James Requeima, and Richard E Turner. Sim2real for environmental neural processes. arXiv preprint arXiv:2310. 19932, 2023.
[54] Anna Vaughan, Stratis Markou, Will Tebbutt, James Requeima, Wessel P Bruinsma, Tom Andersson, Michael Herzog, Nicholas D Lane. J Scott Hosking, and Richard E Turner. Aardvark weather: end-to-end data-driven weather forecasting. arXiv preprint arXiv:2404. 240::2440.00411, 2024.
[55] Anna Vaughan, Will Tebbutt, Scott Hosking, and Richard E Turner. Convolutional conditional neural processes for local climate downscaling. Geoscientific Model Development, 2022.
[56] Wuxin Wang, Weicheng Ni, Tao Han, Lei Bai, Boheng Duan, and Kaijun Ren. Dabench: A benchmark dataset for daiven weather data assimilation. arXiv preprint 1433 2024.
[57] Yueya Wang, Xiaoming Shi, Lili Lei, and Jimmy Chi thi-Hung Fung. Deep learning augmented data assimilation: Reconstructing missing information with convolutional autoencoders. Monthly Weather Review, 2022.
[58] Zhongrui Wang, Lili Lei, Jeffrey L Anderson, Zhe-Min Tan, and Yi Zhang. Convolutional neural network- based adaptive localization for an ensemble kalman filter. Journal of Advances in Modeling Earth Systems, 2023.
[59] Martin Wegmann and Fernando SanteeS Artificial intelligence achieves casy-to-adapp nonlinear global temperature reconstructions using minimal local data. Communications Earth & Environment, 2023.
[60] Haixu Wu, Hang Zhou, Mingsheng Long, and Jianmin Wang. Interpretable weather forecasting for worldwide stations with a unified deep model. Nature Machine Intelligence, 5(6)6022-61,
2023.
13

---

[61] Wanghan Xu, Kang Chen, Tao Han, Hao Chen, Wanli Ouyang, and Lei Bai. Extremecast: Boosting extreme value prediction for global weather forecast. arXiv preprint arX1v:4402.01295 2024.
[62] Xiaoze Xu, Xiuyu Sun, Wei Han, Xiaohui Zhong, Lei Chen, and Hao Li. Fuxi-da: A generalized deep learning data assimilation framework for assimilating satellite observations. arXiv preprint arXiv:2404.08522, 2024.
14

---

A Data Availability
The ERA5 dataset can be downloaded from the official website of Climate Data Store (CDS)
https://cds. http://cc...lm......eooemmmm wel.mate.copenicuus
B More Visualizations for Data Assimilation
Background
Background Emror
Chservations
16
20
45
-10
28
Analys
Analysis
Anolysio
Interpolated Obsevivation
yss
Aralysi
Analysis
Analysic
Analycis Er
Figure 4: Visualization assimilation results with 0.25 resolution for u10.
15

---

Becaground
Background
Observations
Analys of Adas
Analysis increment of Adas
Analysis Enorr of Ados
Interpolated Obsevivations
Adalydi Mean of ConVCNP
Analysis Increment ConVCN?
Analysic Error of ConVCNP
Analys Variance ConVCNP
Analysia Mean of HP
Analysis increment of HN
Analysis broor HNP
Analysis Variar co of Hap
750
780
330
2.3
30
Figure 5: Visualization of assimilation results with 0.25 resolution for t2m.
16

---

Bacaground
Background
Observations
Analys of Ados
Analysia increment of Adas
Analysis Enor of Adis
Interpolated Obsevrvations
58300
Ana ysis Mean of ConVCNP
Analysis Increment ConVEN?
Analysis Erro of COnVCNP
Analyss Variance ConVCNP
2000
58000
Anelysis Mean of MP
Analysis increment of TNP
Analysis trror INP
Analysis Varirce of PNP
13000 30006
52009 50000 o/aa
18399
19300
20C
1392
1206 1299 1900 3000 10Ee anaa
Figure 6: Visualization of assimilation results with 0.25% resolution for z500.
17

---

Beckground
Background
Observatinss
JOME
Kala
Analyss of Adas
Analysia increment of Adas
Analysis Eror of AGGS
Interpolated Obsevrvations
2002 0,34
8110
-1.62
0.00
2022
CCEE
ydil Mean of COnVONP
Analysis Increment CONVENP
Analysic Error of ConVCNP
Analys Variance ConVCNP
0000 3002 0.34
030
-1062
0000
-0.00
3932
050
100
125
byhy
aghg
Agley
Analysia Mean of HP
Analysis increment of HP
Analysis broo HNP
Analysis Var Variarce of HNP
BO00 3003 0.30 5000 0006 5310 kghg
-0.02
0000
6000
D004
-0.044
0.000
5332
0.04
Figure 7: Visualization of assimilation results with 0.703125 resolution for 9000.
18

---

ERAS
Background
Background Error
Cbservatiod
2833 8032
6863 B502
-1.02
8833
Aplp
Apri
kphp
Analysis of Adas
Analysis increment of Adas
Analysis Brrer of Ades
2033 2007 5391 0036 0.06 6n1D
-0,04
2.000
3.003 hgil
0000
0.344
-0.30
-3.00
2.033
0.000
bghp
Agli
Analysis Mean of ConVCNP
Analysis increment ConVENP
Analysis Error of ConVCNP
Analysis Variance ConvCNP
1000 2012 0.04
0006 0.00 0010
-0004
-2.003
3.00 1plg
0003
5334
-000
-2.00
2000 Apli
0000
2014
kglp
Anelysis Mean HNP
Analysis increment
Analysis trror PNP
Analysis Veriance
Z
5003 B002 5991 0000
GO0 D0TD
-CUD0
-0.002
5002
0002
5331
-0.20
-2.02
2022
0002
2091
Ag?
Figure 8: Visualization of assimilation results with 1.40625 resolution for 700.
19

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: The main claims made in the abstract and introduction accurately reflect the paper's contributions and scope.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made in the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made the paper and important assumptions and limitations. A No or NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
It is fine to include aspirational goals as motivation long is clear that these goals are not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
Justification: The paper discusses the limitations of the work performed by the authors in Section 5
Guidelines:
The answer NA means that the paper has limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create a separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (c.g.. independence assumptions, noiseless settings, model sell-speciiiatiinn asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.
The authors should reflect the scope of the claims made, e.g. if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution low or images are taken in low lighting. Or speech-to-tex, system might not be used reliably to provide closed captions for online lectures because it fails handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and faimesss.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't 't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community, Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory Assumptions and Proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
20

---

Answer:
[NA]
Justification:
The paper does not include theoretical results.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced in the statement of any theorems.
The proof's can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide a short proof sketch provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental Result Reproducibility
Does the paper fully disclose all the information needed to reproduce the main ex- perimental results the paper to the extent that affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided not)?
Question:
Answer: [Yes]
Justification: The paper fully discloses all the information needed to reproduce the main experimental results in Section and released code.
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, descríbing the architecture fully might suffice, or if the contribution is specific model and empirical evaluation, it may be necessary to either make possible for others to replicate the model with the same dataset, provide access to the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g.. g., in the case of a large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) the contribution is primarily a new algorithm, the paper should make clear how reproduce that algorithm. (b) If the contribution is primarily new model architecture, the paper should describe the architecture clearly and fully.
(c) the contribution is a new model (e-g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should possible for other researchers to have some path to reproducing verifying the results.
5. Open access to data and code
21

---

The authors should provide scripts to reproduce all experimental results for the new reproducible, they Question: Does the paper provide open access to the data and code, with sufficient instruc- tions faithfully reproduce the main experimental results, as described in supplemental material?
Answer: [Yes]
Justification: The paper provides open access to the code in Appendix ??, and the data used in the paper is open-source dataset.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurIPS code and data submission guidelines (https://nips.cm cc/
public/guides/ Soaeleess/CooeSuuubsssisiiiii
for more details.
While we encourage the release of code and data, we understand that this might be possible, so "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines (https
/ https/ccpppllciggudeeeeeeeeeeeee Cooee//bdeeeSbmmmcia ssi onPolicy) for more details.
The authors should provide instructions data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
proposed method and baselines. only subset of experiments should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended to the paper) recommended, but including URLs to data and code is permitted.
6. Experimental Setting/Details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification: The paper specifies all the training and test details necessary understand the results in Section 4 and released code.
Guidelines:
The answer NA means that the paper does include experiments.
The experimental setting should be presented in the core the paper to level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, as supplemental material.
7. Experiment Statistical Significance
Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Question:
Answer: [No]
Justification: The paper does not report error bars, like other related work.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, or statistical significance tests, least for the experiments that support the main claims of the paper.
The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall with given experimental conditions).
22

---

The method for calculating the error bars should be explained (closed form formula, call a library function, bootstrap, etc.)
The assumptions made should be given (e.g., Normally distributed errors).
It should be clear whether the error bar is the standard deviation or the standard error of the mean.
OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that out of range (e.g. negative error rates).
If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.
8. Experiments Compute Resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time execution) needed to reproduce the experiments?
Question:
Answer: [Yes]
Justification: The paper provides sufficient information on the computer resources needed to reproduce the experiments in Section
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount compute required for each of the individual experimental runs well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make into the paper).
9. Code Of Ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics
Answer: [Yes]
Justification: The research conducted in the paper conforms with the NeurIPS Code of Ethics in every respect.
Guidelines:
The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.
If the authors answer No. they should explain the special cireumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g., if there is special consid- eration due to laws or regulations their jurisdiction).
10. Broader Impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
Justification: The paper discusses both potential positive societal impacts and negative
societal impacts of the work performed Section 5
Guidelines:
The answer NA means that there no societal impact of the work performed.
If the authors answer NA or No, they should explain why their work has no societal impact why the paper does not address societal impact.
23

---

Examples of negative societal impacts include potential malicious unintended uses (e.g., disinformation, generating fake profiles, surveillance), faireess considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional unintentional) misuse of the technology.
If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML).
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification:
The paper poses such risks.
Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere usage guidelines restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Are the creators or original owners of assets (e.g.. code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Question:
Answer: [Yes]
Justification: We cite the original papers websites that produced the code dataset, and the data used in the paper is an open-source dataset.
Guidelines:
The answer NA means that the paper does use existing assets.
The authors should cite the original paper that produced the code package or dataset.
The authors should state which version of the asset is used and, if possible, include URL.
The name of the license (e.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.
24

---

If assets are released, the license, copyright information, and terms of use in the thcode. has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
package should be provided. For popular datasets, paperswi
batasets.papersstt
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
If this information is not available online, the authors encouraged to reach out the asset's s creators.
13. New Assets
Question: Are assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [Yes]
Justification: The code in the paper is well documented and the documentation is provided alongside the code.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/code/moodl as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset is used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsoureing and Research with Human Subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Answer: [NA]
Justification:
The paper does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then much detail as possible should be included the main paper.
According to the NeurlPS Code of Ethics, workers involved in data collection, curation, other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional Review Board (IRB) Approvals or Equivalent for Research with Human Subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [NA]
Justification:
The paper does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does involve crowdsourcing nor research with human subjects.
Depending the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
25

---

We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors adhere the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
26


Reviews:
[{"review_id": "Y9QMOF9mPx", "reviewer": "Reviewer_95HC", "summary": "The authors propose a method for arbitrary-resolution data assimilation, called Fourier Neural Processes (FNP). This approach improves generalization by addressing resolution limitations in existing methods. Key features include unified coordinate transformation, spatial-variable functional representation, and dynamic alignment and merge (DAM). The method effectively integrates diverse observational data without the need for fine-tuning. Experimental results demonstrate its relatively good performance in handling varied data sources.", "questions": "Could the authors clarify the unique contributions of their method in addressing the issue of arbitrary resolution from a design perspective? It seems that the core of your approach relies on interpolation in the feature space. If the generalization ability for arbitrary resolutions is primarily due to this interpolation, does this mean that integrating similar interpolation techniques into any deep learning method could achieve similar improvements in generalization?", "limitations": "The authors point out the key limitation of their method: it has not been tested on a real-world dataset due to the lack of relevant benchmarks and large-scale datasets in the data assimilation community.", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 2, "contribution": 3, "strengths": "(+) This method is the first to address the challenge of data assimilation with arbitrary resolutions.\n\n(+) The authors introduce the neural processes to arbitrary-resolution data assimilation.\n\n(+) Demonstrating excellent generalization, the proposed method can perform observational information reconstruction directly from training on data assimilation, without requiring fine-tuning.\n\n(+) Each proposed component is simple yet effective, as shown in the ablation study.", "weaknesses": "Since the authors aim to address arbitrary-resolution data assimilation with good generalization ability, it would be beneficial to clearly highlight the key observations or components for this problem upfront. The authors quickly delve into the pipeline description without first presenting the key and unique idea of their method for this problem. It takes readers some time to figure out, 'How does this method solve the issue of arbitrary resolution in data assimilation?'", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Dear reviewer,\n\nThank you very much for your detailed review, thoughtful comments, and feedback! We appreciate your recognition of the effectiveness of our method, while deeply regretting and apologizing for any confusion or concerns we may have caused you regarding the motivation and design of our approach. Below, we address each of your questions in turn.\n\n> Since the authors aim to address arbitrary-resolution data assimilation with good generalization ability, it would be beneficial to clearly highlight the key observations or components for this problem upfront.\n\nWeather forecasting is of paramount importance to both science and society. In recent years, AI-based medium-range weather forecasts have advanced rapidly, yet they still rely on traditional data assimilation techniques from conventional numerical weather prediction (NWP) systems to provide initial states. Consequently, data-driven data assimilation methods have garnered increasing attention as a crucial component in constructing end-to-end weather forecasting systems based on AI. These AI-based approaches not only significantly reduce resource consumption but also offer new possibilities for overcoming bottlenecks in traditional numerical methods.\n\nExisting work is primarily designed and trained based on specific resolutions (often matching the background resolution), resulting in models that can only be utilized in the same settings. In this paper, we propose the Fourier Neural Processes (FNP) for arbitrary-resolution data assimilation, whose necessity and importance lie mainly in the following points. Firstly, FNP eliminates the need to interpolate observational data, typically with higher resolution, which can avoid the information loss and improve the performance of data assimilation. Secondly, observational data in practical applications vary greatly in type and resolution; FNP can directly assimilate the diverse data without preprocessing, enhancing its practical application value. Lastly, the challenge of dealing with high-dimensional data is a common issue faced by traditional and AI methods; As our experiments demonstrate, the outstanding generalization ability of FNP enables training at low resolutions and direct application at high resolutions, thereby significantly reducing computational resource consumption.\n\n> The authors quickly delve into the pipeline description without first presenting the key and unique idea of their method for this problem. It takes readers some time to figure out, 'How does this method solve the issue of arbitrary resolution in data assimilation?’\n\nFrom an implementation perspective, the ability to support inputs of arbitrary resolutions lies in the fact that fundamental operations (such as convolution, MLP, etc.) are independent of input size. Neural processes operate on paired (context or target) coordinate-value units, making them highly suitable for data assimilation tasks where observational data may take irregular forms. Therefore, we introduced a convolutional version of neural processes (i.e., ConvCNP) and incorporated custom modules independent of input size to achieve data assimilation with arbitrary resolutions. However, as indicated by our experiments, the performance of ConvCNP is unsatisfactory. Therefore, from a performance standpoint, the key to effectively addressing the issue of arbitrary resolutions lies in the efficacy of our module design (e.g., unified coordinate transformation and dynamic alignment to ensure the model comprehends spatial correspondences between data of different resolutions, and the neural Fourier layer providing significant high-frequency information gain when assimilating high-resolution observations).\n\n> Could the authors clarify the unique contributions of their method in addressing the issue of arbitrary resolution from a design perspective?\n\nBased on the aforementioned ideas, we can categorize the core contributions of our approach in design into two aspects. Firstly, we astutely recognized the unique advantages of neural processes in addressing data assimilation challenges, introducing neural processes into data assimilation tasks to enable data assimilation with arbitrary resolutions. Secondly, through efficient module design, we achieved a significant improvement in performance and generalization, making FNP the only model whose performance demonstrates consistency with theoretical understanding (with the increase of resolution and the increase of observational information, the model's performance continues to improve significantly).\n\n> It seems that the core of your approach relies on interpolation in the feature space. If the generalization ability for arbitrary resolutions is primarily due to this interpolation, does this mean that integrating similar interpolation techniques into any deep learning method could achieve similar improvements in generalization?\n\nBased on the above summary, interpolation in feature space is only a small part of our method. In fact, in the results we presented, ConvCNP also employed interpolation and simple fusion instead of DAM, yet its generalization was poor. Here, we further provide ablation study on generalization of different modules, as shown in Table A from the PDF in global response. The models in the table are all trained at 1.40625° resolution and directly tested at other resolutions to validate their generalization. It can be observed that all modules contribute to enhancing generalization to varying degrees. In addition, not all deep learning methods can support inputs of arbitrary sizes, thereby combining interpolation to achieve data assimilation with arbitrary resolutions. Even if they can, simple interpolation does not necessarily lead to improved generalization.\n\nWe sincerely hope that we have addressed your concerns and questions and look forward to your reading and response. If you have any further questions, please feel free to let us know and we will do our best to answer them.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "LGj744101o", "reviewer": "Reviewer_3aNd", "summary": "This paper proposes a new variant of neural processes called Fourier Neural Processes (FNPs) to solve the data assimilation problem with arbitrary solution, which is an important component in modern weather forecast system. The proposed method based on FNP has better computational efficiency, and achieves state-of-the-art performance in assimilating observations with varying resolutions on large-scale simulated data.", "questions": "See weakness.\n\nI would like to also ask the following:\n1. What do you think is the main reason that FNP outperforms ConvCNPs? What do you think is the most important innovation of FNP for the data assimilation problem, given that ConvNPs (https://proceedings.nips.cc/paper/2020/file/5df0385cba256a135be596dbe28fa7aa-Paper.pdf) already has applications for Environmental Data.\n2. Why the neural Fourier layer (NFL) is more computational efficient than standard conv layers, if NFL contains three branches, one one branch is Conv operations?", "limitations": "The authors have thoroughly discussed the limitations of this work.", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 2, "contribution": 3, "strengths": "This paper focuses on applying neural processes to the data assimilation problem in weather forecast system. Different from many previous work in this area which mainly focus on toy examples, or hand designed problems, this work solves a real application problem that has high impact. I really enjoy this paper and would love to see more work like this in our area.", "weaknesses": "The primary issue with this paper is its technical clarity. The majority of the technical nuances are conveyed through textual descriptions, and at times, the terminology is not adequately explained. It would be beneficial if the authors could provide the precise mathematical formalisms for key modules such as the “Spatial-variable decoupled representation,” the Neural Fourier layer, and DAM. Although one can how these modules work, a definitive mathematical expression would greatly alleviate any confusion.\n\nI am also slightly confused by the following usage of words:\n1. Conditional points/conditional domain. Do you mean \"context\" as supposed to \"target\"?\n2. Variable dimension. Do you mean the dimension across different meteorological variables at the same spatial location?\n3. What do you mean by \"dynamics\" at line 175.\n4. Line 228, the term RMSE is a statistical metric used way beyond geospatial analysis and atmospheric science. So you might want to use something else to refer to the special \"latitude-weighted\" version.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Dear reviewer,\n\nThank you very much for your thorough review, highly constructive comments, and feedback! We greatly appreciate your positive reception of our work and recognition of its practical value. At the same time, we deeply regret any confusion or difficulties you may have encountered regarding our methodological expressions and terminologies. We will address each of your points in the following responses.\n\n> It would be beneficial if the authors could provide the precise mathematical formalisms for key modules such as the “Spatial-variable decoupled representation,” the Neural Fourier layer, and DAM. Although one can how these modules work, a definitive mathematical expression would greatly alleviate any confusion.\n\nExcellent suggestion! Following the description in Figure 1, we will endeavor to provide mathematical formulations for the key modules in FNP.\n\n- Functional representation (FR): $FR(x^c,y^c)=Concate(SetConv(x^c),SetConv(y^c))$\n\n- Spatial-Variable decoupled representation (SVD): As described in Section 3.2, we model a separate spatial FR for each variable and a variable FR for all variables instead of a mixed FR, with each FR following the formula above. \n\n- Neural Fourier layer (NFL): $NFL(·)=\\mathcal{F}^{-1}(Linear(\\mathcal{F}(·)))+Conv(·)+Identity(·)$\n\n- Dynamic Alighment and Merge (DAM): The DAM module aligns the functional representations of the background and observations in the spatial dimension and merges them. The alignment is achieved through interpolation operations, while the merge is implemented by selecting based on similarity to shared features, with the calculation for similarity and selection rules provided in the formula in Section 3.3.\n\n> Conditional points/conditional domain. Do you mean \"context\" as supposed to \"target\"?\n\nConditional points and conditional domain refer to the context points and context domain, corresponding to the input of the model, whlie target points and target domain correspond to the output of the model.\n\n> Variable dimension. Do you mean the dimension across different meteorological variables at the same spatial location?\n\nYes, it is also the channel dimension, because the data of different variables are concatenated on the channel dimension.\n\n> What do you mean by \"dynamics\" at line 175.\n\nIt means the ability to support inputs of varying sizes, i.e., assimilating observations with different resolutions.\n\n> Line 228, the term RMSE is a statistical metric used way beyond geospatial analysis and atmospheric science. So you might want to use something else to refer to the special \"latitude-weighted\" version.\n\nYou are right, in atmospheric science, we usually use RMSE to refer to the latitude-weighted version rather than the standard version. We appreciate and respect your rigor, and we will use the term \"WRMSE\" to replace \"RMSE\" in final version.\n\n> What do you think is the main reason that FNP outperforms ConvCNPs?\n\nThe main difference between FNP and ConvCNP lies in our tailored design for data assimilation task. The SVD reduces the complexity of model training and computational resource consumption while achieving better performance. Unified coordinate transformation and dynamic alignment ensure that the model comprehends the spatial correspondences between data of different resolutions, while dynamic selection and merge mechanism enhances the effectiveness of information fusion. The visualization in Figure 2 demonstrates the enhancement in the model's ability to extract deep features and capture high-frequency information through the NFL. In our supplementary experiments (see Table A from the PDF in global response), as the observational resolution increases, the impact of the NFL in ablation study on generalization becomes increasingly evident, further validating its effectiveness.\n\n>What do you think is the most important innovation of FNP for the data assimilation problem, given that ConvNPs (https://proceedings.nips.cc/paper/2020/file/5df0385cba256a135be596dbe28fa7aa-Paper.pdf) already has applications for Environmental Data.\n\nFNP has the capability to assimilate observational data with arbitrary resolution without the need for prior interpolation of observations that typically have higher resolutions. It avoids information loss, thereby enhancing the performance of data assimilation. The types and resolutions of observational data in practical applications are highly diverse and complex. FNP can directly assimilate such data without pre-processing, offering greater practical value. Additionally, the challenge of dealing with high-dimensional data is a common issue faced by traditional methods and AI approaches. As our experiments have shown, the outstanding generalization ability of FNP enables training at low resolutions and direct application to high resolutions. This not only significantly reduces computational resource consumption but also provides a pathway for data assimilation at higher resolutions.\n\n> Why the neural Fourier layer (NFL) is more computational efficient than standard conv layers, if NFL contains three branches, one one branch is Conv operations?\n\nIn the ablation study presented in Table 3, FNP utilized a structure encoded with 4 NFL blocks, amounting to 67.872G FLOPs. In contrast, the FNP without NFL opted for a replacement comprising 12 convolutional blocks (consistent with the official implementation in [1]), totaling 167.932G FLOPs. FNP's ability to achieve better performance with lower computational complexity demonstrates its higher computational efficiency.\n\n- [1] Yann Dubois, Jonathan Gordon, and Andrew YK Foong. Neural process family. http://yanndubs.github.io/Neural-Process-Family/, 2020.\n\nWe sincerely hope that we have addressed your concerns and questions and look forward to your reading and response. If you have any further questions, please feel free to let us know and we will do our best to answer them.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "P4rUCTMyQE", "reviewer": "Reviewer_Y68H", "summary": "The paper introduces a new approach (Fourier Neural Processes, FNP) for weather data assimilation using data from different resolutions. The authors show that the new approach improves the results over similar data assimilation networks from earlier papers.", "questions": "Figure 4 and section 4.2 refer to fine-tuning the model and present better results for the fine-tuned version. What does the fine-tuning involve? I don't get a good picture of this from the paper.", "limitations": "The paper does an adequate job of explaining the limitations of the current model, and makes useful suggestions for further development. However, the difficulty of understanding the paper makes it harder to assess if the work has additional limitations that are not well addressed.", "rating": 4, "confidence": 2, "soundness": 3, "presentation": 2, "contribution": 3, "strengths": "The paper demonstrates well the advantages of the proposed model to the similar ConvCNP network. The authors perform a thorough analysis of the ability of the method to assimilate the most relevant atmospheric variables. The paper also addresses an important area in the atmospheric ML data chain that has received less attention than e.g. weather forecasting.", "weaknesses": "I found the paper rather hard to follow. Concepts like \"spatial variable functional representation\" and \"dynamic alignment and merge\" are not well explained. As a result, even having previous experience of Fourier neural operators it is difficult to get a sense of what the network actually does. The authors refer the reader to previous papers on the subject, but I feel the current paper would need to better explain these ideas to a reader from outside the specific niche of research to be accessible for the NeurIPS audience.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Dear reviewer,\n\nThank you very much for your detailed review, thoughtful comments, and feedback! We sincerely appreciate your recognition of the effectiveness of our method and its relevance to the research field. At the same time, we deeply regret and apologize for any confusion our writing may have caused you. Below, we will address each of your concerns in sequence.\n\n> I found the paper rather hard to follow. Concepts like \"spatial variable functional representation\" and \"dynamic alignment and merge\" are not well explained. As a result, even having previous experience of Fourier neural operators it is difficult to get a sense of what the network actually does. The authors refer the reader to previous papers on the subject, but I feel the current paper would need to better explain these ideas to a reader from outside the specific niche of research to be accessible for the NeurIPS audience.\n\nSpatial-variable functional representation: All neural process methods share a common core idea of encoding contextual information to model a representation $R$, which is then used for decoding the target values. In CNP, $R$ is a static global representation, while in ANP, $R$ is dynamic and related to the absolute position. In ConvCNP, the encoder's mapping is translation equivariant, resulting in $R$ that is dependent on relative position, which is termed as functional representation (FR). FNP adopts the ConvCNP concept, utilizing SetConv to encode context coordinates and values separately to capture density and signal, which are then concatenated and fed into deep feature extraction module composed of NFL blocks to model FR. Tailored to meteorological data characteristics, we model a distinct spatial FR for each variable and a variable FR for all variables instead of a mixed FR. This is called spatial-variable functional representation, and the modeling process for each FR follows the aforementioned procedure.\n\nThis decoupled modeling offers several advantages. Firstly, the individual spatial FRs diversify and specialize the model's spatial relationship modeling, as different variables may exhibit distinct spatial patterns. Secondly, the variable FR can capture the interrelationships among different variables. Modeling the correlations in these two dimensions is also crucial in traditional data assimilation techniques (i.e., the role of error covariance). Thirdly, it clarifies the objectives of each FR and reduces complexity, thereby easing model training, accelerating convergence, and enhancing performance. Finally, this decoupling allows for lower data embedding dimensions for each FR (i.e., number of channels), which can significantly reduce computational resource consumption. In the ablation study in Table 3, the FLOPs with and without SVD are 67.872G and 167.932G, respectively, corresponding the data embedding dimensions of 128 and 256. The SVD makes the model achieve better performance with lower FLOPs, thus enhancing computational efficiency.\n\nDynamic alignment and merge: The DAM aligns the FR of the background and observations in the spatial dimension and then merges them. Alignment is achieved through interpolation operations, ensuring that the spatial dimensions of observations are consistent with the background in all circumstances for subsequent information fusion. The merge is implemented by selecting based on similarity to shared features, with the calculation for similarity and selection rules provided in the formula in Section 3.3.\n\n> Figure 4 and section 4.2 refer to fine-tuning the model and present better results for the fine-tuned version. What does the fine-tuning involve? I don't get a good picture of this from the paper.\n\nIn our experiments in Sections 4.2 and 4.3, we present the performance of models without fine-tuning and with fine-tuning. \"Without fine-tuning\" refers to training in one setting and directly testing in another, showcasing the model's generalization capability. \"Fine-tuning\" involves further training for a certain number of epochs in the testing setting, demonstrating the model's optimal performance in that specific context. \n\nSpecifically, in Section 4.2, \"without fine-tuning\" denotes assimilating observations all at 1.40625° during training, while assimilating observations at 0.703125° and 0.25° in testing. The performance after fine-tuning refers to the testing results after the model is further trained at the corresponding resolutions. \n\nIn Section 4.3, \"without fine-tuning\" means using the model weights trained from the data assimilation task directly for reconstructing the observational information. We infer the FR of the observations using only the encoding weights of the observation branch, and then infer the output based on the FR using the decoding weights. The performance after fine-tuning refers to the testing results after continuing to train the weights of these two parts in observational information reconstrution task.\n\nWe sincerely hope that we have addressed your concerns and questions and look forward to your reading and response. If you have any further questions, please feel free to let us know and we will do our best to answer them.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "PZVutEivGZ", "reviewer": "Reviewer_oAJc", "summary": "This paper proposes an innovative Fourier Neural Process (FNP) model designed to address the limitations of existing data assimilation methods when handling observational data of varying resolutions. The FNP model combines the characteristics of neural processes and Fourier transforms to effectively assimilate observational data of arbitrary resolutions. Extensive experiments were conducted on the ERA5 dataset, demonstrating the superior performance of FNP in assimilating observational data of different resolutions.", "questions": "none", "limitations": "see above", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. The FNP addresses the limitation of existing data assimilation methods that cannot assimilate observational data of varying resolutions. Additionally, the proposed FNP, based on neural processes, provides uncertainty estimates compared to deterministic data assimilation.\n2. The smoothing of high-frequency information is addressed by using the Fourier neural operator, which preserves high-frequency information. Experimental results show that the proposed Neural Fourier layer improves performance both visually and in terms of metrics.\n3. FNP achieves state-of-the-art performance in arbitrary-resolution data assimilation for key weather variables.", "weaknesses": "1. The ERA5 dataset is global meteorological data with limited high-frequency information. Validation on datasets with richer high-frequency information (e.g., HRRR) could better demonstrate the effectiveness of the Neural Fourier layer and provide more confidence.\n2. The role of FNP in weather forecast tasks is not thoroughly discussed or experimentally validated.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Dear reviewer,\n\nThank you very much for your thorough review, highly constructive comments, and feedback! We sincerely appreciate your recognition of the effectiveness and significance of our method. Below, we will address each of your questions and concerns in sequence.\n\n> The ERA5 dataset is global meteorological data with limited high-frequency information. Validation on datasets with richer high-frequency information (e.g., HRRR) could better demonstrate the effectiveness of the Neural Fourier layer and provide more confidence.\n\nExcellent suggestions! Conducting experiments on the HRRR dataset can better demonstrate the effectiveness of NFL and enhance the experimental content of this paper. Unfortunately, we regret that we are unable to download the HRRR dataset and train models on it within the limited time frame, resulting in the unavailability of experimental results here. We will strive to include experiments on the HRRR dataset in our future work.\n\nNevertheless, we have conducted additional experiments to showcase the effectiveness of NFL from a different perspective. Table A from the PDF in global response provides ablation study on the generalization of different modules at various resolutions. As the resolution increases, the impact of NFL on performance becomes more pronounced. Similarly, in terms of visual perception, FNP's ability to capture high-frequency information strengthens with higher resolutions. This indirectly reflects the correlation between NFL and the high-frequency information gain brought by our method.\n\n> The role of FNP in weather forecast tasks is not thoroughly discussed or experimentally validated.\n\nYour feedback is highly professional! Data assimilation aims to improve weather forecasting results by reducing initial errors. Therefore, it is crucial to explore the impact of different methods on forecast error reduction. Figure A from the PDF in global response provides results on the forecast RMSE improvement of z500 variable over the next ten days through data assimilation, where lead time 0 corresponds to the reduction of initial errors. Darker colors indicate stronger improvements, meaning a greater reduction in forecast errors compared to not using data assimilation.\n\nSimilar to the results of data assimilation, FNP consistently achieves state-of-the-art results in most cases, with its advantage becoming more pronounced as the resolution increases. Moreover, FNP is the only model that strictly enhances forecast improvement with increasing resolution and observational information. Additionally, apart from the accuracy of initial values affecting forecast errors, the physical characteristics of the initial states (e.g., physical balance) also influence the rate of forecast error growth. FNP demonstrates greater improvements in forecast errors at all lead times compared to improvements in initial errors, indicating that FNP not only reduces forecast errors but also slows down the growth rate of forecast errors. Other models do not exhibit the same trend, further highlighting the superior characteristics of the initial states produced by FNP. We appreciate your insights once again and will include this part in the appendix of the final version.\n\nWe sincerely hope that we have addressed your concerns and questions and look forward to your reading and response. If you have any further questions, please feel free to let us know and we will do our best to answer them.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:
Dear reviewers and meta-reviewers,

We greatly appreciate the considerable time you have dedicated to providing us with constructive comments and feedback to further enhance our paper. It is gratifying to see that all reviewers acknowledge the effectiveness and contribution of our method. We have received many thoughtful comments, including some highly professional suggestions, which we find both excited and grateful.

Simultaneously, we sincerely apologize for any confusion or doubts we may have caused you. We have addressed all reviewers' queries with additional explanations and experiments, aiming to offer further insights and alleviate any concerns. In response to the insightful suggestions provided, we will revise and expand the manuscript. We welcome any follow-up discussions!

Last but not least, we thank the PCs, ACs, and all the reviewers again for devoting time and effort to this review!

Meta-review: The paper has received three good support, but also one borderline reject rating that raises concerns/comments about the readability of the paper. While various comments/concerns have been well addressed with more discussions, explanations clarifications, and experimental results, two reviewers who provided positive support were satisfied. The reviewer who rated the paper negatively did not interact with the authors. All the reviewers agree that the proposed techniques have been well validated and deal with an important problem about weather forecasting.
Decision: Accept (poster)