LoRA-EnVar: Parameter-Efficient Hybrid Ensemble Variational Assimilation for Weather Forecasting
Authors: Yi Xiao
Hang Fan
Kun Chen
Ye Cao
Ben Fei
Wei Xue
LEI BAI
Venue: NeurIPS 2025 poster ()
Keywords: AI for science
data assimilation
weather forecasting
low rank adaptation
Primary area: machine_learning_for_sciences

Abstract: Accurate estimation of background error (i.e., forecast error) distribution is critical for effective data assimilation (DA) in numerical weather prediction (NWP). In state-of-the-art operational DA systems, it is common to account for the temporal evolution of background errors by employing hybrid methods, which blend a static climatological covariance with a flow-dependent ensemble-derived component. While effective to some extent, these methods typically assume Gaussian-distributed errors and rely heavily on hand-crafted covariance structures and domain expertise, limiting their ability to capture the complex, non-Gaussian nature of atmospheric dynamics. In this work, we propose LoRA-EnVar, a novel hybrid ensemble variational DA algorithm that integrates low-rank adaptation (LoRA) into a deep generative modeling framework. We first learn a climatological background error distribution using a variational autoencoder (VAE) trained on historical data. To incorporate flow-dependent uncertainty, we introduce LoRA modules that efficiently adapt the learned distribution in response to flow-dependent ensemble perturbations. Our approach supports online finetuning, enabling dynamic updates of the background error distribution without catastrophic forgetting. We validate LoRA-EnVar in high-resolution assimilation settings using the FengWu forecast model and simulated observations from ERA5 reanalysis. Experimental results show that LoRA-EnVar significantly improves assimilation accuracy over models assuming static background error distribution and achieves comparable or better performance than full finetuning while reducing the number of trainable parameters by three orders of magnitude. This demonstrates the potential of parameter-efficient adaptation for scalable, non-Gaussian DA in operational meteorology.

LoRA-EnVar: Parameter-Efficient Hybrid Ensemble Variational Assimilation for Weather Forecasting
Yi Xiao
Hang Fan
Tsinghua University, Beijing
Columbia University. New York hr25260columba..co
Shanghai Artificial Intelligence Laboratory. Shanghai y-xiao22@mails tsinghua. edu. cn
Kun Chen
Ye Cao
Fudan University, Shanghai Shanghai Artificial Intelligence Laboratory, Shanghai kunc33010163.1 com
Tsinghua University, Beijing aaoye541@gmail1 com
Ben Fei
Wei Xue
The Chinese University of Hong Kong, Hong Kong Shanghai Artificial Intelligence Laboratory, Shanghai benfe e eeceekkkckkk.. edu.hk hk
Tsinghua University, Beijing xuewei@tsinghua. edw.g cn
Lei Bai
Shanghai Artificial Intelligence Laboratory, Shanghai baisanshi @ageail.mmm..... com
Abstract
Accurate estimation of background error (i.e., forecast error) distribution is critical for effective data assimilation (DA) in numerical weather prediction (NWP). In state-of-the-r operational DA systems,
common to account for the tempo- ral evolution of background errors by employing hybrid methods, which blend a static climatological covariance with a flow-dependent ensemble-derived compo- nent. While effective to some extent, these methods typically assume Gaussian- distributed errors and rely heavily on hand-crafted covariance structures and domain expertise, limiting their ability to capture the complex, non non- Gaussian nature of this work, we propose LoRA-En Var, a novel hybrid ensemble variational DA algorithm that integrates low-rank adaptation (LoRA) into deep generative modeling framework. We first. learn climatological background error distribution using variational autoencoder (VAE) trained on historical data. To incorporate flow-dependent uncertainty, we introduce LoRA modules that ef- ficiently adapt the learned distribution in response to flow-dependent ensemble perturbations. Our approach supports online finetuning, enabling dynamic updates of the background error distribution without catastrophic forgetting. We validate LoRA- Var high-resolution assimilation settings using the FengWu forecast model and simulated observations from ERA5 reanalysis. Experimental results show that L-RA-En Var significantly improves assimilation accuracy over models assuming static background error distribution and achieves comparable better performance than full finetuning while reducing the number of trainable parameters by three orders of magnitude. This demonstrates the potential of parameter-efficient adaptation for scalable, non-Gaussian DA in operational metcorology.
atmospheric dynamics.
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

Introduction
Data assimilation (DA) plays a crucial role in numerical weather prediction (NWP) by integrating real-world observations with model forecasts to produce accurate initial states for future predictions []] 2 4 In this process, a prior forecast, typically generated by a numerical model, is treated as the background state. The effectiveness of DA depends largely on accurately characterizing this background error distribution, which reflects the uncertainty in prior forecasts and determines how new observations are assimilated into the model.
Early DA systems such as 3DVar typically assume a static Gaussian background error, using fixed covariance matrix throughout the forecast cycle [5] g] While computationally efficient, this assumption fails to capture the fact that background errors evolve over time, especially in dynamically active regions (9. To address this, ensemble-base methods like the Ensemble Kalman Filter (EnKF) estimate flow- depedeendd covariances from short-term ensemble forecasts [1) However, in gigh-dimensional systems like NWP, limited ensemble size often leads to spurious correlations and noisy error estimates [13 12 To mitigate this, hybrid methods have become the dominant approach in modern operational DA. These methods combine static climatological covariance estimated from historical simulations with flowddependent covariance derived from real-time ensembles, leveraging the strength of both [9] [4] Despite their effectiveness, hybrid methods still rely on a Gaussian error assumption [15 16 and involve expert-crafted covariance structures [18] which may limit their ability to model complex, non-Gaussian background uncertainty [19].
Recent advances in Al-based DA methods aim to overcome the limitations of traditional methods. For example, DifIDA [20] and Score-based DA (SDA) 21 [2] utilize diffusion models to model complex non-Gaussian uncertainty, but they do not incorporate climatological priors or flow-dependent ensembles. Other approaches, such as FuXiaEn4DVar (23, introduce ensembles model flow- dependent uncertainty. yet still rely on Gaussian assumptions Similarly, the Ensemble Score Filter (EnSF)
24 25 incorporates flow dependency but scales poorly to high-dimensional systems. Latent- EnSF 226 27 mitigates this issue by learning latent representation of error structure, but it lacks a mechanism to incorporate static background statistics, which can limit its long-term stability and climatological consistency.
To date. no existing method has successfully captured hybrid background error characteristics. combining both static climatological and flow-dependent features, within non-Gaussian generative modeling framework. This leaves gap in current methodologies for flexibly and accurately representing the full spectrum of uncertainty in real-world atmospheric data assimilation.
In this work, we propose LoRA-En Var, hybrid variational assimilation framework that combines deep generative modeling with parameter-efficien adaptation to capture both climatological priors and foow-dependeet background error dynamics, without relying on full retraining or manual covariance design. We first train a variational autoencoder (VAE) 28 on historical forecast error samples to learn low-dimensional latent representation of climatological background uncertainty. To incorporate real-time flow-dependent information, we introduce low-rank adaptation (LoRA) (29 modules into the decoder of the VAE. During each assimilation cycle, these LoRA modules are finetuned online using perturbations from ensemble forecasts, enabling the generative model to adapt dynamically to the evolving atmospheric states. We validate LoRA-En Var high-resolution cyclic assimilation experiments using the Feng Wu (30 forecast model and -.i.---imulated observations, demonstrating its effectiveness large-scale NWP settings. Compared to full decoder finetuning, our approach maintains stability, avoids catastrophic forgetting, and reduces the number of trainable parameters by over three orders of magnitude. Our main contributions are summarized as follows:
We design a hybrid deep generative framework that unifies VAE-based climatological modeling with LoRA-based flow-dependent adaptation.
We introduce online low-rank finetuning during assimilation cycles to enable dynamic, non-Gaussian background error updates with minimal computational cost.
We demonstrate consistent accuracy improvements over static and hybrid baselines in high-resolution cyclic NWP experiments.

---

2 Preliminaries
Variational Assimilation In variational data assimilation, the goal infer the most likely system state X. given a background estimate X₁ and a set of observations y. by maximizing the conditional probability distribution p(x X y). This yields the analysis state X, arg maxx p X x/xx y).
Since the background state x; originates from a numerical forecast and the observations, y are obtained from satellites ground- based instruments, it is reasonable to assume independence between x, and Under this assumption, the posterior distribution can be factorized as [19]
3
arg max p(x[Xx, y) arg max p(y) x p(x()))))
(1)
This formulation leads naturally to the definition of the variational cost function:
[(x) log (oyy)(((((xpppppp) logog x) log log p(x|xx).
(2)
The total cost comprises two components: an observation term (x( = - log p(y|x) and a background term (x(x, Xa) = - px(x))
The observation term primarily reflects the statistical characteristics of measurement errors and is closely tied to the properties of the observing instruments and data quality. In contrast, the background term captures the uncertainty in the model forecast and the structure of prior errors, which are often more complex and system-dependent. This work focuses improving the modeling of the background term to enhance the assimilation accuracy.
Hybrid Background Term Modeling The background term models the uncertainty in the prior forecast and plays critical role variational data assimilation. Its estimation relies on representative error samples that capture the statistical characteristics of forecast errors. These samples typically fall into two categories:
Climatological samples, obtained from long-term historical analysis data (typically using NMC method (17), represent relatively stable and inverinvariant error structures that reflect the background error characteristics the climate state.
Flow- Flow-dependent samples, extracted from short-range raage ensemble forecasts at the current time, capture transient, situation-specific features of forecast uncertainty.
Traditional variational methods are based solely on climatological samples, while ensemble-based methods use flow-depen now-dependent perturbations. To take advantage of both, hybrid methods that combine static and dynamic information have been developed, which have been shown to have better analysis quality and higher assimilation accuracy.
Traditional Hybrid Var Traditional data assimilation methods assume that the background error follows a Gaussian distribution, that is, x - Xs, ~ (O, B). where B is the background error covariance matrix. Hybrid Var, assumed to be linear combination of B, and Bgy that is
B= aB, + (1-a)B.
(3)
where B, and B, denote covariances calculated from the static climatological and ensemble-derived dew-deeendentt samples, respectively, and a E [0, 1] is tunable weighting coefficient. Despite their practical success, most hybrid methods still rely on the Gaussian assumption and linear formulations of B, limiting their expressiveness in highly nonlinear or non Non-Gaussiin regimes. In this work, we retain the core design principle of combining climatological and flowddependent information, but replace the hand-crafted Gaussian formulation with deep, adaptive generative model that can learn and update complex background error distributions more expressively.
3 LoRA-En Var
3.1 Non-Gaussian Hybrid Background Modeling via Online Adaptation
In traditional variational assimilation, the background error is often assumed to follow a Gaussian distribution, allowing its uncertainty structure to be fully characterized by a covariance matrix B This makes it feasible to represent as a linear combination of static and flow-dependent compo- nents. However, in non- oon-Gaussian settings, the background error distribution may exhibit skewness,
3

---

multimodality, or eigher-order dependencies that second-order statistic like covariance matrix cannot capture. In such cases, the distribution must be modeled more expressively, typically through deep generative models such as VAFs 28 normalizing flows [31 32], or diffusion models [R 34, which represent complex distributions in a latent or implicit form. Crucially, these models no longer expose an explicit covariance matrix structure, making it nontrivial to incorporate flow-dependen ensemble samples via simple linear blending, as is done in traditional hybrid methods.
To overcome this limitation, we draw on the concept of online learning in neural networks, where a model pre-trained on historical data can be incrementally updated with new samples. This is analogous to the DA scenario, in which long-term historical samples provide a stable prior, while flow-dependent samples, available during each assimilation cycle, reflect evolving dynamics that require continuous adaptation.
Specifically, build the VAE-Va Var framework to model the background error distribution deep latent-variable generative model. The VAE is first pre-trained using climatological error samples derived from long-term reanalysis data. This training phase captures the relatively stable structure of background uncertainty.
During assimilation, flow-dependent enror samples, which are generated from ensemble forecasts around the current state, become available at each cycle. Rather than discarding prior knowledge and retraining the model from scratch, treat these new samples streaming data and update the model incrementally. This approach allows the generative model to evolve over time, incorporating both global climatological priors and localized, transient dynamics.
3.2 Frrameter-Effiieen Online Adaptation with LoRA
While online adaptation provides principled framework for incorporating flow-dependent informa- tion into the background error model, a key challenge lies in how to update the generative model efficiently and stably during assimilation cycles. Directly finetuning the full set of parameters in the VAE decoder for every new assimilation window is computationally expensive and risks overfitting to transient flow features, potentially leading to catastrophic forgetting of the climatological prior learned during pretraining-
To address these issues, adopt LoRA lightweight, parameter-efficient mechanism for online adaptation. LoRA inserts low-rank trainable matrices into the linear layers of the decoder while keeping the original weights frozen. During online updates, only the LoRA parameters are tuned using flow-dependent samples, while the backbone decoder, trained on climatological data, remains unchanged.
This design offers several advantages:
Efficiency: The number of trainable parameters is significantly reduced (approximately 0.1 1% of the full model), allowing fast updates even in high-resolution settings.
Stability: By preserving the original decoder weights, the model retains its climatological prior knowledge, mitigating the risk catastrophic forgetting.
Flexibility:: Despite its lightweight nature, LoRA introduces enough capacity to fit flow- dependent corrections effectively, enabling the model to adjust its generative outputs to evolving error structures.
By combining LoRA with the online learning structure introduced in Section 3.1 LoRA-En Var achieves a scalable and dynamically adaptable representation of background error distributions, balancing long-term stability with short-term responsiveness.
3.3 LoRA-EnVar Var Pipeline
The overall architecture of LoRA-En Var illustrated in Figure T The framework consists of two phases: offline learning and online adaptation.
In the offline phase, first train VAE using historical background error samples constructed from long-term historical datasets. This allows the decoder D of the VAE to model the climatological distribution of background error, capturing large-scale and relatively stable uncertainty patterns.
Once the offline training is complete, the system enters the online phase, where LoRA modules are introduced into the decoder to enable dynamic adaptation to flow-dependent characteristics.

---

Offline Learning
Historical enor sumples
Training
VAE learning background distribution
Hidden states
sampling
Decoder
Encoder
(backbone)
Std
Online Adaptation
To
Analysis field
Forecast
Background field
Assimilation
Analysis field
Forecast
Background field
Other predictions
Other predictions
Obs
Previous analysis fields
Previous analysis fields
Do Decoder backbone
Flow-dependent etror samples
D Decoder backhone
Flow-dependent error samples
Ao
By (=
Finetuning
Finetuning
Legend:
Background ensemble generation
@ Flow-dependent error sampling
Figure Overview of the LoRA-En Var framework. The method consists of an offline VAE training phase using climatological samples and an online adaptation phase using flow-dependent samples for LoRA finetuning. See Section 3.3 for details.
Following the initialization strategy proposed in the original LoRA paper [29], the low-rank matrices Bo are initialized zero preserve the pretrained decoder's output at the beginning of adaptation.
At each assimilation window, we perform the following steps: (1) Background ensemble generation. The previous analysis states used to produce ensemble of background forecasts for the current time step. This ensemble can be constructed using established techniques, such as time- lagging [35 36] breeding methods [37] 38). (2) Flow-dependent error sampling. From the ensemble, we derive perturbations representing the flw-dependeet background error samples at the current time. (3) Online finetuning. These samples then used to finetune only the LoRA modules in the VAE, while keeping the backhone parameters frozen. (4) Assimilation. The adapted decoder is then used within the variational assimilation framework to compute the analysis field for the current time step. Specifically, we adopt the loss formulation (see Appendix) from VAE-Var, which integrates both the generative model and observation likelihood into unified variational objective.
This process is repeated at each assimilation window, forming a continuous DA cycle that incremen- tally updates the background error distribution and generates the analysis fields.
4 Results
4.1 Experimental Setup
The experiments in this study are conducted within real-world global medium- range weather forecasting context. Specifically, we evaluate our data assimilation framework using FengWu 30) Feng Wu is a learning-based medium-range weather forecasting model trained on the FRA5 reanalysis dataset. It produces six-hour forecasts and simulates a total of 69 meteorological variables, including five upper-air variables across 13 pressure levels and four surface variables. The
Forecasting Model
5

---

MSLP Error Difference (Pa)
Z500 Error Difference (m*/s²)
Z850 Error Difference (m²/s²)
-100 0 100
-100
0
100
-100
0
100
Figure 2: Spatial error difference between assimilation with and without LoRA finetuning, Red areas indicate regions where LoRA-EnVal Var reduces forecast error relative to the frozen VAE baseline. Results are shown for three representative variables, all evaluated at 00:00 UTC on January 1st, 2022. Notable improvements observed over the northeastern Atlantic, particularly west of the UK.
upper-air variables include geopotential height (z), specific humidity (q), zonal wind (u), meridional wind (v), and air temperature (t), denoted using standard short-name and pressure-level conventions (e.g., z500 refers to geopotential at 500 hPa). The four surface variables are 2-meter temperature (12m), 10-meter zonal wind (u10), 10-meter meridional wind (v10), and mean sea level pressure (mslp). In our experiments, all simulations are performed at global resolution of 0.25%.
LoRA Module Design The architecture of the VAE follows the design introduced in VAE-Var 19 where both the encoder and the decoder are built upon the Swin Transformer V2 (39 backbone.
which is well-suited to structured meteorological fields and high-resolution global inputs (39 40) To enable parameter-efficient online adaptation, we incorporate LoRA modules into the VAE decoder. Following the principle of minimal parameter overhead, we insert LoRA modules only into the query projection (q) layers of each self-attention block in the decoder [2]. This selective design preserves the original network's 's structure while allowing low-rank corrections to flow-dependent background error features. Unless otherwise stated, the LoRA rank is fixed at 2 across all experiments (see the Appendix for ablation studies).
4.2 Single-Step Assimilation Experiment
We begin by evaluating the effectiveness of LoRA-EnVar's hybrid modeling ability in a single-step assimilation setting. Specifically, we conduct assimilation experiments targeting 00:00 UTC on January 1st, 2022. The background state is obtained by running FengWu for 8 forecast steps starting from the ERA5 reanalysis field 42 00:00 UTC December 30th, 2021.
To construct the flow-dependent ensemble, we follow time-lagging strategy 36 35
We use ERAS
reanalysis fields at 6-hour intervals from 06:00 UTC on December 30th to 18:00 UTC on December 3sst, 2021. Each these 7 states is forecasted forward using FengWu for 7 to steps respectively, so that all ensemble members align temporally at 00:00 UTC on January 1st. Combined with the original background forecast, we obtain a total of 8 ensemble members. The deviations of these members from their ensemble mean form the flow-dependent perturbation samples used to finetune the LoRA modules in the decoder of the pretrained VAE.
After finetuning, we perform variational data assimilation using simulated observations sampled from ERA5 analysis fields at 1000 random points. To assess the contribution of LoRA-based finetuning, compare two assimilation results: (1) one using the frozen VAE decoder (without LoRA finetuning), (2) the other using the adaptively finetuned decoder (LoRA-En Var). We compute the analysis error for both methods and visualize the spatial error difference between the two, i.c., Errorpo-tune - ErrorI,RA-EMM Eara-- Positive values indicates improvement due to LoRA adaptation.
Figure 2 presents the results for three key variables: mean sea level pressure (mslp), geopotential at hPa (z500), and geopotential at 850 hPa (z850). The most significant improvement is observed over the northeastem Atlantic west of the UK, region known for strong jet streams and frequent mid- latitude cyclones during winter. These dynamically active conditions lead to highly flow-dependent forecast errors. The localized improvement suggests that LoRA-based adaptation helps capture such transient error structures more effectively than static background models, thereby enhancing assimilation accuracy in regions of rapid atmospheric variability.

---

4.3 Cyclic Assimilation and Forecasting
To evaluate the long-term performance of LoR in a near-realistic operational setting, we conduct a eyclic assimilation and forecasting experiment over full month, starting from 00:00 UTC on January 1st, 2022. Following [19] 43), the system performs one assimilation cycle and produces an updated analysis every 6 hours, which is used to initialize the next forecast.
Background Ensemble Generation We follow the framework illustrated in Figure I using an ensemble size of 8 (see the Appendix for experiments with different ensemble size). Unlike the single-step experiment where ensemble members were generated using ERA5 reanalysis fields, here we construct the ensemble entirely from within the system, without access external reanalysis data, Specifically, we use a time-lagging strategy: past analysis fields produced by the system itself are used to initialize forecasts that form the ensemble (see the Appendix for details). This design ensures full self-consistency within the cyclic forecast-assimilatio loop.
Observation Settings The cyclie assimilation experiment uses the same 1000 fixed synthetic observation stations as in the single-step setup, with full-variable measurements derived from ERA.5. We have also conducted experiments on GDAS real-world observations 44. with results reported in the Appendix.
Compute Resources Experiments are performed on a single NVIDIA A100 GPU, with each assimilation cycle requiring approximately 14 seconds for finetuning and 10 seconds for assimilation.
LoRA vs. Traditional Algorithms We compare LoRA-En Var against three baselines: (1) 3D which uses static climatological background covariance following 17: (2) Hybrid 3DEn Var, which combines static and flow-dependent covariances with an optimized coefficient a. following [45); (3) VAE-Var (no finetune) [19], which uses the pretrained VAE decoder without any LoRA finetuning.
Figure 3 reports the RMSE and bias of key variables over the 31-day period. The results show that: LoRA-EnVai outperforms VAE- Var (no finetune) across all variables and lead times, confirming that flow-dependent finetuning of the generative model improves assimilation accuracy: LoRA-En Var achieves lower RMSE and bias than Hybrid 3DEn Var, particularly in geopotential and wind fields, demonstrating its advantage over conventional hybrid schemes; 3DVar performs worst, as expected, due to its inability to adapt to real-time flow conditions. These findings indicate that LoRA-En Var not only effectively integrates climatological and flow-dependent information in a non-Gaussian setting but also remains stable and accurate in long-term forecasting scenarios.
LoRA vs. Full Finetuning To assess the effectiveness of LoRA-based online adaptation, we compare it against two variants of full finetuning strategies applied to the VAE decoder: (1) Full finetune (w/o reset): At each assimilation step, the decoder parameters are inherited from the previous cycle and updated using flow-dependent ensemble samples; (2) Full finetune (w/ reset): At each step, the decoder is reloaded from the original pretrained model before being finetuned using current ensemble samples. These two strategies represent two ends of the full-parameter update spectrum: the former prioritizes continuity but risks overfitting or catastrophic forgetting, while the latter improves stability at the expense of temporal coherence and increased computational cost.
Figure 4 presents a comparative evaluation of the three finetuning strategies across multiple key variables. It can be seen that LoRA-En Var maintains consistently low RMSE and bias over time, while both full finetuning approaches exhibit notable trade-offs. The "Full finetune (w/o reset)" strategy, which accumulates parameter updates across cycles, tends to degrade in performance for certain variables, most prominently Z500 and 12m, likely due to catastrophic forgetting of the climatological structure learned during pretraining. In contrast, the "Full finetune (w/ reset)" strategy avoids such degradation by reloading the pretrained decoder at each step, but this comes at the cost of higher computational burden, it involves updating the full decoder parameters repeatedly.
Despite its simplicity, L-RA-En Var achieves comparable or even superior performance to both full finetuning variants. This demonstrates that low-rank adaptation offers highly efficient way to incorporate flow-dependent corrections while preserving the stability and generalization capabilities inherent to the pretrained model.
7

---

2500 (m²/s²)
t500 (K)
u500 (m/s)
v500 (m/s)
3.0-
0.8
RMSE
75
0.7
2.5
2.5
0.2
Bias
0.0
0.0
0.0
-50
-0.1
-0.2
20
30
10
20
30
10
20
Simulated Time (Days)
Simulated Time (Days)
Simulated Time (Days)
Simulated Time (Days)
u10 (m/s)
v10 (m/s)
t2m (K)
q700 (10 kg/kg)
1.75
RMSE
1.50
1.5
1.2
1.25
1.25
1.0
0.05
0.00
0.1
Bias
0.0
-0.25
0.0
-0.05
-0.1
-0.50
20
30
10
20
20
Simulated Time (Days)
Simulated Time (Days)
Simulated Time (Days)
Simulated Time (Days)
Hybrid 3DEnVar
VAE-Vart finetune)
LoRA-EnVar
Figure 3: RMSE and bias of key atmospheric variables over a 30-day cyclic assimilation experiment. Results shown for 3DVar, Hybrid 3DEn Var, VAE-Var (no finetune), and LoRA- Var, respectively. LoRA-En Var experiments are repeated five times with different random seeds, and we report the mean along with the 1-o standard deviation.
Different Observation Densities Motivated by the results under 1000 observation points, we further evaluate the performance of LoRA-EnVar under varying levels of observation availability by conducting cyclic assimilation experiments with 250, 500, 1000, and 2000 observations per cycle, corresponding to approximately 0.024%, 0.048%, 0.096%, and O. 19% of the total grid points, respectively. In cases, observation locations are fixed and provide values for variables.
Figure 5 shows the RMSE of z500 and u850 under these different settings. Across all observation densities, LoRA-En Var consistently outperforms the non-finetuned VAE-Vas Var model, indicating that flow-dependent adaptation provides benefits regardless of how sparse or dense the observational data are. As expected, the relative performance gain from LoRA-En's Var tends decrease as the observation density increases. When more observations are assimilated, the analysis becomes increasingly constrained by observational data, reducing the marginal impact of more accurate background error modeling. Nonetheless, even in the most observation-rich case (2000 points), LoRA-Ein Var continues to show improvements over both VAE-Var and Hybrid 3DEn Var, demonstrating its strong adaptability and effectiveness in variety of observational scenarios.
4.4 Evaluation of Parameter Efficiency
In our implementation, full finetuning requires updating all decoder parameters every assimilation cycle, totaling 431,772,613 parameters. In contrast, LoRA-En Var only introduces low-rank adapters into the query projection layers of the Swin Transformer blocks. With fixed LoRA rank of 2, the total number of tunable parameters in our setup is just 165,888. This represents reduction of over 99.96% in the number of updated parameters, while still achieving comparable or superior assimilation performance as shown in Section 4.3 This highlights the practical advantage of LoRA- AnEn'nnnV Var for scalable, long-term deployment in operational settings, where both model size and computational efficiency are critical concerns.
5 Related Work
AT for Data Assimilation Beyond the approaches mentioned the introduction, numerous studies have applied AI techniques to improve DA, including latent-space assimilation [46]
47
48
49
50

---

2500 (m²/s²)
t500 (K)
u500 (m/s)
v500 (m/s)
3.0
RMSE
100
2.75
2.5
2.50
0.2
Blas
0.00
-0.25
-100
0.2
10
20
30
10
30
10
20
30
10
30
Simulated (Days)
Simulated (Days)
Simulated (Days)
Simulated (Days)
u1o (m/s)
v10 (m/s)
t2m (K)
4700 (10 kg/kg)
RMSE
14
1.2
1.0
1.0
0.0
Blas
0.0
-0.5
0.0
-0.1
20
20
30
Simulated
(Days)
Simulated Time (Days)
Simulated (Days)
Simulated Time (Days)
VAE Var (no tinetune)
Full rinetune (w/o reset)
Full finetune (w/ reset)
LoRa-EnVar
Figure 4: Comparison of finetuning strategies in cyelic assimilation. RMSE and bias are reported for LoRA-En Var, VAE-Var (no-finetune) and two full-finetune baselines under the same experimental setting. LoRA-En Var and two full-finetune experiments are repeated five times with different random seeds, and we report the mean along with the 1-a standard deviation.
#Obs 250 (0.024%)
#Obs 500 (0.048%)
#Obs 1000 (0.096%)
#Obs 2000 (0.19%)
25.00 RMSE (m'is')
400
Hyorid 3DEnVar
VAE-Var
200
LoRA EnVar
4-
uBSD RMASE
3-
2
10
20
30
10
20
30
0
10
20
30
10
20
30
Simulated Time (Days)
Simulated Time (Days)
Simulated Time (Days)
Simulated Time (Days)
Figure 5: RMSE of z500 and u850 under varying observation densities in cyclic assimilation. Results are shown for four observation counts, comparing four different DA methods.
51. directly learning the assimilation mapping 52 53
neural process (54] and reinforcement learning [55]. These methods aim to enhance assimilation accuracy or computational efficiency. However, most are limited to small-scale problems and often omit the flow-dependent nature of background errors, making them less applicable to high-resolution, operational forecasting scenarios.
LoRA for Dynamical Systems Recent studies have explored the use of LoRA in learning dy- namical systems, primarily in the context of transfer learning for PDE solvers. These approaches leverage structural similarities berween systems to enable parameter-efficient adaptation, and are often evaluated simulated or idealized settings [56] 57 In contrast, our work applies LoRA to real-world atmospheric system, focusing not on solving the governing equations directly, on mod- cling the evolution of forecast uncertainty within DA framework. To the best of our knowledge, this

---

is the first use of LoRA for learning flow-dependent background error dynamics in operational-scale weather forecasting.
Conclusion
We proposed L-RA-En Var, hybrid ensemble variational DA framework that combines deep gen- erative modeling with parameter-efficee flow-dependent adaptation. Our method captures both climatological and real-time error structures, achieving improved assimilation accuracy while reduc- ing the number of trainable parameters by three orders of magnitude. By enabling online adaptation of background error distributions minimal computational cost, LoRA-En Var offers a scalable solu- tion for next- sext-generation data assimilation systems. It is particularly well suited for high-resolution numerical weather prediction settings, where both modeling accuracy and efficiency are critical. This work contributes toward bridging machine learning advances with operational DA workflows, and opens new directions for learning-basee non-Gaussian assimilation methods in atmospheric sciences.
Despite its strong empirical performance, LoRA-Ent Var currently depends on heuristically chosen finetuning parameters, and lacks theoretical characterization of its adaptation dynamics. Moreover, it inherits the intrinsic limitation of variational assimilation in producing only point estimates, leaving uncertainty propagation an open challenge. Future work may address these issues by studying its theoretical properties and extending it toward fully probabilistic assimilation frameworks.
Acknowledgments
This work is supported by the National Natural Science Foundation of China (NO. NO..U224210). This work is also supported by the Shanghai Science and Technology Commission Project and Shanghai Artificial Intelligence Laboratory.
References
[1] Peter Bauer, Alan Thorpe, and Gilbert Brunet. The quiet revolution of numerical weather prediction. Nature, 525(7567):47-55, 2015.
[2] Alberto Carrassi, Mare Bocquet, Laurent Bertino, and Geir Evensen. Data assimilation in the geosciences: An overview of methods, issues, and perspectives. Wiley Interdisciplinary Reviews: Climate Change, 955)e533, 2018.
[3] Eugenia Kalnay. Atmosperiic modeling, data assimilation and predictability. Cambridge university press, 2003.
[4] Dale Barker, Xiang-Yt Yu Huang. Zhiquan Liu, Tom Auligné, Xin Zhang, Steven Rugg, Raji Ajjaji, Al Bourgeois, John Bray, Yongsheng Chen, et al. The weather research and forecasting model's community variational/ensemblle data assimilation system: Wrfda. Bulletin of the American Meteorological Society, 93(6):831-843, 2012.
[5] David F Parrish and John C Derber. The national metcorological center's spectral statistical- interpolation analysis system. Monthly Weather Review, 120(8):1477-1763, 1992.
[6] Philippe Courtier, E Andersson, W Heckley, D Vasiljevic, M Hamrud, A Hollingsworth, F Rabier, M Fisher, and J Pailleux. The ecmwf implementation of three-dimensional variational assimilation (3d-var). i: Formulation. Quarterly Journal of the Royal Meteorological Society, 1283550) 1998.
[7] Pierre Gauthier, C Charette, L Fillion, P Koclas, and S Laroche. Implementation of a 3d variational data assimilation system at the canadian metcorological centre. part i: The global analysis. Ammosphre- Ocean, 37(2):003-156, 1999.
[8] AC Lorene, SP Ballard, RS Bell, NB Ingleby, PLF Andrews, DM Barker, JR Bray, AM Clayton, T Dalby, D Li, et al. The met. office global three-dimensional variational data assimilation scheme. Quarterly Journal of the Royal Meteorological Society, 1261370) 2000.
10

---

[9] Thomas M Hamill and Chris Snyder. A hybrid ensemble kalman filter-3d variational analysis scheme. Monthly Weather Review, 128(8))2905-2911 2000.
[10] Peter L Houtekamer and Herschel L Mitchell. Data assimilation using an ensemble kalman filter technique. Monthly weather review, (26(3):76-811) 1998.
[11] Jeffrey L Anderson. An ensemble adjustment kalman filter for data assimilation. Monthly Weather Review, 129(12):884--203, 2001.
[12] Brian R Hunt, Eric J Kostelich, and Istvan Szunyogh. Efficient data assimilation for spatiotem- poral chaos: A local ensemble transform kalman filter. Physica D: Nonlinear Phenomena, 230( 12-126, 2007.
[13] Edward Ott, Brian R Hunt, Istvan Szunyogh, Aleksey V Zimin, Eric J Kostelich, Matteo Corazza, Eugenia Kalnay, DJ Patil, and James A Yorke. A local ensemble kalman filter for atmospheric data assimilation. Tellus A: Dynamic Meteorology and Oceanography, 36(5):415-423 2004.
[14] Mark Buehner. Ensemble derived stationary and flow-dependent background -eroo covariances: Evaluation in a operaaiopel nwp setting. Quarterly Journal of the Royal Meteorological Sociery: A journal of the atmospheric sciences, applied meteorology and physical oceanography, 131(607):1013-10433 2005.
[15] Dale M Barker, Wei Huang, Yong-Run Guo, AJ Bourgeois, and QN Xiao. three-dimensional variational data assimilation system for mm5: Implementation and initial results. Monthly Weather Review, 132(4)::97-914, 2004.
[16] Ross N Bannister. review of forecast error covariance statistics atmospheric variational data assimilation. i: Characteristics and measurements of forecast error covariances. Quarterly Journal of the Royal Meteorological Society: A journal of the atmospheric sciences, applied meteorology and physical oceanography, 134(637):1951-1977 2008
[17] Gael Descombes, T Auligné, Francois Vandenberghe, DM Barker, and Jerome Barre. Gen- eralized background error covariance matrix model (gen_be be v2. 0). Geoscientific Model Development, 8(3):669-696, 2015.
[18] Xiang-Y Yu Huang, Qingnong Xiao, Dale M Barker, Xin Zhang, John Michalakes, Wei Huang, Tom Henderson, John Bray, Yongsheng Chen, Zaizhong Ma, et al. Four-dimensional variational data assimilation for wrf: Formulation and preliminary results. Monthly Weather Review, 137(1):299-33) -14,,3112221 2009.
[19] Yi Xiao, Qilong Jia, Kun Chen, Lei Bai, and Wei Xue. Vae-var: Variational autoencoder- enhanced variational methods for data assimilation in meteorology. In The Thirteenth Interna- tional Conference on Learning Representations, 2025.
[20] Langwen Huang, Lukas Gianinazzi, Yuejiang Yu, Peter D) Dueben, and Torsten Hoefler. Diffda: a diffusion model for weather-scale data assimilation. arXiv preprint arXiv:2401.05932, 2024.
[21] François Rozet and Gilles Louppe. Score- based data assimilation. Advances in Neural Informa- tion Processing Systems, 36:40521 -05541,, 2023.
[22] François Rozet and Gilles Louppe. Score-based data assimilation for two-layer quasi- geostrophic model. arXiv preprint arXiv:2310.01853 2023.
[23] Yonghui Li. Wei Han, Hao Li, Wansuo Duan, Lei Chen, Xiaohui Zhong, Jincheng Wang, Yongzhu Liu, and Xiuyu Sun. Fuxi-enAdvar: An assimilation system based on machine learning weather forecasting model ensuring physical constraints. Geophysical Research 2024.
Letters, 51(22):c024GLI111
[24] Feng Bao, Zezhong Zhang, and Guannan Zhang. score-based filter for nonlinear data assimilation. Journal of Computational Physics, 514:113207, 2024.
[25] Feng Bao, Zezhong Zhang, and Guannan Zhang. An ensemble score filter for tracking high- dimensional nonlinear dynamical systems. Computer Methods Applied Mechanics and Engineering, 432: 2024.
11

---

[26] Phillip Si and Peng Chen. Latent-ensf: A latent ensemble score filter for high- sigh--imensional data assimilation with sparse observation data, arXiv preprint arXiv:2409.0012) 2024.
[27] Pengpeng Xiao, Phillip Si, and Peng Chen. Ld-ensf: Synergizing latent dynamies with ensemble score filters for fast data assimilation with sparse observations, arXiv preprint arXiv:2411. 2024.
(28) Diederik P Kingma, Max Welling, al. Auto-encoding variational bayes, 2013.
[29] Edward Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu Yuanzhi Li, Shean Wang, Lu Wang, Weizhu Chen, et al. Lora: Lowkrank adaptation of large language models. The Tenth International Conference on Learning Representations, 1(2):3, 2022.
[30] Kang Chen, Tao Han, Junchao Gong, Lei Bai, Fenghua Ling, Jing-Jia Luo, Xi Chen, Leiming Ma, Tianning Zhang, Rui Su, et al. Fengwu: Pushing the skillful global medium-range weather forecast beyond 10 days lead. arXiv preprint 23Xix:3304.02448 2023.
[31] George Papamakarios, Eric Nalisnick, Danilo Jimenez Rezende, Shakir Mohamed, and Balaji Lakshminarayanan. Normalizing flows for probabilistic modeling and inference. Journal of Machine Learning Research, 22(57):1 2021.
[32] Yaron Lipman, Ricky TQ Chen, Heli Ben-Hamu, Maximilian Nickel, and Matt Le. Flow matching for generative modeling. arXiv preprint arXi::2110002747, 2022.
[33] Yang Song, Jascha Sohl-Diekstein, Diederik P Kingma, Abhishek Kumar, Stefano Ermon, and Ben Poole. Score-based generative modeling through stochastic differential equations. arXiv preprint arXiv:2011. 2020.
[34] Jonathan Ho. Ajay Jain, and Pieter Abbeel. Denoising diffusion probabilistic models. Advances in Neural Information Processing Systems, 6158,, 2020.
[35] Čedomir Branković, TN Palmer, F Molteni, S Tibaldi, and U Cubasch. Extended-ramd range predictions with ecmwf models: Time-lagged ensemble forecasting. Quarterly Journal of the Royal Meteorological Society, 116(494):86--112 1990.
[36] Chungu Lu, Huiling Yuan, Barry E Schwartz, and Stanley G Benjamin. Short-range numerical weather prediction using time-lagged ensembles. Weather and Forecasting, 22(3):580-595, 2007.
[37] Zoltan Toth and Eugenia Kalnay. Ensemble forecasting at nmc: The generation of perturbations. Bulletin of the American Meteorological Society, 74(12):2317-2330 1993.
[38] Zoltan Toth and Eugenia Kalnay. Ensemble forecasting at neep and the breeding method. Monthly Weather Review, 125(12):3297-3319 1997.
[39] Ze Liu, Han Hu, Yutong Lin, Zhuliang Yao, Zhenda Xie, Yixuan Wei, Jia Ning, Yue Cao, Zheng Zhang, Li Dong, al. Swin transformer v2: Scaling up capacity and resolution. In Proceedings of the IEFF/CVF Conference on Computer Vision and Pattern Recognition, pages 12009-12019, 2022.
[40] Kaifeng Bi, Lingxi Xie, Hengheng Zhang, Xin Chen, Xiaotao Gu, and Qi Tian. Accurate medium-range global weather forecasting with 3d neural networks. Nature, 619-7970 2023.
[41] Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Lukasz Kaiser, and Illia Polosukhin. Attention is all you need. Advances in Neural Information Processing Systems, 30, 2017.
[42] Hans Hersbach. Bill Bell, Paul Berrisford, Shoji Hirahara, András Horányi, Joaquín Muñoz- Sabater, Julien Nicolas, Carole Peubey, Raluca Radu, Dinand Schepers, et al. The cra5 global reanalysis. Quarterly Journal the Royal Meteorological Society, 146(730):199--2444 2020.
[43] Yi Xiao, Lei Bai, Wei Xue, Kang Chen, Tao Han, and Wanli Ouyang. Fengwu-4dvar: Coupling the data-driven weather forecasting model with 4d variational assimilation. arXiv preprint arXiv:2312.12455, 2023.
12

---

[44] Matthew Rodell, PR Houser, UEA Jambor, J Gottschalck, Kieran Mitchell, C-J Meng, Kristi Arsenault, B Cosgrove, Radakovich, M Bosilovich, al. The global land data assimilation system. Bulletin of the American Meteorological society, 85(3):381-383 2004.
[45] Xuguang Wang, Chris Snyder, and Thomas M Hamill. On the theoretical equivalence of differently proposed ensemble-3dvar hybrid analysis schemes. Monthly Weather Review, (35(1))222-227, 2007.
[46] Maddalena Amendola, Rossella Arcucci, Laetitia Mottet, Cesar Quilodran Casas, Shiwei Fan, Christopher Pain, Paul Linden, and Yi-Ke Guo. Data assimilation the latent space of a neural network. arXiv preprint 201.: 2020.
[47] Mathis Peyron, Anthony Fillion, Selime Gürol, Victor Marchais, Serge Gratton, Pierre Boudier, and Gael Goret. Latent space data assimilation by using deep learning. Quarterly Journal of the Royal Meteorological Society, 147(740):3759-3777 2021.
[48] Hang Fan, Yubao Liu, Zhaoyang Huo, Yuewei Liu, Yueqin Shi, and Yang Li. A Novel Latent Space Data Assimilation Framework with sorenncoder-bbeevatte to Latent Space (AE- 02L) Network. Part I: The Observation-Oni Only Analysis Method. Monthly Weather Review, 153(8):1335-1348 August 2025.
[49] Hang Fan, Yubao Liu, Yuewei Liu, Zhaoyang Huo, Baojun Chen, and Yu Qin. A Novel Latent Space Data Assimilation Framework with Sownnnddddr-Oseraattiee to Latent Space (AE-O2L) Network. Part II: Observation and Background Assimilation with Interpretabilityy Monthly Weather Review, 153(8): 11111349-115 1363, August 2025.
[50] Hang Fan, Lei Bai, Ben Fei, Yi Xiao, Kun Chen, Yubao Liu, Yongquan Qu, Fenghua Ling, and Pierre Gentine, Physically Consistent Global Atmospheric Data Assimilation with Machine Learning in Latent Space, July 2025.
[51] Yongquan Qu, Juan Nathaniel, Shuolin Li, and Pierre Gentine. Deep generative data assimilation in multimodal setting. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 449-459, 2024.
[52] Kun Chen, Lei Bai, Fenghua Ling, Peng Ye, Tao Chen, Jing-Jia Luo, Hao Chen, Yi Xiao, Kang Chen, Tao Han, et al. Towards an end-to-end artificial intelligence driven global weather forecasting system. arXiv preprint arXiv:2312. 122.22 2023.
[53] Xiaoze Xu, Xiuyu Sun, Wei Han, Xiaohui Zhong, Lei Chen, Zhiqiu Gao, and Hao Li. Fuxi-da: A generalized deep learning data assimilation framework for assimilating satellite observations. npj Climate and Atmospheric Science, 8(1):156, 2025.
[54] Kun Chen, Peng Ye, Hao Chen, Tao Han, Wanli Ouyang. Tao Chen, LEI BAL al. Fnp: Fourier neural processes for arbitrary-resolution data assimilation. Advances in Neural Information Processing Systems, 37:137847-137872, 2024.
[55] Mohamad Abed El Rahman Hammoud, Naila Raboudi, Edriss S Titi, Omar Knio, and Ibrahim Hoteit. Data assimilation in chaotic systems using deep reinforcement learning. Journal of Advances in Modeling Earth Systems, 16(3))c2023MS004113 2024.
[56] Jules Berman and Benjamin Peherstorfer. Colora: Continuous low-rank adaptation for re- duced implicit neural modeling of parameterized partial differential equations. arXiv preprint a4Xi 14646, 2024.
[57] Ritam Majumdar, Vishal Sudam Jadhav, Anirudh Deodhar, Shirish Karande, Lovekesh Vig. and Venkataramana Runkana. Pihlora: Physics-informed hypernetworks for low-ranked adaptation. In for Accelerated Materials Design-NeurIPS 2023 Workshop, 2023.
[58] Dong C Liu and Jorge Nocedal. On the limited memory bfgs method for large scale optimization. Mathematical Programming, 45(1):503-555 1989.
[59] Adam Paszke, Sam Gross, Soumith Chintala, Gregory Chanan, Edward Yang, Zachary Vitooo, Zeming Lin, Alban Desmaison, Luca Antiga, and Adam Lerer. Automatic differentiation in pytorch. 2017.
13

---

[60] Andrew C Lorenc. The potential of the ensemble kalman filter for nwp a comparison with 4d-var. Quarterly Journal of the Royal Meteorological Society: A journal of the atmospheric sciences, applied meteorology and physical oceanography, (29(55))3183-320)
2003.
[61] Xuguang Wang, Thomas M Hamill, Jeffrey S Whitaker, Craig H Bishop, and Xuguang Wang. comparison of hybrid ensemble transform kalman filter-oi and ensemble square-root filter analysis schemes. Monthly Weather Review, 2006.
[62] Diederik P Kingma and Jimmy Ba. Adam: A method for stochastic optimization. arXiv preprint arXiv:1422.6900 2014.
14

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification:
The main claims made in the abstract and introduction accurately reflect the paper's contributions and scope.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No
NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
is fine to include aspirational goals as motivation as long as is clear that these goals not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations the work performed by the authors?
Answer: [Yes]
Justification:
We have discussed it in Section
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create a separate "Limitations" section their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution is low images are taken in low lighting. Or speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.
While the authors might fear that complete honesty about limitations might used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't 't acknowledged judgment and recognize that individual actions in favor of transparency play impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
the paper. The authors should use their best
3. Theory assumptions and proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [NA]
15

---

Justification:
The paper does not include theoretical results.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated referenced in the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but they appear in the supplemental material, the authors are encouraged to provide short proof sketch provide intuition.
Inversely, any informal proof provided in the core of the paper should complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental result reproducibility
Question: Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that affects the main claims and/or conclusions of the paper (regardless whether the code and data are provided or not)?
Answer: [Yes]
Justification:
Details for reproducing the main results provided in the Appendix
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g.. in the case of large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) the contribution is primarily a new algorithm, the paper should make clear how to reproduce that algorithm.
(b) the contribution is primarily a new model architecture, the paper should describe the architccture clearly and fully.
(c) If the contribution is new model (e.g., a large language model), then there should either way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome to describe the particular way they provide for reprodueibility. the case of closed-source models, may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers have some path to reproducing or verifying the results.
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described in supplemental material?
16

---

Answer:
[Yes]
Justification:
The code is available at http://ggitubb.cmm/xxxxiy11111/1 VarDA.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurIPS code and data submission guidelines (https://nips... cc/
for more details.
While we encourage the release of code and data, we understand that this might not be possible, so "No" an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines (https
/ /pipp/.cc/pullc//uuui wwwpc//ggue////eeeeeoooc CoodeSubeiiieeee ssi asionnolli onnolicy)))) for more details.
The authors should provide instructions on data access and preparation, including how access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information possible in supplemental material (appended to the paper) recommended, but including URLs to data and code permitted.
6. Experimental setting/details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification:
They are provided in the Appendix
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper to level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, or as supplemental material.
7. Experiment statistical significance
Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Answer: [Yes]
Justification: The main experimental results are accompanied by error bars.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, statistical significance tests, at least for the experiments that support main claims of the paper.
The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, overall run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call a library function, bootstrap, etc.)
The assumptions made should be given (e.g.. Normally distributed errors).
should be clear whether the error bar is the standard deviation the standard error of the mean.
17

---

It is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of enrors is not verified.
For asymmetric distributions, the authors should be careful to show in tables or figures symmetric error bars that would yield results that are out of range (e.g. negative eтror rates).
If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of exccution) needed to reproduce the experiments?
Question:
Answer: [Yes]
Justification:
We provide it in Section
Guidelines:
The answer NA means that the paper does include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, cloud provider, including relevant memory and storage.
The paper should provide the amount compute required for each of the individual experimental runs well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary failed experiments that didn't make into the paper).
9. Code of ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics http:/::////iiss/eeeee http://.....c.b.cccct/ccccccceea
Answer: [Yes]
Justification: The research conducted in the paper conform, in every respect, with the NeurlPS Code of Ethics.
Guidelines:
The answer NA means that the authors have not reviewed the NeurlPS Code of Ethics.
If the authors answer No, they should explain the special cireumstances that require deviation from the Code of Ethics.
The authors should make to preserve anonymity (e.g.. if there is special consid- cration due to laws or regulations their jurisdiction..
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
Justification:
We have discussed in Section
Guidelines:
The answer NA means that there is no societal impact of the work performed.
If the authors answer NA or No, they should explain why their work has no societal impact why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
18

---

The conference expects that many papers will be foundational research and not tied particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology being used intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms monitor how system learns from feedback over time, improving the efficiency and accessibility of MI..
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification: The paper poses no such risks.
Guidelines:
The answer NA means that the paper poses such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Lieenses for existing assets
Are the creators or original owners of assets (e.g., code, data, models), used in the paper. properly credited and are the license and terms of use explicitly mentioned and properly respected?
Question:
Answer: [Yes]
Justification: We cite the FengWu forecasting model and the ERA5 dataset.
Guidelines:
The answer NA means that the paper does existing assets.
The authors should cite the original paper that produced the code package or dataset.
The authors should state which version of the asset is used and, if possible, include URL.
The name of the license (c.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (c.g., website), the copyright and terms of service of that source should be provided.
If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswi thcode. com com/datasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
19

---

If this information is not available online, the authors are encouraged to reach out to the asset's 's creators.
13. New assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [Yes]
Justification: The code the paper is well documented and the documentation is provided alongside the code.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/code/moodl as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent obtained from people whose asset is used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsourcing and research with human subjects
For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Question:
Answer: [NA]
Justification:
The paper does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then much detail as possible should be included the main paper.
According to the NeurlPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals or equivalent for research with human subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or equivalent approval/review based the requirements of your country institution) were obtained?
Answer: [NA]
Justification: The paper does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Depending the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurlPS Code of Ethics and the guidelines for their institution.
For initial submissions, not include any information that would break anonymity (if applicable), such the institution conducting the review.
20

---

16. Declaration of LLM usage
Does the paper describe the usage of LLMs if it is an important, original, or non-standard component of the core methods in this research? Note that if the LLM is used only for writing, editing, or formatting purposes and does not impact the core methodology. scientific rigorousness, originality of the research, declaration is not required.
Question:
Answer: [NA]
Justification: The core method development in this research does not involve LLMs as any important, original, or non-standard components.
Guidelines:
The answer NA means that the core method development in this research does not involve LLMs as any important, original, or non-standard components.
Please refer to our LLM policy buctps://neure wwwparippps..coooeeeeeeeee for what should or should be described.
21

---

A Variational Objective for LoRA-En Var
Once the VAE trained or finetuned within the LoRA-EnVar framework, we employ its decoder for variational data assimilation. Similar traditional variational assimilation, we optimize latent variables z to obtain the posterior estimate. However, in contrast to linear Gaussian models, the mapping from latent space to physical state is non-linear, given by:
D(z)
(4)
where D(z) denotes the output of the trained decoder, and is the background state. The observation term is accordingly reformulated as:
(z(z) L. (D(z) X),
(5)
where L. denotes the original observation mismatch loss, typically based on squared error under Gaussian observation assumptions; in our work, it is formulated as follows:
1 (x y) = (y ( 1 H(x)),
(6)
where H. is an observation operator, mapping variables from the physical space to the observation space.
From a theoretical perspective, a nonlinear transformation from latent to physical space introduces a non-constant Jacobian determinant in the background term. Since this term is intractable in practice, we approximate the background regularization as:
[x(z) Azzz"
where À a positive scalar used to adjust the strength the prior constraint. In our work, the latent dimension is half of the dimension of the physical space; therefore, À is empirically set to 4.
The total variational loss is thus:
[(z) 2 Azz - (
(y 1 (D(z) xb)).
All components this objective are differentiable, allowing efficient gradient- based optimization. During assimilation, we fix the decoder parameters and perform latent-space optimization using the L-BFGS algorithm. The full procedure is summarized in Algorithm [
This latent-space assimilation
Algorithm LoRA-Enn Var Assimilation Step
Require: Trained and finetuned decoder D; at time T,, background state Xs, observation y Initialize latent vector: f- 0
Compute background term: Lz(z) < 212T2
Compute observation term: [z(z) +- (D((i(z) + Xb,
Compute total loss term: L(z) (z(z) + Cz(z)
Obtain the derivative dÉ(x) using auto- mattoddifferentiation
58 and minimize total loss via L-BFGS 59
Return analysis: x, B, (z*) X
framework allows us to leverage the expressive power of deep generative models while maintaining tractable optimization.
B Implementation of Hybrid 3DEn Var
In our experiments, we implement hybrid 3DEn Var system by incorporating both static climatologi- cal and ensemble-derived flow-dependent covariances into the variational assimilation framework. Our implementation is based on the control variable extension method proposed in [60 14, and
theoretically justified in [1I, which proves its equivalence to earlier formulations based on covariance matrix blending [9].
22

---

In hybrid 3DEn Var, the covariance matrix B is expressed as a linear combination:
B = aB; + Be
(9)
where B, is a static, climatological covariance (usually estimated offline), and B, is derived from flow-dependent ensemble perturbations.
To avoid directly computing and inverting the blended B, we follow the extended control variable formulation. The analysis increment áx is expressed as a weighted sum:
åx B,Bl/vv1 B,B!/
(10)
where 11 and V2 are control variables defined in the transformed space, and B and B! are square-root operators of the respective covariance matrices. The corresponding cost function becomes:
1 y H (x6 + 6x)] R y H (xo öx)] 1 vivi + V2
1
Chybrid 3DE (V1, v2)
(11)
This formulation allows preconditioning of each component and efficient optimization, while main- taining the desired covariance blending effect in the physical space. It has been shown that the control variable formulation theoretically equivalent to the covariance blending formulation when the weights satisfy 81 = Va and B3 v1 a 45).. In our experiments, we find that the best performance achieved when 82 = 0.6, corresponding to a 0.64.
In our implementation Hybrid 3DEn both B, and B, are constructed using the GEN_BE tool from the WRF Data Assimilation system (WRFDA) [I7 [9]. The control variables Vi and V2 are optimized jointly using a quasi-Newton solver (L-BFGS). The final analysis increment is reconstructed by combining the transformed increments, and the analysis state is updated
Xa X; 8x.
(12)
C Experimental Details
C.1 Neural Network Architectures
Our VAE architecture is inspired by the design in VAF Var 19), with modifications tailored to realistic high-resolution atmospheric modeling. Specifically, we adopt backbone structure similar to that the Feng Wu forecasting model. As illustrated in Figure 6 both the encoder and decoder are built upon the Swin Transformer V2 [39] architecture.
Each of the encoder and decoder components consists of three stacked modules: an input encoder, central transformer, and an output decoder. All three modules contain attention layers. To enable parameter-efficien adaptation, we insert LoRA modules into the query projections of all attention layers within these components.
Although only the decoder is used during assimilation, both the encoder and decoder equipped with LoRA modules during training, and their parameters are jointly optimized to ensure architectural symmetry and effective latent encoding. During the assimilation phase, the encoder is not used and the decoder, including both its backbone and LoRA components, is kept fixed and used to transform optimized latent variables into physical states.
C.2 Dataset Construction
Historical Error Samples Generation During the offline training phase, we adopt the classical NMC method (originating from the National Metcorological Center) generate historical forecast error samples, as outlined in Algorithm Reanalysis data are treated as the reference (" ("true") states. For each training sample, we select two reanalysis states separated by a time gap and forecast both forward to the same target time-onee once using a single-step forecast from X,, and once using a two-step forecast starting from Xg. The difference between these two forecasts approximates background forecast enror. Repeating this procedure across a long historical time window allows us to construct a representative training set. In our implementation, we use ERA5 reanalysis data from 1979 to 2015,
23

---

VAE Encoder
VAE Decoder
Geopotential
SF-v2
SF-x2 LoRA
Mcan Std
Latent
SF-x2 LoRA
SF-v2 + LoRA
Geopotential
Humnddity
Swin Trnsfonner x2 LolA
Meun 2
Latent Var 2
Swin Trusformer v2 LoRA
Humidity
LoRA
LoRA
LaRA
LoRA
U_ Wind
SF-v2
SF-x2
Mean
Lálent
SF-v2 LaRA
SF-22 +
Wind
LoRA
Sid
Wind
SF-v2
SF-v2
Mcan 4
Latent Var
SF-12
SF-42 LoRA
Winnd
LoRAA
LoRA
Std
LoRA
SF-v2
SF-v2
Mean
Latent
SF-y2
SF-v2
Temperature
LoRA
Std
Temperature
Surface Variables
SF-v2 LoRA
SF-v2 LoRA
Mcan
Latent Var
SF-v2 LoRA
SF-42 LoRA
Surface Variables
Std
Input
Merge
Split
Sampling
Merge
Split
Reconstruction
"Encoder" the VAE Encoder
"Dcodder" VAE Encoder
"Encnder" in the VAE Decoder
"Decdder" the VAE Decoder
Figure 6: Overview of the VAE architecture used in LoRA-En Var. Both the encoder and decoder are based on Swin Transformer V2 blocks, and LoRA modules are inserted in all query projections.
Algorithm 2 NMC-Based Training Set Construction
Require: Forecast model M, number of samples N, forecast interval 7
for i N do
Select two reanalysis states: Xo, X, (separated by r)
1 M (x,)
22 + M o Mo-r((Xo)
Append forecast difference x1 82 to training set
end for
Flow-Dependent Error Samples Generation To generate flow-dependent error samples in self- consistent manner during cycling assimilation, we adopt a time-lagging ensemble strategy, illustrated in Figure 7 For each assimilation cycle, ensemble members
constructed by forecasting from previous analysis times, with varying lead times (e.g.. 1, 2, or 3 steps), such that all members arrive at the same target time. This approach enables ensemble generation without reliance external forecasts or reanalysis data, and allows the system to remain close-loop during long-term assimilation.
However, in practice, forecast errors exhibit magnitude dependence lead time, that is, members with longer lead times tend to have larger absolute deviations. Directly mixing such samples can cause the model to prioritize high-magnitude signals and distort the representation of flow flow-dependent structure.
To address this, we apply two-stage normalization process:
Inter-member normalization: For each ensemble member, we compute the standard deviation of every variable at each vertical level. The corresponding error sample is then divided by this standard deviation, ensuring that all members have comparable amplitude across variables, while preserving their structural differences.
Standard Gaussian normalization: After aligning magnitudes across members, we apply global standardization (zero mean, unit variance) to the full batch of normalized error samples before using them to finetune the decoder via LoRA.

---

T
T₃ time)
Background field
Rackground field
Background field
Flow-dependent
errer samples
Background field (member I)
Forecast
DA
Forecast
DA
Forecast
Error
Analysis field
Analysis field
Analysis field
(umong members)
[stundand Gaussan)
sample
Nornaliation
Nornalization
Background member
(for steps)
sample
Background member 3
Error
Forecast (for three steps)
sample
Ensemble
mean
finetuning
Figure 7: Illustration of time-lagging ensemble construction with normalization. Forecasts with different lead times are aligned to the same target time. Member-specific normalization is applied before finetuning mitigate magnitude imbalance across lead times.
z850 (m²/s²)
t850 (K)
u850 (m/s)
v850 (m/s)
2.4
2.4
RMSE
1.1
2.2
2.0
1.0
1.8
20-
0.2
0.2
0.1
Bias
0.1
0.0
-20
-0.1
-0.1
10
10
20
30
20
30
Simulated Time (Days)
Simulated Time (Days)
Simulated Time (Days)
Simulated Time (Days)
3DVer
Hybrid SDEnVar
VAE-Var ina finetune)
LoRA-EnVar
Figure 8: RMSE and bias comparisons at 850 LoRA-EnVar and three baseline methods (3DVar, hybrid 3DEn Var, and VAE-Var without finetuning) are compared. LoRA-En Var shows consistent advantage in both accuracy and bias reduction.
C.3 Training Finetuning Hyperparameters
The VAE is trained using a composite loss function of the form:
Loss 1 - + LKL.
(13)
where Lree is the reconstruction loss, LKL is the Kullback-Leibler divergence between the latent posterior and prior, and is positive scaling factor. In our experiments,
= 2.0.
Both training and finetuning procedures use the Adam optimizer 662 The learning rate for offline VAE training is set to 10-1, During finetuning, use a learning rate of 10-5 for full model finetuning. both cases, finetuning is performed for five
finetuning, and 10-2 for epochs.
25

---

z850 (m²/s²)
(K)
(m/s)
(m/s)
80
2.2
2.1
RMSE
2.0
2.0
1.0-
1.9
1.8
1.8
20
0.3
0.15
0.2
-20
0.1
-40
0.00
-0.05
10
20
10
10
Simulated Time (Days)
Simulated Time (Days)
simulated Time (Days)
Simulated Time (Days)
VAE-Var finetune)
finetune (wlo reset)
finetune (w/ reset)
LoRA-Eniarr
Figure 9: RMSE and bias comparisons 850 hPa. Different finetuning strategies (LoRA-En Var, full finetuning without resetting, and full finetuning with per-cycle reset) are compared. LoRA-En Var consistently achieves strong accuracy with reduced bias drift.
D
Additional Results
D.1 Supplementary Results for Main Experiments
Cyclic Assimilation and Forecasting To further validate the effectiveness of LoRA-En provide a supplementary assessment on additional atmospheric levels. Figure&report reports the RMSE and bias for four variables at 850 hPa geopotential (z850), temperature (1850), zonal wind (u850) and meridional wind (v850) a 30-day cyclic assimilation experiment.
We compare LoRA-En Var with three baselines: classical 3DVar, hybrid 3DEn and VAE- Var without finetuning. Consistent with the main results at other levels, LoRA-En Var achieves the lowest RMSE and bias across most variables throughout the simulation period. Notably, the improvement is most pronounced for wind components (u850 and v850), highlighting the model's ability to capture flow-dependent structures that are particularly dynamic in the lower troposphere.
To further isolate the effect of different finetuning strategies, we compare LoRA-based adaptation with two variants of full-model finetuning: one that accumulates updates over time (w/o reset) and another that resets the decoder weights at each cycle (w/ reset). As shown in Figure 9 LoRA-En Var achieves comparable or better performance than both full finetuning approaches, with better stability over time and lower bias accumulation.
Different Observation Densities To complement the main paper's analysis, we report additional assimilation results across different observation densities (250, 500, 1000, and 2000 observations, corresponding to approximately 0.024%, 0.048%, 0.096%, and 0.9% of total grid points). These results extend the evaluation in Section 4.3 by including more key atmospheric variables.
Figures 12 show RMSE curves over 30-day assimilation period various vertical levels (surface, 500 and 850 hPa), comparing four methods: 3DVar, Hybrid 3DEn Var, VAE- Var without finetuning, and LoRA-EnVar Var. The comparison includes surface-level variables (t2m, u10, v10, mslp), q)
as well as free-atme ateepphmo variables at 500 hPa and 850 hPa (z,
While performance varies across different variables and observation settings, LoRA-En Var generally outperforms the baselines in most cases, particularly under sparse observation scenarios. In denser cases, it remains competitive with traditional methods while offering better parameter efficiency.
D.2 Impact of Ensemble Generation Strategy
In this section, we investigate the impact of background ensemble generation strategies on the performance of LoRA-En Var. While the main paper uses time-lagging ensemble with 8 members,
26

---

here we explore an extended 20 20-member ensemble that combines two distinct sources of flow-
dependent perturbations.
The first part of the ensemble includes 12 members generated via the standard time-lagging method, expansion over the 8 members used in the main setting. The second part consists of 8 additional members derived from historical forecasts initialized on the same calendar dates one year earlier. Specifically, construct the ensemble for, e.g., January 3, 2022, we collect forecasts targeting this date from initialization times between January 1 and January 2, 2021, using varying lead times (1 steps). This design leverages the seasonal similarity of atmospheric conditions across years to enhance ensemble diversity.
Figures 1 and 14 show the RMSE and bias of LoRA-EnVar and baseline methods under this expanded ensemble setting. Overall, LoR A-En Var continues to outperform traditional assimilation methods and full finetuning variants most variables. The increase in ensemble size also leads to general reduction in assimilation errors for all ensemble-based methods, highlighting the benefit of richer ensemble sampling.
D.3 Impact LoRA Rank
To evaluate the sensitivity of LoRA-En Var to the rank hyperparameter in LoRA modules, we conduct a series of cyclic assimilation experiments with different LoRA ranks: 1, 2 (default in the main paper), 4, 8, and 16. All other configurations, including model architecture, observation density, and ensemble setup, are kept identical to the main setting.
As shown in Figure 15 when the LoRA rank is set to 1, the assimilation accuracy slightly decreases, especially in terms of bias for certain variables such as near-surface temperature and u500. This suggests that a rank of may not provide sufficient representational capacity to capture flow-dependent forecast corrections. In contrast, when the rank set to 2 or higher, the assimilation results remain largely consistent, with negligible differences in both RMSE and bias across ranks 2, 4, 8, and 16.
These findings indicate that LoRA rank of provides adequate modeling capacity for case, while larger ranks offer no significant gain but incur additional computational overhead. This highlights the robustness and parameter efficiency of LoRA-En Var.
D.4 Assimilation with Real Observations
To further evaluate the practical effectiveness of LoRA-En Var, we conduct additional experiments using real-world observations from the NOAA GDAS (Global Data Assimilation System) prepbufr dataset.
D.4.1 Experimental Setup
Dataset Description The GDAS prepbufr dataset provides a comprehensive set of global mete- orological observations curated by the National Centers for Environmental Prediction (NCEP). contains surface and upper-air measurements collected from various sources, including land and marine stations, radiosondes, aircraft reports, and GTS (Global Telecommunications System) trans- missions. The dataset also incorporates advanced remote sensing inputs such as wind profilers, radar-derived wind data from the U.S., satellite-deriver winds from NESDIS, and ocean surface wind estimates from instruments like SSM/I. This rich and operationally relevant dataset serves benchmark in modern weather forecasting systems and thus well-suited for evaluating real-data assimilation performance.
Data Preprocessing Each entry in the prepbufr dataset represents an individual measurement recorded by specific instrument known time and spatial location. Since our assimilation algorithm targets fixed assimilation times, we retain only observations that fall within narrow time window, 30 minutes before and after each assimilation cycle. For example, at an assimilation time of 6 PM, we include only observations recorded between 5:30 PM and 6:30 PM. In addition, these observations are often irregularly located in space and do not lie directly on the forecast model's grid. To resolve this, we construct new, finer observation grid, onto which the raw observations are projected using nearest-neighbor mapping. This intermediate grid shares the horizontal resolution of the Wu model, but contains 40 vertical levels, much denser than the native FengWu Wu grid,
27

---

to preserve the vertical structure while maintaining computational efficiency, After gridding, we perform quality control by discarding any observations that deviate significantly from the co-located FRA5 reanalysis values.
Observation Operator Construction Once the observations are placed on the uniform grid, we define the observation operator V. as the mapping from the FengWu forecast grid to this observation grid. This is implemented using differentiable linear interpolation layer in PyTorch, ensuring full compatibility with our variational assimilation framework.
D.4.2 Evaluation Protocols
To assess the effectiveness of LoRA-En Var under realistic observational settings, we design two complementary evaluation protocols:
ERA5-based Evaluation In this setting, we assimilate all available GDAS observations each assimilation step and compare the resulting analysis fields with the ERAS reanalysis. Specifically, we compute the RMSE between the analysis and ERA5 over the global domain, across multiple variables and levels. This evaluation reflects the system's 's ability to recover large-scale atmospheric structures and serves as a standard benchmark for assessing assimilation fidelity.
-lation-based Evaluation To more directly evaluate the predictive skill of the assimilation system at observation sites, we adopt hold-out validation strategy. At each assimilation time, we randomly exclude 15% of the available observational stations from assimilation. The remaining 85% of observations are used to perform data assimilation as usual. After assimilation, we interpolate the resulting analysis fields to the locations of the held-out 15% stations and compute RMSE between the interpolated values and the actual observations at those sites. This protocol assesses the system's ability generalize unseen observations and is particularly relevant for evaluating performance in sparse or partially observed regions.
D.4.3 Results and Analysis
In this context, compare the analysis fields generated by different assimilation methods against the ERA5 reanalysis. To offer a clearer view of how as- similation improves upon observational input, we include a linear interpolation baseline that directly interpolates observations without consideration of model dynamics. As shown in Figure [6] all assimilation methods, including 3DVar, hybrid 3DEn Var, VAE-Var, and LoRA-En Var, outper- form linear interpolation across nearly all variables and time steps. This highlights the benefit of incorporating physical priors and model dynamics in reducing analysis error. Notably, among the model--ased methods, LoRA Var achieves consistently lower RMSE than VAE Var and hybrid 3DEn Var, especially for variables like wind speed and temperature. These results align well with the trends observed in our earlier simulated observation experiments, reaffirming the effectiveness of LoRA-based flow-dependent adaptation in real-world settings.
ERA5-based Evaluation Results
To evaluate model generalization to unseen observations, we adopt station hold-out strategy, where 15% of the GDAS observation stations are excluded from assimilation and used solely for evaluation. Figure 17 presents the RMSE of interpolated analysis values at these held-out stations. Due to the inherent noise and variability in real-world observations, these RMSE curves exhibit stronger fluctuations compared to ERA5-based evaluation. To highlight the performance difference between LoRA- -nAAEE Var and VAE- Var more clearly, we also plot the RMSE difference between the two methods (gray curves in Figure 17). where positive values indicate that LoRA-En Var performs better at a given time step. Overall, we observe that LoRA-En Var outperforms VAE-Var most time steps and across most variables, indicating its superior ability to integrate sparse and noisy observational data without overfitting.
Station-based Evaluation Results
28

---

#Obs 250 (0.024%)
#Obs 500 (0.048%)
#Obs 1000 (0.096%)
#Obs 2000 (0.19%)
3.5
3.0
Hybrid 3DEnVar
WAE-Vr
2.5
LoRA EnVar
RMSE
2.0
30
20
3.0
RMSE (mis)
10
30
20
20
10
30
2.50
2.25
RMSE 09
2.00
1.75
MUM MWW
1.50
1.00
10
20
30
0
10
20
30
0
10
20
30
10
20
30
350
A00
300
250
150
100
50
10
20
30
0
10
20
30
0
10
20
30
0
10
20
30
AMSE (10-"kg/kg) (kggk)))))
1.6
1.4
0.8
10
20
30
10
20
30
10
20
30
10
20
30
Simulated Time (Days)
Simulated Time (Days)
Simulated Time
Simulated Time (Days)
Figure 10: RMSE comparisons of major surface-level variables and 7000 under different observation densities (250~2000 obs). CoRA-EnVar is compared against 3Dv Hybrid 3DEn Var, and VAE-Var (no finetuning).
29

---

#Obs = 250 (0.024%)
#Obs 500 (0.048%)
#Obs 1000 (0.096%)
#Obs 2000 (0.19%)
400
3DVar
Hlbrid 3DEnVar
(mrlss))
VAE-Vrr
300
LoRA EnVar
200
100
10
20
30
10
20
30
10
20
30
10
20
30
RMSE (10-kk//g)
1.0
0.9
0.6
0.5
10
20
30
0
10
20
30
0
10
20
30
10
20
30
RMSE
3-
10
20
30
10
20
0
10
20
30
10
20
30
v500 RMSE (m/s)
10
30
20
20
10
30
2.00
1.75
1.50
1500 RMSE
1.25
1.00
0.75
0.50
30
Simulated Time (Days)
Simulated Time (Days)
Simulated Time (Days)
Simulated Time (Days)
Figure 11: RMSE comparisons 500 hPa level for multiple variables under varying observation densities. LoRA-En Var is compared against 3DVar, Hybrid 3DEnVar and VAE- Var (no finetuning).
30

---

#Obs = 250 (0.024%)
#Obs 500 (0.048%)
#Obs 1000 (0.096%)
#Obs 2000 (0.19%)
300
3DVar
250
Hlbrid 3DEnVar
VAE-Vrr
200
LoRA EnVar
10
20
30
10
20
30
10
20
30
10
20
30
(20- -kgg/
2.25
2.00
1.75
1.50
1.25
1.00
10
20
30
0
10
20
30
0
10
20
30
0
10
20
30
4.5
uB50 RMSE (mis)
4.0
2.5
1.5
10
20
30
10
20
0
10
20
30
10
20
30
4.5
4.0
v850 RMSE
3.0
1.5
10
30
20
20
10
30
2.25
2.00
1.50
1.25
1.00
0.75
20
30
10
Simulated Time (Days)
Simulated Time (Days)
Simulated Time (Days)
Simulated Time (Days)
Figure 12: RMSE comparisons 850 hPa level for multiple variables under varying observation densities. LoRA-En Var is compared against 3DVar, Hybrid 3DEnVar and VAE- Var (no finetuning).
31

---

u10 (m/s)
vI0 (m/s)
t2m (K)
kg/kg)
1.-
AMSE
1.4
1.4
1.4
1.2
1.2
0.1
0.05
Bea
0.05
0.00
0.0
-0.05
0.4
-0.1
0.05
30
20 Simulated Time (Days)
20
Simulated Time (Days)
Simulated Time (Days)
Simulated Time (Days)
z500 (m²
t500 (K)
uS00 (m/s)
v500 (m/s)
3.0
0.80
AMSE
0.75
2.8
80
0.70
2.6
60
0.65
2.4
2.4-
0.2
Bea
0.0
-0.1
-50-
0.2
20
16
20
(Days)
Simulated (Days)
Simulated (Days)
(Days)
z850 (m 2/5²)
t850 (K)
u850 (m/s)
v850 (m/s)
70
1.2
2.4
2.4
RMSE
2.2
2.2
60
2.0
0.9
1.8
1.8
Maagp Mank
10
0.2
0.1-
Bies
0.1
0.0
0.0
0.0
20
-0.1
20
30
20
10
20
10
20
30
Simulated Time (Days)
Simulated Time (Days)
Simulated Time (Days)
Simulated Time (Days)
JDVar
Flybrid 3DEnVar
WAE-Var (no finetune)
LoRA-EnVar
Figure 13: RMSE and bias comparisons for major variables using a 20-member ensemble, including 12 time-lagged and 8 seasonally aligned historical forecasts. LoRA-En Var is compared with 3DVar, hybrid 3DEn Var, and VAE-Va Var (no finetuning).
32

---

(m/s)
(m/s)
(K)
4700 (10- kg/kg)
1.4
1.4
1.4
1.2
RMSE
1.3
1.3
1.2
1.2
1.2
1.1
8ias
0.0
-0.4-
-0.6
0.1-
-0.8
20
10
20
10
20
30
10
20
30
Simulated Time (Days)
Simulated Time (Days)
Simulated Time (Days)
Simulated Time (Days)
2500 (m?
t500 (K)
u500 (m/s)
v500 (m/s)
RMSE
0.8
2.6
2.6
2.4
0.15
0.4
50
0.2
0.1
Bias
0.05
0.0
0.0
0.00
-50-
0.2
10
20
10
20
10
20
30
10
20
30
Simulated Time (Days)
Simulated Time (Days)
Simulated Time (Days)
Simulated Time (Days)
z850 (m?
t850 (K)
u850 (m/s)
v850 (m/s)
2.2
1.2
2.1
60
2.0
RMSE
2.0
55
1.9
1.0
1.9
0.9
1.8
1.8
45
Mark
0.2
0.2
Blbs
-0
0.0
0.0
10
20
10
20
10
20
30
10
20
30
Simulated Time (Days)
Simulated Time (Days)
Simulated Time (Days)
Simulated Time (Days)
VAE-Ver (no finetune)
Full finetune (wto reset))
Full finetune (w/ reset)
LoR
Figure 14: Comparison of LoRA-En Var and full finetuning methods using the 20-member ensemble. LoRA-En Var remains competitive while using significantly fewer trainable parameters.
33

---

u10 (m/s)
v10 (m/s)
t2m (K)
4700 (10-) kg/kk)
1.4
1.4
1.2
RMSE
1.2
1.3.
1.1
1.2
1.2
1.0
1.1
0.1
0.00
0.05
0.0
-0.05
0.05
-0.1
0.00
-0.2
-0.15
30
Simulated Time (Days)
Simulated Time (Days)
Simuulated Time (Days)
Simulated Time (Days)
2500 (m²/s²)
t500 (K)
us00 (m/s)
v500 (m/s)
2.8
RMSE
2.6
70
2.4
0.6
60
0.4
40
0.1
Bias
20
0.05
0.2
0.0
0.00
0.0
-20
0.2
0.2
10
20
30
10
20
30
10
20
30
10
20
30
Simulated Time (Days)
Simulated Time (Days)
Simulated Time (Days)
Simulated Time (Days)
z850 (m )
t850 (K)
u850 (m/s)
v850 (m/s)
1.2
2.2
2.1
60
2.1
2.0
RMSE
2.0
1.9
1.9
50
0.9
1.0
1.8
0.3
0.2
0.15
0.2
Blas
0.1
0.10
0.1-
0.05
0.0
0.0
-20
-0.1
0.00
-0.1
10
zb
30
10
za
30
10
20
30
10
20
30
Simulated Time (Days)
Simulated Time (Days)
Simulated Time (Days)
Simulated Time (Days)
(no finetune) LoRA-EnVar (rank=16)
trank=8)
LoRWEnVar (rank=2) LoRWEnWVrr (rank=1)
LoRA-EnVar (rank=4)
Figure 15: RMSE and bias comparisons of LoRA-En Var with different LoRA ranks (1, 2, 4, 8, 16). Rank 2 offers a good trade-off between accuracy and efficiency, while rank underperforms slightly some variables.
34

---

u10 (m/s)
v10 (m/s)
t2m (K)
q700 (10 kkg/kg)
3.5
4.0
3.5
3.0
3.5
3.0
RMSE
2.5
2.5
2.0
1.25
1.5
1.5-
1.00
1.0
1.0
0.75
10
20
30
10
20
30
0
20
30
10
20
30
Simulated Time (Days)
Simulated Time (Days)
Simulated Time (Days)
Simulated Time (Days)
2500 (m²
(K)
(m/s)
(m/s)
10
1600
9.
8
1400
RMSE
4
4-
200
2
10
20
30
10
20
30
10
20
10
20
30
Simulated Time (Days)
Simulated Time (Days)
Simulated Time (Days)
Simulated Time (Days)
2850 (m²/s²)
t850 (K)
u850 (m/s)
v850 (m/s)
600
5-
500
4-
RMSE
400
3.
200
100
10
20
10 Simulated (Days)
Simulated (Days)
Simulated (Days)
Simulated (Days)
Hybrid 3DEnvar
VAE-Var finetune)
LoRA-EnVbr
interpolation
Figure 16: ERAS-based evaluation of analysis RMSE for various methods using GDAS real observations. All model-based assimilation approaches outperform linear interpolation, with LoRA- En achieving the best overall accuracy.
35

---

(m/s)
v10 (m/s)
mslp (Pa)
90-
RMSE
0.8
80-
0.6-
0.6
1.2
0.5
0.10
0.10
10
RMSE Difference
0.10
0.05
0.05
0.05
0.00
0.00
0.00
0.05
-0.05
-0.05
10
20
30
10
20
30
10
20
30
0
20
30
Simulated Time (Days)
Simulated Time (Days)
Simulated Time (Days)
Simulated Time (Days)
(m?/s")
t500 (m/s)
us00 (m/s)
v500 (m/s)
400
RMSE
2-
2.5
2.0
200
1.5
100
1.0
1.0
0.50
RASE Difference
0.5
0
-0.5
0.25
-0.50
10
20
10
20
30
10
20
30
10
20
30
Simulated Time (Days)
Simulated Time (Days)
Simulated Time
Simulated Time (Days)
z850 (m?/s?)
t850 (m/s)
u850 (m/s)
v850 (m/s)
120
1.4
RMSE
1.2
80
1.0
0.4
RMSE Difference
0.2
0.1
0.0
0.0
10
-0.4
10
20
10
30
10
30
20
30
Simulated (Days)
Simulated Time (Days)
Simulated
(Days)
Simulated (Days)
VAE-Vrr (no fineture)
LoRA-EnVar
difference
Figure 17: Station-based evaluation using held-out GDAS observations. The bottom half shows RMSE differences between LoRA-En Var and VAE-Var, with positive values indicating better perfor- mance by LoRA-En Var.
36


Reviews:
[{"review_id": "eLvmBBQdtw", "reviewer": "Reviewer_Y4Yp", "summary": "The authors use finetuned LoRA modules and combine them with climate error models (as in VAE-VAR) to conduct data assimilation. They do this by first training a VAE decoder on historical error values and then creating an ensemble of predictions for each cycle and finetuning the LoRA matrices for the flow-dependent ensemble perturbations.", "questions": "Why is are only the query LoRA integrated, as compared to also the K and the V?\n\nIs there a link between the LoRA rank and an assumed effective ensemble size or background covariance rank?", "limitations": "Yes.", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strengths\n\nThe model is efficient in terms of trainable parameters at test time which is only 0.1 % of the model, which is practical for finetuning with limited GPU capacity when running in an online fashion. This allows one to adapt it to more complex transformer architectures for the VAE without much computational overhead.\n\nThe authors do not assume a Gaussian background which allows for more flexible likelihood functions compared to standard data assimilation models which assume Gaussian backgrounds.\n\nThe authors test the model at 0.25 degree global resolution, and they found that LoRA with rank 2 was sufficient to capture the detailed dataset, so method scales without exponential parameter growth. 0.25 degree forecasting is the de-facto standard for MLWP today.\n\nThe model also seems robust to different ensemble sizes, observation densities, and variables. The authors have done quite a few ablation experiments while adjusting these parameters.\n\nThe integration of LoRA into standard diffusion model transformers for image generation has been explored before, but it has not been applied to the weather prediction and assimilation community. This paper presents the first foray into incorporating such a step, and also offers avenues of improvements for future data assimilation techniques which are based on a VAE-Var style of data assimilation. \n\nLORA EnVar seems to scale well with observation sparsity in comparison to the baseline methods. However, the results are not a significant boost over strong baselines such as VAE-Var.\n\nWeaknesses\n\nThe model relies on a Swin-V2 backbone tailored to FengWu; portability to other forecast models is not clear in terms of how it goes.\nAlthough LoRA updates are light, the decoder is still a large transformer that must run in each inner-loop gradient evaluation.\n\nSummary\n\nThe paper is well-structured and clear, and offers significant novelty in terms of introducing LoRA to data assimilation with MLWP, and opens up future avenues of improvement for VAE-Var style data assimilation approaches. The results seem like a modest improvement upon existing data assimilation techniques like VAE-Var. As a result, I vote accept for this paper.", "was_revised": true, "final_justification": "I believe that this work is a good contribution towards the field of data assimilation, and I will argue for acceptance. The experiments seem to be robust on ERA5. However, since its improvements over the standard VAE-Var approach seems to be somewhat incremental, its impact is slightly limited to be considered for a strong acceptance.", "rebuttal": "Thank you very much for your constructive and detailed comments. We greatly appreciate your positive assessment of the methodology, novelty, and overall presentation of our work. \n\n___\n\n> **Response to Q1: Selective LoRA integration on queries**\n\nThank you for the insightful question regarding the selective integration of LoRA modules into the query projections only.\n\nTo address this, we conducted additional ablation experiments where LoRA was applied to different combinations of the query (Q), key (K), and value (V) projections. Specifically, we tested:\n(1) finetuning Q only (the default in our main results),\n(2) finetuning both Q and K,\n(3) finetuning both Q and V, and\n(4) finetuning Q, K, and V simultaneously.\n\nThe results over the first 30 days of January are summarized in the tables below. For each 3-day period, we computed the average RMSE across twelve 6-hour assimilation cycles. These experiments were conducted for two representative variables: z500 and u500. As shown, finetuning Q alone already achieves strong performance, and while incorporating additional LoRA modules into K and/or V leads to minor improvements in some periods, the relative performance fluctuates across different 3-day windows. Overall, the average gains are marginal and often fall within the variability observed across repeated runs.\n\nImportantly, extending LoRA to K or V increases the number of trainable parameters by $2\\times$ or $3\\times$, which goes against our design goal of efficient online adaptation with minimal computational overhead. Thus, we chose to finetune only Q in our main experiments.\n\n| z500 RMSE ($\\mathrm{m^2}/\\mathrm{s^2}$) | Finetune Q | Finetune Q, K | Finetune Q, V | Finetune Q, K, V |\n| --------------------------------------- | ---------- | ------------- | ------------- | ---------------- |\n| Day 1-3                                 | 56.79      | 56.46         | 56.25         | 54.62            |\n| Day 4-6                                 | 59.77      | 60.46         | 66.67         | 65.49            |\n| Day 7-9                                 | 61.24      | 61.47         | 62.10         | 60.80            |\n| Day 10-12                               | 65.34      | 64.95         | 65.43         | 64.72            |\n| Day 13-15                               | 65.97      | 66.41         | 64.57         | 64.79            |\n| Day 16-18                               | 63.85      | 65.14         | 63.59         | 65.34            |\n| Day 19-21                               | 67.88      | 68.72         | 64.95         | 65.30            |\n| Day 22-24                               | 65.17      | 66.33         | 69.48         | 71.20            |\n| Day 25-27                               | 67.31      | 69.27         | 62.24         | 65.58            |\n| Day 28-30                               | 67.92      | 71.07         | 65.00         | 66.90            |\n| Average                                 | 64.12      | 65.03         | 64.03         | 64.47            |\n\n| u500 RMSE ($\\mathrm{m}/\\mathrm{s}$) | Finetune Q | Finetune Q, K | Finetune Q, V | Finetune Q, K, V |\n| ----------------------------------- | ---------- | ------------- | ------------- | ---------------- |\n| Day 1-3                             | 2.403      | 2.398         | 2.384         | 2.384            |\n| Day 4-6                             | 2.599      | 2.595         | 2.582         | 2.567            |\n| Day 7-9                             | 2.605      | 2.592         | 2.598         | 2.551            |\n| Day 10-12                           | 2.701      | 2.675         | 2.722         | 2.659            |\n| Day 13-15                           | 2.554      | 2.584         | 2.571         | 2.547            |\n| Day 16-18                           | 2.634      | 2.630         | 2.612         | 2.589            |\n| Day 19-21                           | 2.781      | 2.741         | 2.753         | 2.730            |\n| Day 22-24                           | 2.683      | 2.656         | 2.700         | 2.693            |\n| Day 25-27                           | 2.657      | 2.612         | 2.649         | 2.608            |\n| Day 28-30                           | 2.674      | 2.706         | 2.672         | 2.661            |\n| Average                             | 2.629      | 2.619         | 2.624         | 2.599            |\n\nThese findings are also consistent with the original LoRA paper [1], which reports that finetuning Q alone often suffices, and that additional gains from finetuning K and V are typically small. This reinforces our decision to prioritize efficiency by limiting updates to the query projections only.\n\n> **Response to Q2: Relationship between the LoRA rank and an effective ensemble size**\n\nThank you for the insightful question regarding the relationship between LoRA rank and ensemble size or background covariance rank.\n\nWhile both the ensemble size in traditional data assimilation and LoRA rank in our framework reflect constraints on the representational capacity of uncertainty, they operate in fundamentally different ways. In traditional ensemble-based methods, ensemble size determines the number of samples used to estimate the flow-dependent structure of the background error. A small ensemble size can lead to an under-represented covariance with artificially low rank, failing to capture key dynamical modes. In general, assuming sufficient computational resources are available, larger ensembles lead to better performance, as they provide more accurate and higher-rank estimates of the background error.\n\nIn contrast, LoRA rank does not directly correspond to ensemble size, nor does it determine the rank of any explicit background covariance matrix. Instead, it controls the maximum number of directions in parameter space along which dynamical corrections can be introduced into a climatologically trained model. If the LoRA rank is too low, even a large ensemble cannot be fully leveraged, as the model lacks sufficient degrees of freedom to incorporate flow-dependent corrections. Conversely, an overly high rank may introduce redundant parameters with little benefit. In our experiments, a LoRA rank of 2  achieved a optimal balance between expressiveness and efficiency. Moreover, in LoRA-EnVar, flow-dependent information is incorporated via online learning, enabling the model to extract meaningful dynamical information even with a small ensemble, as long as the LoRA rank is sufficiently expressive. \n\nExploring a more explicit theoretical connection between LoRA rank, ensemble size, and the effective background error structure remains an interesting direction for future work.\n\n> **Response to W1: Portability to backbones beyond Swin-V2**\n\nWe thank the reviewer for raising the question of portability. Our current implementation of LoRA-EnVar is based on the Swin-V2, but the approach is in principle easily adaptable to other Transformer-based architectures. This flexibility is because LoRA modules operate over standard linear projections, such as query/key/value matrices, which are widely shared across Transformer variants.\n\nFor models like GraphCast, which are based on graph neural networks (GNNs) and do not contain Transformer-style attention mechanisms, a direct application of LoRA-EnVar is not straightforward. Such architectures lack the projection layers that LoRA typically targets. Adapting our framework to GNNs would require non-trivial modifications, such as developing alternative fine-tuning mechanisms tailored for message-passing networks. We consider this a promising direction for future work.\n\n> **Response to W2: Inner-loop gradient updates requiring full transformer backpropagation**\n\nWe appreciate the reviewer’s insightful observation. Indeed, our method requires running the Swin-V2 decoder in each inner-loop gradient evaluation during assimilation, reflecting the inherent computational cost of variational approaches that rely on gradient optimization. However, by updating only the lightweight LoRA modules, we significantly reduce the number of **trainable parameters**. While the full decoder remains part of the backward graph to allow gradient flow into the LoRA parameters, all backbone weights remain frozen, eliminating gradient storage for the backbone itself. This approach reduces memory usage and simplifies optimization, even though backpropagation still traverses the full network. In future work, we plan to explore further optimizations, such as updating only a subset of decoder layers or employing lighter-weight backbone architectures. \n\n> **Response to S6 (last sentence): Comparison with other baselines, such as VAE-Var**\n\nWe appreciate the reviewer’s comment. Indeed, even in traditional data assimilation methods, the gain from incorporating flow-dependent (dynamical) error structures is often incremental, particularly on some variables, when compared to strong climatological baselines (e.g., Hybrid 3DEnVar vs. 3DVar, as shown in Figure 3). Our goal is to introduce such capabilities in a lightweight and extensible way, and we believe LoRA-EnVar achieves this with minimal overhead.\n\n___\n\nWe appreciate your feedback once again and it helped us strengthen the overall presentation and completeness of the paper.\n\n**References:**\n\n[1] Hu, E. J., Shen, Y., Wallis, P., Allen-Zhu, Z., Li, Y., Wang, S., ... & Chen, W. (2022). Lora: Low-rank adaptation of large language models. *ICLR*, *1*(2), 3."}, {"review_id": "NN1V8J7Sfr", "reviewer": "Reviewer_2YwJ", "summary": "The submission introduces a new data assimilation approach which uses a pretrained VAE to learn an approximation of a non-Gaussian background covariance which is then updated online via LoRA in order to incorporate information from the local flow. This approach is empirically validated against several traditional approaches and the recently proposed VAE-Var on both assimilation from reanalysis ICs and simulated operational ICs across a variety of observation densities.", "questions": "Q1 - How does the calibration of the ensembles resulting from this approach compare to those of the competitive approaches?\n\nQ2 - It is often easy to reduce RMSE by reducing high-frequency details and extremes in the forecast. Does the distribution of the resulting field values match the target?\n\nQ3 - Does this significant competitive advantage persist when compared against modern DA tools?", "limitations": "Negative societal impact yes. However, discussion of limitations largely focuses on trivial limitations without discussing limitation in experimental methodology.", "rating": 3, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 2, "originality": 2, "strengths_and_weaknesses": "Strengths:\n\n- S1 - The challenge of efficiently capturing non-Gaussian variation in data assimilation is a key challenge in modern weather forecasting. \n- S2 - The combination of experiments progressively challenges the model. The use of restricted observations in space and time are very important and relevant to practical usage and greatly strengthen the potential impact. \n- S3 - The extensive ablations are really interesting and provide some somewhat counter-intuitive information (LoRA providing the biggest gains for slower moving fields like Z500 in Fig 3, larger amounts of finetunable parameters hurting performance). This could be explored more strengthening the paper. \n- S4 - Performance at smaller ensemble sizes is extremely promising. In an operation setting this could significantly reduce the cost of the forecast.\n\nWeaknesses:\n- W1 - The major weaknesses of the paper are centered on the evaluation and comparisons.\n    - W1.A - The comparisons in general are very weak. The proposed method utilizes a 300M parameter SWINv2 architecture while 3DVar is largely a teaching example. This is not a fair comparison. While it's not expected that the experiments use a full operational DA system, a variational-to-variational comparison should include 4DVar (and associated ensemble-equipped methods) and likely at least one EnKF-based method (as they tend to be a bit easier to tune). Given the small ensemble sizes, it's likely that these would require some parameter tuning.\n    - W1.B - On the ML side, the related work discussed a number of different approaches while stating without evidence why these approaches are unsuitable. This is a largely empirical submission, so these claims need to be shown rather than just stated. If scale is an issue, it's fine to just point to an appendix section showing why those approaches shouldn't scale. If the issue is instead that methods perform poorly at scale, then this can be shown experimentally. \n    - W1.C - Right now the paper is motivated by fixing an error in the probabilistic formulation of variational DA. However, there is then no follow-up on calibration of forecasts. The paper states that this is not available for variational methods which is true for pure variational approaches where all spread information is provided a priori, but with hybrid methods, this can be estimated empirically from the ensemble.\n - W2 - All experiments are currently performed on a fairly small time window. 4.2 is on January 1, 2022. 4.3 is on that same month. There does not seem to be a reason why more extensive backtesting would be impossible. \n- W3 - (Minor) Parameter efficiency is interesting, but not really the relevant comparison given your experiments show LoRA outperforming full rank training. Since online training is involved, LoRA should actually provide some real hardware efficiency gains either on runtime for fixed hardware or memory usage. These are more relevant.\n- W4 - (Minor) For cases where a term or definition is vital to understanding the paper, it's usually better to find a way to include it in the main text rather than pointing readers to citations or the appendix. The variational procedure is the most notable case of this in the text currently.\n\nOverall, this feels quite close to the line. Maintaining results with nontrivial numerics baselines and performing more extensive testing across time is likely enough that I'd be comfortable moving to weak accept while including stronger ML baselines, including analysis of the field distributions (appendix is fine as long as it's pointed out and discussed in the main body), and more probabilistic analysis would result in a stronger evaluation.", "was_revised": true, "final_justification": "On first read, my primary concern was the extent of validation. Including only one month of data and only a trivial baseline made it difficult to place the paper in context. \n\nHowever, discussion seemed to bring up more issues. The ensembles were generally poorly calibrated and the inclusion of 21st century baselines produced results that did not match up with accepted knowledge in the domain. The authors were unable to produce satisfying explanations for these inconsistencies and it on deeper read, it seems experimental flaws are the most likely culprit. \n\nWhile I think the paper on the whole is well motivated and contains many interesting ideas, I am uncomfortable recommending acceptance given the issues identified in the experimental setup and the lack of discussion of these issues in the submission and rebuttal.", "rebuttal": "Thank you very much for your constructive and detailed comments. We greatly appreciate your positive assessment of the methodology of our work.\n\n___\n\n> **Response to W1.A&W1.B&Q3: Comparison with modern DA tools**\n\n**Comparison with traditional DA methods**\n\nWe fully agree that stronger baselines, such as 4DVar and EnKF-based methods, are essential for a meaningful evaluation. In response, we extended our experiments to include:\n\n- **4DVar and Hybrid 4DEnVar:** To evaluate 4D variants, we introduce additional observations at 1h, 7h, 13h, etc., alongside standard 6-hourly observations. We use a 1-hour-resolution FengWu model to represent flow evolution within each assimilation window.\n- **LETKF:** We implemented the Local Ensemble Transform Kalman Filter with 200 km horizontal localization and 0.3 vertical tapering. Ensemble size was kept consistent with other EnVar-based methods.\n\n**Comparison with learning-based DA methods**\n\nWe carefully considered several recent ML-based DA baselines:\n\n- **DiffDA** learns $p(x \\mid x_b)$ via diffusion models, which requires paired $(x_b, x)$ data for training and adaptation. However, such labeled data is difficult to obtain online, making flow-dependent adaptation nontrivial. Our method instead models the error distribution $p(e = x - x_b)$, which is easier to sample in operational settings. We nonetheless include DiffDA (without flow-dependent ensembles) in our comparisons.\n- **SDA** does not incorporate $x_b$ and cannot perform cyclical assimilation, making it incompatible with our setting.\n- **FuXi-En4DVar** is a variant of Hybrid 4DEnVar with FuXi used for ensemble generation. We include a comparable Hybrid 4DEnVar baseline using FengWu.\n- **EnSF and Latent-EnSF**: EnSF assumes complete observations and is not applicable to high-dimensional sparse settings. Latent-EnSF extends EnSF to sparse observations, but its results are only shown at low spatial resolution ($21 \\times 72 \\times 144$), and no official code is currently available for reproduction or scaling tests. We will include it in future work once the implementation becomes accessible.\n\n**Results**\n\nIn the updated tables below, we report RMSEs for key variables (z500 and u500) across **ten methods**, including five traditional and five learning-based approaches (including both 3D and 4D variants). These comparisons demonstrate that our proposed LoRA-EnVar performs competitively against both advanced DA systems and state-of-the-art ML-based approaches. \n\n| z500 RMSE ($\\mathrm{m^2}/\\mathrm{s^2}$) | 3DVar  | Hybrid 3DEnVar | 4DVar  | Hybrid 4DEnVar | LETKF | VAE-3DVar | LoRA-3DEnVar | VAE-4DVar | LoRA-4DEnVar | DiffDA |\n| - | - | - | - | - | - | - | - | - | - | - |\n| Day 1-10 | 80.434 | 80.937 | 70.331 | 72.894 | 102.83 | 66.724    | 60.334       | 60.609    | 54.200 | 125.12 |\n| Day 11-20 | 86.261 | 88.631 | 74.870 | 78.942 | 106.99 | 77.401    | 64.602       | 71.187    | 59.024 | 140.40 |\n| Day 21-30 | 85.746 | 84.153 | 72.278 | 76.135 | 119.77 | 85.516    | 67.135       | 73.638    | 61.156 | 149.41 |\n| Average | 84.147 | 83.820 | 72.493 | 75.990 | 109.86 | 76.547    | 64.024       | 68.478    | 58.127 | 138.31 |\n\n| u500 RMSE ($\\mathrm{m}/\\mathrm{s}$) | 3DVar | Hybrid 3DEnVar | 4DVar | Hybrid 4DEnVar | LETKF | VAE-3DVar | LoRA-3DEnVar | VAE-4DVar | LoRA-4DEnVar | DiffDA |\n| - | - | - | - | - | - | - | - | - | - | - |\n| Day 1-10 | 2.761 | 2.611 | 2.725 | 2.565 | 2.773 | 2.595     | 2.539        | 2.445     | 2.440 | 3.104 |\n| Day 11-20 | 2.918 | 2.730 | 2.879 | 2.680 | 2.518 | 2.729     | 2.649        | 2.536     | 2.533 | 3.306 |\n| Day 21-30 | 2.923 | 2.691 | 2.826 | 2.638 | 2.497 | 2.743     | 2.670        | 2.592     | 2.582 | 3.354 |\n| Average | 2.867 | 2.653 | 2.810 | 2.628 | 2.596 | 2.689     | 2.619        | 2.524     | 2.519 | 3.255 |\n\n> **Response to W1.C&Q1: Analysis of the calibration of the ensembles**\n\nThank you for raising this important point. Our current work focuses on improving the analysis given a background ensemble, and we did not initially emphasize calibration analysis. To address this, we now include a new experiment to evaluate ensemble calibration. We ran both LoRA-EnVar and Hybrid 3DEnVar systems starting from July 1st and assessed the ensemble forecast initialized at 00:00 on July 15th, allowing sufficient spin-up for both methods. The results are summarized below:\n\n| z500 | Hybrid 3DEnVar | LoRA-EnVar |\n| - | - | - |\n| RMSE | 113.50 | 82.20 |\n| Spread | 70.61 | 76.10 |\n| Spread-to-RMSE Ratio | 62.2% | 92.6% |\n\n| u500 | Hybrid 3DEnVar | LoRA-EnVar |\n| - | - | - |\n| RMSE | 2.840 | 2.587 |\n| Spread  | 1.423 | 1.385 |\n| Spread-to-RMSE Ratio | 50.1% | 53.5% |\n\nThese results show that LoRA-EnVar not only improves forecast accuracy (lower RMSE), but also yields better-calibrated ensembles, as indicated by the higher spread-to-RMSE ratios. This suggests that the flow-dependent latent prior learned by LoRA-EnVar more effectively captures uncertainty in the analysis compared to traditional hybrid methods. We will include additional calibration diagnostics in the revised version to further support this finding.\n\n> **Response to W2: More extensive testing across time**\n\nThank you for the suggestion. We fully agree that evaluating the system solely on January data is insufficient to demonstrate generalizability. In response, we have conducted an additional experiment using data from July, which represents a different seasonal regime.\n\n| z500 RMSE ($\\mathrm{m^2}/\\mathrm{s^2}$) | 3DVar | Hybrid 3DEnVar | LoRA-EnVar | VAE-Var (no finetune) | Full finetune (w/o reset) | Full finetune (w/ reset) |\n| - | - | - | - | - | - | - |\n| Day 1-10 | 84.310 | 85.128 | 62.146 | 70.980 | 93.606 | 59.597 |\n| Day 11-20 | 88.768 | 91.054 | 68.911 | 87.594 | 137.540 | 68.526 |\n| Day 21-30 | 87.170 | 87.920 | 73.072 | 87.623 | 130.110 | 71.267 |\n| Average | 86.749 | 88.034 | 68.043 | 82.066 | 120.419 | 66.463 |\n\n| u500 RMSE ($\\mathrm{m}/\\mathrm{s}$) | 3DVar | Hybrid 3DEnVar | LoRA-EnVar | VAE-Var (no finetune) | Full finetune (w/o reset) | Full finetune (w/ reset) |\n| - | - | - | - | - | - | - |\n| Day 1-10 | 2.725 | 2.581 | 2.514 | 2.550 | 2.598 | 2.489 |\n| Day 11-20 | 2.815 | 2.713 | 2.707 | 2.709 | 3.043 | 2.764 |\n| Day 21-30 | 2.827 | 2.713 | 2.747 | 2.715 | 2.898 | 2.824 |\n| Average | 2.789 | 2.669 | 2.656 | 2.658 | 2.846 | 2.692 |\n\n> **Response to W3: Hardware efficiency gains**\n\nThank you for the comment. While LoRA does not significantly reduce runtime (from $\\sim$10s to $\\sim$9s per cycle) or memory (~40.5GB to ~39.1GB) compared to full-finetune methods, its main advantage lies in dramatically lower storage cost. A single LoRA checkpoint is only ~648KB, enabling long-term retention of fine-tuned states (e.g., ~2.5GB for one year at 6-hour cycles), compared to ~2.3TB for full fine-tuning, which is over 1000× smaller. This makes LoRA-EnVar much more practical for retrospective analysis or downstream applications.\n\n> **Response to W4: Key formulations not presented in main body**\n\nThank you for the helpful suggestion. We agree that the variational procedure is important and should be briefly clarified in the main text. In variational data assimilation, the goal is to estimate the state that minimizes a cost function balancing prior and observational constraints, equivalent to a regularized negative log-posterior. In our case, this is formulated in a latent space with learned priors. We will add concise explanations in the revised version to improve clarity and self-containment.\n\n> **Response to Q2: Concerns about realism of forecast distribution**\n\nThank you for raising this important point. Our LoRA-EnVar framework does not rely on reducing high-frequency features or extremes to lower RMSE. To examine whether our analysis field preserves realistic distributional characteristics, rather than artificially smoothing the output, we compute two standard statistical metrics: standard deviation and skewness, which reflect the overall spread and asymmetry of the field values, respectively.\n\nDue to limits on image uploads, we report numerical values at a representative time (2022.01.01 00:00) in the table below. It can be observed that:\n\n- The standard deviation of the analysis field is similar to than that of the background field, indicating that the assimilation process does not suppress variance.\n- The skewness values of both z500 and u500 are well aligned with those of the ground truth, suggesting that extreme values are retained, and the distributional shape is preserved.\n\nThe relative differences between the analysis and the ground truth are below 1%, confirming that our method does not distort the statistical structure of the field. In the revised version, we will include additional visualizations and statistical analyses in the appendix to further support this point.\n\n| | Background Field | Analysis Field | Ground Truth |\n| - | - | - | - |\n| z500 standard deviation | 3145.8 | 3152.6 | 3171.3 |\n| u500 standard deviation | 11.516 | 11.503 | 11.554 |\n| z500 skew | -0.1582 | -0.1586 | -0.1573 |\n| u500 skew | 0.6580 | 0.6601 | 0.6505 |\n\n> **Response to S3: Interesting but underexplored ablation insights**\n\nThank you for highlighting this interesting finding. We also observed that LoRA yields the largest improvements for Z500, a relatively slow-evolving and spatially coherent variable. One possible reason is that Z500 reflects the large-scale circulation and tends to integrate information from multiple physical processes, making it more amenable to the structured, low-rank adaptation provided by LoRA. Conversely, the decline in performance with more finetunable parameters suggests potential overfitting or interference with pretrained representations. We will explore this further and add discussions in the revised version to strengthen the interpretation of these results.\n\n___\n\nWe appreciate your feedback once again and it helped us strengthen the overall presentation and completeness of the paper."}, {"review_id": "7OK8v6S5KN", "reviewer": "Reviewer_Zgna", "summary": "The paper aims to address limitations in current variational data assimilation (DA) systems, most notably, their reliance on using a static background error learned from climatological data, and the limitation to using Gaussian errors to generate analysis. While hybrid DA methods exist that combine background error statistics with \"error-of-the-day\" statistics obtained by ensemble simulations, traditionally, this still assumes the error to be Gaussian to be used within the traditional variational framework. The paper proposes an algorithm that performs variational data assimilation in latent space using VAEs trained on climatological error samples; assuming a Gaussian bottleneck for the VAE, the use of Gaussian error statistics is justified in latent space by construction. Furthermore, the VAE is fine-tuned using LoRA on \"errors-of-the-day\" to achieve a dynamically evolving error statistics. Experiments on a weather forecasting model trained on ERA5 demonstrate the added benefits of using non-Gaussian error estimates (achieved by operating in latent space) and dynamically evolving error by LoRA finetuning.", "questions": "- How are the ensemble obtained to fine-tune LoRA (i.e., what are the \"Other predictions\" in Figure 1)? Are they based on different initial conditions or different forecasting models?", "limitations": "The authors acknowledge limitations of the approach, in particular, lack of theoretical characterisation and intrinsic limitation of only being able to produce point estimates.", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 4, "strengths_and_weaknesses": "__Strengths:__\n- Overall, I believe that the idea presented in this work is very elegant, providing an interesting approach to tackle both the issues of using static and Gaussian error covariances in traditional data assimilation using a single clean framework.\n- The experiment section is quite strong. In particular, the ablation studies convincingly demonstrate the added benefit of using both non-Gaussian error statistics and dynamical error statistics in data assimilation, at different observation densities.\n- Experiments to compare LoRA vs full fine-tuning demonstrate that we are not losing much by only fine-tuning the VAE; in fact, the experiments show that LoRA is not only sufficient, but even outperforms the results of full fine-tuning. in general, I believe that all of the components that are introduced in the model is justified.\n\n__Weaknesses:__\n- One clear limitation of the approach is that the method explicitly uses the fact that the VAE has a Gaussian bottleneck to justify the use of 3DVAR in latent space. Thus, it cannot accommodate the use of more general autoencoders such as vector-quantised VAEs (VQ-VAEs), which can achieve better reconstruction compared to standard VAEs.\n- A minor gripe that I have is that some of the colors used in the plots are very similar to each other, which makes it hard to see which curve corresponds to which model.\n- I believe the proposed methodology can be better explained. For example, I believe it would be useful to have Appendix A in the main body instead of the appendix. Otherwise, we are only left with a vague description of the methodology and Figure 1 to understand the idea, which I initially found difficult to grasp until reading Appendix A.", "was_revised": true, "final_justification": "The methodology introduced by the paper is, in my opinion, quite an interesting extension to classical hybrid ensemble variational methods, which, traditionally, are limited to using Gaussian error covariances. I found the experiments to be well-conducted and presented, although Reviewer 2YwJ points out the possibility that it may be flawed due to concerns about how the ensembles are calibrated. Nonetheless, I find it to be an interesting direction in data assimilation with potential for further improvements down the line, which is why I would recommend an acceptance.", "rebuttal": "Thank you very much for your constructive and detailed comments. We greatly appreciate your positive assessment of the overall methodology and experimental evaluation. \n\n___\n\n> **Response to Q1: How the ensemble is obtained to finetune LoRA**\n\nThank you for the question regarding the source of ensemble forecasts used to fine-tune the LoRA modules (“Other predictions” in Figure 1). The LoRA-EnVar framework is flexible and can support different strategies for ensemble forecast generation, such as perturbing initial conditions or using multiple forecast models. In our current implementation, we adopt a time-lagged ensemble approach, which is well adopted in use and fundamentally falls into the first category—based on different initial conditions. Specifically, all ensemble members are generated using the same forecast model (FengWu), but from earlier analysis times with different lead times (e.g., 1, 2, or 3 steps), such that they all arrive at the same target time. This strategy offers a computationally efficient way to represent flow-dependent uncertainty without requiring multiple models or explicit perturbations. We have provided the detailed implementation of this scheme in Appendix C.2, and the ensemble structure is illustrated in Figure 7.\n\n> **Response to W1: Support for other autoencoder architectures**\n\nWe fully agree that our current method relies on the assumption of a Gaussian latent space, which underpins the validity of applying 3DVar in latent space. While LoRA can be incorporated into more expressive autoencoders such as VQ-VAEs, particularly those based on Transformer architectures, we have not yet developed a data assimilation framework that is compatible with vector-quantized latent representations. Unlike VAEs with continuous latent spaces, the discrete nature of VQ-VAEs poses unique challenges when integrated with variational assimilation, including the absence of gradient flow in the discrete latent space and the difficulty of formulating background error priors. We will explicitly clarify this limitation and consider this an important direction for future work in the revised version.\n\n> **Response to W2: Colors used in the plots**\n\nThank you for pointing this out. We acknowledge that some color choices in the figures may make it difficult to distinguish between different curves. In the revised version, we will improve the figure design by optimizing the color palette and line styles to enhance clarity and accessibility.\n\n> **Response to W3: Better explanation of the proposed methodology**\n\nWe appreciate your suggestion to improve the presentation of the methodology. In the revised version, we will move the description of VAE-Var (currently in Appendix A) into the main text, so that the algorithmic pipeline can be more easily understood without requiring readers to consult the appendix.\n\n___\n\nWe appreciate your feedback once again and it helped us strengthen the overall presentation and completeness of the paper."}, {"review_id": "nz5EdhJK89", "reviewer": "Reviewer_scAN", "summary": "This work addresses the problem of data assimilation by proposing a data-driven method based on the VAE-VAR architecture. A key contribution is the explicit separation of two types of model error: climatological error and flow-dependent error. These are handled differently: the former through offline training, and the latter via online fine-tuning of a small subset of network weights using the LoRA (Low-Rank Adaptation) technique. Experimental results show the effectiveness of both online adaptation in general, and the LoRA-based method in particular, compared to other fine-tuning approaches.\n\nIf I understood correctly, the core components of the proposed method are VAE-VAR and LoRA. The main novelty compared to existing machine learning-based approaches lies in the ability to adapt the prior model online through LoRA.", "questions": "1. How does the proposed model compare with other architectures such as DiffDA? In line 45, the authors state: “but they do not incorporate climatological priors or flow-dependent ensembles.” Could the authors elaborate on this distinction and explain this mathematically, and how it translate into architectural or algorithmic differences?\n\n2. The training procedure uses the NMC method (mentioned in line 59 and described in the Appendix). Could the authors clarify how training error samples are constructed using this method, and how these samples capture 'climatological' rather than 'flow-dependent' error?\n\n3. Could the authors discuss the computational cost of their method? For instance:\n   - Does the VAE structure reduce cost by enabling assimilation in latent space?\n   - How does the LoRA-based online tuning compare in efficiency to other fine-tuning strategies?", "limitations": "Yes", "rating": 4, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Overall, the paper is clearly written and the idea is original. However, I found some clarity issues and a general lack of context.\n\n### Strengths\n\n- Data assimilation is a crucial challenge, and leveraging deep learning has the potential to significantly improve weather and climate forecasting.\n- The paper is clearly written and easy to follow.\n- The proposed technique allows low-cost parameter adaptation.\n- The idea of adapting neural network parameters online is novel in this domain and appears both intuitive and powerful.\n- The experimental results are promising and support the main claims.\n\n### Weaknesses\n\n1. The paper focuses exclusively on spatial data assimilation, where observations are assimilated once per time step. There is no mention of 4D-Var, which is the state-of-the-art approach and would be a natural point of comparison.\n\n2. The distinction between climatological and flow-dependent error is not clearly defined. From my understanding, both arise from model forecast errors but differ in their time scales. Since this conceptual separation underpins the method, it would be valuable to define both types of error mathematically, for example, as expectations over model trajectories, and to clarify how they differ in structure and treatment.\n\n3. The paper lacks technical depth in describing its main components, VAE-VAR and LoRA. While the plain English explanations are helpful, the mathematical formulation is minimal. VAE-VAR is central to the model, yet it is only briefly described, with most details deferred to Appendix A. Similarly, LoRA is treated as well-known, which may not be the case in the scientific machine learning community. The problem of efficiently adapting a parametric model that defines a distribution, given samples from that distribution, is nontrivial and deserves formal treatment. This could benefit from connections to frameworks like online learning, adaptive neural networks, and meta-learning. While much of the existing literature in these areas focuses on prediction and forecasting tasks, rather than on estimating prior distributions, there are strong conceptual overlaps regarding online model adaptation. In particular, recent works on low-rank meta-learning architectures for physical systems [1, 2] may offer useful insights and parallels.\n\n4. Relatedly, the comparison to existing machine learning-based data assimilation methods (e.g., DiffDA) is somewhat superficial. A clearer description of the proposed architecture and its treatment of model error would enable more precise comparisons to alternative approaches. See also my Question 1.\n\n5. In the paragraph beginning at line 103, no references are provided for traditional hybrid EnVar approaches. A citation would strengthen the context and situate the contribution within established literature.\n\n### References\n\n[1] Serrano, L., Koupaï, A. K., Wang, T. X., ERBACHER, P., & Gallinari, P. Zebra: In-Context and Generative Pretraining for Solving Parametric PDEs.\n\n[2] Blanke, M., & Lelarge, M. Interpretable Meta-Learning of Physical Systems. In The Twelfth International Conference on Learning Representations.", "was_revised": true, "final_justification": "The idea of an adaptive neural network for data assimilation is neat and original. While the rebuttal clarified most the clarity issues, the paper still lacks a proper mathematical formulation for the tackled problem. Therefore I have increased my score from 3 to 4.", "rebuttal": "Thank you very much for your constructive and detailed comments. We greatly appreciate your positive assessment of the novelty and overall presentation of our work.\n\n___\n\n> **Response to Q1&W4: comparison with other architectures such as DiffDA**\n\nBy stating that \"they do not incorporate climatological priors or flow-dependent ensembles,\" we mean that SDA does neither, while DiffDA incorporates a learned climatological prior but does not leverage flow-dependent ensembles.\n\nMore formally, data assimilation aims to estimate the posterior $p(x^i \\mid x_b^i, y^i)$, given the prior state $x_b^i$ and observations $y^i$ at time step $i$. One common approximation is to replace the time-specific distribution with a time-averaged conditional prior $p(x \\mid x_b)$, which encodes climatological structure. DiffDA follows this approach and uses a diffusion model to learn a fixed mapping from $x_b$ to $x$, approximating $p(x \\mid x_b)$. However, this mapping is static and cannot reflect the time-varying nature of $p(x^i \\mid x_b^i)$ under dynamically evolving conditions.\n\nIn contrast, VAE-Var, inspired by traditional variational data assimilation methods, assumes the forecast error $e^i = x^i - x_b^i$ is independent of $x_b^i$ and instead learns a time-averaged climatological error distribution $p(e)$. LoRA-EnVar builds upon this by incorporating flow-dependent adjustments to the error distribution. Specifically, it constructs online error samples using time-lagged forecasts and encodes them through low-rank updates, allowing the model to approximate time-varying distributions $p_i(e^i)$ and better capture regime-dependent uncertainties (e.g., stronger jet activity in winter).\n\nImportantly, this modeling choice has architectural and practical implications. DiffDA requires paired data $(x_b^i, x^i)$ to train its diffusion model, which can be difficult or even infeasible to obtain in online or operational settings. In contrast, LoRA-EnVar only requires access to error samples $e^i$, which can be easily estimated from ensembles using established methods such as time-lagging. Therefore, LoRA-EnVar offers greater flexibility and is better suited for real-time deployment.\n\n> **Response to Q2: How training error samples are constructed using the NMC method**\n\nTo construct training error samples using the NMC method, we first select historical ERA5 reanalysis states at 12-hour intervals. For each selected state, we run the forecast model to generate two forecasts for the same target time: one with a 12-hour lead time and another with a 24-hour lead time. The difference between these two forecasts forms an error sample. This process is repeated over a 37-year span of ERA5 data, resulting in a large and diverse training set. Because the errors are aggregated over many years and meteorological regimes, the learned distribution reflects averaged, climatological characteristics rather than case-specific, flow-dependent variations. In contrast, the samples used during finetuning are constructed online from recent ensemble forecasts and are therefore sensitive to current atmospheric conditions. These are referred to as “flow-dependent” errors in our framework.\n\n> **Response to Q3: Discussion on the computational cost**\n\nThank you for the question.\n\n- The primary benefit of VAE-Var over traditional 3DVar is its ability to capture non-Gaussian error structures via a learned latent representation. While 3DVar applies a linear transformation to a Gaussian control variable, VAE-Var maps a latent variable to physical space using a neural network decoder. As such, the computational complexity of VAE-Var is not lower; in fact, it involves more floating-point operations. However, VAE-Var, implemented with optimized neural network operators, is highly compute-efficient on GPUs. In our experiments, one assimilation cycle using VAE-Var takes approximately 18 seconds on a single NVIDIA A100 GPU, compared to about 4 minutes per cycle using a CPU-based 3DVar implementation running on 16 cores. \n- Although LoRA updates only a small subset of parameters, the backward pass still traverses the entire network to compute gradients. In practice, LoRA reduces the fine-tuning time modestly, from ~10s per cycle in full fine-tuning to ~9s with LoRA. Nevertheless, The key advantage of LoRA lies not in time savings but in memory efficiency and modularity. Because LoRA modifies only a few low-rank matrices, the adapted model can be stored and transferred compactly, which is especially valuable when tracking adaptations over time or switching between flow regimes. Moreover, LoRA enables efficient online updates without altering the backbone, simplifing deployment and mitigating catastrophic forgetting.\n\n> **Response to W1: Comparison with 4DVar**\n\nTo evaluate our framework under 4DVar settings, we constructed an extended observation scenario where, in addition to the standard 6-hourly observations (e.g., at 0h, 6h, 12h), we introduce intermediate observations at 1h, 7h, 13h, etc. We employ a 1-hour resolution version of the FengWu forecast model to resolve temporal dynamics within each 6-hour assimilation window.\n\nWe compared eight methods and the results over the first 30 days of January are summarized in the tables below. As expected, all 4D variants outperform their 3D counterparts, reflecting the benefit of temporal assimilation. Importantly, LoRA-4DEnVar consistently outperforms 4DVar and Hybrid 4DEnVar, demonstrating both the extensibility and effectiveness of our framework in fully 4D assimilation settings.\n\n|z500 RMSE ($\\mathrm{m^2}/\\mathrm{s^2}$)|3DVar|Hybrid 3DEnVar|4DVar|Hybrid 4DEnVar|VAE-3DVar|LoRA-3DEnVar|VAE-4DVar|LoRA-4DEnVar|\n|-|-|-|-|-|-|-|-|-|\n|Day 1-10|80.434|80.937|70.331|72.894|66.724|60.334|60.609|54.200|\n|Day 11-20|86.261|88.631|74.870|78.942|77.401|64.602|71.187|59.024|\n|Day 21-30|85.746|84.153|72.278|76.135|85.516|67.135|73.638|61.156|\n|Average|84.147|83.820|72.493|75.990|76.547|64.024|68.478|58.127|\n\n|u500 RMSE ($\\mathrm{m}/\\mathrm{s}$)|3DVar|Hybrid 3DEnVar|4DVar|Hybrid 4DEnVar|VAE-3DVar|LoRA-3DEnVar|VAE-4DVar|LoRA-4DEnVar|\n|-|-|-|-|-|-|-|-|-|\n|Day 1-10|2.761|2.611|2.725|2.565|2.595|2.539|2.445|2.440|\n|Day 11-20|2.918|2.730|2.879|2.680|2.729|2.649|2.536|2.533|\n|Day 21-30|2.923|2.691|2.826|2.638|2.743|2.670|2.592|2.582|\n|Average|2.867|2.653|2.810|2.628|2.689|2.619|2.524|2.519| \n\n> **Response to W2: Mathematical definition of two types of error**\n\nWe thank the reviewer for the valuable suggestion. To clarify the conceptual distinction, we provide a more precise mathematical formulation. We denote the forecast error at time step $i$ as $e^i = x^i - x_b^i$, where $x^i$ is the true state and $x_b^i$ is the background forecast. Then:\n\n- The **flow-dependent error distribution** refers to the conditional error distribution at a particular time $i$, given the background flow conditions $\\mathcal{F}_i$ (e.g., the synoptic-scale state of the atmosphere): $p_i(e) := p(e^i \\mid \\mathcal{F}_i)$. This formulation acknowledges that the forecast error structure depends on the current dynamical regime. For instance, during periods with strong jet streams or tropical cyclones, the forecast uncertainty tends to be larger and more anisotropic.\n- The **climatological error distribution** is defined as the time-averaged distribution of forecast errors: $p_{\\text{clim}}(e) := \\mathbb{E}_i \\, p_i(e)$. This captures the average statistical structure of forecast errors across a long historical period and is often used in traditional climatology-based assimilation schemes.\n\nWhile both types of error indeed arise from model forecast errors, we respectfully note that the distinction is not solely based on time scales. The key difference, particularly in the context of ensemble-based assimilation, lies in whether the error statistics are conditioned on the current flow state. Flow-dependent errors reflect state-specific uncertainties, while climatological errors represent the marginal distribution averaged over many flow regimes. \n\n> **Response to W3: Scope to strengthen mathematical formulation and conceptual links**\n\nThank you for the thoughtful suggestion. While many recent works in the SciML community focus on general-purpose physical surrogates or PDE solvers, our work centers on improving probabilistic data assimilation in a large-scale, operational setting—namely, numerical weather prediction. Our adoption of LoRA is driven not only by considerations of efficiency, but also by the need for spatially localized, flow-dependent adaptation within a fixed VAE backbone. We agree that connecting our approach more clearly to the broader frameworks of online learning and meta-learning would enhance the conceptual framing of the paper. In particular, we appreciate the pointer to recent work on low-rank meta-learning architectures for physical systems, which indeed parallels our use of LoRA as a mechanism for cycle-wise adaptation. Although our primary motivation is practical—capturing flow-dependent uncertainty efficiently in high-dimensional systems—we recognize that the underlying problem structure closely aligns with meta-adaptation paradigms, where compact parameter updates enable generalization across evolving regimes. We will revise the manuscript to clarify these connections, formalize the VAE and LoRA components more precisely, and cite the suggested references accordingly.\n\n> **Response to W5: Missing Hybrid EnVar citation**\n\nThank you for pointing this out. The implementation we adopt for Hybrid 3DEnVar follows the formulation described in [1]. We will include this citation in the revised version.\n\n___\n\nWe appreciate your feedback once again and it helped us strengthen the overall presentation and completeness of the paper.\n\n[1] Wang, X., Snyder, C., & Hamill, T. M. (2007). On the theoretical equivalence of differently proposed ensemble–3DVAR hybrid analysis schemes. *Monthly Weather Review*, *135*(1), 222-227."}, {"review_id": "LQzLeCL1dH", "reviewer": "Reviewer_pe2M", "summary": "The authors develop a new variational assimilation method to support numerical weather prediction. First, a VAE is trained on historical forecast errors to serve as a baseline. Then, LoRA modules are inserted into the decoder. These LoRA modules are tuned at each time point using uncertainty information from AI-based ensemble forecasts. The resulting framework is intended to efficiently incorporate new information while keeping the historical background of the model.", "questions": "The space/time summaries of forecast errors seem to favor your method, does this still hold up when reduced to a single spatial/temporal mean? Appears to be the case but it’s not always clear to me (especially in parts of Figure 3).", "limitations": "Yes.", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 4, "clarity": 4, "significance": 4, "originality": 3, "strengths_and_weaknesses": "Strengths: The online training of LoRA modules in the decoder serves a double purpose to vastly improve efficiency over full fine tuning while preserving the decoder’s original parameters. Empirical results show strong improvement over previous approaches in a vitally important application.\n\nWeaknesses: While the included spatial and temporal summaries are of greater interest and importance, it may also be interesting to see overall accuracy metrics to provide full context (perhaps just for your main results). While the incorporation of LoRA modules is novel, the overall VAE approach to data assimilation is familiar.", "was_revised": true, "final_justification": "This work adds the novel contribution of LoRA modules to the VAE-VAR architecture for climate data assimilation. A comprehensive spatiotemporal evaluation shows that the model offers a clear benefit for an important application area.", "rebuttal": "Thank you very much for your thoughtful and encouraging review. We are glad that you found the proposed approach and its evaluation to be of high quality and significance.\n\n___\n\n> **Response to Q1&W1: Spatial/temporal mean for overall evaluation** \n\nIn response to your helpful suggestion, we have included overall accuracy metrics in the table below, where each entry represents the spatial-temporal mean RMSE for a key variable (corresponding to Figure 3&4 in the main text and Figure 8&9 in the Appendix). **Bold** values indicate the best performance, and *italic* values indicate the second-best. As shown, LoRA-EnVar consistently ranks either first or second, closely matching or outperforming both Full Finetune (w/ reset) and Hybrid 3DEnVar, depending on the variable. This further confirms that the improvements observed in the space/time summaries also hold under global scalar evaluation.\n\n| Overall RMSE                            | 3DVar   | Hybrid 3DEnVar      | LoRA-EnVar          | VAE-Var (no finetune) | Full finetune (w/o reset) | Full finetune (w/ reset) |\n| --------------------------------------- | ------- | ------------------- | ------------------- | --------------------- | ------------------------- | ------------------------ |\n| z500 ($\\mathrm{m}^2/\\mathrm{s}^2$)      | $98.17$ | $84.79$             | $\\underline{64.11}$ | $76.75$               | $94.93$                   | $\\mathbf{61.68}$         |\n| z850 ($\\mathrm{m}^2/\\mathrm{s}^2$)      | $58.23$ | $54.92$             | $\\underline{53.58}$ | $55.10$               | $67.98$                   | $\\mathbf{53.56}$         |\n| u500 ($\\mathrm{m}/\\mathrm{s}$)          | $2.811$ | $\\underline{2.681}$ | $\\mathbf{2.623}$    | $2.691$               | $2.754$                   | $2.708$                  |\n| u850 ($\\mathrm{m}/\\mathrm{s}$)          | $2.272$ | $2.103$             | $\\mathbf{1.971}$    | $2.011$               | $2.037$                   | $\\underline{1.972}$      |\n| v500 ($\\mathrm{m}/\\mathrm{s}$)          | $2.786$ | $2.642$             | $\\mathbf{2.523}$    | $2.592$               | $2.622$                   | $\\underline{2.549}$      |\n| v850 ($\\mathrm{m}/\\mathrm{s}$)          | $2.247$ | $2.080$             | $\\mathbf{1.908}$    | $1.948$               | $1.959$                   | $\\mathbf{1.908}$         |\n| t500 ($\\mathrm{K}$)                     | $0.757$ | $\\mathbf{0.728}$    | $\\underline{0.755}$ | $0.771$               | $0.793$                   | $0.776$                  |\n| t850 ($\\mathrm{K}$)                     | $1.136$ | $\\underline{1.075}$ | $\\underline{1.075}$ | $1.101$               | $1.103$                   | $\\mathbf{1.064}$         |\n| u10 ($\\mathrm{m}/\\mathrm{s}$)           | $1.539$ | $1.449$             | $\\underline{1.275}$ | $1.308$               | $1.332$                   | $\\mathbf{1.268}$         |\n| v10 ($\\mathrm{m}/\\mathrm{s}$)           | $1.593$ | $1.507$             | $\\mathbf{1.314}$    | $1.347$               | $1.377$                   | $\\underline{1.315}$      |\n| t2m ($\\mathrm{K}$)                      | $1.499$ | $1.347$             | $\\mathbf{1.179}$    | $1.194$               | $1.468$                   | $\\underline{1.269}$      |\n| q700 ($10^{-3}\\mathrm{kg}/\\mathrm{kg}$) | $1.146$ | $\\mathbf{1.129}$    | $\\underline{1.136}$ | $1.208$               | $1.166$                   | $1.152$                  |\n\n> **Response to W1: Novelty of the VAE approach** \n\nRegarding your note on the familiarity of the VAE-based approach, we fully agree that the use of VAEs in data assimilation is established. Our contribution focuses on incorporating LoRA modules for efficient online adaptation, which we believe adds a novel and practically valuable enhancement to this VAE-Var framework.\n\n___\n\n\nWe appreciate your feedback once again and it helped us strengthen the overall presentation and completeness of the paper."}]

Author rebuttal:

Meta-review: The average rating is 4.6, with almost all reviewers recommending acceptance (4, 5, 6, 5, 3). The discussion has been constructive and improvements have been made to the paper, to the satisfaction of the reviewers. Reviewer 2Ywj (3) has raised concerns about the realism of the observational model considered in the experiments and the choice of baselines. Given the overall positive reviews, I recommend acceptance. However, I strongly encourage the authors to follow Reviewer 2Ywj's suggestions and discuss the realism or artificiality of the experimental setup and adjust claims accordingly. New results presented during the discussion are helpful and should be included in the final version of the paper.

Recommendation: acceptance.
Decision: Accept (poster)