## 4 Approach
### DRL-Based Formulation for the VarDA System
* **Action \(a_{t}\)** is a spatially adaptive scaling vector (\(\mathbf{W}=[w_{1},w_{2},\ldots,w_{CK}]\)), where each element corresponds to a rescaling factor applied within a specific spatial domain. These factors are selected from a bounded interval \((l,u)\) to ensure computational efficiency. Here, \(l\) and \(u\) denote the shared lower and upper bounds of the scaling vector, where each \(w_{ck}\in(l,u)\) corresponds to a spatial chunk and modulates the background error variances within that region. * **Transition \(s_{t+1}\)** is obtained as the numerical solution that minimizes the cost function given in Equation (1) in our formulation (see Algorithm 1): \[s_{t+1}=\mathcal{M}_{t\to t+1}(s_{t})+\mathbf{B}_{\mathbf{W}}\mathbf{H}^{T}( \mathbf{H}\mathbf{B}_{\mathbf{W}}\mathbf{H}^{T}+\mathbf{R})^{-1}(\mathbf{y}^ {o}-\mathcal{H}(\mathcal{M}_{t\to t+1}(s_{t}))),\] (16) where \(\mathcal{M}_{t\to t+1}\) denotes the outcomes of the Lorenz-96 model when it runs from \(t\) to \(t+1\). * **Reward \(r(s_{t},a_{t},s_{t+1})\)** quantifies the immediate benefit of applying action \(a_{t}\) at state \(s_{t}\), resulting in the new state \(s_{t+1}\). This study aims to learn an adaptive variance rescaling policy that improves the quality of assimilation, thereby providing initial conditions that enhance the subsequent forecast performance. Since the effectiveness of data assimilation depends on multiple factor, the detailed formulation of \(r\) is provided in subsequent content. * **Discount factor \(\gamma\)** is set close to 1, as our task requires long-term planning over sequential assimilation cycles. **Setting of the Reward Function:** as we have mentioned previously, the purpose of the DA system is to provide the optimal initial state for numerical weather models, thereby improving the accuracy of NWP. Therefore, the indicator for measuring the optimal initial state is the value that directly describes the performance of the DA system, while the indicator for measuring the accuracy of NWP is the value that assists in describing the quality of the result of the DA system, both of which are important components of the reward function. To evaluate the performance of the DA system, the root mean square error (RMSE) between \(\mathbf{x}^{a}\) and \(\mathbf{x}^{t}\) is a general evaluation metric, where \(\mathbf{x}^{t}\) can be calculated by integrating the numerical model, which is considered perfect and without model errors in our task. Thus, the following function represents the first reward:  
\[r_{a}=RMSE_{a}=\sqrt{\frac{1}{JT}\sum_{i=0}^{T}\sum_{j=1}^{J}\Bigl{(}\mathbf{ x}_{ij}^{t}-\mathbf{x}_{ij}^{a}\Bigr{)}^{2}} \tag{17}\]  
where \(J\) is the number of model state variables, \(T\) is the total time of the DA system, and \(x_{ij}\) represents the \(j\)th state variable in the \(i\)th timestep. To evaluate the accuracy of the NWP forecast, the RMSE between \(\mathbf{x}^{f}\) (the forecast state predicted by the numerical model) and \(\mathbf{x}^{t}\) is a typical index:  
\[r_{f}=RMSE_{f}=\sqrt{\frac{1}{JT}\sum_{i=0}^{T}\sum_{j=1}^{J}\left(\left( \mathbf{x}^{t}_{j}\right)_{i}-\left(\mathbf{x}^{f}_{j}\right)_{i}\right)^{2}} \tag{18}\]  
Overall, the reward function \(R\) is defined as follows:  
\[R=-r_{a}-r_{f} \tag{19}\]  
```
1:Transition Model, consisting of: \(\mathbf{y}^{o}\), \(\mathbf{R}\), \(\mathbf{H}\), \(\mathcal{H}\), \(\mathbf{B}_{\text{NMC}}\) (1), number of assimilation cycles \(N_{\text{assim}}\), integration timestep \(dt\), number of spatial chunks \(CK\), Total time \(T\). 2:Current State \(s_{t}\), consisting of: \(\mathbf{x}^{t}_{t}\) based on the Lorenz-96 model. 3:Action \(a_{t}\), representing a spatio-temporal adaptive scaling factor vector \(\mathbf{W}\). 4:Output: State transition from \(s_{t}\) to \(s_{t+1}\) via the VarDA W-ST environment.