# Deep Data Assimilation: Integrating Deep Learning with Data Assimilation
## 2 Data Assimilation Process  
Let  
\[\dot{u}=\mathcal{M}(u,t,\theta) \tag{1}\]  
be a forecasting model, where \(u\) denotes the state, \(\mathcal{M}\) is a nonlinear map, \(\theta\) is a parameter of interest, and \(t\in[0,T]\) denotes the time. Let  
\[v=\mathcal{H}(u)+\epsilon \tag{2}\]  
be an observation of the state \(u\), where \(\mathcal{H}\) is an observation function and \(\epsilon\) denotes the measurement error. Data Assimilation is concerned with how to use the observations in (2) and the model in (1) in order to obtain the best possible knowledge of the system as a function of time.The Bayes theorem conducts the estimation of a function \(u^{DA}\), which maximizes a probability density function, given the observation \(v\) and a prior from \(u\)[1,35]. For a fixed a time step \(t_{k}\in[0,T]\), let \(u_{k}\) denote the estimated system state at time \(t_{k}\):  
\[u_{k}=M\,u_{k-1} \tag{3}\]  
where the operator \(M\) represents a discretization of a first order approximation of \(\mathcal{M}\) in (1). Let \(v_{k}\) be an observation of the state at time \(t_{k}\) and let  
\[H:u_{k}\mapsto v_{k}. \tag{4}\]  
be the discretization of a first order approximation of the linearization of \(\mathcal{H}\) in (2). The DA process consists in finding \(u^{DA}\) (called analysis) as an optimal tradeoff between the prediction made based on the estimated system state (called background) and the available observation. The state \(u\) is then replaced by the function \(u^{DA}\), which includes the information from the observation \(v\) in a prediction-correction cycle that is shown in Figure 1.  
The analysis \(u^{DA}\) is computed as inverse solution of  
\[v_{k}=Hu^{DA}+e_{R_{k}}, \tag{5}\]  
subject to the constraint that  
\[u^{DA}=u_{k}+e_{B_{k}}. \tag{6}\]  
where \(e_{R_{k}}=\mathcal{N}(u_{R},R_{k})\) and \(e_{B_{k}}=\mathcal{N}(u_{B},B_{k})\) denote the observation error and model error, respectively, \(\mathcal{N}(\cdot)\) denotes Gaussian distribution [1], and \(e_{R_{k}}\) includes the measurement error \(\epsilon\) introduced in (2). Both of the errors include the discretization, approximation and the other numerical errors. Because \(H\) is typically rank deficient, the (5) is an ill posed inverse problem [36; 37]. The Tikhonov formulation [38] leads to an unconstrained least square problem, where the term in (6) ensures the existence of a unique solution of the (5). The DA process can be then described in Algorithm 1, where:  
\[u^{DA}=argmin_{u}\bigg{\{}\|u-u_{k}\|_{B_{k}^{-1}}^{2}+\|v_{k}-Hu\|_{R_{k}^{-1 }}^{2}\bigg{\}} \tag{7}\]  
where \(R_{k}\) and \(B_{k}\) are the observation and model error covariance matrices, respectively:  
\[R_{k}:=\sigma_{0}^{2}I, \tag{8}\]  
with \(0\leq\sigma_{0}^{2}\leq 1\) and \(I\) be the identity matrix,  
\[B_{k}=V_{k}V_{k}^{T} \tag{9}\]  
with \(V_{k}\) background error deviation matrix [13] being computed by a set of \(n_{k}\) historical data \(S=\{u_{j}\}_{j=1,\ldots,n_{k}}\) available at time \(t_{k}\) and such that  
\[v=\mathcal{H}(u)+\epsilon \tag{2}\]0  
where  
\[v=\mathcal{H}(u)+\epsilon \tag{2}\]1  
and  
\[v=\mathcal{H}(u)+\epsilon \tag{2}\]2  
\[v=\mathcal{H}(u)+\epsilon \tag{2}\]3 The prediction-correction cycle. The DA process, as defined in (7), can be solved by several methods. Two main approaches that have gained acceptance as powerful methods for data assimilation on the last years are the variational approach and Kalman Filter (KF). The variational approach [39] is based on the minimization of a functional, which estimates the discrepancy between numerical results and measures. The Kalman Filter [40] is a recursive filtering instead. In both cases we seek an optimal solution. Statistically, KF seeks a solution with minimum variance. Variational methods seek a solution that minimizes a suitable cost function. In certain cases, the two approaches are identical and they provide exactly the same solution [1].