# Data-Completion and Model Correction by Means of Evanescent Regularization
## 3 A Hybrid Modeling Approach  
### A Single Unmeasurable Quantity  
When it comes to hybrid modeling, the approach is similar to the previously proposed one; using the same methods described in Section 2.1. However, this approach assumes the existence of another model \(\mathbf{C}\) existing in parallel to the simulation, and integrating the results simultaneously. For instance, in the linear case for example, the hybrid model solution \(\mathbf{y}_{i+1}^{h}\) at any time steps can be written as:  
\[\left\{ \begin{array}{l}\mathbf{y}_{i+1}^{e}=\mathbf{y}_{i}^{e}+\Delta t\; \mathbf{A}\;\mathbf{y}_{i}^{e}\\ \mathbf{y}_{i+1}^{e}=\mathbf{y}_{i}^{e}+\Delta t\;\mathbf{B}\;\mathbf{y}_{i}^ {e}\\ \mathbf{y}_{i+1}^{e}=\mathbf{y}_{i}^{e}+\Delta t\;\mathbf{B}\;\mathbf{y}_{i}^ {h}+\Delta t\;\mathbf{C}\;\mathbf{y}_{i}^{h}\end{array} \right. \tag{13}\]  
Figure 2: Final output of the trained neural network for both the measurable and unmeasurable outputs. In the non-linear situation, the hybrid model would be written as:  
\[\frac{d\mathbf{y}^{h}}{dt}=\mathbf{h}(t,p,\mathbf{y}^{h})+\mathbf{g}(t,p,\mathbf{ y}^{h}) \tag{14}\]  
where \(\mathbf{g}\) is the correction term and \(\mathbf{h}\) is the initial simulation model assumed non-trainable in this case. The loss function is changed in the hybrid modeling to include the \(L_{2}\) norm regularization with respect to the hyperparameters of the trainable model \(\mathbf{g}(t,p,\mathbf{y})\), because it is expected that the correction is not large with respect to \(\mathbf{B}\). For instance, the new loss function \(\mathcal{J}\) is written in the linear case by:  
\[ \begin{split}\mathcal{J}&=\sum\limits_{i=1}^{n_{t}} \left(\sum\limits_{j=1}^{N_{m}}\left(y_{i}^{e,j}-y_{i}^{h,j}\right)^{2}\right)+ \lambda_{1}\big{(}C^{m,m}-\hat{C}^{m,m}\big{)}^{2}+\lambda_{2}\big{(}C^{m,nm}- \hat{C}^{m,nm}\big{)}^{2}+\\ &\quad+\lambda_{3}\left(\mathbf{y}_{0}^{h,nm}-\hat{\mathbf{y}}_{0 }^{h,nm}\right)^{2}+\lambda_{4}\sum\limits_{i=1}^{N}\sum\limits_{j=1}^{N} \big{(}\hat{\mathbb{C}}_{ij}\big{)}^{2}+\lambda_{5}\big{(}\|C^{m,nm}\|-\|\hat{ C}^{m,m}\|\big{)}^{2}\end{split}  \tag{15}\]  
where \(\hat{\mathbb{C}}_{ij}\) includes all of the components of \(\mathbf{C}\), and enforce the sparsity in the correction, even if a stronger enforcement could be applied by using the \(L_{1}\) norm. The new loss function can be generalized for the non-linear case using:  
\[ \begin{split}\mathcal{J}&=\sum\limits_{i=1}^{n_{t}} \left(\sum\limits_{j=1}^{N_{m}}\left(y_{i}^{e,j}-y_{i}^{h,j}\right)^{2}\right) +\lambda_{1}\big{(}\boldsymbol{\theta}^{m,m}-\hat{\boldsymbol{\theta}}^{m,m} \big{)}^{2}+\lambda_{2}\big{(}\boldsymbol{\theta}^{m,nm}-\hat{\boldsymbol{ \theta}}^{m,nm}\big{)}^{2}+\\ &\quad+\lambda_{3}\left(\mathbf{y}_{0}^{h,nm}-\hat{\mathbf{y}}_{0 }^{h,nm}\right)^{2}+\lambda_{4}\sum\limits_{k=1}^{N_{q}}\left(\hat{\boldsymbol {\theta}}_{k}\right)^{2}+\lambda_{5}\big{(}\|\boldsymbol{\theta}^{m,nm}\|-\| \hat{\boldsymbol{\theta}}^{m,m}\|\big{)}^{2},\end{split}  \tag{16}\]  
with \(\boldsymbol{\theta}\) being the trainable parameters of the non-linear model \(\mathbf{g}\), having a total number of \(N_{\theta}\) trainable parameters. The exact error matrix \(\mathbf{C}\) used in this example is shown in Appendix A. The last term multiplying \(\lambda_{5}\) couples the amplitudes of the equations computing the measurable part and the non-measurable part, to reduce the possibility of having a scale bias. The optimization problem is now written as:  
\[ \begin{split}\left(\boldsymbol{\theta},\mathbf{y}_{0}^{h,nm} \right)=&\underset{\mathbf{g}^{h,nm}\in\mathbf{N}^{2}}{arg\min} \quad\{\mathcal{J}\},\end{split}  \tag{17}\]  
where \(\boldsymbol{\theta}\) are the hyperparameters to optimize. The training is performed on a standard laptop within few minutes, even when starting from a very wrong initial condition \(\mathbf{y}_{0}^{h,nm}\).