# Data-Completion and Model Correction by Means of Evanescent Regularization
## 2 Correction of the Available Model
### Methods
#### 2.1.3 Time Integration and Adjoint-Free Neural ODE  
The use of neural Ordinary Differential Equation (ODE), with the adjoint method, involves the integration of another differential equation, backward in time, to compute the gradients. This integration can involve very low time step requirements and increase dramatically the computation time. Therefore, in this work, this issue is alleviated through proposing an adjoint-free neural ODE. This adjoint-free neural ODE uses a neural network that estimates the derivatives of the quantities of interest \(\mathbf{y}\), while computing analytically, through automatic differentiation, their gradients with respect to the neural network training variables. In this case, the matrix \(\mathbf{B}\) appearing in Equation (9) is interpreted as the weights of a single layer dense neural network, with no bias. To illustrate the adjoint-free neural ODE, the derivative of the quantities of interest is written as:  
\[\frac{d\mathbf{y}}{dt}=\mathbf{h}(t,\mathbf{y};p), \tag{10}\]  
with \(h(t,\mathbf{y};p)\) is a neural network and \(p\) are any input parameters. The feed-forward evaluation is possible though explicit integration using:  
\[ \begin{gathered}\mathbf{y}_{i+1}=\mathbf{y}_{i}+\Delta\mathbf{h} (t_{i},p_{i},\mathbf{y}_{i})\\ \mathbf{y}_{i+2}=\mathbf{y}_{i+1}+\Delta\mathbf{h}(t_{i+1},p_{i+1 },\mathbf{y}_{i+1})=\\ =\mathbf{y}_{i}+\Delta\mathbf{h}(t_{i},p_{i},\mathbf{y}_{i})+ \Delta\mathbf{h}(t_{i+1},p_{i+1},\mathbf{y}_{i}+\Delta\mathbf{h}(t_{i},p_{i}, \mathbf{y}_{i}))\\ \vdots\end{gathered}  \tag{11}\]  
The derivation of \(\mathbf{y}_{i+n_{t}}\) with respect to the trainable parameters of \(\mathbf{h}\), \(\boldsymbol{\theta}\) for instance, can be performed through a straightforward manner by the means of the chain rule, and using the property \((f\circ g(\boldsymbol{\theta}))^{\prime}=f^{\prime}(g(\boldsymbol{\theta})) \cdot g^{\prime}(\boldsymbol{\theta})\). Thus, if \(\boldsymbol{\theta}\) is the vector of the neural network trainable variables, one can write \((\mathbf{h}\circ\mathbf{h}(\boldsymbol{\theta}))^{\prime}=\mathbf{h}^{\prime }(\mathbf{h}(\boldsymbol{\theta}))\cdot\mathbf{h}^{\prime}(\boldsymbol{\theta})\). The gradients of \(\mathbf{y}_{i+n_{t}}\) are therefore easily computed, using the chain rule, while \(\mathbf{h}^{\prime}(\boldsymbol{\theta})\) is computed through automatic differentiation. The neural ODE can therefore compute several values for \(\mathbf{y}_{i+n_{t}}\), to be used in the loss function and the optimization problem defined in Equation (9), which can be re-written in its general form:  
\[ \begin{gathered}\left(\boldsymbol{\theta},\mathbf{y}_{0}^{s,nm} \right)=\begin{array}{c}\underset{\boldsymbol{\theta}\in\mathbf{R}^{N^{2}} }{\arg\min}\ \ \{\mathcal{L}\},\end{gathered}  \tag{12}\]  
\(\boldsymbol{\theta}\) being the model hyperparameters to optimize, which are the values of the matrix \(\mathbf{B}\) of dimensionality \(N^{2}\) in our selected example (\(\boldsymbol{\theta}\equiv\mathbf{B}\) in our example). The final algorithm for updating the simulation model and training the model weights is illustrated in Algorithm 1. The ADAM stochastic optimization algorithm [15] is used for the gradient descent in our examples. ```
Initialize \(\boldsymbol{\theta}\) as \(\hat{\boldsymbol{\theta}},\mathbf{y}_{0}^{s,nm}\) as \(\hat{\mathbf{y}}_{0}^{s,nm}\) while\(\mathcal{L}\geq\epsilon\)do while\(i<n_{t}\)do  Compute \(\mathbf{y}_{i}\) using adjoint-free Neural ODE integration, Equation (11) endwhile  Compute \(\mathcal{L}\)  Evaluate \(\frac{\partial\mathcal{L}}{\partial\boldsymbol{\theta}}\) and \(\frac{\partial\mathcal{L}}{\partial\mathbf{y}_{0}^{s,nm}}\) \(\left( \begin{array}{c}\boldsymbol{\theta}\\ \mathbf{y}_{0}^{s,nm}\end{array} \right)\gets ADAM\Bigg{(}\left( \begin{array} []{c}\boldsymbol{\theta}\\ \mathbf{y}_{0}^{s,nm}\end{array} \right);\left( \begin{array}{c}\frac{\partial \mathcal{L}}{\partial\boldsymbol{\theta}}\\ \frac{\partial\mathcal{L}}{\partial\boldsymbol{\theta}}\end{array}