# A Novel Reconstruction Method for Irregularly Sampled Observation Sequences for Digital Twin
## 3 Irregularly Sampled Observation Sequence Reconstruction Method Based on PRN-NCDE
### NCDE-Based Decoder  
The decoder predicts the desired regular data sequence from the fused initial latent feature \(\mathbf{x}_{0}^{fus}\). Generally, most real systems are controlled, e.g., the coal-fired power plants. However, the prediction results of the NODE method depend only on the initial hidden state without considering the impact of control inputs on state changes. The NCDE proposed by [PERSON] et al. [28] takes the impact of control inputs into consideration during the prediction process. The solution of the NCDE is defined as follows:  
\[ \begin{array}{ll}\mathbf{x}_{t}&=\mathbf{x}_{0}+\int_{t_{0}}^{t}f_{cd}(\mathbf{x}_{w},w; \mathbf{W}_{cd})\mathrm{d}\mathbf{U}_{w}\\ &=\mathbf{x}_{0}+\int_{t_{0}}^{t}f_{cd}(\mathbf{x}_{w},w;\mathbf{W}_{cd})\frac{\mathrm{d} \mathbf{U}_{w}(w)}{\mathrm{d}w}\mathrm{d}w\end{array}  \tag{12}\]  
where \(\mathbf{x}_{t}\in\mathbb{R}^{r}\) is the hidden state at time \(t\in(t_{0},t_{N^{\prime}}]\), \(\mathbf{U}_{w}:[t_{0},t_{N^{\prime}}]\rightarrow\mathbb{R}^{c}\) is a thrice-spline curve based on the control sequence \(\{\mathbf{u}_{n}^{\prime}\}_{n=0}^{N^{\prime}}\) over the interval \([t_{0},t_{N^{\prime}}]\), and \(f_{cd}(\mathbf{x}_{t},t;\mathbf{W}_{cd}):\mathbb{R}^{r}\rightarrow\mathbb{R}^{r\times c}\) is a neural network with learnable parameters \(\mathbf{W}_{cd}\). Consider a generative model defined by the NCDE network with the initial latent feature \(\mathbf{x}_{0}^{\prime fus}\) and desired sampling times \(\{t_{n}^{\prime}\}_{n=0}^{N^{\prime}}\). The initial latent feature \(\mathbf{x}_{0}^{\prime fus}\) follows a normal distribution given by  
\[\mathbf{x}_{0}^{\prime fus}\sim p_{e}\Big{(}\mathbf{x}_{0}^{\prime fus}\Big{)}=\mathcal{ N}\Big{(}\mathbf{x}_{0}^{\prime fus}\Big{|}\mathbf{\mu}_{0}^{\prime fus},\mathbf{\sigma}_{0}^{ \prime fus}\Big{)} \tag{13}\]  
where  
\[ \begin{array}{ll}\mathbf{\mu}_{0}^{\prime fus}=(1-Q)\cdot\mathbf{\mu}_{0}^{obs}+Q \cdot\mathbf{\mu}_{0}^{\prime ref}\\ \mathbf{\sigma}_{0}^{2,\prime fus}=(1-Q)^{2}\cdot\mathbf{\sigma}_{0}^{2,obs}+Q^{2} \cdot\mathbf{\sigma}_{0}^{2,ref}\end{array} \]  
Using Formula (12), we can compute the latent features at all desired time points. The neural network \(f_{cd}\) consists of multiple channels of multilayer feedforward networks, with the number of channels determined by the system's control dimension \(c\). Taking the boiler system as an example, since it has two control inputs (coal feed and secondary air flow), \(f_{cd}\) has two channels as follows:  
\[(\mathbf{h}_{1},\mathbf{h}_{2})=f_{cd}(\mathbf{x}_{t},t;\mathbf{W}_{cd})=\text{mlfnn}(\mathbf{x}_{ t},\mathbf{x}_{t}) \tag{14}\]  
where \(\text{mlfnn}(\bullet)\) denotes a multilayer feedforward network cascaded with fully connected layers and LeakyReLU activation functions, and \(\mathbf{h}_{1},\mathbf{h}_{2}\in\mathbb{R}^{r\times 1}\) are the outputs of the \(\text{mlfnn}(\bullet)\) for each channel. Multiplying \(\mathbf{h}_{1}\) and \(\mathbf{h}_{2}\) with their respective control derivatives and summing the results yields the latent feature of the NCDE network at the desired time \(t_{n+1}^{\prime}\) given by  
\[\left.\mathbf{x}_{t}\right|_{t=t_{n+1}^{\prime}}=\mathbf{h}_{1}\cdot\left.\frac{d\mathbf{ U}^{1}(t)}{dt}\right|_{t=t_{n}^{\prime}}+\mathbf{h}_{2}\cdot\left.\frac{d\mathbf{U}^{2}(t)}{ dt}\right|_{t=t_{n}^{\prime}} \tag{15}\]  
where \(\mathbf{U}^{1}\) and \(\mathbf{U}^{2}\) are the cubic spline curves formed by the sequences of control input 1 (coal feed) and control input 2 (secondary air flow), respectively. After obtaining the latent features at all desired time points, the output layer generates the desired observation sequence as follows:  
\[(\mathbf{y}_{0}^{\prime},\cdots,\mathbf{y}_{N^{\prime}}^{\prime})=\text{fnn}((\mathbf{x}_ {0},\cdots,\mathbf{x}_{N^{\prime}})) \tag{16}\]  
where \(\mathbf{y}_{n}^{\prime}\sim p_{d}(\mathbf{y}_{n}^{\prime}|\mathbf{x}_{n})\), \(n=0,1,\cdots,N^{\prime}\).