# A Novel Reconstruction Method for Irregularly Sampled Observation Sequences for Digital Twin
## 3 Irregularly Sampled Observation Sequence Reconstruction Method Based on PRN-NCDE
### Overall Network Framework of PRN-NCDE  
The overall network framework of the sequence reconstruction model, as shown in Figure 2, consists primarily of an encoder and a decoder. The encoder comprises two identical network structures that map the actual observational sequence and the reference observational sequence into latent features, subsequently merging them through weighted fusion based on the noise level of the actual observational data. The decoder, consisting of an NCDE network and an output layer, utilizes these fused latent features to reconstruct the desired regular sequence. Initially, the input of the encoder is made up by the actual irregular observation sequence and the reference observation sequence, each passing through the identical neural network architecture to produce the actual and reference latent features, respectively. In order to enhance reconstruction accuracy under sparse measurement conditions, a multi-head self-attention mechanism is adopted to process both sequences, focusing on the position information of the measurement data and capturing the correlations among them. The multi-channel self-attention mechanism specifically addresses the irregularity and non-alignment issues in actual observation sequences. Taking the coal-fired power plant boiler system as an illustrative example, multivariate irregular observational sequences are first expanded to equal-length sequences based on desired sampling intervals to deal with irregular sampling intervals. Furthermore, discrepancies in sensor sampling times and frequencies lead to non-alignment among variables such as gas density, gas temperature, and gas oxygen content, causing inconsistent  
Figure 1: Schematic diagram of non-aligned irregularly sampled sequence reconstruction. positions of missing values across different variables. Therefore, the expanded sequence is segmented along variable dimensions and input into corresponding masked multi-head self-attention modules, where masks indicate missing values, preventing negative impacts during network training and prediction. Subsequently, the output sequence from the self-attention module undergo group normalization and reversal before being fed into an LSTM network for reverse-time inference from time \(t_{N^{\prime}}\) to \(t_{0}\), obtaining the hidden state at the initial time step \(t_{0}\). A fully connected layer maps this hidden state to the mean and variance of the latent feature distribution, and by combining with noise sampled from a standard Gaussian distribution can generate latent features. Similarly, reference observation sequences generated by a digital twin model undergo processing through an identical PRN structure to obtain reference latent features. Incorporating PRN to acquire the reference latent features provides prior information, thereby enhancing the reconstruction model's fitting performance under higher noise conditions. Figure 2: PRN-NCDE-based sequence reconstruction network framework. Higher noise levels would increase the noise information in the latent features from the actual observation sequences, significantly impairing the decoder's effectiveness. Hence, weights assigned to latent features are set as a nonlinear function of observation sequence noise variance. With noise level increasing, we should reduce the weight of latent features of the actual observation sequence and rely more on the reference latent features. Noise variance is approximated from the difference between actual and reference sequences. The resulting weighted fusion feature \(\mathbf{x}_{0}^{fus}\) is subsequently used as the decoder input. The decoder can be viewed as the inverse process of the encoder, reconstructing observation data from latent features through neural networks. Compared with NODE models that rely solely on the initial value \(\mathbf{x}_{0}^{fus}\), the NCDE-based decoder, which takes the influence of the control inputs into consideration during sequence reconstruction, can deal with the sequence reconstruction of a controlled system better.