## 4 Dynamic Feature Based XGBoost Model
### Dynamic Feature Extractor and Feature Transfer  
The autoencoder (AE) proposed in [31] is used to learn the main rules of the data to extract features. Based on the unsupervised AE, a series of supervised and semi-supervised stacked auto-encoder models have been developed, such as discussed in [12; 32]. The structure of this type of soft sensor model is still the ordinary fully connected neural network, and the semi-supervised static models such as SS-SAE are only capable of dealing with missing samples and do not account for the process dynamics. However, the mismatch problem between process dynamics and the number of process quality variables often coexists, so it is hoped that the soft sensor model can fully use unlabeled samples and extract process dynamic information.  
Based on AE, considering the dynamic characteristics of the industrial process, a dynamic feature extractor [32] can be designed. Different from the traditional autoencoder, the input of the dynamic feature extractor is the sequence data in a period-of-time window, and the goal of the decoder is to try to restore the output sequence to the input sequence so as to maximize the learning of the dynamic information in the industrial process.  
SCINet can effectively extract the relationship features between samples. Limited by the ordinary fully connected network of regressors, SCINet has insufficient generalization ability, and the model is easy to overfit when the number of training samples is small. Based on the feature capture capability of SCINet, considering the generalization performance of the model and the existing problem of mismatch in the number of process quality variables, the unsupervised dynamic feature extractor was designed, as shown in Figure 3.  
For the encoder, the input data are expressed as a \(K\) time sequence, \(X_{seq}=\left[X^{1},\ X^{2},\ \dots,\ X^{K}\right]\). The decoder output sequence can be expressed as \(\hat{X}_{seq}=\left[\hat{X}^{1},\ \hat{X}^{2},\ \dots,\ \hat{X}^{K}\right]\), whose dimension is consistent with the input sequence. The training loss function of the dynamic feature extractor is the reconstruction error of the input sequence and the output sequence, defined as follows:  
\[Loss=\frac{1}{K}\sum_{k=1}^{K}\|\hat{X}^{k}-X^{k}\| \tag{6}\]  
where \(K\) is the number of sequences.  
The encoding part of SCINet is used as the encoder of the dynamic feature extractor, and the sequence processed by the encoder is represented by a vector (hv). Combining the SCINet loss function (3) and the loss function of the dynamic feature extractor (6), the optimization objective of the feature extractor is designed as follows:  
Figure 3: Dynamic Feature Extraction and Transfer to XGBoost.  
\[J=\frac{1}{K}\|g_{\theta}\big{(}d_{\theta}\big{(}X_{seq}\big{)}\big{)}-X_{seq}\| \tag{7}\]  
where \(d_{\theta}\) and \(g_{\theta}\), respectively, feature the extractor encoder and the decoder.  
After the training of the dynamic feature extractor, the sequence corresponding to the labeled samples is input to the encoder of the dynamic feature extractor, and the output hidden layer vector is the dynamic feature corresponding to the sequence hv. The dynamic features are transferred to XGBoost to construct the SCI-XGBoost soft sensor model. Since the unsupervised training is not affected by the quality labels, the extracted dynamic features can be provided to XGBoost with stronger generalization ability for quality variable prediction.