## 3 Ensemble Soft Sensor Model Construction Based on Diversity Enhancement and Ensemble Pruning
### Ensemble Pruning Based on Multi-Objective GWO Optimization  
The selection of base models before the ensemble is another important factor that affects the final performance of the ensemble soft sensor model. This study takes into account the diversity and accuracy of the two indicators; in the ensemble pruning of the base models, the problem of base model selection is formalized as a multi-objective optimization problem, and an evolutionary algorithm is used to solve the problem in order to better weigh these two criteria, to achieve the goal of \"both good and different\". The basic idea is as follows:  
Figure 3: The modeling framework of MOSE- MLV-VSPLS. First, define a small-scale subensemble model set for the final ensemble, which is called SEM\({}_{\text{sel}}\). The RMSE is proposed as the accuracy index to measure the comprehensive prediction performance of SEM\({}_{\text{sel}}\), which is formed as RMSE\({}_{\text{t}al}\). Take the distance value between the predicted outputs of the models as the model similarity index to measure the diversity of models in SEM\({}_{\text{sel}}\), which is formed as CED\({}_{\text{t}al}\). Then, transform the model pruning problem into a multi-objective optimization problem that is formulated as follows:  
\[\text{minimize }F(\mathbf{x})=[f_{1}(\mathbf{x}),f_{2}(\mathbf{x})]^{ \text{T}} \tag{18}\]  
\[\text{subject to }\mathbf{x}\in[a,b] \tag{19}\]  
where \(\mathbf{x}\) is the decision variable vector, and \(F(\mathbf{x})\) is the object vector. The specific steps to build the SEM\({}_{\text{sel}}\) base on MOGWO that takes into account both accuracy and diversity parameters are as follows:  
(1) Define the decision variables. The construction of the SEM\({}_{\text{sel}}\) essentially corresponds to the question of whether all the base models are selected. Number all base models from \(1\) to MS\({}_{\text{L}VM}\), and use a 0-1 code to describe the selection status of those base models. Code 1 is assigned to those selected base models. Code 0 is assigned to the unselected base models. Then a binary number of MS\({}_{\text{L}VM}\) bits represents the selecting status of the base models. Therefore, the decision variable of the multi-objective optimization problem in this study is defined as a binary number of MS\({}_{\text{L}VM}\) bits. (2) Define the constraints. The purpose of ensemble pruning is to select appropriate base models to form a subensemble model set. Therefore, the size of the subensemble models set \(m_{sem}\) is taken as the constraint condition. (3) Define the objective function. **Definition 1**. _Prediction accuracy objective function._  
On the verification set \(D_{val}=\{Q_{val},T_{val}\}\), RMSE is used as the prediction accuracy index. First, the prediction accuracy of each base model is estimated as RMSE\({}_{val}^{i}\); then the average value of RMSE of all base models is used as the overall prediction accuracy. The ensemble pruning accuracy objective function is defined as follows:  
\[f_{1}(M)=\text{RMSE}_{val}=\frac{\sum_{i=1}^{m_{sem}}\text{RMSE}_{val}^{i}}{m _{sem}} \tag{20}\]  
where \(m_{sem}\) represents the number of actually selected model sets SEM\({}_{\text{sel}}\), and RMSE\({}_{val}^{i}\) represents the RMSE of the \(i\)th base model on the validation set. **Definition 2**. _Diversity objective function._  
In ensemble learning modeling, there is no unified definition of quantitative parameters of diversity. This study defines a comprehensive diversity quantification parameter called CED based on the Euclidean distance of the representation of the base model on the verification set. On the verification set \(D_{val}=\{Q_{val},T_{val}\}\), the output predictive value of the base model is named as \(yh_{val}\). The smaller the similarity between the output predictive values of any two base models, the greater the difference between the characterization base models.