# Adaptive Hybrid Soft-Sensor Model of Grinding Process Based on Regularized Extreme Learning Machine and Least Squares Support Vector Machine Optimized by Golden Sine Harris Hawk Optimization Algorithm

Authors: Wei Xie; Jie-sheng Wang; Cheng Xing; Sha-Sha Guo; Meng-wei Guo; Ling-feng Zhu
Journal: Complexity
 (2020)

---

## Abstract

Soft-sensor technology plays a vital role in tracking and monitoring the key production indicators of the grinding and classifying process. Least squares support vector machine (LSSVM), as a soft-sensor model with strong generalization ability, can be used to predict key production indicators in complex grinding processes. The traditional crossvalidation method cannot obtain the ideal structure parameters of LSSVM. In order to improve the prediction accuracy of LSSVM, a golden sine Harris Hawk optimization (GSHHO) algorithm was proposed to optimize the structure parameters of LSSVM models with linear kernel, sigmoid kernel, polynomial kernel, and radial basis kernel, and the influences of GSHHO algorithm on the prediction accuracy under these LSSVM models were studied. In order to deal with the problem that the prediction accuracy of the model decreases due to changes of industrial status, this paper adopts moving window (MW) strategy to adaptively revise the LSSVM (MW-LSSVM), which greatly improves the prediction accuracy of the LSSVM. The prediction accuracy of the regularized extreme learning machine with MW strategy (MW-RELM) is higher than that of MW-LSSVM at some moments. Based on the training errors of LSSVM and RELM within the window, this paper proposes an adaptive hybrid soft-sensing model that switches between LSSVM and RELM. Compared with the previous MW-LSSVM, MW-neural network trained with extended Kalman filter(MW-KNN), and MW-RELM, the prediction accuracy of the hybrid model is further improved. Simulation results show that the proposed hybrid adaptive soft-sensor model has good generalization ability and prediction accuracy.


---

## Body

## 1. Introduction

As one of the most important operating procedures in a beneficiation plant, the grinding and classifying process is to grind larger-sized metal ore to a reasonable size, expose useful metal components in the ore, and prepare for the next stage of the flotation process [1]. Some production indicators (such as granularity, impurity content, and iron content) become decisive indicators of whether the grinding process can be performed normally and will have an important impact on subsequent ore processing processes (especially flotation operations) [2]. Therefore, it is essential to monitor key production indicators in real time. Due to technical and economic conditions and the harsh industrial environment, there is no effective sensor to monitor the variables related to product quality in real time, which may reduce industrial control performance, increase production costs, and cause obstacles to green and healthy production [3]. With the development of data acquisition technologies and data processing algorithms, data-driven models have established it as an effective tool to solve this problem by estimating online variables that are difficult to measure directly [4]. These data-driven models are often referred to as “data-driven soft sensor,” which is an important application of data analysis techniques designed to estimate difficult-to-measure variables by using easy-to-measure variables [5]. In the past ten years, soft-sensor technology has been widely used in industrial data modeling due to its low cost, high implementation efficiency, and easy operation [6–9].In the grinding and classifying process, many experts and scholars have made great contributions to the development of soft-sensor models for the grinding process. In the initial stage of soft-sensor modeling of the grinding process, due to the limitations of theoretical knowledge and calculation tools, some researchers (such as Casali) tried to establish a mechanism model to predict the granularity of the grinding process [10–12]. When establishing the mechanism model, the entire grinding process needs to be carefully analyzed, and a large number of algebraic and differential equations are used to describe the entire grinding process. The mechanism model needs to assume that all external conditions need to be in an ideal state. However, the actual industrial environment is complex and changeable, which will cause a large deviation in the mechanism model and the prediction accuracy will be greatly reduced. Therefore, in recent years, few researchers are willing to use mechanism soft-sensor models for grinding and classifying process. Some scholars use the idea of case-based reasoning to build soft-sensing models [13–15]. The principle of case-based reasoning is to compare the similarity between the test samples and the existing samples in the database and infer the output value of the test samples from the most similar training samples in the database. This method is simple in theory, high in implementation efficiency (less time required to calculate the output value of the test sample), strong in antiinterference, and has certain practicability. This method requires accurate classification of training samples based on their numerical characteristics. When there are many samples in the database, it is difficult to find a suitable clustering algorithm to accurately classify the samples. This inaccurate classification will cause case-based reasoning to have a large error when inferring the output of the test sample, which will reduce the prediction accuracy of the model. Similar to the case-based reasoning model, Yan et al. built a soft-sensing model of uncertainty reasoning relationships based on the cloud model [16], which is more robust, especially when part of the training data is missing and still guarantees that its prediction accuracy will not decrease significantly. However, this method still faces the problem of fuzzy membership function partitioning of the sample. When the membership of the samples is not accurate, the prediction accuracy of the model will still decrease. With the continuous progress of statistical theory, some scholars have established soft-sensor models of the grinding process using a nonlinear multiple regression model. Partial least squares (PLS) model is used by some experts in soft-sensor modeling of grinding process due to its nonlinear fitting ability [17–19]. A common disadvantage of multiple regression models is that they are particularly sensitive to changes in the external environment. When the external environment changes, the existing model parameters will not accurately reflect the current state of industrial production, which will result in a decrease in the prediction accuracy of the model and a problem of “model degradation” [20]. Tang et al. selected multiple PLS models as the soft-sensor model on the grinding process to predict the load parameters of the ball mill [21, 22], which determines the number of PLS submodels based on multiple features of the sample set, uses an adaptive weighted fusion algorithm to integrate multiple PLS results, and obtains output so as to improve the adaptability of the soft-sensing model and the adjustment ability when the industrial status changes suddenly. However, the model parameters that need to be determined by this method will increase geometrically, which increases the computational burden of the model, and the training time of the model is also greatly increased.Compared with other types of soft-sensor models, a neural network soft-sensor shows greater advantages in which the neural network model only needs to focus on the input and output of the model and does not need to pay too much attention to the specific industrial production process. Most neural network models have strong generalization ability. Once the number of neurons is determined, only the appropriate model parameters need to be found to track key industrial production indicators [23]. Therefore, more experts and scholars are willing to study neural network models to set up soft-sensing models for grinding and classifying process. Common neural network soft-sensing models include BP neural network [15, 24, 25], RBF neural network [26–30], support vector machine (SVM) [31, 32], and extreme learning machine (ELM) [33]. The traditional training methods of these neural networks often cannot find suitable model parameters, which makes the prediction accuracy of the model unable to be further improved. Natural heuristic algorithms provide better ideas for solving this problem. The neural network is treated as a nonlinear function, the natural heuristic algorithm optimizes the error function formed by the nonlinear function, and the model parameters corresponding to the optimal value of the error function are reasonable. Some scholars applied this kind of algorithms to find the parameters of neural network models and obtained better experimental results [34–36]. Xie et al. proposed an improved black hole algorithm (GSLBH) with strong optimization ability [37] and used GSLBH to find suitable weights and biases for ELM models with different kernel functions [23]. The experimental results show that GSLBH-ELM has better prediction ability under the condition of stable industrial status. In many industrial environments, the characteristics of the equipment and other processing behaviors frequently change, such as equipment aging, and the change of industrial raw materials [6]. This working condition change will make the optimization algorithm no longer able to find suitable structural parameters for the neural network model and the accuracy of the model is difficult to improve by this method. Wang et al. introduced the idea of model migration to wavelet neural network, which can revise structural parameters as the environment changes [38]. This method requires frequent updating of model parameters, which increases the computational burden. Dai et al. [39] proposed a robust random vector functional link network, which can maintain its generalization ability without major changes in the face of changes of the industrial status. Some other scholars have proposed a hybrid soft-sensing model combining multiple neural networks [2, 40], whose main advantage is that it can play the prediction capabilities of different neural networks in different states so that the prediction accuracy of the model is always maintained at a high level. However, the fatal disadvantage of this model are that there are too many model parameters, the training time of the model is too long, and the working efficiency of model is extremely low, which are contrary to the efficiency of the soft-sensor model. How to make the most of the advantages of the hybrid model while ensuring the implementation efficiency of the soft sensor has become a problem to be solved.Compared with the traditional SVM model, the least squares SVM (LSSVM) is widely used to predict key production indicators in the steel and chemical industries due to its small training volume, low training difficulty, and strong generalization ability [41]. LSSVM transforms the quadratic programming problem of SVM parameters into linear equations by constructing the constraints of the equation, which reduces the difficulty of model solving [42]. However, the two types of parameters (the penalty factor and kernel parameter) which affect the prediction accuracy of the model in LSSVM need to be set manually [43]. In the LSSVM model, the reasonable selection of penalty factors and kernel function parameters becomes the key to whether the prediction accuracy of the model can meet the requirements. The most common parameter selection method is crossvalidation [44], the model parameters obtained by this method have strong limitations, and the optimal parameter combination is likely to fall far short of prediction accuracy. In contrast, the natural heuristic algorithm provides a better solution for finding the optimal model parameters of LSSVM.The Harris Hawk optimization (HHO) algorithm is a new type of natural heuristic algorithm proposed by Mirjalili et al. [45]. This algorithm has strong exploration and weak exploitation, and it has been proved in Xie [37] that the golden sine operator can expand the search range of agents, so this paper introduces the golden sine operator to the HHO (GSHHO) when the HHO is switched to the exploitation phase, which expands exploitation range of the algorithm and enhances the exploitation of the algorithm. This paper uses GSHHO algorithm to optimize the error function of LSSVM, and the model parameters corresponding to the minimum value of the obtained error function are used as the optimal parameter of LSSVM. In order to explore the optimization ability of GSHHO algorithm for LSSVM with different kernel functions, GSHHO algorithm is used to optimize LSSVM models with linear kernel, sigmoid kernel, polynomial kernel, and radial basis kernel, and the prediction results of different LSSVMs are compared. The model with the best prediction result is currently the most ideal LSSVM soft-sensor model. Through the observation and analysis of the dataset, the industrial status selected in this paper has been in dynamic changes. Therefore, local LSSVM models are established by using moving window (MW) strategy to predict the ore granularity at different periods. This MW-LSSVM model can respond well to changes of industrial status. At the same time, it was found that the regularized extreme learning machine (RELM) based on MW strategy (MW-RELM) also obtained similar prediction results as MW-LSSVM. In order to take advantage of different neural network soft-sensor models in different situations, based on the MW strategy, this paper proposes a hybrid soft-sensor model with LSSVM and RELM continuously switching (MW-LSSVM-RELM). Because the RELM model has the characteristics of extremely short training time, the introduction of RELM into MW-LSSVM does not affect the implementation efficiency of the soft-sensing model, and at the same time improves the prediction accuracy of the soft-sensing model. Neural network trained with extended Kalman filter (KNN) has proven to be a predictive model with strong generalization ability [46, 47]. This paper compares the experimental results of KNN with LSSVM and RELM and also compares the experimental results of MW-KNN with MW-LSSVM-RELM. Finally, simulation experiments prove that this hybrid model based on moving window strategy is more practical. The rest of this paper is organized as follows. Section 2 introduces LSSVM and RELM, Section 3 introduces GSHHO algorithm, Section 4 introduces the working principle of the hybrid model proposed in this paper, Section 5 conducts simulation experiments, and Section 6 draws conclusions.


## 2. Neural Network Soft-Sensor Models

### 2.1. Least Squares Support Vector Machine Regression Model

The core idea of the support vector machine is to derive an estimatefx of the unknown relationship y=fx between the input variable x and the output variable y in a given training set:(1)fx=wTφx+b,where x∈RN×p, N is the number of samples, p is the dimension of the variables, fx∈RN×1, and φx represents a high-dimensional nonlinear mapping of x, w, and b are constants. Let the given training sample set x=x1,y1,…,xN,yN, xi and yi are the corresponding input and output of the ith sample. The optimization problem of SVR can be expressed as follows:(2)minJw,ξ,ξ∗=12wTw+C∑i=1Nξi+ξi∗.The unequal constraints of this function can be expressed as follows:(3)yi−wTφxi−b≤ε+ξi,wTφxi+b−yi≤ε+ξi∗,ξi,ξi∗≥0,i=1,2,…,l,where ξi and ξi∗ are slack variables, ε>0 is the allowable error threshold, and C is the penalty factor.Compared with the original SVR, LSSVM regression replaces the inequality constraint with the linear least squares criterion and applies it to the loss function, whose objective function can be expressed as follows:(4)minJw,e=12wTw+12C∑i=1Nei2.At this time, the constraints of the objective function become(5)yi=wTφxi+b+ei,i=1,2,…,N.The following Lagrange function can be constructed from mathematical theoretical knowledge:(6)Lw,b,e,α=ψw,e−∑i=1NαiwTφxi+b+ei−yi,where αii=1,2,…,N is the Lagrangian multiplier. So, the optimal KKT condition of Lw,b,e,a can be expressed as follows:(7)∂L∂w=0⟹w=∑i=1Nαiφxi,∂L∂b=0⟹∑i=1Nαi=0,∂L∂ei=0⟹αi=Cei,i=1,2,…,N,∂L∂αi=0⟹wTφxi+b+ei−yi=0,i=1,2,…,N.By calculating the variablesw and ei, the optimization problem can be transformed into the following linear equations:(8)01NT1NΩ+C−1Ibα=0y,where 1N=1,…,1T, α=α1,…,αlT, and y=y1,…,yNT. The matrix Ω=ΩijN×N should satisfy the Mercer condition, that is to say,(9)Ωij=φxiTφxj=Kxi,xj.After solving the equations shown in equation (8), the final result of LSSVM regression can be expressed as follows:(10)fx=∑i=1NαiKx,xi+b,where αi and b are obtained by solving the linear equations in equation (8). In equation (10), Kx,xi is a kernel function that satisfies the Mercer condition. The kernel functions commonly used in LSSVM are shown in Table 1 [48], which give the parameters that LSSVM needs to be manually adjusted in the case of different kernel functions. C in Table 1 is the penalty factor for LSSVM.Table 1
Kernel functions used by LSSVM.
Kernel functionExpressionParameters to be adjustedLinearKx,xi=xkTxCPolynomialKx,xi=xkTx+θ2C, θSigmoidKx,xi=tanhkxkTxC, kRBFKx,xi=exp−x−xk22/σ2C, σ2The method of adjusting these parameters is generally crossvalidation. This method has a simple idea and is easy to operate. However, the limitation of this method is large, and the obtained structural parameters are probably not optimal solutions. Natural heuristic algorithm provides a good solution for finding the optimal parameters of the model. It works by optimizing the error function of the model, and the parameter corresponding to the minimum value of the obtained error function is the optimal parameter. Commonly used algorithms include genetic algorithm (GA) and particle swarm optimization (PSO) algorithm, but a large number of experiments have proved that many algorithms have better optimization capabilities than GA and PSO [37, 45, 49]. This paper proposes a golden sine Harris Hawk optimization algorithm to select reasonable structural parameters for the LSSVM soft-sensor model. This paper will use the golden sine Harris Hawk algorithm to optimize the LSSVM with the four kernel functions in Table 1 and compare the experimental results of these four LSSVMs. The specific implementation steps are described in the following sections.


### 2.2. Regularized Extreme Learning Machine Regression Model

Extreme learning machine (ELM) is a single hidden layer feed-forward network (SLFN) proposed by Huang et al. [50]. This neural network has a simple structure, extremely short training time, efficient execution, and good generalization ability [4, 23, 51, 52]. Principles of ELM training and prediction have been explained in details in [23]. The key work of training ELM is to solve the weight β between the hidden layer and the output layer, and the calculation of β is realized by(11)β=H+T′,where H+ is the Moore–Penrose generalized inverse of the hidden layer output matrix H, H+ can also be expressed as HT⋅H−1⋅HT, and T represents the output matrix. After solving the structural parameters, the trained model can be used to predict the output variables. The main disadvantage of ELM is its instability, it is too sensitive to the interference of data noise, and it is prone to overfitting. To solve this difficulty, Deng et al. [53] introduced regularization parameters when calculating the weight β, which greatly enhanced the generalization ability of the ELM model and made the ELM model more practical.According to statistical learning theory, the actual predicted risk of learning is composed of empirical risk and structural risk. A model with good generalization capabilities should make the best trade-off between the two risks. Therefore, the actual risk can be expressed by the weighted sum of these two risks. By introducing a weighting factorγ for empirical risks, their ratio can be adjusted [53]. Empirical risk can be expressed by squared error ε2. Structural risk is represented by variables β2 that maximize the distance to the interface [54–56]. The mathematical model of the regularized ELM model can be expressed as follows:(12)min12β2+12γε2s.t.∑i=1Nβigwixi+bi−tj=εj,j=1,2,…,N,where N˜ is the number of samples, N is the number of neurons, and ε=ε1,ε2,…,εN. To solve the optimization problem shown in equation (12), the following Lagrangian equation can be constructed:(13)Lβ,ε,α=γ2ε2+12β2−∑j=1Nαj∑i=1N˜βigwi⋅xj+bi−tj−εj=γ2ε2+12β2−αHβ−T−ε,where αi is the Lagrangian multiplier. The optimal KKT condition of Lβ,ε,α can be expressed as follows:(14)∂L∂β=0⟹βT=αH,∂L∂ε=0⟹γεT+α=0,∂L∂α=0⟹Hβ−T−ε=0.Three sets of equations in equation (14) can be combined to calculate the final expression of β:(15)β=Iγ+HTH−1HTT.The ELM that obtains the weightβ between the hidden layer and the output layer through equation (15) is called a regularized ELM (RELM). Adjustment of γ can adjust the ratio between empirical risk and structural risk. The model will obtain the best generalization ability when these two risks reach the optimal compromise [53]. Therefore, compared with traditional ELM, RELM has better anti-interference ability, stronger generalization ability of neural network, and higher prediction accuracy. The implementation steps of RELM are described as follows:Step 1: determine the number of hidden layer neuronsN of the RELM model according to the size of the samples, and determine the activation function gx.Step 2: given a training set of samples, randomly initialize weightswii=1,2,…,N˜ and biases values bii=1,2,…,N˜ between the input layer and the hidden layer and calculate the output matrix H of the hidden layer.Step 3: calculate the weightβ between the hidden layer and the output layer according to equation (15).Step 4: input the query set into the RELM model to obtain the output.


## 2.1. Least Squares Support Vector Machine Regression Model

The core idea of the support vector machine is to derive an estimatefx of the unknown relationship y=fx between the input variable x and the output variable y in a given training set:(1)fx=wTφx+b,where x∈RN×p, N is the number of samples, p is the dimension of the variables, fx∈RN×1, and φx represents a high-dimensional nonlinear mapping of x, w, and b are constants. Let the given training sample set x=x1,y1,…,xN,yN, xi and yi are the corresponding input and output of the ith sample. The optimization problem of SVR can be expressed as follows:(2)minJw,ξ,ξ∗=12wTw+C∑i=1Nξi+ξi∗.The unequal constraints of this function can be expressed as follows:(3)yi−wTφxi−b≤ε+ξi,wTφxi+b−yi≤ε+ξi∗,ξi,ξi∗≥0,i=1,2,…,l,where ξi and ξi∗ are slack variables, ε>0 is the allowable error threshold, and C is the penalty factor.Compared with the original SVR, LSSVM regression replaces the inequality constraint with the linear least squares criterion and applies it to the loss function, whose objective function can be expressed as follows:(4)minJw,e=12wTw+12C∑i=1Nei2.At this time, the constraints of the objective function become(5)yi=wTφxi+b+ei,i=1,2,…,N.The following Lagrange function can be constructed from mathematical theoretical knowledge:(6)Lw,b,e,α=ψw,e−∑i=1NαiwTφxi+b+ei−yi,where αii=1,2,…,N is the Lagrangian multiplier. So, the optimal KKT condition of Lw,b,e,a can be expressed as follows:(7)∂L∂w=0⟹w=∑i=1Nαiφxi,∂L∂b=0⟹∑i=1Nαi=0,∂L∂ei=0⟹αi=Cei,i=1,2,…,N,∂L∂αi=0⟹wTφxi+b+ei−yi=0,i=1,2,…,N.By calculating the variablesw and ei, the optimization problem can be transformed into the following linear equations:(8)01NT1NΩ+C−1Ibα=0y,where 1N=1,…,1T, α=α1,…,αlT, and y=y1,…,yNT. The matrix Ω=ΩijN×N should satisfy the Mercer condition, that is to say,(9)Ωij=φxiTφxj=Kxi,xj.After solving the equations shown in equation (8), the final result of LSSVM regression can be expressed as follows:(10)fx=∑i=1NαiKx,xi+b,where αi and b are obtained by solving the linear equations in equation (8). In equation (10), Kx,xi is a kernel function that satisfies the Mercer condition. The kernel functions commonly used in LSSVM are shown in Table 1 [48], which give the parameters that LSSVM needs to be manually adjusted in the case of different kernel functions. C in Table 1 is the penalty factor for LSSVM.Table 1
Kernel functions used by LSSVM.
Kernel functionExpressionParameters to be adjustedLinearKx,xi=xkTxCPolynomialKx,xi=xkTx+θ2C, θSigmoidKx,xi=tanhkxkTxC, kRBFKx,xi=exp−x−xk22/σ2C, σ2The method of adjusting these parameters is generally crossvalidation. This method has a simple idea and is easy to operate. However, the limitation of this method is large, and the obtained structural parameters are probably not optimal solutions. Natural heuristic algorithm provides a good solution for finding the optimal parameters of the model. It works by optimizing the error function of the model, and the parameter corresponding to the minimum value of the obtained error function is the optimal parameter. Commonly used algorithms include genetic algorithm (GA) and particle swarm optimization (PSO) algorithm, but a large number of experiments have proved that many algorithms have better optimization capabilities than GA and PSO [37, 45, 49]. This paper proposes a golden sine Harris Hawk optimization algorithm to select reasonable structural parameters for the LSSVM soft-sensor model. This paper will use the golden sine Harris Hawk algorithm to optimize the LSSVM with the four kernel functions in Table 1 and compare the experimental results of these four LSSVMs. The specific implementation steps are described in the following sections.


## 2.2. Regularized Extreme Learning Machine Regression Model

Extreme learning machine (ELM) is a single hidden layer feed-forward network (SLFN) proposed by Huang et al. [50]. This neural network has a simple structure, extremely short training time, efficient execution, and good generalization ability [4, 23, 51, 52]. Principles of ELM training and prediction have been explained in details in [23]. The key work of training ELM is to solve the weight β between the hidden layer and the output layer, and the calculation of β is realized by(11)β=H+T′,where H+ is the Moore–Penrose generalized inverse of the hidden layer output matrix H, H+ can also be expressed as HT⋅H−1⋅HT, and T represents the output matrix. After solving the structural parameters, the trained model can be used to predict the output variables. The main disadvantage of ELM is its instability, it is too sensitive to the interference of data noise, and it is prone to overfitting. To solve this difficulty, Deng et al. [53] introduced regularization parameters when calculating the weight β, which greatly enhanced the generalization ability of the ELM model and made the ELM model more practical.According to statistical learning theory, the actual predicted risk of learning is composed of empirical risk and structural risk. A model with good generalization capabilities should make the best trade-off between the two risks. Therefore, the actual risk can be expressed by the weighted sum of these two risks. By introducing a weighting factorγ for empirical risks, their ratio can be adjusted [53]. Empirical risk can be expressed by squared error ε2. Structural risk is represented by variables β2 that maximize the distance to the interface [54–56]. The mathematical model of the regularized ELM model can be expressed as follows:(12)min12β2+12γε2s.t.∑i=1Nβigwixi+bi−tj=εj,j=1,2,…,N,where N˜ is the number of samples, N is the number of neurons, and ε=ε1,ε2,…,εN. To solve the optimization problem shown in equation (12), the following Lagrangian equation can be constructed:(13)Lβ,ε,α=γ2ε2+12β2−∑j=1Nαj∑i=1N˜βigwi⋅xj+bi−tj−εj=γ2ε2+12β2−αHβ−T−ε,where αi is the Lagrangian multiplier. The optimal KKT condition of Lβ,ε,α can be expressed as follows:(14)∂L∂β=0⟹βT=αH,∂L∂ε=0⟹γεT+α=0,∂L∂α=0⟹Hβ−T−ε=0.Three sets of equations in equation (14) can be combined to calculate the final expression of β:(15)β=Iγ+HTH−1HTT.The ELM that obtains the weightβ between the hidden layer and the output layer through equation (15) is called a regularized ELM (RELM). Adjustment of γ can adjust the ratio between empirical risk and structural risk. The model will obtain the best generalization ability when these two risks reach the optimal compromise [53]. Therefore, compared with traditional ELM, RELM has better anti-interference ability, stronger generalization ability of neural network, and higher prediction accuracy. The implementation steps of RELM are described as follows:Step 1: determine the number of hidden layer neuronsN of the RELM model according to the size of the samples, and determine the activation function gx.Step 2: given a training set of samples, randomly initialize weightswii=1,2,…,N˜ and biases values bii=1,2,…,N˜ between the input layer and the hidden layer and calculate the output matrix H of the hidden layer.Step 3: calculate the weightβ between the hidden layer and the output layer according to equation (15).Step 4: input the query set into the RELM model to obtain the output.


## 3. Improved Harris Hawk Optimization Algorithm

### 3.1. Harris Hawk Optimization Algorithm

The Harris Hawk optimization (HHO) algorithm is a new natural heuristic algorithm proposed by Mirjalili et al. inspired by the hunting behavior of Harris Hawk [23]. It has the characteristics of strong exploration and extremely high efficiency. Therefore, once HHO algorithm was proposed, it attracted the attention of many scholars [57, 58]. According to the current physical energy consumption of the prey, Harris Hawk will adopt different hunting strategies to capture the prey. Let energy of the current prey be E, and it can be calculated by(16)E=2E01−tT,where E0 is a random number between −1,1, t is the current number of iterations, and T is the specified total number of iterations. When E0 is between −1 and 0, prey is physically flagging. When E0 is between 0 and 1, prey is strengthening. It can be seen from equation (16) and Figure 1 that as t increases, the prey energy E decreases. Harris Hawk will judge whether it should keep exploration phase or switch to the exploitation phase based on the E.Figure 1
Energy change curve of the prey.WhenE≥1, the prey has more energy, the Harris Hawk is in the exploration phase, and the position update strategy in the exploration phase can be realized by(17)Xt+1=Xrandt−r1Xrandt−2r2Xtq≥0.5,Xperyt−Xmt−r3LB+r4UB−LBq<0.5,where LB and UB represent upper and lower bounds, r1,r2,r3,r4,andq are random numbers between 0 and 1, Xt and Xt+1 represent the current location of the Harris Hawk and the location of the next iteration, respectively, Xrandt represents the current position of an agent in the Harris Hawk population, Xperyt represents the current position of the prey (the current optimal location), and Xmt represents the coordinate average of all agents in the Harris Hawk population, whose calculation formula is described as follows:(18)Xmt=1N∑i=1NXit,where N is the population size.WhenE<1, Harris Hawk transforms into the exploitation phase and began to capture the prey. Scientific research shows that Harris Hawk has seven ways to attack the prey [59]. In HHO, only four methods of attacking prey were selected: “Soft besiege,” “Hard besiege,” “Soft besiege with progressive rapid dives,” and “Hard besiege with progressive rapid dives.” A random number r∈0,1 is also defined in the process of the eagle attacking the prey, which indicates whether the prey successfully escaped.In the exploitation phase, whenr≥0.5andE≥0.5, the prey has enough energy, but still fails to escape. At this time, the Harris Hawk attacked the prey with a “soft besiege” method. The movement of Harris Hawk is expressed as follows:(19)Xt+1=ΔXt−EJXpreyt−Xt,J=21−r5,ΔXt=Xpreyt−Xt,where r5 is a random number between 0 and 1.The other parameters have the same meaning as those in equation (17).Whenr≥0.5andE<0.5, the prey has no enough energy to escape and the Harris Hawk uses a “hard besiege” method to attack the prey. The movement of Harris Hawk can be expressed as follows:(20)Xt+1=Xpreyt−EΔXt,where ΔXt has the same meaning as the parameter in equation (19).Whenr<0.5andE≥0.5, the prey is difficult to be caught by the Harris Hawk. Harris Hawk uses a more clever method “Soft besiege with progressive rapid dives” to capture the prey. The movement pattern of this attack method is described as follows:(21)Y=Xpreyt−EJXpreyt−Xt,Z=Y+S×LevyD,where J,Xprey,andXt have the same meaning as in equation (19), D represents the dimension of the agents, and S represents a 1×D random matrix, and Levy means Levy flight function. The laws of movement of most animals in nature conform to the Levy flight function [60], and the expression of the Levy flight function is expressed as follows:(22)LFx=0.01×u×σv1/β,σ=Γ1+β×sinπβ/2Γ1+β/2×β×2β−1/21/β,where uandv are a random number between 0 and 1 and the value of β defaults to 1.5. In the end, the Harris Hawk determined the movement method of the next iteration according to the fitness value of the objective function as follows:(23)Xt+1=Y,ifFY<FXt,Z,ifFZ<FXt,where Fx is the calculated fitness function value, and other parameters have the same meanings as the parameters in equation (21).Whenr<0.5andE<0.5, although the prey is exhausted, it still tries to escape the capture of Harris Hawk. At this time, Harris Hawk uses “Hard besiege with progressive rapid dives”. The movement pattern of this attack method is expressed as follows:(24)Y=Xpreyt−EJXpreyt−Xmt,Z=Y+S×LevyD,(25)Xt+1=Y,ifFY<FXt,Z,ifFZ<FXt,where Xmt has the same meaning as in equation (17), and the remaining parameters have the same meaning as in equation (21). Similar to “Soft besiege with progressive rapid dives”, in the end, the Harris Hawk determines the movement pattern of the next iteration according to the fitness value of the objective function shown in equation (25). The implementation flowchart of Harris Hawk optimization algorithm is shown in Figure 2.Figure 2
Procedure of HHO algorithm.


### 3.2. Golden Sine Harris Hawk Optimization Algorithm

#### 3.2.1. Golden Sine Operator

The golden sine operator is a search strategy derived from the golden sine algorithm (Gold-SA) [60]. This operation can expand the search range of agents. From the geometric meaning of the trigonometric function, it can be known that the process of continuously moving a point on a sine function is equivalent to the process of continuously scanning this point on a unit circle, but in Gold-SA, agents do not move according to a standard sine curve, but move according to a gold sine curve [37]. During each iteration, the ability that agents search within a certain local improved is expanded, when the agents move according to this golden sine curve. This golden sine curve path shape in two-dimensional space is shown by the red curves in Figure 3 [37].Figure 3
Golden sine curve in two-dimensional space.In Gold-SA, the agents to search according to the golden sine curve can be expressed as follows:(26)Xt+1=Xt∗sinr1−r2∗sinr1∗m1∗Xbestt−m2∗Xt,where r1 is a random number between 0 and 2π, r2 is a random number between 0 and π, Xbestt is the current optimal location, and m1andm2 is the coefficient obtained by the gold selection method. This method can reduce the search space of the agents, improve the search efficiency of the agents, and make the agents move to the target location faster. In order to simplify the operation and improve the execution efficiency of the algorithm, Xie et al. [37] assign m1andm2 in equation (26) with suitable constants. This approach guarantees that the search ability of the agents will not weaken, while increasing the stability of the algorithm. The constant values given to m1andm2 are set as follows:(27)m1=−π+1−τ,m2=−π+τ∗π,τ=1−52,when m1andm2 in equation (25) becomes constants in equation (27), the movement mode shown in equation (26) becomes the golden sine operator.


#### 3.2.2. Golden Sine Harris Hawk Optimization (GSHHO) Algorithm

HHO is an algorithm with very strong exploration. Its global search mechanism can effectively reduce the possibility of the algorithm falling into a local optimum. However, when HHO enters the exploitation phase, its exploitation is not particularly strong. In order to expand the exploitation range of HHO, this paper introduces the golden sine operator into the exploitation phase of HHO to enhance its exploitation. During each iteration, whenE<1, all agents first perform the golden sine operation in equation (26) and then select one of four exploitation methods for the next operation according to the size of r and E. Compared with the implementation steps of the original HHO in Figure 2, GSHHO only adds a golden sine search operation to the exploitation phase, but the exploitation of the algorithm has been greatly improved. The pseudocode of GSHHO algorithm is described as follows:Determine the number of populations asN and the number of iterations as T;Initialize populationXii=1,2,…,N;whilet<TDetermine the optimal position according to the fitness value of all agents, and specify this optimal position as the position of the current prey;Calculate the value ofE according to formula (16);ifE≥1The agents perform exploration according to formula (17);ifE<1The agents perform golden sine search according to formula (26);ifr≥0.5,E≥0.5The agents perform exploitation according to formula (19);else ifr≥0.5,E<0.5The agents perform exploitation according to formula (20);else ifr<0.5,E≥0.5The agents perform exploitation according to formula (23);else ifr<0.5,E<0.5The agents perform exploitation according to formula (25)t=t+1Output the optimal value and optimal location found by HHO


#### 3.2.3. Golden Sine Algorithm to Optimize LSSVM Model

As mentioned at the end of Section2.1, using natural heuristics algorithm to select the structural parameters of LSSVM is a good idea. This paper adopts GSHHO algorithm to optimize the structural parameters of LSSVM with 4 kernel functions listed in Table 1. It can be seen from Table 1 that when the kernel function of LSSVM is different, the parameters to be adjusted will also be different, and the different parameters are the internal parameters of the kernel function. For each different kernel function, the parameter set to be adjusted is Θ. For example, when the kernel function is rbf function, Θ=C,σ2. Let the error function formed by LSSVM be(28)error=1n∑i=1nfXi,Θ−Yi,where n represents the number of training samples, Xi represents the input of the training samples, Yi represents the actual output corresponding to the training sample Xi, and f represents the unknown nonlinear relationship derived from input Xi and output Yi in the LSSVM model.When using GSHHO algorithm, the size ofΘ determines the dimensions of the solution space, and error function is the objective function that GSHHO algorithm needs to be optimized. The smaller the objective function value found in the solution space, the smaller the training error of the LSSVM model and the higher the prediction accuracy of model for the query samples. When the iteration is completed, the optimal position obtained is optimal Θ found by GSHHO algorithm. When the different kernel functions listed in Table 1 are optimized, the results that GSHHO optimize LSSVM models will definitely be different. In the subsequent simulation experiments, this paper will compare the prediction results of GSHHO-LSSVM models with different kernel functions and find the optimal GSHHO-LSSVM model and the corresponding optimal parameters. The whole process of GSHHO-LSSVM model predicting query sample is divided into the following steps:Step 1: construct the error function using the training sample setX1,Y1,…,Xn,Yn and any given Θ according to the function construction rule of f in Section 2.1Step 2: find the minimum value of the error function and the coordinateΘbest corresponding to the minimum value according to the GSHHO algorithm implementation rule shown in Section 3.2.2Step 3: after the iteration, output the optimal positionΘbest and use Θbest as the optimal model parameter of the LSSVMStep 4: train the LSSVM model again using the optimal parameterΘbest and the training sample set X1,Y1,…,Xn,YnStep 5: input the test samples to the trained LSSVM model to obtain prediction results


## 3.1. Harris Hawk Optimization Algorithm

The Harris Hawk optimization (HHO) algorithm is a new natural heuristic algorithm proposed by Mirjalili et al. inspired by the hunting behavior of Harris Hawk [23]. It has the characteristics of strong exploration and extremely high efficiency. Therefore, once HHO algorithm was proposed, it attracted the attention of many scholars [57, 58]. According to the current physical energy consumption of the prey, Harris Hawk will adopt different hunting strategies to capture the prey. Let energy of the current prey be E, and it can be calculated by(16)E=2E01−tT,where E0 is a random number between −1,1, t is the current number of iterations, and T is the specified total number of iterations. When E0 is between −1 and 0, prey is physically flagging. When E0 is between 0 and 1, prey is strengthening. It can be seen from equation (16) and Figure 1 that as t increases, the prey energy E decreases. Harris Hawk will judge whether it should keep exploration phase or switch to the exploitation phase based on the E.Figure 1
Energy change curve of the prey.WhenE≥1, the prey has more energy, the Harris Hawk is in the exploration phase, and the position update strategy in the exploration phase can be realized by(17)Xt+1=Xrandt−r1Xrandt−2r2Xtq≥0.5,Xperyt−Xmt−r3LB+r4UB−LBq<0.5,where LB and UB represent upper and lower bounds, r1,r2,r3,r4,andq are random numbers between 0 and 1, Xt and Xt+1 represent the current location of the Harris Hawk and the location of the next iteration, respectively, Xrandt represents the current position of an agent in the Harris Hawk population, Xperyt represents the current position of the prey (the current optimal location), and Xmt represents the coordinate average of all agents in the Harris Hawk population, whose calculation formula is described as follows:(18)Xmt=1N∑i=1NXit,where N is the population size.WhenE<1, Harris Hawk transforms into the exploitation phase and began to capture the prey. Scientific research shows that Harris Hawk has seven ways to attack the prey [59]. In HHO, only four methods of attacking prey were selected: “Soft besiege,” “Hard besiege,” “Soft besiege with progressive rapid dives,” and “Hard besiege with progressive rapid dives.” A random number r∈0,1 is also defined in the process of the eagle attacking the prey, which indicates whether the prey successfully escaped.In the exploitation phase, whenr≥0.5andE≥0.5, the prey has enough energy, but still fails to escape. At this time, the Harris Hawk attacked the prey with a “soft besiege” method. The movement of Harris Hawk is expressed as follows:(19)Xt+1=ΔXt−EJXpreyt−Xt,J=21−r5,ΔXt=Xpreyt−Xt,where r5 is a random number between 0 and 1.The other parameters have the same meaning as those in equation (17).Whenr≥0.5andE<0.5, the prey has no enough energy to escape and the Harris Hawk uses a “hard besiege” method to attack the prey. The movement of Harris Hawk can be expressed as follows:(20)Xt+1=Xpreyt−EΔXt,where ΔXt has the same meaning as the parameter in equation (19).Whenr<0.5andE≥0.5, the prey is difficult to be caught by the Harris Hawk. Harris Hawk uses a more clever method “Soft besiege with progressive rapid dives” to capture the prey. The movement pattern of this attack method is described as follows:(21)Y=Xpreyt−EJXpreyt−Xt,Z=Y+S×LevyD,where J,Xprey,andXt have the same meaning as in equation (19), D represents the dimension of the agents, and S represents a 1×D random matrix, and Levy means Levy flight function. The laws of movement of most animals in nature conform to the Levy flight function [60], and the expression of the Levy flight function is expressed as follows:(22)LFx=0.01×u×σv1/β,σ=Γ1+β×sinπβ/2Γ1+β/2×β×2β−1/21/β,where uandv are a random number between 0 and 1 and the value of β defaults to 1.5. In the end, the Harris Hawk determined the movement method of the next iteration according to the fitness value of the objective function as follows:(23)Xt+1=Y,ifFY<FXt,Z,ifFZ<FXt,where Fx is the calculated fitness function value, and other parameters have the same meanings as the parameters in equation (21).Whenr<0.5andE<0.5, although the prey is exhausted, it still tries to escape the capture of Harris Hawk. At this time, Harris Hawk uses “Hard besiege with progressive rapid dives”. The movement pattern of this attack method is expressed as follows:(24)Y=Xpreyt−EJXpreyt−Xmt,Z=Y+S×LevyD,(25)Xt+1=Y,ifFY<FXt,Z,ifFZ<FXt,where Xmt has the same meaning as in equation (17), and the remaining parameters have the same meaning as in equation (21). Similar to “Soft besiege with progressive rapid dives”, in the end, the Harris Hawk determines the movement pattern of the next iteration according to the fitness value of the objective function shown in equation (25). The implementation flowchart of Harris Hawk optimization algorithm is shown in Figure 2.Figure 2
Procedure of HHO algorithm.


## 3.2. Golden Sine Harris Hawk Optimization Algorithm

### 3.2.1. Golden Sine Operator

The golden sine operator is a search strategy derived from the golden sine algorithm (Gold-SA) [60]. This operation can expand the search range of agents. From the geometric meaning of the trigonometric function, it can be known that the process of continuously moving a point on a sine function is equivalent to the process of continuously scanning this point on a unit circle, but in Gold-SA, agents do not move according to a standard sine curve, but move according to a gold sine curve [37]. During each iteration, the ability that agents search within a certain local improved is expanded, when the agents move according to this golden sine curve. This golden sine curve path shape in two-dimensional space is shown by the red curves in Figure 3 [37].Figure 3
Golden sine curve in two-dimensional space.In Gold-SA, the agents to search according to the golden sine curve can be expressed as follows:(26)Xt+1=Xt∗sinr1−r2∗sinr1∗m1∗Xbestt−m2∗Xt,where r1 is a random number between 0 and 2π, r2 is a random number between 0 and π, Xbestt is the current optimal location, and m1andm2 is the coefficient obtained by the gold selection method. This method can reduce the search space of the agents, improve the search efficiency of the agents, and make the agents move to the target location faster. In order to simplify the operation and improve the execution efficiency of the algorithm, Xie et al. [37] assign m1andm2 in equation (26) with suitable constants. This approach guarantees that the search ability of the agents will not weaken, while increasing the stability of the algorithm. The constant values given to m1andm2 are set as follows:(27)m1=−π+1−τ,m2=−π+τ∗π,τ=1−52,when m1andm2 in equation (25) becomes constants in equation (27), the movement mode shown in equation (26) becomes the golden sine operator.


### 3.2.2. Golden Sine Harris Hawk Optimization (GSHHO) Algorithm

HHO is an algorithm with very strong exploration. Its global search mechanism can effectively reduce the possibility of the algorithm falling into a local optimum. However, when HHO enters the exploitation phase, its exploitation is not particularly strong. In order to expand the exploitation range of HHO, this paper introduces the golden sine operator into the exploitation phase of HHO to enhance its exploitation. During each iteration, whenE<1, all agents first perform the golden sine operation in equation (26) and then select one of four exploitation methods for the next operation according to the size of r and E. Compared with the implementation steps of the original HHO in Figure 2, GSHHO only adds a golden sine search operation to the exploitation phase, but the exploitation of the algorithm has been greatly improved. The pseudocode of GSHHO algorithm is described as follows:Determine the number of populations asN and the number of iterations as T;Initialize populationXii=1,2,…,N;whilet<TDetermine the optimal position according to the fitness value of all agents, and specify this optimal position as the position of the current prey;Calculate the value ofE according to formula (16);ifE≥1The agents perform exploration according to formula (17);ifE<1The agents perform golden sine search according to formula (26);ifr≥0.5,E≥0.5The agents perform exploitation according to formula (19);else ifr≥0.5,E<0.5The agents perform exploitation according to formula (20);else ifr<0.5,E≥0.5The agents perform exploitation according to formula (23);else ifr<0.5,E<0.5The agents perform exploitation according to formula (25)t=t+1Output the optimal value and optimal location found by HHO


### 3.2.3. Golden Sine Algorithm to Optimize LSSVM Model

As mentioned at the end of Section2.1, using natural heuristics algorithm to select the structural parameters of LSSVM is a good idea. This paper adopts GSHHO algorithm to optimize the structural parameters of LSSVM with 4 kernel functions listed in Table 1. It can be seen from Table 1 that when the kernel function of LSSVM is different, the parameters to be adjusted will also be different, and the different parameters are the internal parameters of the kernel function. For each different kernel function, the parameter set to be adjusted is Θ. For example, when the kernel function is rbf function, Θ=C,σ2. Let the error function formed by LSSVM be(28)error=1n∑i=1nfXi,Θ−Yi,where n represents the number of training samples, Xi represents the input of the training samples, Yi represents the actual output corresponding to the training sample Xi, and f represents the unknown nonlinear relationship derived from input Xi and output Yi in the LSSVM model.When using GSHHO algorithm, the size ofΘ determines the dimensions of the solution space, and error function is the objective function that GSHHO algorithm needs to be optimized. The smaller the objective function value found in the solution space, the smaller the training error of the LSSVM model and the higher the prediction accuracy of model for the query samples. When the iteration is completed, the optimal position obtained is optimal Θ found by GSHHO algorithm. When the different kernel functions listed in Table 1 are optimized, the results that GSHHO optimize LSSVM models will definitely be different. In the subsequent simulation experiments, this paper will compare the prediction results of GSHHO-LSSVM models with different kernel functions and find the optimal GSHHO-LSSVM model and the corresponding optimal parameters. The whole process of GSHHO-LSSVM model predicting query sample is divided into the following steps:Step 1: construct the error function using the training sample setX1,Y1,…,Xn,Yn and any given Θ according to the function construction rule of f in Section 2.1Step 2: find the minimum value of the error function and the coordinateΘbest corresponding to the minimum value according to the GSHHO algorithm implementation rule shown in Section 3.2.2Step 3: after the iteration, output the optimal positionΘbest and use Θbest as the optimal model parameter of the LSSVMStep 4: train the LSSVM model again using the optimal parameterΘbest and the training sample set X1,Y1,…,Xn,YnStep 5: input the test samples to the trained LSSVM model to obtain prediction results


## 3.2.1. Golden Sine Operator

The golden sine operator is a search strategy derived from the golden sine algorithm (Gold-SA) [60]. This operation can expand the search range of agents. From the geometric meaning of the trigonometric function, it can be known that the process of continuously moving a point on a sine function is equivalent to the process of continuously scanning this point on a unit circle, but in Gold-SA, agents do not move according to a standard sine curve, but move according to a gold sine curve [37]. During each iteration, the ability that agents search within a certain local improved is expanded, when the agents move according to this golden sine curve. This golden sine curve path shape in two-dimensional space is shown by the red curves in Figure 3 [37].Figure 3
Golden sine curve in two-dimensional space.In Gold-SA, the agents to search according to the golden sine curve can be expressed as follows:(26)Xt+1=Xt∗sinr1−r2∗sinr1∗m1∗Xbestt−m2∗Xt,where r1 is a random number between 0 and 2π, r2 is a random number between 0 and π, Xbestt is the current optimal location, and m1andm2 is the coefficient obtained by the gold selection method. This method can reduce the search space of the agents, improve the search efficiency of the agents, and make the agents move to the target location faster. In order to simplify the operation and improve the execution efficiency of the algorithm, Xie et al. [37] assign m1andm2 in equation (26) with suitable constants. This approach guarantees that the search ability of the agents will not weaken, while increasing the stability of the algorithm. The constant values given to m1andm2 are set as follows:(27)m1=−π+1−τ,m2=−π+τ∗π,τ=1−52,when m1andm2 in equation (25) becomes constants in equation (27), the movement mode shown in equation (26) becomes the golden sine operator.


## 3.2.2. Golden Sine Harris Hawk Optimization (GSHHO) Algorithm

HHO is an algorithm with very strong exploration. Its global search mechanism can effectively reduce the possibility of the algorithm falling into a local optimum. However, when HHO enters the exploitation phase, its exploitation is not particularly strong. In order to expand the exploitation range of HHO, this paper introduces the golden sine operator into the exploitation phase of HHO to enhance its exploitation. During each iteration, whenE<1, all agents first perform the golden sine operation in equation (26) and then select one of four exploitation methods for the next operation according to the size of r and E. Compared with the implementation steps of the original HHO in Figure 2, GSHHO only adds a golden sine search operation to the exploitation phase, but the exploitation of the algorithm has been greatly improved. The pseudocode of GSHHO algorithm is described as follows:Determine the number of populations asN and the number of iterations as T;Initialize populationXii=1,2,…,N;whilet<TDetermine the optimal position according to the fitness value of all agents, and specify this optimal position as the position of the current prey;Calculate the value ofE according to formula (16);ifE≥1The agents perform exploration according to formula (17);ifE<1The agents perform golden sine search according to formula (26);ifr≥0.5,E≥0.5The agents perform exploitation according to formula (19);else ifr≥0.5,E<0.5The agents perform exploitation according to formula (20);else ifr<0.5,E≥0.5The agents perform exploitation according to formula (23);else ifr<0.5,E<0.5The agents perform exploitation according to formula (25)t=t+1Output the optimal value and optimal location found by HHO


## 3.2.3. Golden Sine Algorithm to Optimize LSSVM Model

As mentioned at the end of Section2.1, using natural heuristics algorithm to select the structural parameters of LSSVM is a good idea. This paper adopts GSHHO algorithm to optimize the structural parameters of LSSVM with 4 kernel functions listed in Table 1. It can be seen from Table 1 that when the kernel function of LSSVM is different, the parameters to be adjusted will also be different, and the different parameters are the internal parameters of the kernel function. For each different kernel function, the parameter set to be adjusted is Θ. For example, when the kernel function is rbf function, Θ=C,σ2. Let the error function formed by LSSVM be(28)error=1n∑i=1nfXi,Θ−Yi,where n represents the number of training samples, Xi represents the input of the training samples, Yi represents the actual output corresponding to the training sample Xi, and f represents the unknown nonlinear relationship derived from input Xi and output Yi in the LSSVM model.When using GSHHO algorithm, the size ofΘ determines the dimensions of the solution space, and error function is the objective function that GSHHO algorithm needs to be optimized. The smaller the objective function value found in the solution space, the smaller the training error of the LSSVM model and the higher the prediction accuracy of model for the query samples. When the iteration is completed, the optimal position obtained is optimal Θ found by GSHHO algorithm. When the different kernel functions listed in Table 1 are optimized, the results that GSHHO optimize LSSVM models will definitely be different. In the subsequent simulation experiments, this paper will compare the prediction results of GSHHO-LSSVM models with different kernel functions and find the optimal GSHHO-LSSVM model and the corresponding optimal parameters. The whole process of GSHHO-LSSVM model predicting query sample is divided into the following steps:Step 1: construct the error function using the training sample setX1,Y1,…,Xn,Yn and any given Θ according to the function construction rule of f in Section 2.1Step 2: find the minimum value of the error function and the coordinateΘbest corresponding to the minimum value according to the GSHHO algorithm implementation rule shown in Section 3.2.2Step 3: after the iteration, output the optimal positionΘbest and use Θbest as the optimal model parameter of the LSSVMStep 4: train the LSSVM model again using the optimal parameterΘbest and the training sample set X1,Y1,…,Xn,YnStep 5: input the test samples to the trained LSSVM model to obtain prediction results


## 4. Adaptive Hybrid Soft-Sensor Model Based on LSSVM and RELM

### 4.1. Moving Window Strategy

In the actual grinding and classifying process, changes in various external conditions will cause the industrial process to change at any time (such as different types of ore, aging of processing equipment, and frequent switching of equipment working modes), and historical databases cannot contain all possible states and conditions in the future industrial process [61]. When the above problems occur, the existing soft-sensor models can no longer accurately describe the current status of industrial production, which will cause the prediction accuracy of the model to decrease. In order to cope with the frequent changes of industrial status, this paper introduces moving window (MW) strategy to adaptively revise the parameters of the soft-sensor model. The idea of MW strategy was proposed by Michalski et al. [62], and its main work is to select a specific set of data to adjust the parameters of the soft-sensor model. In the vast majority of cases, a fixed amount of the latest data is considered the most relevant to the current state of the industry [20]. Therefore, when predicting the current production indicators, some training samples that are most relevant to the current moment can be selected from the database to build a local soft-sensor model, and the prediction value of such a local model for the current production indicator is likely to be closest to the real value. The implementation process of the MW strategy is shown in Figure 4.Figure 4
Schematic diagram of moving window.Suppose the size of the window isL and the moving step is S, the training set in the window at the initial moment is(29)Dori=x1,y1,…,xL,yL.Modely=forix is trained according to Dori, and the predicted value of a test sample t1 that is closest to Dori in time is(30)p^1=forit1.When new samples are added, the window will slide forward according to the step sizeS. At this time, the training set in the window can be expressed as follows:(31)D1=x1+s,y1+s,…,xL+s,yL+s.Modely=f1x is trained according to D1, the predicted value of a test sample t2 that is closest to D1 in time is(32)p^2=forit2.The model training phase of the MW strategy can be repeatedly applied to the online phase without the need for other online algorithms, which means that it can be combined with all existing soft-sensor models [20] and these advantages also make the MW strategy become a very popular adaptive strategy.


### 4.2. Hybrid Soft-Sensor Model Based on Moving Window Strategy

Section3.2.3 details the process of GSHHO algorithm to optimize LSSVM. Compared with the traditional crossvalidation method, GSHHO algorithm can find more suitable parameters for LSSVM, which further improves the prediction accuracy of the LSSVM soft-sensor model. However, in the face of changes in the working conditions, the ability of algorithm to optimize LSSVM has a certain limit. Because a hyperplane formed by single LSSVM model cannot characterize all industrial status in the training sample, this will cause a bottleneck in improving the accuracy of the soft-sensor model. MW strategy can effectively solve this problem. Based on the optimal LSSVM structural parameters discovered by GSHHO algorithm, the introduction of MW strategy into LSSVM will further improve the model prediction accuracy. When a current query sample is needed to be predicted, LSSVM selects the window data closest to the query sample to build a local model and uses this local model to predict the query sample. When the window slides m times, m local models of LSSVM will be established, which also means that m hyperplanes are used to characterize the industrial status at different time periods, and the hyperplanes of these local models can more realistically reflect the industrial conditions of certain periods and improve the prediction accuracy of the models. The specific implementation process of MW-LSSVM is as described in formulas (29)–(23) in Section 4.1. However, the MW strategy has a major drawback: the computational burden increases when training the model. If the window slides m times, the soft-sensing model needs to be retrained m times, so it is necessary to maintain a balance between improving the prediction accuracy of the model and reducing the calculation burden. For the regression model of SVM, one of its biggest advantages is that the training time of the model is very short after the structural parameters are determined. LSSVM is more simplified and training time is shorter than traditional SVM training process. Therefore, although the calculation load of LSSVM is increased after the introduction of MW strategy, it is far less than the acceptable training time. This advantage of LSSVM further enhances the utility of MW-LSSVM. It is found that the RELM model can obtain good prediction results similar to the MW-LSSVM model after the MW strategy is introduced. In order to further improve the prediction accuracy of MW-LSSVM, this paper proposes a hybrid model of LSSVM and RELM based on MW (MW-LSSVM-RELM), which can take advantage of LSSVM and RLEM under different conditions. The implementation mechanism of this hybrid model is shown in Figure 5.Figure 5
Schematic diagram of the implementation of the hybrid soft-sensor model.When using this hybrid model, the computational burden of the model will further increase, but RELM is also a model with extremely high execution efficiency [53]. Although the data in each window requires training of the two models, the time required to train two models is still very short, much less than the acceptable time. The computational efficiency of the hybrid model will be explained later in the simulation experiment section. The operation of this hybrid soft-sensor model is described as follows:Step 1: establish local LSSVM model1 and RELM model2 according to the training samples in the windowStep 2: compare the training errors of model1 and model2, and select a model with a smaller training error from these two local models as the current soft-senor modelStep 3: predict query samples with the model identified in Step 2Step 4: when a new sample is added, the window slides according to the specified step size, and then repeat the operations of Steps 1–3


## 4.1. Moving Window Strategy

In the actual grinding and classifying process, changes in various external conditions will cause the industrial process to change at any time (such as different types of ore, aging of processing equipment, and frequent switching of equipment working modes), and historical databases cannot contain all possible states and conditions in the future industrial process [61]. When the above problems occur, the existing soft-sensor models can no longer accurately describe the current status of industrial production, which will cause the prediction accuracy of the model to decrease. In order to cope with the frequent changes of industrial status, this paper introduces moving window (MW) strategy to adaptively revise the parameters of the soft-sensor model. The idea of MW strategy was proposed by Michalski et al. [62], and its main work is to select a specific set of data to adjust the parameters of the soft-sensor model. In the vast majority of cases, a fixed amount of the latest data is considered the most relevant to the current state of the industry [20]. Therefore, when predicting the current production indicators, some training samples that are most relevant to the current moment can be selected from the database to build a local soft-sensor model, and the prediction value of such a local model for the current production indicator is likely to be closest to the real value. The implementation process of the MW strategy is shown in Figure 4.Figure 4
Schematic diagram of moving window.Suppose the size of the window isL and the moving step is S, the training set in the window at the initial moment is(29)Dori=x1,y1,…,xL,yL.Modely=forix is trained according to Dori, and the predicted value of a test sample t1 that is closest to Dori in time is(30)p^1=forit1.When new samples are added, the window will slide forward according to the step sizeS. At this time, the training set in the window can be expressed as follows:(31)D1=x1+s,y1+s,…,xL+s,yL+s.Modely=f1x is trained according to D1, the predicted value of a test sample t2 that is closest to D1 in time is(32)p^2=forit2.The model training phase of the MW strategy can be repeatedly applied to the online phase without the need for other online algorithms, which means that it can be combined with all existing soft-sensor models [20] and these advantages also make the MW strategy become a very popular adaptive strategy.


## 4.2. Hybrid Soft-Sensor Model Based on Moving Window Strategy

Section3.2.3 details the process of GSHHO algorithm to optimize LSSVM. Compared with the traditional crossvalidation method, GSHHO algorithm can find more suitable parameters for LSSVM, which further improves the prediction accuracy of the LSSVM soft-sensor model. However, in the face of changes in the working conditions, the ability of algorithm to optimize LSSVM has a certain limit. Because a hyperplane formed by single LSSVM model cannot characterize all industrial status in the training sample, this will cause a bottleneck in improving the accuracy of the soft-sensor model. MW strategy can effectively solve this problem. Based on the optimal LSSVM structural parameters discovered by GSHHO algorithm, the introduction of MW strategy into LSSVM will further improve the model prediction accuracy. When a current query sample is needed to be predicted, LSSVM selects the window data closest to the query sample to build a local model and uses this local model to predict the query sample. When the window slides m times, m local models of LSSVM will be established, which also means that m hyperplanes are used to characterize the industrial status at different time periods, and the hyperplanes of these local models can more realistically reflect the industrial conditions of certain periods and improve the prediction accuracy of the models. The specific implementation process of MW-LSSVM is as described in formulas (29)–(23) in Section 4.1. However, the MW strategy has a major drawback: the computational burden increases when training the model. If the window slides m times, the soft-sensing model needs to be retrained m times, so it is necessary to maintain a balance between improving the prediction accuracy of the model and reducing the calculation burden. For the regression model of SVM, one of its biggest advantages is that the training time of the model is very short after the structural parameters are determined. LSSVM is more simplified and training time is shorter than traditional SVM training process. Therefore, although the calculation load of LSSVM is increased after the introduction of MW strategy, it is far less than the acceptable training time. This advantage of LSSVM further enhances the utility of MW-LSSVM. It is found that the RELM model can obtain good prediction results similar to the MW-LSSVM model after the MW strategy is introduced. In order to further improve the prediction accuracy of MW-LSSVM, this paper proposes a hybrid model of LSSVM and RELM based on MW (MW-LSSVM-RELM), which can take advantage of LSSVM and RLEM under different conditions. The implementation mechanism of this hybrid model is shown in Figure 5.Figure 5
Schematic diagram of the implementation of the hybrid soft-sensor model.When using this hybrid model, the computational burden of the model will further increase, but RELM is also a model with extremely high execution efficiency [53]. Although the data in each window requires training of the two models, the time required to train two models is still very short, much less than the acceptable time. The computational efficiency of the hybrid model will be explained later in the simulation experiment section. The operation of this hybrid soft-sensor model is described as follows:Step 1: establish local LSSVM model1 and RELM model2 according to the training samples in the windowStep 2: compare the training errors of model1 and model2, and select a model with a smaller training error from these two local models as the current soft-senor modelStep 3: predict query samples with the model identified in Step 2Step 4: when a new sample is added, the window slides according to the specified step size, and then repeat the operations of Steps 1–3


## 5. Simulation Experiments and Result Analysis

### 5.1. Technique of Grinding and Classifying Process

Based on the processing of iron ore in a beneficiation plant, this paper studies the grinding and classifying process of iron ore. Xie et al. have introduced the grinding classification process in details [23], and this article only briefly describes the grinding and classifying process, whose schematic diagram of the grinding and classifying process is shown in Figure 6 [23].Figure 6
Schematic diagram of grinding and classification process.The grinding and classifying process is mainly composed of two parts: the first part is the first stage of closed-circuit grinding process, and the second part is the second stage of closed-circuit grinding process. The iron ore is first sent to the first ball mill for grinding through a belt. After a period of grinding, the pulp produced by the first ball mill is sent to the spiral classifier to classify the ore. The ore with larger granularity (size of the ore) is sent to the first ball mill for grinding again; the ore with smaller granularity flows out with the overflow product and then enters the second stage ball mill for fine grinding after fine sieve work. The hydrocyclone will separate the ore particles whose granularity meets the production standards. For the particles that do not meet the requirements, they will enter the second stage ball mill to continue grinding until the size of the ore meets the standards. Similar to Ref. [23], this paper still selects 10 easy-to-measure variables as secondary variables for the soft-sensor model and adopts these variables to predict the ore granularity during the grinding process. Information of secondary variables and objective variables are shown in Tables 2 and 3 [23]. Table 3 shows some data of the grinding process soft-sensor modeling. The collection positions of these secondary variables have been indicated in Figure 6.Table 2
Information on various variables of the grinding process.
Variable typeVariableVariable symbolUnitSecondary variableFeeding capacityFC-1kgInlet water flowIWF-1m3/sOutlet water flowIWF-2m3/sPump pool levelPPL-1mPump pressurePP-1kpaCurrentBMP-1AFeed flow of cycloneIWF-3m3/sOverflow concentrationOC-1%Ball mill powerBMP-2kwFeed concentrationOC-2%Objective variableGranularity%Table 3
Soft-sensor modeling data for the grinding process.
Secondary variableObjective variableFeeding capacityInlet water flowOutlet water flowPump pool levelPump pressureCurrentFeed flow of cycloneOverflow concentrationBall mill powerFeed concentrationGranularity1134.79681.3562.2531.5840.04263.000128.15674.3261119.31170.36458.6212134.57882.54763.4241.6090.04358.000127.71773.3251118.63869.31458.9923133.63382.46563.7181.5950.04266.000127.91174.3641121.66369.41557.6944135.70282.47564.6871.5920.04163.000127.56174.6311118.56167.31458.3235135.41282.45762.4131.6170.04160.000127.45475.2541119.87868.14159.9226134.79483.34763.0811.6060.04359.000128.93273.3441118.74970.56454.773………………………………1700136.36482.14584.3411.5340.04360.000126.34772.3841120.37970.36456.314


### 5.2. Data Dimension Reduction

For the neural network soft-sensor model, too large dimension of the input variables makes the network topology very large and the training process becomes complicated. At the same time, there may be redundancy between the variables, which will interfere with the prediction process of the neural network, so high-dimensional data information needs to be reduced [23, 63]. In the case of unstable industrial status, this paper also uses the KPCA dimensionality reduction method used in Ref. [23] to reduce the data in Table 3, and the results are listed in Table 4. The contribution percentage of the first 5 variables in Table 4 reached 85%. The input data after KPCA dimension reduction can improve the calculation efficiency of the soft-sensor model and reduce the complexity of the model.Table 4
Contribution percentage of the principal components.
dVariance percentage (%)Cumulative variance percentage (%)10.33350.333520.21320.546730.18320.729940.08210.81250.05010.862160.04820.910370.04130.951680.03220.983890.00910.9929100.00711


### 5.3. Simulation Experiments

In this paper, five input variables after KPCA processing are used to predict the content of ore granularity to200 mesh during the grinding and classifying process. This paper collected 1,700 samples from an existing database, of which 1500 were used as training samples and 200 were used as test samples. In order to make it clear that the prediction accuracy of the soft-sensor models proposed in this paper is higher than the prediction accuracy of other models, three quantitative indicators were selected to describe the prediction accuracy of the soft-sensor model, whose calculation methods of these three quantitative indicators are listed in Table 5. Many scholars use these three quantitative indicators to evaluate the predictive performance of the model [24, 25, 29, 30]. In Table 5, n represents the number of test samples, y^i represents the predicted value of the objective variables, yi represents the true value of the objective variables, and y¯ represents the average of all objective variables. The smaller MAE and RMSE, the higher the prediction accuracy of the model. The larger R2, the stronger the fitting ability of the soft-sensor model.Table 5
Quantitative indicators of soft-sensor models.
IndexFunctionMean absolute errorMAE=1/n∑i=1ny^i−yiRoot mean square errorRMSE=1/n∑i=1ny^i−yi21/2R2R2=∑i=1ny^i−y¯2/∑i=1nyi−y¯2


#### 5.3.1. GSHHO Algorithm to Optimize LSSVM Model

In order to study the optimization capability of GSHHO algorithm for LSSVM with different kernel functions, this paper uses GSHHO algorithm to optimize the four different LSSVMs listed in Table1. The traditional three-layer crossvalidation method (CV) and HHO-optimized LSSVM were used as comparative experimental results to prove the superiority of GSHHO algorithm in this paper. The population of GSHHO algorithm and HHO algorithm is set to 30 and the number of iterations is set to 100. Figures 7–10 show the results that these three methods optimize the LSSVM parameters, where (a) of each figure represents the prediction results of the soft-sensor model, and (b) represents the prediction error of the soft-sensor model. In order to reflect the optimization capabilities of GSHHO algorithm and HHO algorithm, the convergence curves that GSHHO algorithm and HHO algorithm optimize different error functions are shown in Figure 11. The quantitative indicator results of ore granularity predicted by CV-LSSVM, GSHHO-LSSVM, and HHO-LSSVM are listed in Table 6 to further illustrate the prediction accuracy of different soft-sensor models.Figure 7
Prediction results under LSSVM model withlinear function. (a) Prediction results under different LSSVM models. (b) Prediction errors under different LSSVM models.
(a)(b)Figure 8
Prediction results under LSSVM model withploy function. (a) Prediction results under different LSSVM models. (b) Prediction errors under different LSSVM models.
(a)(b)Figure 9
Prediction results under LSSVM model withrbf function. (a) Prediction results under different LSSVM models. (b) Prediction errors under different LSSVM models.
(a)(b)Figure 10
Prediction results under LSSVM model withsigmoid function. (a) Prediction results under different LSSVM models. (b) Prediction errors under different LSSVM models.
(a)(b)Figure 11
Convergence curves of error functions under four different LSSVM models.
(a)(b)(c)(d)Table 6
Quantitative indicators under different soft-sensor models.
Kernel functionAlgorithmMAERMSER2LinearCV0.6860.73270.202HHO0.68610.73270.2024GSHHO0.68610.73270.2024PloyCV0.53680.4360.5763HHO0.53630.43480.5786GSHHO0.53630.43470.5787rbfCV0.36420.22960.7812HHO0.34120.23510.7731GSHHO0.34530.21910.79SigmoidCV0.64691.18390.3368HHO0.72331.44880.2213GSHHO0.58410.58390.4942It can be seen from the experimental results in these figures and tables that LSSVM with different kernel functions show different prediction capabilities. From Figures8 and 9 and Table 6, it can be seen that compared with traditional crossvalidation, the optimization capabilities of GSHHO algorithm and HHO algorithm for LSSVM with linear functions and ploy function do not show great advantages, the prediction results of these three methods are not ideal, and it is impossible to perform real-time monitoring of the particle size of the ore. From Figure 10 and Table 6, for LSSVM with sigmoid function, the optimization capabilities of GSHHO algorithm are significantly better than HHO and CV. Compared with traditional HHO algorithm, GSHHO algorithm has both strong exploration and exploitation. It can be seen from the convergence curve of Figure 11(d) that GSHHO can find more suitable minimum value for the error function in equation (28), which makes prediction accuracy of GSHHO-LSSVM higher. However, compared with the LSSVM with rbf function, the prediction result of LSSVM with sigmoid function is still not optimal. Combining the quantitative indicators in Table 5 and the results in Figures 9 and 10, it can be seen that the prediction result of LSSVM with rbf optimized by CV is better than with sigmoid optimized by GSHHO algorithm. Looking further at the results in Figure 10 and Table 6, it can be seen that compared with HHO-LSSVM and CV-LSSVM, GSHHO-LSSVM with rbf function shows better performance. By the way, in the convergence curve of Figure 11, when GSHHO algorithm optimizes the error function, the convergence speed is faster and the convergence accuracy is higher. Table 7 shows the parameter values found by GSHHO algorithm and HHO algorithm when optimizing different soft-sensor models. Combining all the above conclusions, it can be concluded that GSHHO algorithm has a great advantage in optimizing the LSSVM model, and the optimization result of the LSSVM model with rbf function is the best.Table 7
Parameter values found by the algorithms.
Kernel functionAlgorithmParameter valueLinearHHOC=124.36546GSHHOC=148.61254PloyHHOC=85.31564,θ=3.2146GSHHOC=98.63345,θ=1.36543rbfHHOC=53.24514,σ2=1.12283GSHHOC=61.34664,σ2=0.71341SigmoidHHOC=253.54376,k=0.86543GSHHOC=257.77324,k=0.36424


#### 5.3.2. Simulation Experiments of Regularized Extreme Learning Machine Model

As explained in Section2.2, RELM is a neural network model with strong generalization ability and high implementation efficiency. This section studies the prediction accuracy of RELM for ore granularity. GSHHO-LSSVM with rbf function has been proved to be a prediction model with good prediction accuracy, so this GSHHO-LSSVM will be compared with the prediction result of the RELM model. The traditional ELM experimental results will be used as a comparison of the RELM model. The number of hidden layer neurons in the ELM model and RELM model is 45.sigma function is used as the kernel function of ELM and RELM, and regularized parameter 1/λ of RELM is 0.001. In the case of stable industrial status, the GSLBH-ELM proposed by Xie et al. [23] has been proven to be an effective soft-sensor model. When predicting the ore granularity, GSLBH-ELM with arctan function has the highest prediction accuracy. This paper will also use GSLBH-ELM with arctan function to predict the ore granularity when this industrial status is constantly changing, and the results obtained will be compared with the experimental results of RELM, ELM, and LSSVM. The experimental results of these four models are shown in Figure 12 and Table 8.Figure 12
Experimental results under different ELM and LSSVM models. (a) Prediction results under different soft-sensor models. (b) Prediction errors under different soft-sensor models. (c) Predicted scatter plots under different soft-sensor models.
(a)(b)(c)Table 8
Quantitative indicators under different soft-sensor models.
ModelMAERMSER2ELM0.45250.31830.7101RELM0.44880.30670.7209LSSVM0.34530.21910.7901GSLBH-ELM0.74030.0267−0.5167KNN0.35320.23070.7904It can be seen from the experimental results that the GSLBH-ELM model does not show good prediction ability. The main reason is that the number of hidden layer neurons selected by GSLBH-ELM is too small. The small number of neurons does not reflect the dynamic industrial characteristics of this experiment. So, GSLBH algorithm cannot find more reasonable weights and biases. If the number of neurons in the GSLBH-ELM is increased, the parameters that the algorithm needs to be optimized will increase geometrically and the time required to train the model will be too long, which is contrary to the efficiency of the soft-sensor model. Therefore, when the industrial process changes dynamically, GSLBH-ELM is not suitable for predicting the ore granularity. Combining the results shown in Figure12 and Table 8, the prediction performance of RELM is slightly better than ELM, especially in Figure 12(c), and it can be clearly seen that the prediction value of RELM is more accurate than ELM at some moments. However, the prediction accuracy of GSHHO-LSSVM is still higher than that of RELM and ELM.In order to study the ability of the neural network model trained with the extended Kalman filter (KNN) to predict the granularity in the grinding and classifying process, this paper compares the prediction results of KNN with RELM and LSSVM. In this experiment, the structural parameters of the KNN model and the hyperparameters of the Kalman filter method are shown in Table9, where I represents the identity matrix, the size of the matrix P is the square of the number of model parameters, and std=0.01, the size of the matrix Q is the square of the number of output. The experimental results are shown in Figure 13, and the quantitative indicators of the prediction results are shown in Table 8. As can be seen from the results in Figure 13 and Table 8, compared with RELM and LSSVM, KNN does not show a large advantage, and its prediction accuracy is better than RELM, but still worse than LSSVM.Table 9
The structural parameters of the KNN model and the hyperparameters of the Kalman filter method.
(a) Neural network model structure parametersParameterNumber of model inputsNumber of hidden neuronsNumber of model outputNumber of iterationsValue5101200(b) Kalman filter parametersParameterStep of update weightInitial weight covariancePData covarianceRProcess covarianceQValue1P=0.5∗IR=0.1+std2∗IQ=0Figure 13
Experimental results under KNN and other models. (a) Prediction results under different soft-sensor models. (b) Prediction errors under different soft-sensor models. (c) Predicted scatter plots under different soft-sensor models.
(a)(b)(c)


#### 5.3.3. Simulation Experiment of Hybrid Soft-Sensor Model Based on Moving Window Strategy

Under the circumstances that the industrial status is changing at any time, this paper introduces moving window strategy into the hybrid soft-sensor model to improve the prediction accuracy of the model. The operation of this hybrid model has been described in details in Section4.2. The window size in this experiment is 90, and the step size is 15. The LSSVM in the hybrid model uses the best performing model (the kernel function is rbf kernel, and the structural parameters are those found by GSHHO in Table 6) described in 5.3.2, the parameters of the RLEM model are those selected in Section 5.3.3. At the same time, the moving window strategy is introduced into the LSSVM model (MW-LSSVM), the RELM model (MW-RELM), and the ELM model (MW-ELM) as comparative experiments. The experimental results are all shown in Figures 14 and 15 and Tables 10 and 11.Figure 14
Prediction results under MW-ELM model and MW-RELM model. (a) Prediction results under different soft-sensor models. (b) Prediction errors under different soft-sensor models. (c) Predicted scatter plots under different soft-sensor models.Figure 15
Experimental results under different hybrid models. (a) Prediction results under different soft-sensor models. (b) Prediction errors under different soft-sensor models. (c) Predicted scatter plots under different soft-sensor models.
(a)(b)(c)Table 10
Quantitative indicators under different soft-sensor models.
ModelMAERMSER2MW-ELM0.41041.54810.5021MW-RELM0.28590.15520.8765Table 11
Quantitative indicators under different models.
ModelMAERMSER2MW-RELM0.28590.15520.8765MW-LSSVM0.24390.15340.8651MW-LSSVM-RELM0.20940.10030.9175MW-KNN0.23860.13730.8886From the experimental results in Figure14 and Table 10, it can be seen that when building a local soft-sensing model, the MW-ELM model is very prone to overfitting due to the small number of training samples, there are some abnormal points in the test results of MW-ELM, and the prediction accuracy of the model is greatly reduced, even worse than the ELM prediction results in Section 5.3.3. MW-RELM overcomes the shortcomings of MW-ELM, and the introduction of regularization parameters 1/λ makes the model avoid the interference of noise and greatly improves the generalization ability of the model. Because MW-RELM can adaptively adjust the structural parameters of the model according to changes in the working conditions, compared with the single model in Section 5.3.3, the prediction accuracy of MW-RELM has been greatly improved. From the results of Figure 15 and Table 11, it can be seen that the prediction results of MW-LSSVM are slightly better than MW-RELM. The hybrid model MW-LSSVM-RELM combines the advantages of LSSVM and RELM under different industrial status which can further improve the prediction accuracy of the model and achieve the purpose of online prediction of the ore granularity in a dynamic industrial environment.In order to find out which model is selected every time the window slides, this article shows the model switching process in Figure16, where “1” indicates that the LSSVM model is used as the local soft-sensor model and “−1” indicates that the RELM model is used as the local soft-sensing model. It can be seen from the results in Figure 16 that the LSSVM model is used as the local soft-sensing model is the majority, and the role of the RELM model is to “revise” when LSSVM performs poorly. In order to explore the working efficiency of the MW-LSSVM-RELM model, the training time required for the MW-LSSVM-RELM model to predict a new query sample is listed in Table 12. The results in Table 12 show that although the training time of MW-LSSVM-RELM is longer than that of MW-RELM and MW-LSSVM, its training time is still very short and the model is extremely efficient, much less than the acceptable training time.Figure 16
Switching process of different models.Table 12
Training time under deterrent soft-sensor models.
ModelTraining time(s)MW-RELM0.0043MW-LSSVM0.0086MW-LSSVM-RELM0.0111MW-KNN2.0076Based on the above experimental results, it can be seen that the MW-LSSVM-RELM model has better prediction ability than other single soft-senor models and models that introduce the MW strategy into a single soft-sensor model. The MW-LSSVM-RELM model has better prediction accuracy and higher work efficiency and can achieve online prediction of ore granularity.It can be seen from the experimental results in Figure13 and Table 9 that after the extended Kalman filtering method is used to adjust the parameters of the neural network, the neural network has good generalization ability, and its prediction accuracy is even better than RELM. In this section, in order to study the influence of moving window strategy on KNN prediction accuracy, this paper introduces moving window strategy into KNN. The window size and sliding steps are the same as MW-LSSVM-ELM. In order to improve the working efficiency of the model, the number of training iterations is set to 100, and other parameters remain unchanged. The experimental results of MW-KNN are shown in Figure 17 and Table 11. It can be seen from the results that the prediction accuracy of KNN has been greatly improved after the introduction of the sliding window strategy. Prediction accuracy of KNN is better than MW-RELM and MW-LSSVM, but still not as good as MW-LSSVM-ELM. Compared with MW-LSSVM-ELM, MW-KNN has a major disadvantage, that is, it is less efficient. From the results in Table 12, it can be seen that although the number of iterations of KNN changes from 200 to 100, the time for MW-KNN to predict a test sample is still much longer than MW-LSSVM-ELM. From two aspects of work efficiency and prediction accuracy, the performance of the MW-LSSVM-ELM model is better than that of MW-KNN.Figure 17
Experimental results under different hybrid models.
(a)(b)(c)Combining all the above results, it can be concluded that the predictive ability of the MW-LSSVM-ELM model is superior to ELM, RELM, LSSVM, and KNN, even if these four models introduce moving window strategy, the prediction accuracy of MW-LSSVM-ELM is still better than them, and MW-LSSVM-ELM is more efficient than other models.


## 5.1. Technique of Grinding and Classifying Process

Based on the processing of iron ore in a beneficiation plant, this paper studies the grinding and classifying process of iron ore. Xie et al. have introduced the grinding classification process in details [23], and this article only briefly describes the grinding and classifying process, whose schematic diagram of the grinding and classifying process is shown in Figure 6 [23].Figure 6
Schematic diagram of grinding and classification process.The grinding and classifying process is mainly composed of two parts: the first part is the first stage of closed-circuit grinding process, and the second part is the second stage of closed-circuit grinding process. The iron ore is first sent to the first ball mill for grinding through a belt. After a period of grinding, the pulp produced by the first ball mill is sent to the spiral classifier to classify the ore. The ore with larger granularity (size of the ore) is sent to the first ball mill for grinding again; the ore with smaller granularity flows out with the overflow product and then enters the second stage ball mill for fine grinding after fine sieve work. The hydrocyclone will separate the ore particles whose granularity meets the production standards. For the particles that do not meet the requirements, they will enter the second stage ball mill to continue grinding until the size of the ore meets the standards. Similar to Ref. [23], this paper still selects 10 easy-to-measure variables as secondary variables for the soft-sensor model and adopts these variables to predict the ore granularity during the grinding process. Information of secondary variables and objective variables are shown in Tables 2 and 3 [23]. Table 3 shows some data of the grinding process soft-sensor modeling. The collection positions of these secondary variables have been indicated in Figure 6.Table 2
Information on various variables of the grinding process.
Variable typeVariableVariable symbolUnitSecondary variableFeeding capacityFC-1kgInlet water flowIWF-1m3/sOutlet water flowIWF-2m3/sPump pool levelPPL-1mPump pressurePP-1kpaCurrentBMP-1AFeed flow of cycloneIWF-3m3/sOverflow concentrationOC-1%Ball mill powerBMP-2kwFeed concentrationOC-2%Objective variableGranularity%Table 3
Soft-sensor modeling data for the grinding process.
Secondary variableObjective variableFeeding capacityInlet water flowOutlet water flowPump pool levelPump pressureCurrentFeed flow of cycloneOverflow concentrationBall mill powerFeed concentrationGranularity1134.79681.3562.2531.5840.04263.000128.15674.3261119.31170.36458.6212134.57882.54763.4241.6090.04358.000127.71773.3251118.63869.31458.9923133.63382.46563.7181.5950.04266.000127.91174.3641121.66369.41557.6944135.70282.47564.6871.5920.04163.000127.56174.6311118.56167.31458.3235135.41282.45762.4131.6170.04160.000127.45475.2541119.87868.14159.9226134.79483.34763.0811.6060.04359.000128.93273.3441118.74970.56454.773………………………………1700136.36482.14584.3411.5340.04360.000126.34772.3841120.37970.36456.314


## 5.2. Data Dimension Reduction

For the neural network soft-sensor model, too large dimension of the input variables makes the network topology very large and the training process becomes complicated. At the same time, there may be redundancy between the variables, which will interfere with the prediction process of the neural network, so high-dimensional data information needs to be reduced [23, 63]. In the case of unstable industrial status, this paper also uses the KPCA dimensionality reduction method used in Ref. [23] to reduce the data in Table 3, and the results are listed in Table 4. The contribution percentage of the first 5 variables in Table 4 reached 85%. The input data after KPCA dimension reduction can improve the calculation efficiency of the soft-sensor model and reduce the complexity of the model.Table 4
Contribution percentage of the principal components.
dVariance percentage (%)Cumulative variance percentage (%)10.33350.333520.21320.546730.18320.729940.08210.81250.05010.862160.04820.910370.04130.951680.03220.983890.00910.9929100.00711


## 5.3. Simulation Experiments

In this paper, five input variables after KPCA processing are used to predict the content of ore granularity to200 mesh during the grinding and classifying process. This paper collected 1,700 samples from an existing database, of which 1500 were used as training samples and 200 were used as test samples. In order to make it clear that the prediction accuracy of the soft-sensor models proposed in this paper is higher than the prediction accuracy of other models, three quantitative indicators were selected to describe the prediction accuracy of the soft-sensor model, whose calculation methods of these three quantitative indicators are listed in Table 5. Many scholars use these three quantitative indicators to evaluate the predictive performance of the model [24, 25, 29, 30]. In Table 5, n represents the number of test samples, y^i represents the predicted value of the objective variables, yi represents the true value of the objective variables, and y¯ represents the average of all objective variables. The smaller MAE and RMSE, the higher the prediction accuracy of the model. The larger R2, the stronger the fitting ability of the soft-sensor model.Table 5
Quantitative indicators of soft-sensor models.
IndexFunctionMean absolute errorMAE=1/n∑i=1ny^i−yiRoot mean square errorRMSE=1/n∑i=1ny^i−yi21/2R2R2=∑i=1ny^i−y¯2/∑i=1nyi−y¯2


### 5.3.1. GSHHO Algorithm to Optimize LSSVM Model

In order to study the optimization capability of GSHHO algorithm for LSSVM with different kernel functions, this paper uses GSHHO algorithm to optimize the four different LSSVMs listed in Table1. The traditional three-layer crossvalidation method (CV) and HHO-optimized LSSVM were used as comparative experimental results to prove the superiority of GSHHO algorithm in this paper. The population of GSHHO algorithm and HHO algorithm is set to 30 and the number of iterations is set to 100. Figures 7–10 show the results that these three methods optimize the LSSVM parameters, where (a) of each figure represents the prediction results of the soft-sensor model, and (b) represents the prediction error of the soft-sensor model. In order to reflect the optimization capabilities of GSHHO algorithm and HHO algorithm, the convergence curves that GSHHO algorithm and HHO algorithm optimize different error functions are shown in Figure 11. The quantitative indicator results of ore granularity predicted by CV-LSSVM, GSHHO-LSSVM, and HHO-LSSVM are listed in Table 6 to further illustrate the prediction accuracy of different soft-sensor models.Figure 7
Prediction results under LSSVM model withlinear function. (a) Prediction results under different LSSVM models. (b) Prediction errors under different LSSVM models.
(a)(b)Figure 8
Prediction results under LSSVM model withploy function. (a) Prediction results under different LSSVM models. (b) Prediction errors under different LSSVM models.
(a)(b)Figure 9
Prediction results under LSSVM model withrbf function. (a) Prediction results under different LSSVM models. (b) Prediction errors under different LSSVM models.
(a)(b)Figure 10
Prediction results under LSSVM model withsigmoid function. (a) Prediction results under different LSSVM models. (b) Prediction errors under different LSSVM models.
(a)(b)Figure 11
Convergence curves of error functions under four different LSSVM models.
(a)(b)(c)(d)Table 6
Quantitative indicators under different soft-sensor models.
Kernel functionAlgorithmMAERMSER2LinearCV0.6860.73270.202HHO0.68610.73270.2024GSHHO0.68610.73270.2024PloyCV0.53680.4360.5763HHO0.53630.43480.5786GSHHO0.53630.43470.5787rbfCV0.36420.22960.7812HHO0.34120.23510.7731GSHHO0.34530.21910.79SigmoidCV0.64691.18390.3368HHO0.72331.44880.2213GSHHO0.58410.58390.4942It can be seen from the experimental results in these figures and tables that LSSVM with different kernel functions show different prediction capabilities. From Figures8 and 9 and Table 6, it can be seen that compared with traditional crossvalidation, the optimization capabilities of GSHHO algorithm and HHO algorithm for LSSVM with linear functions and ploy function do not show great advantages, the prediction results of these three methods are not ideal, and it is impossible to perform real-time monitoring of the particle size of the ore. From Figure 10 and Table 6, for LSSVM with sigmoid function, the optimization capabilities of GSHHO algorithm are significantly better than HHO and CV. Compared with traditional HHO algorithm, GSHHO algorithm has both strong exploration and exploitation. It can be seen from the convergence curve of Figure 11(d) that GSHHO can find more suitable minimum value for the error function in equation (28), which makes prediction accuracy of GSHHO-LSSVM higher. However, compared with the LSSVM with rbf function, the prediction result of LSSVM with sigmoid function is still not optimal. Combining the quantitative indicators in Table 5 and the results in Figures 9 and 10, it can be seen that the prediction result of LSSVM with rbf optimized by CV is better than with sigmoid optimized by GSHHO algorithm. Looking further at the results in Figure 10 and Table 6, it can be seen that compared with HHO-LSSVM and CV-LSSVM, GSHHO-LSSVM with rbf function shows better performance. By the way, in the convergence curve of Figure 11, when GSHHO algorithm optimizes the error function, the convergence speed is faster and the convergence accuracy is higher. Table 7 shows the parameter values found by GSHHO algorithm and HHO algorithm when optimizing different soft-sensor models. Combining all the above conclusions, it can be concluded that GSHHO algorithm has a great advantage in optimizing the LSSVM model, and the optimization result of the LSSVM model with rbf function is the best.Table 7
Parameter values found by the algorithms.
Kernel functionAlgorithmParameter valueLinearHHOC=124.36546GSHHOC=148.61254PloyHHOC=85.31564,θ=3.2146GSHHOC=98.63345,θ=1.36543rbfHHOC=53.24514,σ2=1.12283GSHHOC=61.34664,σ2=0.71341SigmoidHHOC=253.54376,k=0.86543GSHHOC=257.77324,k=0.36424


### 5.3.2. Simulation Experiments of Regularized Extreme Learning Machine Model

As explained in Section2.2, RELM is a neural network model with strong generalization ability and high implementation efficiency. This section studies the prediction accuracy of RELM for ore granularity. GSHHO-LSSVM with rbf function has been proved to be a prediction model with good prediction accuracy, so this GSHHO-LSSVM will be compared with the prediction result of the RELM model. The traditional ELM experimental results will be used as a comparison of the RELM model. The number of hidden layer neurons in the ELM model and RELM model is 45.sigma function is used as the kernel function of ELM and RELM, and regularized parameter 1/λ of RELM is 0.001. In the case of stable industrial status, the GSLBH-ELM proposed by Xie et al. [23] has been proven to be an effective soft-sensor model. When predicting the ore granularity, GSLBH-ELM with arctan function has the highest prediction accuracy. This paper will also use GSLBH-ELM with arctan function to predict the ore granularity when this industrial status is constantly changing, and the results obtained will be compared with the experimental results of RELM, ELM, and LSSVM. The experimental results of these four models are shown in Figure 12 and Table 8.Figure 12
Experimental results under different ELM and LSSVM models. (a) Prediction results under different soft-sensor models. (b) Prediction errors under different soft-sensor models. (c) Predicted scatter plots under different soft-sensor models.
(a)(b)(c)Table 8
Quantitative indicators under different soft-sensor models.
ModelMAERMSER2ELM0.45250.31830.7101RELM0.44880.30670.7209LSSVM0.34530.21910.7901GSLBH-ELM0.74030.0267−0.5167KNN0.35320.23070.7904It can be seen from the experimental results that the GSLBH-ELM model does not show good prediction ability. The main reason is that the number of hidden layer neurons selected by GSLBH-ELM is too small. The small number of neurons does not reflect the dynamic industrial characteristics of this experiment. So, GSLBH algorithm cannot find more reasonable weights and biases. If the number of neurons in the GSLBH-ELM is increased, the parameters that the algorithm needs to be optimized will increase geometrically and the time required to train the model will be too long, which is contrary to the efficiency of the soft-sensor model. Therefore, when the industrial process changes dynamically, GSLBH-ELM is not suitable for predicting the ore granularity. Combining the results shown in Figure12 and Table 8, the prediction performance of RELM is slightly better than ELM, especially in Figure 12(c), and it can be clearly seen that the prediction value of RELM is more accurate than ELM at some moments. However, the prediction accuracy of GSHHO-LSSVM is still higher than that of RELM and ELM.In order to study the ability of the neural network model trained with the extended Kalman filter (KNN) to predict the granularity in the grinding and classifying process, this paper compares the prediction results of KNN with RELM and LSSVM. In this experiment, the structural parameters of the KNN model and the hyperparameters of the Kalman filter method are shown in Table9, where I represents the identity matrix, the size of the matrix P is the square of the number of model parameters, and std=0.01, the size of the matrix Q is the square of the number of output. The experimental results are shown in Figure 13, and the quantitative indicators of the prediction results are shown in Table 8. As can be seen from the results in Figure 13 and Table 8, compared with RELM and LSSVM, KNN does not show a large advantage, and its prediction accuracy is better than RELM, but still worse than LSSVM.Table 9
The structural parameters of the KNN model and the hyperparameters of the Kalman filter method.
(a) Neural network model structure parametersParameterNumber of model inputsNumber of hidden neuronsNumber of model outputNumber of iterationsValue5101200(b) Kalman filter parametersParameterStep of update weightInitial weight covariancePData covarianceRProcess covarianceQValue1P=0.5∗IR=0.1+std2∗IQ=0Figure 13
Experimental results under KNN and other models. (a) Prediction results under different soft-sensor models. (b) Prediction errors under different soft-sensor models. (c) Predicted scatter plots under different soft-sensor models.
(a)(b)(c)


### 5.3.3. Simulation Experiment of Hybrid Soft-Sensor Model Based on Moving Window Strategy

Under the circumstances that the industrial status is changing at any time, this paper introduces moving window strategy into the hybrid soft-sensor model to improve the prediction accuracy of the model. The operation of this hybrid model has been described in details in Section4.2. The window size in this experiment is 90, and the step size is 15. The LSSVM in the hybrid model uses the best performing model (the kernel function is rbf kernel, and the structural parameters are those found by GSHHO in Table 6) described in 5.3.2, the parameters of the RLEM model are those selected in Section 5.3.3. At the same time, the moving window strategy is introduced into the LSSVM model (MW-LSSVM), the RELM model (MW-RELM), and the ELM model (MW-ELM) as comparative experiments. The experimental results are all shown in Figures 14 and 15 and Tables 10 and 11.Figure 14
Prediction results under MW-ELM model and MW-RELM model. (a) Prediction results under different soft-sensor models. (b) Prediction errors under different soft-sensor models. (c) Predicted scatter plots under different soft-sensor models.Figure 15
Experimental results under different hybrid models. (a) Prediction results under different soft-sensor models. (b) Prediction errors under different soft-sensor models. (c) Predicted scatter plots under different soft-sensor models.
(a)(b)(c)Table 10
Quantitative indicators under different soft-sensor models.
ModelMAERMSER2MW-ELM0.41041.54810.5021MW-RELM0.28590.15520.8765Table 11
Quantitative indicators under different models.
ModelMAERMSER2MW-RELM0.28590.15520.8765MW-LSSVM0.24390.15340.8651MW-LSSVM-RELM0.20940.10030.9175MW-KNN0.23860.13730.8886From the experimental results in Figure14 and Table 10, it can be seen that when building a local soft-sensing model, the MW-ELM model is very prone to overfitting due to the small number of training samples, there are some abnormal points in the test results of MW-ELM, and the prediction accuracy of the model is greatly reduced, even worse than the ELM prediction results in Section 5.3.3. MW-RELM overcomes the shortcomings of MW-ELM, and the introduction of regularization parameters 1/λ makes the model avoid the interference of noise and greatly improves the generalization ability of the model. Because MW-RELM can adaptively adjust the structural parameters of the model according to changes in the working conditions, compared with the single model in Section 5.3.3, the prediction accuracy of MW-RELM has been greatly improved. From the results of Figure 15 and Table 11, it can be seen that the prediction results of MW-LSSVM are slightly better than MW-RELM. The hybrid model MW-LSSVM-RELM combines the advantages of LSSVM and RELM under different industrial status which can further improve the prediction accuracy of the model and achieve the purpose of online prediction of the ore granularity in a dynamic industrial environment.In order to find out which model is selected every time the window slides, this article shows the model switching process in Figure16, where “1” indicates that the LSSVM model is used as the local soft-sensor model and “−1” indicates that the RELM model is used as the local soft-sensing model. It can be seen from the results in Figure 16 that the LSSVM model is used as the local soft-sensing model is the majority, and the role of the RELM model is to “revise” when LSSVM performs poorly. In order to explore the working efficiency of the MW-LSSVM-RELM model, the training time required for the MW-LSSVM-RELM model to predict a new query sample is listed in Table 12. The results in Table 12 show that although the training time of MW-LSSVM-RELM is longer than that of MW-RELM and MW-LSSVM, its training time is still very short and the model is extremely efficient, much less than the acceptable training time.Figure 16
Switching process of different models.Table 12
Training time under deterrent soft-sensor models.
ModelTraining time(s)MW-RELM0.0043MW-LSSVM0.0086MW-LSSVM-RELM0.0111MW-KNN2.0076Based on the above experimental results, it can be seen that the MW-LSSVM-RELM model has better prediction ability than other single soft-senor models and models that introduce the MW strategy into a single soft-sensor model. The MW-LSSVM-RELM model has better prediction accuracy and higher work efficiency and can achieve online prediction of ore granularity.It can be seen from the experimental results in Figure13 and Table 9 that after the extended Kalman filtering method is used to adjust the parameters of the neural network, the neural network has good generalization ability, and its prediction accuracy is even better than RELM. In this section, in order to study the influence of moving window strategy on KNN prediction accuracy, this paper introduces moving window strategy into KNN. The window size and sliding steps are the same as MW-LSSVM-ELM. In order to improve the working efficiency of the model, the number of training iterations is set to 100, and other parameters remain unchanged. The experimental results of MW-KNN are shown in Figure 17 and Table 11. It can be seen from the results that the prediction accuracy of KNN has been greatly improved after the introduction of the sliding window strategy. Prediction accuracy of KNN is better than MW-RELM and MW-LSSVM, but still not as good as MW-LSSVM-ELM. Compared with MW-LSSVM-ELM, MW-KNN has a major disadvantage, that is, it is less efficient. From the results in Table 12, it can be seen that although the number of iterations of KNN changes from 200 to 100, the time for MW-KNN to predict a test sample is still much longer than MW-LSSVM-ELM. From two aspects of work efficiency and prediction accuracy, the performance of the MW-LSSVM-ELM model is better than that of MW-KNN.Figure 17
Experimental results under different hybrid models.
(a)(b)(c)Combining all the above results, it can be concluded that the predictive ability of the MW-LSSVM-ELM model is superior to ELM, RELM, LSSVM, and KNN, even if these four models introduce moving window strategy, the prediction accuracy of MW-LSSVM-ELM is still better than them, and MW-LSSVM-ELM is more efficient than other models.


## 5.3.1. GSHHO Algorithm to Optimize LSSVM Model

In order to study the optimization capability of GSHHO algorithm for LSSVM with different kernel functions, this paper uses GSHHO algorithm to optimize the four different LSSVMs listed in Table1. The traditional three-layer crossvalidation method (CV) and HHO-optimized LSSVM were used as comparative experimental results to prove the superiority of GSHHO algorithm in this paper. The population of GSHHO algorithm and HHO algorithm is set to 30 and the number of iterations is set to 100. Figures 7–10 show the results that these three methods optimize the LSSVM parameters, where (a) of each figure represents the prediction results of the soft-sensor model, and (b) represents the prediction error of the soft-sensor model. In order to reflect the optimization capabilities of GSHHO algorithm and HHO algorithm, the convergence curves that GSHHO algorithm and HHO algorithm optimize different error functions are shown in Figure 11. The quantitative indicator results of ore granularity predicted by CV-LSSVM, GSHHO-LSSVM, and HHO-LSSVM are listed in Table 6 to further illustrate the prediction accuracy of different soft-sensor models.Figure 7
Prediction results under LSSVM model withlinear function. (a) Prediction results under different LSSVM models. (b) Prediction errors under different LSSVM models.
(a)(b)Figure 8
Prediction results under LSSVM model withploy function. (a) Prediction results under different LSSVM models. (b) Prediction errors under different LSSVM models.
(a)(b)Figure 9
Prediction results under LSSVM model withrbf function. (a) Prediction results under different LSSVM models. (b) Prediction errors under different LSSVM models.
(a)(b)Figure 10
Prediction results under LSSVM model withsigmoid function. (a) Prediction results under different LSSVM models. (b) Prediction errors under different LSSVM models.
(a)(b)Figure 11
Convergence curves of error functions under four different LSSVM models.
(a)(b)(c)(d)Table 6
Quantitative indicators under different soft-sensor models.
Kernel functionAlgorithmMAERMSER2LinearCV0.6860.73270.202HHO0.68610.73270.2024GSHHO0.68610.73270.2024PloyCV0.53680.4360.5763HHO0.53630.43480.5786GSHHO0.53630.43470.5787rbfCV0.36420.22960.7812HHO0.34120.23510.7731GSHHO0.34530.21910.79SigmoidCV0.64691.18390.3368HHO0.72331.44880.2213GSHHO0.58410.58390.4942It can be seen from the experimental results in these figures and tables that LSSVM with different kernel functions show different prediction capabilities. From Figures8 and 9 and Table 6, it can be seen that compared with traditional crossvalidation, the optimization capabilities of GSHHO algorithm and HHO algorithm for LSSVM with linear functions and ploy function do not show great advantages, the prediction results of these three methods are not ideal, and it is impossible to perform real-time monitoring of the particle size of the ore. From Figure 10 and Table 6, for LSSVM with sigmoid function, the optimization capabilities of GSHHO algorithm are significantly better than HHO and CV. Compared with traditional HHO algorithm, GSHHO algorithm has both strong exploration and exploitation. It can be seen from the convergence curve of Figure 11(d) that GSHHO can find more suitable minimum value for the error function in equation (28), which makes prediction accuracy of GSHHO-LSSVM higher. However, compared with the LSSVM with rbf function, the prediction result of LSSVM with sigmoid function is still not optimal. Combining the quantitative indicators in Table 5 and the results in Figures 9 and 10, it can be seen that the prediction result of LSSVM with rbf optimized by CV is better than with sigmoid optimized by GSHHO algorithm. Looking further at the results in Figure 10 and Table 6, it can be seen that compared with HHO-LSSVM and CV-LSSVM, GSHHO-LSSVM with rbf function shows better performance. By the way, in the convergence curve of Figure 11, when GSHHO algorithm optimizes the error function, the convergence speed is faster and the convergence accuracy is higher. Table 7 shows the parameter values found by GSHHO algorithm and HHO algorithm when optimizing different soft-sensor models. Combining all the above conclusions, it can be concluded that GSHHO algorithm has a great advantage in optimizing the LSSVM model, and the optimization result of the LSSVM model with rbf function is the best.Table 7
Parameter values found by the algorithms.
Kernel functionAlgorithmParameter valueLinearHHOC=124.36546GSHHOC=148.61254PloyHHOC=85.31564,θ=3.2146GSHHOC=98.63345,θ=1.36543rbfHHOC=53.24514,σ2=1.12283GSHHOC=61.34664,σ2=0.71341SigmoidHHOC=253.54376,k=0.86543GSHHOC=257.77324,k=0.36424


## 5.3.2. Simulation Experiments of Regularized Extreme Learning Machine Model

As explained in Section2.2, RELM is a neural network model with strong generalization ability and high implementation efficiency. This section studies the prediction accuracy of RELM for ore granularity. GSHHO-LSSVM with rbf function has been proved to be a prediction model with good prediction accuracy, so this GSHHO-LSSVM will be compared with the prediction result of the RELM model. The traditional ELM experimental results will be used as a comparison of the RELM model. The number of hidden layer neurons in the ELM model and RELM model is 45.sigma function is used as the kernel function of ELM and RELM, and regularized parameter 1/λ of RELM is 0.001. In the case of stable industrial status, the GSLBH-ELM proposed by Xie et al. [23] has been proven to be an effective soft-sensor model. When predicting the ore granularity, GSLBH-ELM with arctan function has the highest prediction accuracy. This paper will also use GSLBH-ELM with arctan function to predict the ore granularity when this industrial status is constantly changing, and the results obtained will be compared with the experimental results of RELM, ELM, and LSSVM. The experimental results of these four models are shown in Figure 12 and Table 8.Figure 12
Experimental results under different ELM and LSSVM models. (a) Prediction results under different soft-sensor models. (b) Prediction errors under different soft-sensor models. (c) Predicted scatter plots under different soft-sensor models.
(a)(b)(c)Table 8
Quantitative indicators under different soft-sensor models.
ModelMAERMSER2ELM0.45250.31830.7101RELM0.44880.30670.7209LSSVM0.34530.21910.7901GSLBH-ELM0.74030.0267−0.5167KNN0.35320.23070.7904It can be seen from the experimental results that the GSLBH-ELM model does not show good prediction ability. The main reason is that the number of hidden layer neurons selected by GSLBH-ELM is too small. The small number of neurons does not reflect the dynamic industrial characteristics of this experiment. So, GSLBH algorithm cannot find more reasonable weights and biases. If the number of neurons in the GSLBH-ELM is increased, the parameters that the algorithm needs to be optimized will increase geometrically and the time required to train the model will be too long, which is contrary to the efficiency of the soft-sensor model. Therefore, when the industrial process changes dynamically, GSLBH-ELM is not suitable for predicting the ore granularity. Combining the results shown in Figure12 and Table 8, the prediction performance of RELM is slightly better than ELM, especially in Figure 12(c), and it can be clearly seen that the prediction value of RELM is more accurate than ELM at some moments. However, the prediction accuracy of GSHHO-LSSVM is still higher than that of RELM and ELM.In order to study the ability of the neural network model trained with the extended Kalman filter (KNN) to predict the granularity in the grinding and classifying process, this paper compares the prediction results of KNN with RELM and LSSVM. In this experiment, the structural parameters of the KNN model and the hyperparameters of the Kalman filter method are shown in Table9, where I represents the identity matrix, the size of the matrix P is the square of the number of model parameters, and std=0.01, the size of the matrix Q is the square of the number of output. The experimental results are shown in Figure 13, and the quantitative indicators of the prediction results are shown in Table 8. As can be seen from the results in Figure 13 and Table 8, compared with RELM and LSSVM, KNN does not show a large advantage, and its prediction accuracy is better than RELM, but still worse than LSSVM.Table 9
The structural parameters of the KNN model and the hyperparameters of the Kalman filter method.
(a) Neural network model structure parametersParameterNumber of model inputsNumber of hidden neuronsNumber of model outputNumber of iterationsValue5101200(b) Kalman filter parametersParameterStep of update weightInitial weight covariancePData covarianceRProcess covarianceQValue1P=0.5∗IR=0.1+std2∗IQ=0Figure 13
Experimental results under KNN and other models. (a) Prediction results under different soft-sensor models. (b) Prediction errors under different soft-sensor models. (c) Predicted scatter plots under different soft-sensor models.
(a)(b)(c)


## 5.3.3. Simulation Experiment of Hybrid Soft-Sensor Model Based on Moving Window Strategy

Under the circumstances that the industrial status is changing at any time, this paper introduces moving window strategy into the hybrid soft-sensor model to improve the prediction accuracy of the model. The operation of this hybrid model has been described in details in Section4.2. The window size in this experiment is 90, and the step size is 15. The LSSVM in the hybrid model uses the best performing model (the kernel function is rbf kernel, and the structural parameters are those found by GSHHO in Table 6) described in 5.3.2, the parameters of the RLEM model are those selected in Section 5.3.3. At the same time, the moving window strategy is introduced into the LSSVM model (MW-LSSVM), the RELM model (MW-RELM), and the ELM model (MW-ELM) as comparative experiments. The experimental results are all shown in Figures 14 and 15 and Tables 10 and 11.Figure 14
Prediction results under MW-ELM model and MW-RELM model. (a) Prediction results under different soft-sensor models. (b) Prediction errors under different soft-sensor models. (c) Predicted scatter plots under different soft-sensor models.Figure 15
Experimental results under different hybrid models. (a) Prediction results under different soft-sensor models. (b) Prediction errors under different soft-sensor models. (c) Predicted scatter plots under different soft-sensor models.
(a)(b)(c)Table 10
Quantitative indicators under different soft-sensor models.
ModelMAERMSER2MW-ELM0.41041.54810.5021MW-RELM0.28590.15520.8765Table 11
Quantitative indicators under different models.
ModelMAERMSER2MW-RELM0.28590.15520.8765MW-LSSVM0.24390.15340.8651MW-LSSVM-RELM0.20940.10030.9175MW-KNN0.23860.13730.8886From the experimental results in Figure14 and Table 10, it can be seen that when building a local soft-sensing model, the MW-ELM model is very prone to overfitting due to the small number of training samples, there are some abnormal points in the test results of MW-ELM, and the prediction accuracy of the model is greatly reduced, even worse than the ELM prediction results in Section 5.3.3. MW-RELM overcomes the shortcomings of MW-ELM, and the introduction of regularization parameters 1/λ makes the model avoid the interference of noise and greatly improves the generalization ability of the model. Because MW-RELM can adaptively adjust the structural parameters of the model according to changes in the working conditions, compared with the single model in Section 5.3.3, the prediction accuracy of MW-RELM has been greatly improved. From the results of Figure 15 and Table 11, it can be seen that the prediction results of MW-LSSVM are slightly better than MW-RELM. The hybrid model MW-LSSVM-RELM combines the advantages of LSSVM and RELM under different industrial status which can further improve the prediction accuracy of the model and achieve the purpose of online prediction of the ore granularity in a dynamic industrial environment.In order to find out which model is selected every time the window slides, this article shows the model switching process in Figure16, where “1” indicates that the LSSVM model is used as the local soft-sensor model and “−1” indicates that the RELM model is used as the local soft-sensing model. It can be seen from the results in Figure 16 that the LSSVM model is used as the local soft-sensing model is the majority, and the role of the RELM model is to “revise” when LSSVM performs poorly. In order to explore the working efficiency of the MW-LSSVM-RELM model, the training time required for the MW-LSSVM-RELM model to predict a new query sample is listed in Table 12. The results in Table 12 show that although the training time of MW-LSSVM-RELM is longer than that of MW-RELM and MW-LSSVM, its training time is still very short and the model is extremely efficient, much less than the acceptable training time.Figure 16
Switching process of different models.Table 12
Training time under deterrent soft-sensor models.
ModelTraining time(s)MW-RELM0.0043MW-LSSVM0.0086MW-LSSVM-RELM0.0111MW-KNN2.0076Based on the above experimental results, it can be seen that the MW-LSSVM-RELM model has better prediction ability than other single soft-senor models and models that introduce the MW strategy into a single soft-sensor model. The MW-LSSVM-RELM model has better prediction accuracy and higher work efficiency and can achieve online prediction of ore granularity.It can be seen from the experimental results in Figure13 and Table 9 that after the extended Kalman filtering method is used to adjust the parameters of the neural network, the neural network has good generalization ability, and its prediction accuracy is even better than RELM. In this section, in order to study the influence of moving window strategy on KNN prediction accuracy, this paper introduces moving window strategy into KNN. The window size and sliding steps are the same as MW-LSSVM-ELM. In order to improve the working efficiency of the model, the number of training iterations is set to 100, and other parameters remain unchanged. The experimental results of MW-KNN are shown in Figure 17 and Table 11. It can be seen from the results that the prediction accuracy of KNN has been greatly improved after the introduction of the sliding window strategy. Prediction accuracy of KNN is better than MW-RELM and MW-LSSVM, but still not as good as MW-LSSVM-ELM. Compared with MW-LSSVM-ELM, MW-KNN has a major disadvantage, that is, it is less efficient. From the results in Table 12, it can be seen that although the number of iterations of KNN changes from 200 to 100, the time for MW-KNN to predict a test sample is still much longer than MW-LSSVM-ELM. From two aspects of work efficiency and prediction accuracy, the performance of the MW-LSSVM-ELM model is better than that of MW-KNN.Figure 17
Experimental results under different hybrid models.
(a)(b)(c)Combining all the above results, it can be concluded that the predictive ability of the MW-LSSVM-ELM model is superior to ELM, RELM, LSSVM, and KNN, even if these four models introduce moving window strategy, the prediction accuracy of MW-LSSVM-ELM is still better than them, and MW-LSSVM-ELM is more efficient than other models.


## 6. Conclusions

This paper first introduces the structure and implementation procedure of LSSVM model and RELM model in details, then introduces the HHO algorithm and proposes the GSHHO algorithm, and also introduces the optimization process of LSHVM by GSHHO. In order to improve the adaptive ability of soft-sensor models, this paper adopts MW strategy into the LSSVM model and the RELM model to revise the structural parameters of the models. Based on MW technology, this paper proposes the MW-LSSVM-RELM model to further improve the prediction accuracy of the models. The following conclusions are drawn through the final simulation experiments results:(1)
Compared with the traditional CV method and HHO algorithm, GSHHO algorithm can find more suitable structural parameters for LSSVM, which makes the prediction accuracy of LSSVM higher(2)
The introduction of the MW strategy into the LSSVM model, the RELM model, and KNN model can greatly improve the ability of models to respond to changes of the industrial status and make these models maintain good generalization ability in different states(3)
The MW-LSSVM-RELM model proposed in this paper combines the advantages of the LSSVM model and the RELM model in different industrial status, which can make this hybrid model have better prediction capabilities than a single soft-sensor modelIn the database of the grinding and classifying process, there are many samples where data are missing for various reasons. Conventional soft-sensor models cannot take advantage of this data-missing sample, which contains a lot of key information that can characterize the state of the industry. Abandoning these samples means losing a lot of useful information and wasting data resources. Based on the existing research work, this team will focus on deep neural network models because deep neural networks can better describe the strong nonlinear and dynamic changes of industrial processes, and the team will study the semisupervised model of the existing soft-sensor model so that these data-missing samples can be used for high-quality online prediction of key variables in the grinding and classification process.



---

*Source: 6457517-2020-05-28.xml*