# Soft Sensor Development Based on Quality-Relevant Slow Feature Analysis and Bayesian Regression with Application to Propylene Polymerization

Authors: Miao Zhang; Le Zhou; Jing Jie; Xiaoli Wu
Journal: Journal of Sensors
 (2021)

---

## Abstract

Data-driven soft sensors are widely used to predict quality indices in propylene polymerization processes to improve the availability of measurements and efficiency. To deal with the nonlinearity and dynamics in propylene polymerization processes, a novel soft sensor based on quality-relevant slow feature analysis and Bayesian regression is proposed in this paper. The proposed method can handle the dynamics of the process better by extracting quality-relevant slow features, which present both the slowly varying characteristic and the correlations with quality indices. Meanwhile, a Bayesian inference model is developed to predict the quality indices, which takes advantages of a probability framework with iterative maximum likelihood techniques for parameter estimation and a sparse constraint for avoiding overfitting. Finally, a case study is conducted with data sampled from a practical industrial propylene polymerization process to demonstrate the effectiveness and superiority of the proposed method.


---

## Body

## 1. Introduction

As modern process industries become larger scale and more integrated, pivotal key performance indices about product quality, process safety, and pollution reduction should be closely monitored [1–4]. Besides, real-time measuring of quality indices is important for process monitoring, control, and optimizing products [5–7]. Thermoplastic polymers such as polypropylene are important materials and have been used in various sectors. The melt index (MI) which determines the grade of the polymer product is considered one of the most crucial indicators in quality control for industrial propylene polymerization (PP) processes. Up to date, an accurate first-principle MI model is still not available. The MI is usually evaluated offline with an analytical procedure in the laboratory which takes 1.5-2 h to complete. This can result in a delay in the quality control system as there is no available quality indicator at this time. In the absence of an economical or effective online measurement, soft sensors could serve as an alternative solution [8–14]. Additionally, with the wide availability of process data in PP processes, increasing data-driven soft sensors have been adopted to predict the MI.Data-driven approaches are used as an alternative to the mathematical model. The data reading from sensors can often contain a large number of variables that describe the same process phenomena. When encountering high-dimensional data, it is useful to project the data into the latent space, which is more compact than the original space. For MI prediction, the principal component analysis (PCA) [15] and partial least squares (PLS) [16] have been applied. Unfortunately, as the model is linear, it is inadequate when the soft sensing variables are nonlinear. To this end, nonlinear modeling methods are used in MI prediction, such as artificial neural networks (ANNs) [17], support vector machines (SVMs) [18], Gaussian process regression (GPR) [19, 20], and relevance vector machine (RVM) [21] [22–25]. Recently, Liu et al. proposed an adversarial transfer learning- (ATL-) based soft sensor [26] and a domain adaptation transfer learning soft sensor for product quality prediction [7]. As classical methods, PCA and PLS have achieved great successes with respect to quality prediction by extracting latent variables (LVs) [27–29]. However, they can only be used to extract static LVs, which are unsuitable for modeling dynamic processes because they are limited by containing temporally related process information.In practical chemical processes, frequent fluctuations of equipment characteristics with time always result in dynamic processes. For soft sensing of quality indices, dynamic data characteristic could play an important role in regression, the result of which may highly influence the performance for the quality control system. A new merged unsupervised algorithm, slow feature analysis (SFA) [30], is applied in our work for its remarkable ability to extract slowly varying and temporally related features for modeling dynamical processes. SFA has been extensively used for various reasons, such as blind source separation [31], signal processing [32], and process monitoring [33–36]. Unlike the static models generated by PCA or PLS, SFA-based models can better describe process temporal behaviors through the extraction of slowness of LVs. When used for modeling, the extracted slowly varying LVs will be termed as process intrinsic properties, and the fast-varying LVs are seen as process noise inversely. Up to now, some SFA-based methods have been applied in prediction modeling. Shang et al. [37] proposed probabilistic slow feature analysis (PSFA) based a soft sensor model for quality prediction. Fan et al. [38] proposed a robust PSFA- (RPSFA-) based regression model that models outliers in the observation data using Student’s t-distribution. Zhong et al. [39] put forward an online quality prediction method based on modified regularized SFA (ReSFA). Jiang et al. [40] proposed a real-time semisupervised predictive modeling strategy for industrial continuous catalytic reforming process using SFA. However, traditional slow features are calculated only considering slowness of process variations, while their correlations with quality indices are neglected for feature extraction. Slow features are extracted and then used for quality prediction by performing regression using the ordinary least square, which means that they may not describe nonlinear relationship among variables well. Considering the nonlinearity of the propylene polymerization process, using nonlinear regression modeling method is quite necessary.Hence, to deal with the nonlinearity and dynamics in PP processes, a soft sensor based on quality-relevant slow feature analysis and Bayesian regression is proposed in this paper. First, the quality-relevant slow feature analysis (QSFA) is adopted to extract slow features which present both the slowly varying characteristic and the correlations with quality indices. Then, the extracted quality-relevant slow features (QSFs) are sorted according to the correlation index and selected for regression modeling by the cross-validation method. Based on the selected QSFs, a Bayesian inference framework named relevance vector regression (RVR) is developed to predict the quality variable, i.e., MI for the polypropylene products. It takes advantages of a probability framework with iterative maximum likelihood techniques for parameter estimation and a sparse constraint for avoiding overfitting. Finally, the effectiveness of the proposed method (QSFA-RVR) is confirmed through real data obtained from industrial propylene polymerization plants.The rest of the paper is organized as follows. Section2 presents the methods for the development of the proposed soft sensor. First, a brief introduction of the traditional SFA is presented. Then, the formulation for the QSFA-RVR prediction model is presented, which contains the QSFA method, the relevance vector regression, and the quality prediction method based on QSFA-RVR. Experiment results on real data from practical industry processes are analyzed and discussed in Section 3. Finally, the conclusion is drawn in Section 4.


## 2. Methods

### 2.1. Overview of Slow Feature Analysis

SFA is an unsupervised algorithm proposed by Wiskott and Sejnowski [30], which extracts invariant features from quickly varying signals. The slow variations are always termed as intrinsic features of signals. Meanwhile, rapid variations often allude to measurement noise. The detailed methodology of SFA is illustrated as follows [41].Given aJ-dimensional input signal X, SFA is aimed at finding mapping functions from input to slow features sj=gjX,j∈1,2,⋯,J that minimize
(1)mingj⋅Δsj≔ṡj2,under the constraints
(2)ṡjgj⋅j=1Jsj=0zeromean,sj2=1unitvariance,∀i≠j,sisj=0decorrelationandorder,where⋅ denotes time averaging and denotes the first-order derivative of sj with respect to time.To simplify this optimization problem, it is assumed that the SFs are a linear combination of input variables. The linear mapping is(3)sj=Xwj.The optimization problem in (1) can be solved by two steps of singular value decomposition (SVD) [42], which is tantamount to a numerical solution to the generalized eigenvalue decomposition (GED) problem in
(4)AW=BWΩ,where A=ẊẊT denotes the covariance matrix of the first-order derivative of X and B=XXT denotes the covariance matrix of X; W=w1,w2,⋯wJT is the matrix of generalized eigenvectors, which are coefficient vectors of linear mappings; and Ω=diagλ1,⋯,λJ is a diagonal matrix with λj=ṡj2 being generalized eigenvalues, which are arranged in an ascending order.By performing a first SVD onB with equation (5), the original input X is whitened as Eq. (6).
(5)B=UΛUT,(6)Z=Λ−1/2UTX,where U is an orthogonal matrix, Λ is a diagonal matrix, and covZ=ZZT=IJ. Then, a second SVD on the covariance of the first-order derivative of Z yields
(7)ŻŻT=PTΩP,where P is an orthogonal matrix.Thus, the weighting matrixW can be calculated, and the slow features s can be obtained using equation (3).


### 2.2. Quality-Relevant Slow Feature Analysis

Conventional SFA only considers slowness of process variations when extracting slow features. SFA’s competence to retain process information containing quality indices is limited. An intuition is presented to integrate the relationships between slow features and quality indices into SFA’s optimization problem. In this way, an improved SFA algorithm, named quality-relevant slow feature analysis (QSFA), can extract quality-relevant slow features (QSF) from a process, and these slow features can better reveal the essence of the process. Moreover, a new objective function for SFA should be designed to enable the QSFA that considers both slowness of process variations and their correlations with quality indices. Given the normalized process variablesX and the quality index y, a new objective function Jw composed of three subobjective functions is designed. The first subobjective function J1 which considers that the QSF should change slowly is given as follows:
(8)minJ1w=Δṡ=ẊwTẊw,where s is the QSF and w is the corresponding weighting vector. Considering that the QSF should be highly correlated with the quality index, the second subobjective function J2 is aimed at maximizing the correlation between s and y, which is given as follows:
(9)maxJ2w=sTyyTsvars=XwTyyTXwXwTXw.The third subobjective functionJ3 is used to maximize the variance of s which represents the variation information of the process.
(10)maxJ3w=vars=XwTXw.Thus, a new optimization problem of QSFA can be described as(11)maxJw=J2J3J1=XwTyyTXw,s.t.ẊwTẊw=1wTw=1.By introducing Lagrange multipliersλ and δ, equation (14) can be easily obtained. Obviously, the value of the objective function Jw is equal to λ+δ.
(12)Lw=wTXTyyTXw−λwTẊTẊw−1−δwTw−1,(13)∂Lw∂w=2XTyyTXw−2λẊTẊw−2δw=0,(14)wTXTyyTXw=λ+δ,Therefore, we can rewrite the solution of equation (13) as the following GED problem [43]:
(15)XTyyTXw=λẊTẊ+δIw,where I is a J-dimensional identity matrix.By performing SVD decomposition on the covariance matrixẊTẊ, we have
(16)ẊTẊ=UΛUT.The whitening transformation with the whitening matrixH is described as follows:
(17)Z=Λ−1/2UTẊT=HTẊT.Obviously, it holds thatZZT=HTẊTẊH=I. Thus, finding a weighting vector w is identical to finding a vector Hφ that satisfies s=XHφ. The GED problem in equation (15) can be reformulated as
(18)XTyyTXHφ=λẊTẊ+δIHφ.Left multiplying both sides of equation (18) by HT and substituting HTẊTẊH=I and HTH=Λ−1, we obtain
(19)HTXTyyTXHφ=λI+δΛ−1φ.It is an eigenvalue decomposition problem, and the eigenvectorφ is calculated. Finally, the QSF s can thus be computed as
(20)s=Xw=XHφ=XUΛ−1/2φ.


### 2.3. Relevance Vector Regression

After the QSFs have been extracted and selected, a sparse Bayesian framework named relevance vector regression (RVR) is developed for the prediction model. Given a databasexn,tnn=1N with the input data xn and the corresponding target, the target can be expressed as follows [44, 45]:
(21)tn=yxn+εn,where εn~N0,σ2. The function yx is defined as follows:
(22)yx=∑i=1NωiKx,xi+ω0=∑i=1Nωiϕix,where ϕix=Kx,xi is the kernel function and ω=ω0,ω1,⋯,ωN is the weighted parameter vector of the kernel function.The targets is given asptn∣xn=Ntn∣yxn,σ2. The likelihood function of target values t=t1,t2,⋯,tN is written as
(23)pt∣x,ω,σ2=2πσ2−N/2exp−12σ2t−Φω2,where Φ is a design matrix consisting of kernel functions. A zero-mean Gaussian distribution is used to constrain the weight parameters ω:
(24)pω∣α=∏i=0NNωi∣0,αi−1,where α is a vector of hyperparameters (α=α0,α1,⋯,αNT).Then, the posterior distribution overweights can be calculated through the Bayesian rule:(25)pω∣t,α,σ2=pt∣ω,σ2pω∣αpt∣α,σ2=2π−N+1/2Σ−1/2exp−12ω−μT∑−1ω−μ,where the posterior mean and covariance are given by
(26)μ=σ−2ΣΦTt,(27)Σ=σ−2ΦTΦ+A−1,with A=diagα0,α1,⋯,αN.For a new test samplex∗ and the corresponding target t∗, the predictive distribution is given by
(28)pt∗∣x∗,x,t,αMP,σMP2=∫pt∗∣x∗,ω,σMP2pω∣x,t,αMP,σMP2dω,where αMP and σMP2 are most probable values for hyperparameters and variance, respectively. Using equations (23) and (25), the predictive model is presented as
(29)pt∗∣t=Nt∗∣μTϕx∗,σ∗2,where the variance is σ∗2=σMP2+ϕx∗TΣϕx∗.Thus, a Bayesian prediction model is developed. The sparseness of the model is guaranteed by the constrained weights that follow zero-mean prior distribution. Theα and σ2 are estimated by maximizing pt∣α,σ2 given by tipping [44]:
(30)pt∣α,σ2=∫pt∣ω,σ2⋅pω∣αdω=2π−N/2C−1/2exp12tTC−1t,where the covariance is given by C=σ2I+ΦA−1ΦT.Values ofα and σ2 can be calculated by the iterative method:
(31)αinew=1−αiΣiiμi2,(32)σ2new=t−Φμ2N−∑i=1N1−α1Σii.


### 2.4. Quality Prediction Method Based on QSFA-RVR

As plenty of data have been sampled from the propylene polymerization process, a soft sensor model based on quality-relevant slow feature analysis and Bayesian regression is constructed to predict the MI value for quality monitoring. The model extracts quality-relevant slow features from the sampled data and built a conditional probability distribution over sparse weight parameters; then, MI values are predicted according to the Bayesian inference. The parameters of the model are estimated by a recursive expectation maximum technique. It is recognized that different QSFs may have different significances and only some of them are critical to quality prediction. The noncritical QSFs should be removed from model development to avoid undesirable disturbance. Here, the cross-validation method is adopted to determine the retained QSFs for Bayesian regression. The flowchart of the QSFA-RVR soft sensing approach for quality prediction is shown in Figure1. The step-by-step procedures for constructing the quality prediction model are described as follows:Step 1.
Classify the modeling data into two groups, namely, the training datasetX~tr,ytr and testing dataset X~te,yte.Step 2.
Normalize the training datasetX~tr,ytr with X~tr−u/σ to obtain Xtr,ytr, where u and σ are the mean and standard deviation of X~tr.Step 3.
Conduct quality-relevant slow feature analysis withXtr,ytr and obtain a series of QSFs s=str1,str2,⋯,strJ according to equations (15)–(20).Step 4.
Sort the QSFss=str1,str2,⋯,strJ in descending order evaluated by the correlation index Ri:
(33)Ri=striTyvarstrivary.
For brevity, the sorted QSFs are still noteds=str1,str2,⋯,strJ, and the corresponding weighting matrix is saved as w~.Step 5.
Determine the retained QSFsstr=str1,str2,⋯,strM for regression modeling using the cross-validation method, where M is the number of retained features. The corresponding weighting matrix is saved as w.Step 6.
Initialize the parametersα and σ2 of the quality prediction model.Step 7.
Given the values of parametersα and σ2 and the training set str,ytr, compute the posterior mean μ and covariance Σ of the model according to equations (26) and (27).Step 8.
Given the values of posterior statisticsμ and Σ, determine the model parameters α and σ2 by equations (31) and (32).Step 9.
If values of parametersα and σ2 are converged to their optima, continue to Step 10. Otherwise, go back to Step 7.Step 10.
For the testing dataX~te, it is normalized as Xte according to the mean and variance of the training dataset.Step 11.
Calculate the corresponding QSFsste=ste1,ste2,⋯,steM by projecting the normalized Xte onto the weighting matrix w, that is, ste=Xtew.Step 12.
A quality prediction model is formulated by equation (29) with input ste. An estimated value of quality index y^ could be obtained by the prediction model.Figure 1
Flowchart of the QSFA-RVR soft sensing approach for quality prediction.To quantify the prediction accuracy, five performance indices are used for comparisons, namely, the mean absolute error (MAE), the mean relative error (MRE), the root mean square error (RMSE), Theil’s inequality coefficient (TIC), and standard deviation of absolute error (STD). The error indicators are defined as follows:(34)MAE=1n∑i=1nyi−y^i,(35)MRE=1n∑i=1nyi−y^iyi×100%,(36)RMSE=1n∑i=1nyi−y∧i2,(37)TIC=∑i=1nyi−y∧i2∑i=1nyi2+∑i=1ny^i2,(38)STD=1n−1∑i=1nei−e¯2,where ei=yi−y^i, e¯=1/n∑i=1nei, yi is the real MI value, y^i is the predicted MI value, and y¯ is the mean value of the output. The MAE, MRE, and RMSE confirm the prediction accuracy of the prediction models. The smaller the value of these indicators is, the higher the accuracy of the prediction model is. The STD indicates the stability of the prediction model. The smaller the value is, the more stable the prediction model is. The TIC indicates a good level of agreement between the proposed model and the studied process.


## 2.1. Overview of Slow Feature Analysis

SFA is an unsupervised algorithm proposed by Wiskott and Sejnowski [30], which extracts invariant features from quickly varying signals. The slow variations are always termed as intrinsic features of signals. Meanwhile, rapid variations often allude to measurement noise. The detailed methodology of SFA is illustrated as follows [41].Given aJ-dimensional input signal X, SFA is aimed at finding mapping functions from input to slow features sj=gjX,j∈1,2,⋯,J that minimize
(1)mingj⋅Δsj≔ṡj2,under the constraints
(2)ṡjgj⋅j=1Jsj=0zeromean,sj2=1unitvariance,∀i≠j,sisj=0decorrelationandorder,where⋅ denotes time averaging and denotes the first-order derivative of sj with respect to time.To simplify this optimization problem, it is assumed that the SFs are a linear combination of input variables. The linear mapping is(3)sj=Xwj.The optimization problem in (1) can be solved by two steps of singular value decomposition (SVD) [42], which is tantamount to a numerical solution to the generalized eigenvalue decomposition (GED) problem in
(4)AW=BWΩ,where A=ẊẊT denotes the covariance matrix of the first-order derivative of X and B=XXT denotes the covariance matrix of X; W=w1,w2,⋯wJT is the matrix of generalized eigenvectors, which are coefficient vectors of linear mappings; and Ω=diagλ1,⋯,λJ is a diagonal matrix with λj=ṡj2 being generalized eigenvalues, which are arranged in an ascending order.By performing a first SVD onB with equation (5), the original input X is whitened as Eq. (6).
(5)B=UΛUT,(6)Z=Λ−1/2UTX,where U is an orthogonal matrix, Λ is a diagonal matrix, and covZ=ZZT=IJ. Then, a second SVD on the covariance of the first-order derivative of Z yields
(7)ŻŻT=PTΩP,where P is an orthogonal matrix.Thus, the weighting matrixW can be calculated, and the slow features s can be obtained using equation (3).


## 2.2. Quality-Relevant Slow Feature Analysis

Conventional SFA only considers slowness of process variations when extracting slow features. SFA’s competence to retain process information containing quality indices is limited. An intuition is presented to integrate the relationships between slow features and quality indices into SFA’s optimization problem. In this way, an improved SFA algorithm, named quality-relevant slow feature analysis (QSFA), can extract quality-relevant slow features (QSF) from a process, and these slow features can better reveal the essence of the process. Moreover, a new objective function for SFA should be designed to enable the QSFA that considers both slowness of process variations and their correlations with quality indices. Given the normalized process variablesX and the quality index y, a new objective function Jw composed of three subobjective functions is designed. The first subobjective function J1 which considers that the QSF should change slowly is given as follows:
(8)minJ1w=Δṡ=ẊwTẊw,where s is the QSF and w is the corresponding weighting vector. Considering that the QSF should be highly correlated with the quality index, the second subobjective function J2 is aimed at maximizing the correlation between s and y, which is given as follows:
(9)maxJ2w=sTyyTsvars=XwTyyTXwXwTXw.The third subobjective functionJ3 is used to maximize the variance of s which represents the variation information of the process.
(10)maxJ3w=vars=XwTXw.Thus, a new optimization problem of QSFA can be described as(11)maxJw=J2J3J1=XwTyyTXw,s.t.ẊwTẊw=1wTw=1.By introducing Lagrange multipliersλ and δ, equation (14) can be easily obtained. Obviously, the value of the objective function Jw is equal to λ+δ.
(12)Lw=wTXTyyTXw−λwTẊTẊw−1−δwTw−1,(13)∂Lw∂w=2XTyyTXw−2λẊTẊw−2δw=0,(14)wTXTyyTXw=λ+δ,Therefore, we can rewrite the solution of equation (13) as the following GED problem [43]:
(15)XTyyTXw=λẊTẊ+δIw,where I is a J-dimensional identity matrix.By performing SVD decomposition on the covariance matrixẊTẊ, we have
(16)ẊTẊ=UΛUT.The whitening transformation with the whitening matrixH is described as follows:
(17)Z=Λ−1/2UTẊT=HTẊT.Obviously, it holds thatZZT=HTẊTẊH=I. Thus, finding a weighting vector w is identical to finding a vector Hφ that satisfies s=XHφ. The GED problem in equation (15) can be reformulated as
(18)XTyyTXHφ=λẊTẊ+δIHφ.Left multiplying both sides of equation (18) by HT and substituting HTẊTẊH=I and HTH=Λ−1, we obtain
(19)HTXTyyTXHφ=λI+δΛ−1φ.It is an eigenvalue decomposition problem, and the eigenvectorφ is calculated. Finally, the QSF s can thus be computed as
(20)s=Xw=XHφ=XUΛ−1/2φ.


## 2.3. Relevance Vector Regression

After the QSFs have been extracted and selected, a sparse Bayesian framework named relevance vector regression (RVR) is developed for the prediction model. Given a databasexn,tnn=1N with the input data xn and the corresponding target, the target can be expressed as follows [44, 45]:
(21)tn=yxn+εn,where εn~N0,σ2. The function yx is defined as follows:
(22)yx=∑i=1NωiKx,xi+ω0=∑i=1Nωiϕix,where ϕix=Kx,xi is the kernel function and ω=ω0,ω1,⋯,ωN is the weighted parameter vector of the kernel function.The targets is given asptn∣xn=Ntn∣yxn,σ2. The likelihood function of target values t=t1,t2,⋯,tN is written as
(23)pt∣x,ω,σ2=2πσ2−N/2exp−12σ2t−Φω2,where Φ is a design matrix consisting of kernel functions. A zero-mean Gaussian distribution is used to constrain the weight parameters ω:
(24)pω∣α=∏i=0NNωi∣0,αi−1,where α is a vector of hyperparameters (α=α0,α1,⋯,αNT).Then, the posterior distribution overweights can be calculated through the Bayesian rule:(25)pω∣t,α,σ2=pt∣ω,σ2pω∣αpt∣α,σ2=2π−N+1/2Σ−1/2exp−12ω−μT∑−1ω−μ,where the posterior mean and covariance are given by
(26)μ=σ−2ΣΦTt,(27)Σ=σ−2ΦTΦ+A−1,with A=diagα0,α1,⋯,αN.For a new test samplex∗ and the corresponding target t∗, the predictive distribution is given by
(28)pt∗∣x∗,x,t,αMP,σMP2=∫pt∗∣x∗,ω,σMP2pω∣x,t,αMP,σMP2dω,where αMP and σMP2 are most probable values for hyperparameters and variance, respectively. Using equations (23) and (25), the predictive model is presented as
(29)pt∗∣t=Nt∗∣μTϕx∗,σ∗2,where the variance is σ∗2=σMP2+ϕx∗TΣϕx∗.Thus, a Bayesian prediction model is developed. The sparseness of the model is guaranteed by the constrained weights that follow zero-mean prior distribution. Theα and σ2 are estimated by maximizing pt∣α,σ2 given by tipping [44]:
(30)pt∣α,σ2=∫pt∣ω,σ2⋅pω∣αdω=2π−N/2C−1/2exp12tTC−1t,where the covariance is given by C=σ2I+ΦA−1ΦT.Values ofα and σ2 can be calculated by the iterative method:
(31)αinew=1−αiΣiiμi2,(32)σ2new=t−Φμ2N−∑i=1N1−α1Σii.


## 2.4. Quality Prediction Method Based on QSFA-RVR

As plenty of data have been sampled from the propylene polymerization process, a soft sensor model based on quality-relevant slow feature analysis and Bayesian regression is constructed to predict the MI value for quality monitoring. The model extracts quality-relevant slow features from the sampled data and built a conditional probability distribution over sparse weight parameters; then, MI values are predicted according to the Bayesian inference. The parameters of the model are estimated by a recursive expectation maximum technique. It is recognized that different QSFs may have different significances and only some of them are critical to quality prediction. The noncritical QSFs should be removed from model development to avoid undesirable disturbance. Here, the cross-validation method is adopted to determine the retained QSFs for Bayesian regression. The flowchart of the QSFA-RVR soft sensing approach for quality prediction is shown in Figure1. The step-by-step procedures for constructing the quality prediction model are described as follows:Step 1.
Classify the modeling data into two groups, namely, the training datasetX~tr,ytr and testing dataset X~te,yte.Step 2.
Normalize the training datasetX~tr,ytr with X~tr−u/σ to obtain Xtr,ytr, where u and σ are the mean and standard deviation of X~tr.Step 3.
Conduct quality-relevant slow feature analysis withXtr,ytr and obtain a series of QSFs s=str1,str2,⋯,strJ according to equations (15)–(20).Step 4.
Sort the QSFss=str1,str2,⋯,strJ in descending order evaluated by the correlation index Ri:
(33)Ri=striTyvarstrivary.
For brevity, the sorted QSFs are still noteds=str1,str2,⋯,strJ, and the corresponding weighting matrix is saved as w~.Step 5.
Determine the retained QSFsstr=str1,str2,⋯,strM for regression modeling using the cross-validation method, where M is the number of retained features. The corresponding weighting matrix is saved as w.Step 6.
Initialize the parametersα and σ2 of the quality prediction model.Step 7.
Given the values of parametersα and σ2 and the training set str,ytr, compute the posterior mean μ and covariance Σ of the model according to equations (26) and (27).Step 8.
Given the values of posterior statisticsμ and Σ, determine the model parameters α and σ2 by equations (31) and (32).Step 9.
If values of parametersα and σ2 are converged to their optima, continue to Step 10. Otherwise, go back to Step 7.Step 10.
For the testing dataX~te, it is normalized as Xte according to the mean and variance of the training dataset.Step 11.
Calculate the corresponding QSFsste=ste1,ste2,⋯,steM by projecting the normalized Xte onto the weighting matrix w, that is, ste=Xtew.Step 12.
A quality prediction model is formulated by equation (29) with input ste. An estimated value of quality index y^ could be obtained by the prediction model.Figure 1
Flowchart of the QSFA-RVR soft sensing approach for quality prediction.To quantify the prediction accuracy, five performance indices are used for comparisons, namely, the mean absolute error (MAE), the mean relative error (MRE), the root mean square error (RMSE), Theil’s inequality coefficient (TIC), and standard deviation of absolute error (STD). The error indicators are defined as follows:(34)MAE=1n∑i=1nyi−y^i,(35)MRE=1n∑i=1nyi−y^iyi×100%,(36)RMSE=1n∑i=1nyi−y∧i2,(37)TIC=∑i=1nyi−y∧i2∑i=1nyi2+∑i=1ny^i2,(38)STD=1n−1∑i=1nei−e¯2,where ei=yi−y^i, e¯=1/n∑i=1nei, yi is the real MI value, y^i is the predicted MI value, and y¯ is the mean value of the output. The MAE, MRE, and RMSE confirm the prediction accuracy of the prediction models. The smaller the value of these indicators is, the higher the accuracy of the prediction model is. The STD indicates the stability of the prediction model. The smaller the value is, the more stable the prediction model is. The TIC indicates a good level of agreement between the proposed model and the studied process.


## 3. Results and Discussion

The proposed QSFA-RVR model is applied to predict the MI in an industrial propylene polymerization process located in China. Figure2 shows the schematic diagram of the industrial process. The process consists of four reactors in series: the first two continuous stirred tank reactors (CSTR) and the last two fluidized-bed reactors (FBR). The feed to the reactor is comprised of propylene, hydrogen, and the Ziegler-Natta catalyst. In the first two reactors, the polymerization reaction takes place in a liquid phase, and in the third and fourth reactors, the reaction is completed in the vapor phase to produce the powdered polymer products. The MI, which depends on the catalyst properties, reactant composition, reactor temperature, and so on, can determine different brands of the products. To develop a prediction model to estimate the MI, a total of nine process variables are chosen as input variables according to the workers’ experience and reaction mechanism analysis. The selected variables are listed in Table 1, including the flow rates, temperature, and pressure. The MI is analyzed at a sampling time of 2 hours in the laboratory, and the dataset of nine process variables is acquired from the distributed control system of the polypropylene process. There are 170 datasets, and they are divided into training and testing datasets. The first 119 datasets (about 70%) are used for training, and the remaining 51 datasets (about 30%) are used for testing. It should be noted that the current MI data are from the same grade of polypropylene production, which is a slowly varying dynamic process. The data are filtered to discard abnormal situations and to improve the quality of the prediction results. The variables are normalized with the method of statistical normalization.Figure 2
Schematic diagram of the industrial propylene polymerization process.Table 1
Process variables in the propylene polymerization process used for modeling.
No.Measured variablesNo.Measured variables1Temperature in the first CSTR6Flow rate of feedstock in the first FBR2Pressure in the first CSTR7Concentration of hydrogen in the first CSTR3Liquid level in the first CSTR8Flow rate of the main catalyst4Flow rate of propylene in the first CSTR9Flow rate of the second catalyst5Flow rate of feedstock in the second CSTRFirst, QSFs are extracted based on the proposed method. To provide a visual picture of extracted QSFs, Figure3 visualizes all QSFs extracted on the training set for the PP process. It is observed that the first six QSFs change more slowly and the slowness decreases. To investigate these QSFs in detail, two quantitative indices are defined, one of which measures the slowness of each QSF and the other measures the correlation with quality. The slowness index is calculated as Si=Δṡi, which has been defined in equation (10). The correlation index Ri defined in equation (38) is used to evaluate the correlation between a QSF and a quality index. Then, the slowness of QSFs and correlation coefficients with respect to the quality index, i.e., MI, are calculated and shown in Figure 4. These QSFs are sorted in a descending order and their values of slowness are correspondingly arranged. As can be seen from the figure, the slowness of QSFs is consistent with their quality interpretation. In particular for the first three QSFs, as the correlation weakened, the slowness increased. Compared to the first seven QSFs, the 8th and the 9th QSFs are fast time-varying features and almost irrelevant to the quality index. It reveals that the MI is almost all determined by slowly time-varying QSFs. Thus, a good prediction performance can be achieved based on the extracted QSFs which present both the slowly varying characteristic and the correlations with MI.Figure 3
QSFs extracted on the training set for PP process.Figure 4
Correlation and slowness of QSFs using training data with respect to MI.For prediction purpose, the cross-validation method is adopted to determine the retained QSFs for Bayesian regression. The number of retained features isM=5. Then, the proposed QSFA-RVR model is built based on the selected QSFs. At the same time, the RVR model, the quality-relevant slow feature regression (QSFR) model, and the PCA-based RVR (PCA-RVR) model have also been developed in comparison with the proposed model. Performances of different models for predicting MI values on the testing dataset are shown in Figure 5. The analytic values obtained from the laboratory are marked with points, and prediction values of RVR, QSFR, PCA-RVR, and QSFA-RVR are marked with triangle, asterisks, squares, and pentagrams, respectively. As can be seen from the figure, the proposed QSFA-RVR model performance is the best among all models. It not only can achieve more satisfactory expected results but also can trace process variations well.Figure 5
Prediction performance of different models on the testing dataset.Moreover, another qualitative comparison is displayed in Figure6. The scattered plots give exhibition on how the predicted values gather around the real values. If the plots are on the black diagonal line, the predicted results are more accurate and vice versa. As illustrated in Figure 6, the proposed QSFA-RVR model has less dispersion of the prediction than the other three models. It generates more precise estimated values as these estimated values almost stay around the black diagonal line.Figure 6
Prediction performance of different models on the testing dataset.For comparison, the traditional feature selection method LASSO is considered for soft sensing. The prediction results of different models on the testing dataset are listed in Table2. It is obvious that the QSFA-RVR model has the best performance over all. In detail, the QSFR model obtains an MAE of 0.0168, an MRE of 0.64%, a RMSE of 0.0189, a TIC of 0.0036, and a STD of 0.0139. The PCA-RVR model gives an MAE of 0.0119, an MRE of 0.45%, a RMSE of 0.0144, a TIC of 0.0027, and a STD of 0.0126. The QSFA-RVR model shows even better results. The MAE, MRE, RMSE, TIC, and STD are 0.0051, 0.19%, 0.0069, 0.0013, and 0.0069, respectively. The error measurement of the proposed model has percentage decreases of 69.6%, 70.3%, 63.5%, 63.9%, and 50.4% compared to the QSFR model and percentage decreases of 57.1%, 57.8%, 52.1%, 51.9%, and 45.2% compared to the PCA-RVR model. The comparison results prove that the soft sensor based on the QSFA-RVR model can achieve a good performance in the MI prediction for the propylene polymerization process.Table 2
Quality prediction results of different models on the testing dataset.
ModelMAEMRE (%)RMSETICSTDLASSO0.01870.710.02330.00450.0212RVR0.01740.660.02190.00420.0217QSFR0.01680.640.01890.00360.0139PCA-RVR0.01190.450.01440.00270.0126QSFA-RVR0.00510.190.00690.00130.0069The above experiments are performed on a personal computer with the configuration shown as follows: operating system: Windows 10 (64-bit); CPU: Intel Core i5-7200U (2.70 GHz); RAM: 8.00 GB; and MATLAB 2017b software. The computation time of the QSFR, PCA-RVR, and QSFA-RVR models is 0.96 s, 1.65 s, and 1.94 s, respectively. The prediction time for testing data only needs less than 2 s. Since the sampling time of the industrial MI prediction is about 2 h, the proposed method qualifies the online soft sensor for the MI prediction. The time lags of MI in the form of average residence time do exist and have been considered in our research work.


## 4. Conclusions

In this paper, a novel soft sensor based on quality-relevant slow feature analysis and Bayesian regression is proposed for quality prediction in industrial PP processes. The proposed method can handle the dynamics of the process better by extracting quality-relevant slow features and deal with the nonlinearity by introducing the Bayesian inference model. A case study about the MI prediction in a real industrial PP plant is carried out to evaluate the performance of the proposed QSFA-RVR method. For comparison, the RVR model, the QSFR model, and the PCA-RVR model are also developed and evaluated. The application of the proposed model to the testing dataset demonstrates its superiority. The QSFA-RVR model predicts the MI with an MRE of 0.19%, which is much more accurate than the PCA-RVR model with an MRE of 0.45%, while much better than the QSFR model with an MRE of 0.64%. The research results reveal the prediction accuracy and validity of the proposed model, which indicate that the QSFA-RVR modeling approach can be a promising and efficient methodology for industrial MI prediction.



---

*Source: 9985747-2021-12-30.xml*