# Fault Diagnosis for Hydraulic Servo System Using Compressed Random Subspace Based ReliefF

Authors: Yu Ding; Fei Wang; Zhen-ya Wang; Wen-jin Zhang
Journal: Complexity
 (2018)

---

## Abstract

Playing an important role in electromechanical systems, hydraulic servo system is crucial to mechanical systems like engineering machinery, metallurgical machinery, ships, and other equipment. Fault diagnosis based on monitoring and sensory signals plays an important role in avoiding catastrophic accidents and enormous economic losses. This study presents a fault diagnosis scheme for hydraulic servo system using compressed random subspace based ReliefF (CRSR) method. From the point of view of feature selection, the scheme utilizes CRSR method to determine the most stable feature combination that contains the most adequate information simultaneously. Based on the feature selection structure of ReliefF, CRSR employs feature integration rules in the compressed domain. Meanwhile, CRSR substitutes information entropy and fuzzy membership for traditional distance measurement index. The proposed CRSR method is able to enhance the robustness of the feature information against interference while selecting the feature combination with balanced information expressing ability. To demonstrate the effectiveness of the proposed CRSR method, a hydraulic servo system joint simulation model is constructed by HyPneu and Simulink, and three fault modes are injected to generate the validation data.


---

## Body

## 1. Introduction

Hydraulic servo system plays a crucial role in electromechanical systems, like engineering machinery, metallurgical machinery, ships, and other equipment. Failures of hydraulic servo system caused by severe and complex conditions may lead to catastrophic accidents and enormous economic losses. Fault diagnosis based on monitoring and sensory signals is able to classify the current state of complex systems, which plays a key role in performance evaluation [1]. Feature set extracted from signals is an important index to reflect the fault mechanism and performance evolution laws. The quality of feature set plays a key role in improving the generalization ability of fault identification [2]. The common feature extraction methods are time-frequency index extraction, wavelet analysis, Hilbert transform, Duffing oscillator, and so on. Despite their respective applicable conditions and limitations, those methods are able to mine the health characteristics of the system from multiaspect [3, 4]. Same as machine learning, features extracted from images, speeches, and other signals often have certain correlations and hidden mutual influences. Information expressed by a single feature is usually inadequate, which can be greatly improved when the single feature is aggregated with others [5]. Similarly, due to the nonlinearity, instability, and nonconformity of complex electromechanical systems, the expression of the information on individual feature is often one-sided. Thus, a new challenge is how to utilize those features more effectively and efficiently, in other words, how to obtain the feature set that expresses the information sufficiently by eliminating the redundant and negatively correlated features [6–9].To tackle the challenge mentioned above, on the premise of existing feature extraction techniques, feature processing techniques including feature selection and dimension reduction have gradually become an important research focus. Both feature selection and dimension reduction can reduce the scale of feature set by obtaining a set of principal variables. Such techniques often use a variety of feature extraction methods to integrate the features into a comprehensive representation of the signal [10]. For the purpose of enhancing the expressing ability of core information on multiclass feature sets, spatial transformation or importance measurement methods are used [11]. Such methods are able to reduce redundancy existing in features and improve learning efficiency while retaining the performance advantages. The data transformation may be linear, such as principal component analysis (PCA). But many nonlinear dimension reduction techniques also exist. The common feature processing techniques include linear dimensionality reduction methods (FDA, LPP, etc.), kernel function based dimension reduction methods (KPCA, KFDA, etc.), manifold based dimension reduction methods (Isomap, LLE, MDS, etc.), filter method based feature selection methods (Relief, Focus, information gain, CBFS, etc.), wrapper method based feature selection methods (genetic algorithm, distribution estimation, differential algorithm, etc.), and embedded method based feature selection methods (SVN-RFE, RF, etc.) [12–14].For the nonlinear signals of complex electromechanical systems, although the dimension reduction methods can reduce the scale of input features for fault diagnosis, they change the basic attributes of the feature set. Such situation makes it difficult to give a clear understanding of the obtained feature subset. Meanwhile, existing feature selection methods sort the importance degree of the features according to the independent feature evaluation result. They ignore the interaction among features, which would lead to information loss in processing the data of electromechanical systems [15]. Aiming at the shortcomings of feature selection and dimension reduction techniques, such as low expandability, unclear evaluation indexes, and strong tendency, this study proposes compressed random space based ReliefF (CRSR) method. Based on ReliefF method, CRSR introduces ensemble strategy on feature level based on compressed random space. Furthermore, CRSR optimizes the objective function using information entropy and fuzzy theory. The main contributions are as follows:(i)
This study analyzes the feasibility of the ReliefF based feature selection architecture for the fault diagnosis of complex electromechanical system. Meanwhile, the basic mechanism of measuring the contribution of the features based on ReliefF is also demonstrated.(ii)
By converting the assessment process of ReliefF, which takes the entire feature set as object, into the construction and ensemble process of feature subspace, CRSR can improve the global optimization ability of ReliefF.(iii)
Considering ReliefF based feature selection as a problem of maximizing the distance, CRSR replaces the traditional spatial distance with fuzzy membership degree, which is able to obtain a robust and steady objective function.This paper is structured as follows: In Section2, ReliefF method based feature selection structure is introduced. Then feature integration method based on compressed random subspace is described. Objective function optimization method based on information entropy and fuzzy theory is also introduced. Section 3 presents the overall diagnosis procedure for hydraulic servo systems, the details of construction and fault injection of the hydraulic servo system, and analysis and comparison on feature selection results using the proposed CRSR method which are discusses numerically. Section 4 concludes the paper.


## 2. Related Theories

### 2.1. ReliefF Method Based Feature Selection Structure

ReliefF is the extension of Relief method by estimating probabilities more reliably, which is able to handle incomplete and multiclass data sets while the complexity remains the same [16–19]. By calculating the distances between the sample distributions, ReliefF can obtain the correlation weight coefficients of the features which is similar to Relief.For a specific feature from the feature set, if its difference in same class is much smaller than that in different class, it is considered that this feature contributes to class discrimination [20]. Given a sample set S=s1,…,si,…,sni∈1,n with n instances, each sample, si=si1,…,sit,…,simt∈1,m, hasm-dimensional features. Meanwhile, the samples in S only belong to two classes which are tagged as ci∈C1,C2. The difference between each two samples (si and sj) on feature t is defined as(1)difft,si,sj=1,sit≠sjt0,sit=sjt,where the attribute of feature t is discrete value. If the attribute of feature t is continuous value,(2)difft,si,sj=sit-sjtmaxt-mint.Features extracted from condition monitoring data of electromechanical system mostly are continuous data. Meanwhilemaxt and mint represent the maximum value and minimum value of the entire sample, respectively. The closest same-class instance of sample si is called “near-hit (NH),” and the closest different-class instance of sample si is called “near-miss (NM).” Meanwhile, the weight of feature t is denoted as ωt, and ωt is updated by(3)ωt=ωt-old-difft,si,NH+difft,si,NM,where the initial value of ωt is 0, and ωt-old is the last value of ωt.For reducing the randomness in feature evaluation, the whole process needs to repeatr times to obtain the average value being the final weight. Although Relief method is very efficient in estimating the quality of attributes, it cannot deal with incomplete data and is limited to two-class problem. Thus ReliefF method is utilized in the paper to deal with the multiclass classification and regression problems for continuous data.For a multiclass classification problem, assuming that the samples inS belong to multiple classes and the tags for S are ci∈C1,…,Cl, ReliefF updates ωt on sample si by taking k near hits (NHS) and k near misses (NMS) into consideration, which is different from Relief. Similarly, the weight of feature t, which is ωt, can be updated through(4)ωt=ωt-old-∑j=1kdifft,si,NHSjr∗k+∑c∉classsipci/1-pclasssi∑j=1kdifft,si,NMSjr∗k,where pci/1-pclasssi represents the ratio of the entire samples in class ci to all the heterogeneous samples in si. Furthermore, ReliefF method equalizes the differentiation of NHs and calculates the average differences between si and other classes on feature t to evaluate the classification ability of the samples nearby.


### 2.2. Feature Integration Method Based on Compressed Random Subspace

The purpose of CRSR method is finding the balance between the differences and the correlations of features. Specifically, in the premise of fully mining the correlations of features using ReliefF method, CRSR method is applied to make each feature subset keep a certain degree of difference. Based on random subspace and feature sorting strategy, the feature sets with higher contribution can be obtained in various feature combinations [21, 22]. RS1 and RS2 are denoted as two random subspaces, so the difference, denoted as D_RSRS1,RS2, can be calculated as(5)D_RSRS1,RS2=RS1∪RS2-RS1∩RS2,where symbol · denotes the dimension of random subspaces.The right side of (5) obtains the noncoincident features of RS1 and RS2. D_RSRS1,RS2 is plus one when there is an unrepeated feature fD. The average difference between two random subspaces on all the features is defined as(6)ED_RS=∑q=1QPfD∈RS1Pf∉RS2+PfD∉RS1Pf∈RS2,where Q is the dimension after feature ordering compression. It can be simplified as Q=m-m1. Concretely, according to sorting strategy, RS12, the probability of the first m0 strongly related features being selected is 2m0/Q, and the relatively poorly related m0 features being selected are m1/Q-m0. The average difference of feature evaluation based on compressed random subspace method, which is ED_RS, can be denoted as follows:(7)ED_RS=212Qm01/2Q1-m01/2Q+Q-m0m1Q-m01-m1Q-m0=2m0Q-2m0Q+m1Q-m1-m0Q-m0.Equation (7) shows that the ranking result can balance the difference of ReliefF to determine the dominant features that are crucial for classification, which would improve the feature selection efficiency.Based on compressed random subspace method, this study proposes redundancy analysis method from statistics to reduce the redundancy of feature. The features are checked in pairs using redundancy analysis method [23, 24]. Firstly, two sets of feature vectors, F1 and F2, are selected randomly from the feature set obtained by ReliefF. Then, the selected feature vectors are regarded as independent variable and dependent variable separately. The covariance matrixes, denoted as Uand V, can be calculated, respectively, as(8)U=EF1-EF1F1-EF1T,V=EF2-EF2F2-EF2T,where EX represents mathematical expectation of vector X. Then, the correlation coefficient of F1 and F2, denoted as ρU,V, can be formulated as(9)ρU,V=CovU,VVar⁡UVar⁡V,where CovU,V denotes the covariance of U and V. If the correlation coefficient is greater than a presetting threshold, only the one with larger weight from F1 and F2 will be added to the final selected feature set. It is noticed that redundancy analysis based on matrix transformation focuses on the correlation between features instead of the similarity of data. Thus, CRSR can reduce the influence of numerical confusion existing in feature selection. Furthermore, compared with traditional methods based on data similarity, CRSR is able to obtain higher confidence level.


### 2.3. Objective Function Optimization Method Based on Information Entropy and Fuzzy Theory

From the aspect of maximizing the distance, Relief method can be seen as a distance optimization algorithm using feature weighting method [25]. Under this condition, the optimization objective function, denoted as maxω⁡J, can be described as [26](10)maxωJ=∑i=1nρiω,where ρiω denotes the distance of the ith sample. Based on (4), ρiω can be converted as(11)ρiω=∑t=1mωt∑c∉classsipci1-pclasssi∑j=1kdifft,si,NMSj-∑j=1kdifft,si,NHSj.For complex electromechanical system signals, two problems occur when using (10) as the optimization objective function of CRSR. One problem is that the objective function concentrates the weight on one or some of the features, which leads to the result that assessment value of the remaining features tends to 0. Meanwhile, (10) regards the samples with stochastic volatility and noise similar to the normal sample. Another problem is the lack of consideration on the influence from the quality of samples on the feature selection process.Aiming at the first problem, the information entropy theory is proposed based on compressed random subspace method, which combines the maximization of entropy together with maximization of distance to reduce the over tendency problem of the existing ReliefF method. After adding a sample estimation factorκi, the optimization objective function is denoted as(12)maxω,κ⁡J=∑i=1nκiρiω.Supposing thatκi and ωi follow probability distribution, Shannon entropy is used to adjust the sample distribution, as shown below:(13)Jeκi=-∑i=1nκiln⁡κi,Jeωi=-∑i=1mωiln⁡ωi.Aiming at the second problem, the fuzzy membership degree is chosen to replace the traditional nearest neighbor distance. The fuzzy membership degree has the advantage of being insensitive to sample fluctuations and noise and the ability of updating adaptively while changing the feature weights [27]. In a sample space of same class, the fuzzy membership degree of the feature sit, denoted as νij, can be calculated as [28, 29](14)νij=Qν,ij1/1-m∑i=1NHSQν,ij1/1-m,where NHS is the sample set of same class, and Qν,ij is the fuzzy difference between feature sit and feature sjt, as shown below:(15)Qνi=∑j=1Mωjdifft,si,sj,sj∈NHS.Then the same fuzzy distance, denoted asm_difft,si,NHS, is calculated as(16)m_difft,si,NHS=∑sj∈NHSνijdifft,si,sj.Similarly, the fuzzy difference and the fuzzy membership degree of the heterogeneous sample sets ofsi, which are denoted as Qμi and μij separately, can be calculated as follows:(17)Qμi=∑j=1Mωjdifft,si,sj,sj∈NMS,μij=Qμ,ij1/1-m∑i=1NMSQμ,ij1/1-m.Therefore, the heterogeneous fuzzy distance, denoted asm_difft,si,NMS, is calculated as(18)m_difft,si,NMS=∑sj∈NMSμijdifft,si,sj.Based on the fuzzy distances obtained by (16) and (18), updated (11) can be formulated as follows:(19)ρf,iω=∑t=1mωt∑c∉classsipci1-pclasssi∑j=1km_difft,si,NMSj-∑j=1km_difft,si,NHSj.Based on the formulas above, the objective function, which ismaxω,κ⁡J, can be denoted as(20)maxω,κJ=∑i=1nκiρf,iω+λκJeκi+λωJeωi,where the first item on the right side is the maximum distance of ReliefF, which is meant to determine the feature set that contributes most to classification. Meanwhile, from the aspect of entropy maximization, the second and the third items denote the sample evaluation operator and Shannon entropy of feature weight, respectively, which are used to avoid the over tendency problem of objective function. λκ and λω are the balance coefficients for adjusting the differences between features. When the maximum point of objective function is achieved, the constraint condition of the sample evaluation operator is defined below [30]:(21)κi=exp⁡∑j=1mωjdi,j/λκ∑i=1nexp⁡∑j=1mωjdi,j/λκ,where di,j denotes the partial derivative of ρf,iω.In the process of feature selection for mechanical system, the information entropy and fuzzy theory based optimization objective function ensures that the evaluation process of each subspace using ReliefF is adaptive and robust. Such advantage provides a new thought for feature processing of complex monitoring signal, in other words, under the premise of maintaining the high calculating efficiency of ReliefF method, reducing the bias and redundancy caused by methodological defects and external disturbances [31].


## 2.1. ReliefF Method Based Feature Selection Structure

ReliefF is the extension of Relief method by estimating probabilities more reliably, which is able to handle incomplete and multiclass data sets while the complexity remains the same [16–19]. By calculating the distances between the sample distributions, ReliefF can obtain the correlation weight coefficients of the features which is similar to Relief.For a specific feature from the feature set, if its difference in same class is much smaller than that in different class, it is considered that this feature contributes to class discrimination [20]. Given a sample set S=s1,…,si,…,sni∈1,n with n instances, each sample, si=si1,…,sit,…,simt∈1,m, hasm-dimensional features. Meanwhile, the samples in S only belong to two classes which are tagged as ci∈C1,C2. The difference between each two samples (si and sj) on feature t is defined as(1)difft,si,sj=1,sit≠sjt0,sit=sjt,where the attribute of feature t is discrete value. If the attribute of feature t is continuous value,(2)difft,si,sj=sit-sjtmaxt-mint.Features extracted from condition monitoring data of electromechanical system mostly are continuous data. Meanwhilemaxt and mint represent the maximum value and minimum value of the entire sample, respectively. The closest same-class instance of sample si is called “near-hit (NH),” and the closest different-class instance of sample si is called “near-miss (NM).” Meanwhile, the weight of feature t is denoted as ωt, and ωt is updated by(3)ωt=ωt-old-difft,si,NH+difft,si,NM,where the initial value of ωt is 0, and ωt-old is the last value of ωt.For reducing the randomness in feature evaluation, the whole process needs to repeatr times to obtain the average value being the final weight. Although Relief method is very efficient in estimating the quality of attributes, it cannot deal with incomplete data and is limited to two-class problem. Thus ReliefF method is utilized in the paper to deal with the multiclass classification and regression problems for continuous data.For a multiclass classification problem, assuming that the samples inS belong to multiple classes and the tags for S are ci∈C1,…,Cl, ReliefF updates ωt on sample si by taking k near hits (NHS) and k near misses (NMS) into consideration, which is different from Relief. Similarly, the weight of feature t, which is ωt, can be updated through(4)ωt=ωt-old-∑j=1kdifft,si,NHSjr∗k+∑c∉classsipci/1-pclasssi∑j=1kdifft,si,NMSjr∗k,where pci/1-pclasssi represents the ratio of the entire samples in class ci to all the heterogeneous samples in si. Furthermore, ReliefF method equalizes the differentiation of NHs and calculates the average differences between si and other classes on feature t to evaluate the classification ability of the samples nearby.


## 2.2. Feature Integration Method Based on Compressed Random Subspace

The purpose of CRSR method is finding the balance between the differences and the correlations of features. Specifically, in the premise of fully mining the correlations of features using ReliefF method, CRSR method is applied to make each feature subset keep a certain degree of difference. Based on random subspace and feature sorting strategy, the feature sets with higher contribution can be obtained in various feature combinations [21, 22]. RS1 and RS2 are denoted as two random subspaces, so the difference, denoted as D_RSRS1,RS2, can be calculated as(5)D_RSRS1,RS2=RS1∪RS2-RS1∩RS2,where symbol · denotes the dimension of random subspaces.The right side of (5) obtains the noncoincident features of RS1 and RS2. D_RSRS1,RS2 is plus one when there is an unrepeated feature fD. The average difference between two random subspaces on all the features is defined as(6)ED_RS=∑q=1QPfD∈RS1Pf∉RS2+PfD∉RS1Pf∈RS2,where Q is the dimension after feature ordering compression. It can be simplified as Q=m-m1. Concretely, according to sorting strategy, RS12, the probability of the first m0 strongly related features being selected is 2m0/Q, and the relatively poorly related m0 features being selected are m1/Q-m0. The average difference of feature evaluation based on compressed random subspace method, which is ED_RS, can be denoted as follows:(7)ED_RS=212Qm01/2Q1-m01/2Q+Q-m0m1Q-m01-m1Q-m0=2m0Q-2m0Q+m1Q-m1-m0Q-m0.Equation (7) shows that the ranking result can balance the difference of ReliefF to determine the dominant features that are crucial for classification, which would improve the feature selection efficiency.Based on compressed random subspace method, this study proposes redundancy analysis method from statistics to reduce the redundancy of feature. The features are checked in pairs using redundancy analysis method [23, 24]. Firstly, two sets of feature vectors, F1 and F2, are selected randomly from the feature set obtained by ReliefF. Then, the selected feature vectors are regarded as independent variable and dependent variable separately. The covariance matrixes, denoted as Uand V, can be calculated, respectively, as(8)U=EF1-EF1F1-EF1T,V=EF2-EF2F2-EF2T,where EX represents mathematical expectation of vector X. Then, the correlation coefficient of F1 and F2, denoted as ρU,V, can be formulated as(9)ρU,V=CovU,VVar⁡UVar⁡V,where CovU,V denotes the covariance of U and V. If the correlation coefficient is greater than a presetting threshold, only the one with larger weight from F1 and F2 will be added to the final selected feature set. It is noticed that redundancy analysis based on matrix transformation focuses on the correlation between features instead of the similarity of data. Thus, CRSR can reduce the influence of numerical confusion existing in feature selection. Furthermore, compared with traditional methods based on data similarity, CRSR is able to obtain higher confidence level.


## 2.3. Objective Function Optimization Method Based on Information Entropy and Fuzzy Theory

From the aspect of maximizing the distance, Relief method can be seen as a distance optimization algorithm using feature weighting method [25]. Under this condition, the optimization objective function, denoted as maxω⁡J, can be described as [26](10)maxωJ=∑i=1nρiω,where ρiω denotes the distance of the ith sample. Based on (4), ρiω can be converted as(11)ρiω=∑t=1mωt∑c∉classsipci1-pclasssi∑j=1kdifft,si,NMSj-∑j=1kdifft,si,NHSj.For complex electromechanical system signals, two problems occur when using (10) as the optimization objective function of CRSR. One problem is that the objective function concentrates the weight on one or some of the features, which leads to the result that assessment value of the remaining features tends to 0. Meanwhile, (10) regards the samples with stochastic volatility and noise similar to the normal sample. Another problem is the lack of consideration on the influence from the quality of samples on the feature selection process.Aiming at the first problem, the information entropy theory is proposed based on compressed random subspace method, which combines the maximization of entropy together with maximization of distance to reduce the over tendency problem of the existing ReliefF method. After adding a sample estimation factorκi, the optimization objective function is denoted as(12)maxω,κ⁡J=∑i=1nκiρiω.Supposing thatκi and ωi follow probability distribution, Shannon entropy is used to adjust the sample distribution, as shown below:(13)Jeκi=-∑i=1nκiln⁡κi,Jeωi=-∑i=1mωiln⁡ωi.Aiming at the second problem, the fuzzy membership degree is chosen to replace the traditional nearest neighbor distance. The fuzzy membership degree has the advantage of being insensitive to sample fluctuations and noise and the ability of updating adaptively while changing the feature weights [27]. In a sample space of same class, the fuzzy membership degree of the feature sit, denoted as νij, can be calculated as [28, 29](14)νij=Qν,ij1/1-m∑i=1NHSQν,ij1/1-m,where NHS is the sample set of same class, and Qν,ij is the fuzzy difference between feature sit and feature sjt, as shown below:(15)Qνi=∑j=1Mωjdifft,si,sj,sj∈NHS.Then the same fuzzy distance, denoted asm_difft,si,NHS, is calculated as(16)m_difft,si,NHS=∑sj∈NHSνijdifft,si,sj.Similarly, the fuzzy difference and the fuzzy membership degree of the heterogeneous sample sets ofsi, which are denoted as Qμi and μij separately, can be calculated as follows:(17)Qμi=∑j=1Mωjdifft,si,sj,sj∈NMS,μij=Qμ,ij1/1-m∑i=1NMSQμ,ij1/1-m.Therefore, the heterogeneous fuzzy distance, denoted asm_difft,si,NMS, is calculated as(18)m_difft,si,NMS=∑sj∈NMSμijdifft,si,sj.Based on the fuzzy distances obtained by (16) and (18), updated (11) can be formulated as follows:(19)ρf,iω=∑t=1mωt∑c∉classsipci1-pclasssi∑j=1km_difft,si,NMSj-∑j=1km_difft,si,NHSj.Based on the formulas above, the objective function, which ismaxω,κ⁡J, can be denoted as(20)maxω,κJ=∑i=1nκiρf,iω+λκJeκi+λωJeωi,where the first item on the right side is the maximum distance of ReliefF, which is meant to determine the feature set that contributes most to classification. Meanwhile, from the aspect of entropy maximization, the second and the third items denote the sample evaluation operator and Shannon entropy of feature weight, respectively, which are used to avoid the over tendency problem of objective function. λκ and λω are the balance coefficients for adjusting the differences between features. When the maximum point of objective function is achieved, the constraint condition of the sample evaluation operator is defined below [30]:(21)κi=exp⁡∑j=1mωjdi,j/λκ∑i=1nexp⁡∑j=1mωjdi,j/λκ,where di,j denotes the partial derivative of ρf,iω.In the process of feature selection for mechanical system, the information entropy and fuzzy theory based optimization objective function ensures that the evaluation process of each subspace using ReliefF is adaptive and robust. Such advantage provides a new thought for feature processing of complex monitoring signal, in other words, under the premise of maintaining the high calculating efficiency of ReliefF method, reducing the bias and redundancy caused by methodological defects and external disturbances [31].


## 3. Method for Hydraulic Servo System Fault Diagnosis Based on CRSR

The CRSR based fault diagnosis method for hydraulic servo system consists of following successive steps: first, the average, standard deviation, skewness, and wavelet singular entropy features are extracted to form a feature matrix as the input of ReliefF model. Second, the initial contribution of the features sampled randomly is measured by calculating the inner-class distance and between-class distance, and the features of high contribution and features of low contribution are determined for fault classification. Theith iteration operation is based on the result of the previous iteration, and the iteration stops as long as i reaches the preset threshold. Third, based on the sorting result of the second step, the compressed evaluation of the features is realized using supervised sampling method in current iteration. Finally, keeping the iteration running until certain terms is satisfied to acquire the difference value as the criterion of feature selection. The detailed process of feature selection method using CRSR for hydraulic servo pump is shown in Figure 1.Figure 1
Process of CRSR based feature selection method for hydraulic servo pump.Compared with the traditional feature processing methods [32–35], the CRSR method is designed to meet the robustness and accuracy requirements of fault diagnosis with smaller feature set and lower resource consumption. This study demonstrates the advantages of the proposed method by extracting a variety of categories of features from the simulation data of hydraulic servo system and optimizes the feature set extracted by CRSR method to verify the feasibility in feature selection technique.


### 3.1. Description of the Simulation Environment of Hydraulic Servo System

The data used in this case is generated in HyPneu and Simulink joint simulation environment. The joint simulation process of hydraulic servo system can be divided into mechanical part and control part considering the characteristics of the hydraulic components [36–38]. The mechanical hydraulic physical part, including hydraulic pumps, servo valves, and actuators, is modeled using component library in HyPneu. The failure is realized by adding the modules of fault injection for the relevant components. The dynamic control part, including feedback sensors and electronic amplifiers, is established using the relevant model in Simulink. Meanwhile, the control of the hydraulic servo system and the simulation of fault injection can be realized by transmitting the signal data through relevant interface files. Thus, the model architectures of hydraulic servo system constructed using HyPneu and Simulink are shown in Figures 2 and 3, respectively.Figure 2
The failure injection model of mechanical hydraulic part using HyPneu.Figure 3
The failure injection model of dynamic control part using Simulink.As shown in Figure3, the Simulink model includes input signals, comparison elements, control elements, amplifying elements, and HyPneu module. The gain parameter of the electronic amplifying element is 80, and the control parameter of PID is set as proportional parameter of 1500, integral parameter of 0, and differential parameter of 5. To simulate the actual operating environment of hydraulic system, random noises in the range of −0.01 to 0.01 are added on the output of the HyPneu module.The process of joint simulation based on HyPneu and Simulink is as follows. First, the actuator receives the feedback signal converted by servo valve and right after that drives the load to do reciprocated motion. Then, the real-time displacement information of the load is collected by the sensor and transferred to the control circuit. Finally, the input signal and the amplified displacement signal are compared and inputted into the servo valve to build the negative feedback control logic of closed loop. The joint simulation model has a good ability to match the hydraulic system and to realize the fault injection of the hydraulic system effectively.


### 3.2. Fault Injection for Hydraulic Servo System and Multidimensional Feature Extraction

This study simulates four kinds of state of the hydraulic system including normal state, electronic amplifier fault, sensor constant deviation fault, and hydraulic pump wear fault. The detailed fault injection scheme is shown in Table1.Table 1
Fault injection scheme.
Fault mode
Fault phenomenon
Fault injection mode
Normal state
-
-
Electronic amplifier fault
Magnification notably increases
Multiplying the gain by 0.25
Sensor fault
Constant gain
Setting the feedback of the sensor to 0.5
Hydraulic wear fault
Output flow decreases
Setting flow-pressure coefficient of the variable orifice valve to 0.03The input of all fault modes is sinusoidal signal with amplitude being 1 and frequency being4πrad/s. As the expression of the input signal is y=sin⁡(4πt), the sampling frequency is 100 Hz and the sampling time is 70 s. The input-output relationships of the hydraulic servo system under circumstances of normal state and three fault states are collected, and every signal of the corresponding state contains 7000 points. The original output signals are shown in Figure 4.Figure 4
Original output signals of the hydraulic servo system.
(a)
Normal state
(b)
Amplifier fault
(c)
Sensor fault
(d)
Hydraulic pump wear faultAs mentioned above, the input of the proposed CRSR method is a feature matrix, and each column of the matrix represents a feature vector extracted from the original signal. As the prerequisite of feature selection, multidimensional features should be extracted for all of the signals. It should be noted that the cycle of hydraulic servo system signal collected for the feature selection should be an integer multiple of the input waveform cycle. The cycle of the input signal is 0.5 seconds and one input-output cycle contains 50 points, which means the object points of the feature extraction should be a geometric multiple of 50. In addition, in consideration of the fact that pump wear fault is a gradual degradation process and the degree of fault in the early phase may be weak, this study takes the last 4000 points of the original signal to extract the feature matrix. Moreover, a sampling window of 500-point length is used (with 5-point interval) to obtain 700 feature vectors.Furthermore, this study adds 15 dB Gaussian white noise to all the original signals to validate the performance of CRSR method. The details of the extracted features are shown in Table2.Table 2
Extracted features of the output signals of hydraulic servo system.
Feature
Feature magnitude
Normal
Amplifier fault
Sensor fault
Hydraulic pump wear fault
Average
−0.0446~−0.0411
0.0004~0.0036
−0.0734~−0.0698
−0.0100~−0.0063
Standard deviation
0.2715~0.2728
0.2745~0.2756
0.2715~0.2727
0.3068~0.3080
Mean square root
0.2746~0.2760
0.2742~0.2754
0.2806~0.2819
0.3067~0.3079
Skewness
0.0080~0.0148
0.0028~0.0099
0.0082~0.0149
0.0085~0.0174
Crest factor
1.4603~1.4608
1.4620~1.4685
1.4599~1.4675
1.8089~1.8189
Maximum amplitude of FFT
95.8092~96.288
96.8611~97.2778
95.8319~96.2598
107.5320~107.94
Total energy of wavelet
99.4925~99.746
99.4695~99.7350
99.5243~99.7664
98.5909~99.0743
Wavelet singular entropy
0.7682~0.8193
0.7701~0.8113
0.7857~0.8401
0.8579~0.9178
Maximum coefficient of wavelet
0.0012~0.0035
0.0005~0.0669
0.0004~0.0034
0.0005~0.0543As can be seen from Table2, the quantitative range of different fault features fluctuates much, which could lead to the bias problem in the feature selection process of ReliefF. Therefore, this study normalizes the features to the range of zero to one. The visualization of the feature vectors of different states is shown in Figure 5.Figure 5
Feature vectors of the hydraulic servo system in different states.As is shown in Figure5, data confusion exists in the features of different fault modes, which makes it difficult to select satisfied feature subsets through artificial observation. Thus, CRSR method is designed to select the feature combination with higher contribution to the fault diagnosis adaptively.


### 3.3. Analysis and Comparison on Feature Selection Result

The input of CRSR feature selection model is a9×700 matrix acquired by regularizing the 9 sets of the obtained feature vectors. One of the advantages of CRSR is the ability to calculate the distances of either same class or different classes adaptively and iteratively. Based on the ReliefF feature selection method, the corresponding distribution vectors of feature weight can be obtained.During the iteration process of ReliefF, feature selection constraint is established using compressed stochastic subspace method, which optimizes the performance and efficiency of CRSR. The parameters are as follows: number of iterations: 30; feature base of the subspace: 6. And the threshold condition of feature selection is that the difference between a feature being selected as “one with high contribution” and “one with low contribution” is greater than 10. WithF1–F9 representing the feature parameters in Figure 5, the statistical result of the features being selected as “one with high contribution” and “one with lo contribution” is shown in Table 3.Table 3
Statistical results based on compressed stochastic subspace.
Feature
F
1
F
2
F
3
F
4
F
5
F
6
F
7
F
8
F
9
High contribution
28
26
22
0
12
26
3
14
1
Low contribution
0
0
0
17
1
0
2
6
21
Difference
28
26
22
−17
11
26
1
8
−20It can be seen from Table3 that the feature combination selected by CRSR method contains average (F1), standard deviation (F2), mean square root (F3), crest factor (F5), and the maximum amplitude of FFT (F6). Although wavelet singular entropy (F8) was selected as “one with high contribution” for 14 times during the iteration, it was selected as “one with low contribution” for 6 times. Such result reveals that wavelet singular entropy is able to contribute to classification sometimes, but the ensemble performance is not stable. Even in some feature subspace matrixes, it exerts a negative impact on the classification. The final statistical result of feature weights assessed by CRSR is shown in Figure 6.Figure 6
Statistical results based on compressed stochastic subspace.As is shown in Figure6, if the weight basis for ReliefF is set to 0.75, the feature combination selected by CRSR method will append features F7 (total energy of wavelet) and F8 (wavelet singular entropy). However, according to the statistical results from the CRSR model, the contribution of these two features in the integration process is unstable. In particular, as for the total energy of wavelet, the reason why its average weight coefficient is high is that the weight coefficient reaches 0.92 and 0.88 in the 8th and 22nd iterations, respectively, while in the remaining subspaces its contributions are lower than the average slightly. Such situation reveals that the total energy of wavelet could not meet the requirement of stability in feature selection. The result shows that the CRSR method is more reasonable than the traditional ReliefF, which reflects the advantages of ensemble learning in generalization.Based on the feature selection result using CRSR, redundancy analysis for the feature sequence is carried out in this study. The threshold of correlation coefficient is set to 0.8, and the correlation matrix of feature is shown in Table4.Table 4
Times of feature selections based on compressed random subspace.
Dependence analysis
F
1
F
2
F
3
F
5
F
6
F
1
1
0.42
0.69
0.17
0.16
F
2
-
1
0.56
0.09
0.44
F
3
-
-
1
0.31
0.61
F
5
-
-
-
1
0.82
F
6
-
-
-
-
1It can be seen from Table4 that the correlation coefficient of peak factor (F5) and the maximum amplitude of FFT transform (F6) are greater than the preset threshold, which indicates that redundancy exists in the fault diagnosis information provided by them. Therefore, only the maximum amplitude of FFT transform (F6) is retained. In summary, the features selected by CRSR are average (F1), standard deviation (F2), mean square root (F3), and the maximum amplitude of FFT (F6).The purpose of introducing CRSR method is improving the performance of fault diagnosis. In other words, for a classifier, the diagnostic performance using selected feature set should not be less than that using the original feature set. The comparative models used in this study contain the classical ReliefF algorithm, the Mean Impact Value (MIV) algorithm, the Locally Linear Embedding (LLE) algorithm, and Kernel Principal Component Analysis (KPCA). In this study, the feature sets selected by different feature selection methods are used as inputs for the classifier based on Radial Basis Function (RBF) neural network. The ratio of training samples to test samples is 50%. The fault diagnosis accuracy of the hydraulic servo system is obtained by using the 10-fold cross validation method as shown in Table5.Table 5
Fault diagnosis results with different combinations of features.
Feature combination
Fault diagnosis accuracy of the 10-fold cross validation method (%)
F
1
+
F
2
+
F
3
+
F
6
94.62
95.17
94.29
94.40
93.81
94.11
93.54
95.12
95.01
93.98
F
1
+
F
2
+
F
3
+
F
5
+
F
6
94.48
93.61
93.01
93.77
94.15
95.25
93.99
94.58
94.62
93.71
F
1
+
F
2
+
F
3
+
F
5
+
F
6
+
F
7
+
F
8
92.04
93.62
93.11
93.25
91.37
94.54
93.27
92.32
93.29
93.41
F
1
+
F
3
+
F
4
+
F
5
93.02
93.57
92.15
92.89
92.46
94.19
93.32
93.55
92.53
92.77
KPCA1~KPCA4
93.08
93.78
93.07
93.57
92.17
93.04
92.80
92.42
93.48
93.36
LLE1~LLE4
94.10
93.59
94.51
94.22
93.98
94.10
93.99
93.44
93.98
94.78
F
1 ~F9
92.67
91.80
93.54
92.47
92.11
93.64
92.97
93.62
94.01
93.00In Table5, the first row (A1) represents the final feature set selected by CRSR; the second row (A2) represents the feature set selected by CRSR before the redundancy analysis; the third row (A3) represents the feature set selected by the classical ReliefF method; the fourth line (A4) represents the feature set selected by MIV; the fifth row (A5) and the sixth row (A6) are the high-dimensional feature sequences obtained by dimension reduction using KPCA and LLE, respectively, where the dimension reduction target is set to be same as the feature numbers determined by CRSR. The last row (A7) is the collection of original features. Taking the feature combinations mentioned above as input, the average and variance of fault diagnosis accuracy of hydraulic servo system calculated are shown in Figure 7, respectively.Figure 7
Fault diagnosis of hydraulic system under different combinations of features.According to the RBF network fault diagnostic results from Table5 and Figure 7, the following can be obtained.(
1
) Compared with using the original feature set directly, the feature selection methods are able to improve the precision of fault diagnosis based on RBF classifier, as the averages of the 10-fold cross validation from A1 to A7 are 94.405%, 94.217%, 93.022%, 93.045%, 93.077%, 94.069%, and 92.983%, respectively. The result indicates that the feature selection process plays a positive role in the classification task using high-dimensional data.(
2
) Compared with the original ReliefF method, the introduction of compressed random subspace method eliminates those features with unstable contribution in the process of subspace integration and improves the performance and efficiency of feature selection. Eventually, higher fault diagnosis accuracy can be achieved with fewer input features.(
3
) The variances calculated for A1 (with redundancy analysis) and A2 (without redundancy analysis) are 0.29 and 0.37, respectively. It can be seen that although the average diagnostic accuracy of them is close, the redundancy analysis can optimize the information repetition among the features and reduce the computational resource consumption.(
4
) Compared with the MIV-based feature selection method, CRSR has a great advantage in the diagnosis performance, which indicates that it has higher confidence in feature selection. Compared with KPCA and LLE, although the latter two (especially the LLE reduction algorithm) achieve high diagnostic accuracy as well, the variance of the 10-fold cross validation results for KPCA and LLE is higher than that of CRSR (0.29 and 0.31, resp.), which indicates that a certain degree of volatility exists in the results obtained by dimension reduction algorithms. In addition, due to the lack of clear interpretability and the existence of ambiguity in optimal target parameter setting, it is difficult to determine the best reduction method among the dimension reduction algorithms. Thus adaptive distance metrics based CRSR method has better applicability.To further illustrate the rationality of CRSR method, the diagnostic results were tested by rank-based nonparametric Kruskal-Wallis test, which is based on fault diagnosis accuracy as a basic indicator in Table5. The Kruskal-Wallis test was used to determine whether the different features sets have significant influence on the diagnostic performance. The significance threshold is set to 0.05, and the test results returned by Kruskal-Wallis function are shown in Figure 8. The p value returned by Kruskal-Wallis function is 0.00002 which is far less than 0.05. It indicates that the different combinations of features make significant impact on the fault diagnosis performance for hydraulic servo system.Figure 8
Nonparametric Kruskal-Wallis significance test results.Moreover, another improvement of CRSR is that the target optimization function is constructed based on the information entropy and the fuzzy theory. Compared with the distance measurement method used in the classical ReliefF model, the robustness of the feature selection process is improved in a complex environment. To illustrate the improvement of CRSR mentioned above, this study analyzes the original output signal and the output signal with 15 dB noise. Meanwhile, PCA is used to project two sets of the selected feature sequences into three-dimensional space. Furthermore, theK-means method is used to cluster the feature sets of different fault modes. The results are shown in Figure 9.Figure 9
Fault feature vector clustering results under different noise conditions.
(a)
CRSR cluster result without noise
(b)
CRSR cluster result with 15 dB noise
(c)
ReliefF cluster result without noise
(d)
ReliefF cluster result with 15 dB noiseIt can be seen from Figure9 that the feature qualities selected by CRSR and ReliefF are higher before the noise is added, and the feature sets corresponding to each failure mode can be clearly distinguished byK-means method. After adding 15 dB of noise, due to the influence of external disturbances, the data points of the amplifier gain fault and the sensor constant deviation fault are obviously confused in the clustering result of ReliefF, which indicates that the feature differentiation declines. However, in the result of CRSR method, spatial distribution boundaries between the four fault modes are still identifiable, which proves that the proposed objective function optimization method has good robustness against noise. Thus, CRSR method is a promising technique for feature selection and subsequent fault diagnosis of hydraulic servo systems.


## 3.1. Description of the Simulation Environment of Hydraulic Servo System

The data used in this case is generated in HyPneu and Simulink joint simulation environment. The joint simulation process of hydraulic servo system can be divided into mechanical part and control part considering the characteristics of the hydraulic components [36–38]. The mechanical hydraulic physical part, including hydraulic pumps, servo valves, and actuators, is modeled using component library in HyPneu. The failure is realized by adding the modules of fault injection for the relevant components. The dynamic control part, including feedback sensors and electronic amplifiers, is established using the relevant model in Simulink. Meanwhile, the control of the hydraulic servo system and the simulation of fault injection can be realized by transmitting the signal data through relevant interface files. Thus, the model architectures of hydraulic servo system constructed using HyPneu and Simulink are shown in Figures 2 and 3, respectively.Figure 2
The failure injection model of mechanical hydraulic part using HyPneu.Figure 3
The failure injection model of dynamic control part using Simulink.As shown in Figure3, the Simulink model includes input signals, comparison elements, control elements, amplifying elements, and HyPneu module. The gain parameter of the electronic amplifying element is 80, and the control parameter of PID is set as proportional parameter of 1500, integral parameter of 0, and differential parameter of 5. To simulate the actual operating environment of hydraulic system, random noises in the range of −0.01 to 0.01 are added on the output of the HyPneu module.The process of joint simulation based on HyPneu and Simulink is as follows. First, the actuator receives the feedback signal converted by servo valve and right after that drives the load to do reciprocated motion. Then, the real-time displacement information of the load is collected by the sensor and transferred to the control circuit. Finally, the input signal and the amplified displacement signal are compared and inputted into the servo valve to build the negative feedback control logic of closed loop. The joint simulation model has a good ability to match the hydraulic system and to realize the fault injection of the hydraulic system effectively.


## 3.2. Fault Injection for Hydraulic Servo System and Multidimensional Feature Extraction

This study simulates four kinds of state of the hydraulic system including normal state, electronic amplifier fault, sensor constant deviation fault, and hydraulic pump wear fault. The detailed fault injection scheme is shown in Table1.Table 1
Fault injection scheme.
Fault mode
Fault phenomenon
Fault injection mode
Normal state
-
-
Electronic amplifier fault
Magnification notably increases
Multiplying the gain by 0.25
Sensor fault
Constant gain
Setting the feedback of the sensor to 0.5
Hydraulic wear fault
Output flow decreases
Setting flow-pressure coefficient of the variable orifice valve to 0.03The input of all fault modes is sinusoidal signal with amplitude being 1 and frequency being4πrad/s. As the expression of the input signal is y=sin⁡(4πt), the sampling frequency is 100 Hz and the sampling time is 70 s. The input-output relationships of the hydraulic servo system under circumstances of normal state and three fault states are collected, and every signal of the corresponding state contains 7000 points. The original output signals are shown in Figure 4.Figure 4
Original output signals of the hydraulic servo system.
(a)
Normal state
(b)
Amplifier fault
(c)
Sensor fault
(d)
Hydraulic pump wear faultAs mentioned above, the input of the proposed CRSR method is a feature matrix, and each column of the matrix represents a feature vector extracted from the original signal. As the prerequisite of feature selection, multidimensional features should be extracted for all of the signals. It should be noted that the cycle of hydraulic servo system signal collected for the feature selection should be an integer multiple of the input waveform cycle. The cycle of the input signal is 0.5 seconds and one input-output cycle contains 50 points, which means the object points of the feature extraction should be a geometric multiple of 50. In addition, in consideration of the fact that pump wear fault is a gradual degradation process and the degree of fault in the early phase may be weak, this study takes the last 4000 points of the original signal to extract the feature matrix. Moreover, a sampling window of 500-point length is used (with 5-point interval) to obtain 700 feature vectors.Furthermore, this study adds 15 dB Gaussian white noise to all the original signals to validate the performance of CRSR method. The details of the extracted features are shown in Table2.Table 2
Extracted features of the output signals of hydraulic servo system.
Feature
Feature magnitude
Normal
Amplifier fault
Sensor fault
Hydraulic pump wear fault
Average
−0.0446~−0.0411
0.0004~0.0036
−0.0734~−0.0698
−0.0100~−0.0063
Standard deviation
0.2715~0.2728
0.2745~0.2756
0.2715~0.2727
0.3068~0.3080
Mean square root
0.2746~0.2760
0.2742~0.2754
0.2806~0.2819
0.3067~0.3079
Skewness
0.0080~0.0148
0.0028~0.0099
0.0082~0.0149
0.0085~0.0174
Crest factor
1.4603~1.4608
1.4620~1.4685
1.4599~1.4675
1.8089~1.8189
Maximum amplitude of FFT
95.8092~96.288
96.8611~97.2778
95.8319~96.2598
107.5320~107.94
Total energy of wavelet
99.4925~99.746
99.4695~99.7350
99.5243~99.7664
98.5909~99.0743
Wavelet singular entropy
0.7682~0.8193
0.7701~0.8113
0.7857~0.8401
0.8579~0.9178
Maximum coefficient of wavelet
0.0012~0.0035
0.0005~0.0669
0.0004~0.0034
0.0005~0.0543As can be seen from Table2, the quantitative range of different fault features fluctuates much, which could lead to the bias problem in the feature selection process of ReliefF. Therefore, this study normalizes the features to the range of zero to one. The visualization of the feature vectors of different states is shown in Figure 5.Figure 5
Feature vectors of the hydraulic servo system in different states.As is shown in Figure5, data confusion exists in the features of different fault modes, which makes it difficult to select satisfied feature subsets through artificial observation. Thus, CRSR method is designed to select the feature combination with higher contribution to the fault diagnosis adaptively.


## 3.3. Analysis and Comparison on Feature Selection Result

The input of CRSR feature selection model is a9×700 matrix acquired by regularizing the 9 sets of the obtained feature vectors. One of the advantages of CRSR is the ability to calculate the distances of either same class or different classes adaptively and iteratively. Based on the ReliefF feature selection method, the corresponding distribution vectors of feature weight can be obtained.During the iteration process of ReliefF, feature selection constraint is established using compressed stochastic subspace method, which optimizes the performance and efficiency of CRSR. The parameters are as follows: number of iterations: 30; feature base of the subspace: 6. And the threshold condition of feature selection is that the difference between a feature being selected as “one with high contribution” and “one with low contribution” is greater than 10. WithF1–F9 representing the feature parameters in Figure 5, the statistical result of the features being selected as “one with high contribution” and “one with lo contribution” is shown in Table 3.Table 3
Statistical results based on compressed stochastic subspace.
Feature
F
1
F
2
F
3
F
4
F
5
F
6
F
7
F
8
F
9
High contribution
28
26
22
0
12
26
3
14
1
Low contribution
0
0
0
17
1
0
2
6
21
Difference
28
26
22
−17
11
26
1
8
−20It can be seen from Table3 that the feature combination selected by CRSR method contains average (F1), standard deviation (F2), mean square root (F3), crest factor (F5), and the maximum amplitude of FFT (F6). Although wavelet singular entropy (F8) was selected as “one with high contribution” for 14 times during the iteration, it was selected as “one with low contribution” for 6 times. Such result reveals that wavelet singular entropy is able to contribute to classification sometimes, but the ensemble performance is not stable. Even in some feature subspace matrixes, it exerts a negative impact on the classification. The final statistical result of feature weights assessed by CRSR is shown in Figure 6.Figure 6
Statistical results based on compressed stochastic subspace.As is shown in Figure6, if the weight basis for ReliefF is set to 0.75, the feature combination selected by CRSR method will append features F7 (total energy of wavelet) and F8 (wavelet singular entropy). However, according to the statistical results from the CRSR model, the contribution of these two features in the integration process is unstable. In particular, as for the total energy of wavelet, the reason why its average weight coefficient is high is that the weight coefficient reaches 0.92 and 0.88 in the 8th and 22nd iterations, respectively, while in the remaining subspaces its contributions are lower than the average slightly. Such situation reveals that the total energy of wavelet could not meet the requirement of stability in feature selection. The result shows that the CRSR method is more reasonable than the traditional ReliefF, which reflects the advantages of ensemble learning in generalization.Based on the feature selection result using CRSR, redundancy analysis for the feature sequence is carried out in this study. The threshold of correlation coefficient is set to 0.8, and the correlation matrix of feature is shown in Table4.Table 4
Times of feature selections based on compressed random subspace.
Dependence analysis
F
1
F
2
F
3
F
5
F
6
F
1
1
0.42
0.69
0.17
0.16
F
2
-
1
0.56
0.09
0.44
F
3
-
-
1
0.31
0.61
F
5
-
-
-
1
0.82
F
6
-
-
-
-
1It can be seen from Table4 that the correlation coefficient of peak factor (F5) and the maximum amplitude of FFT transform (F6) are greater than the preset threshold, which indicates that redundancy exists in the fault diagnosis information provided by them. Therefore, only the maximum amplitude of FFT transform (F6) is retained. In summary, the features selected by CRSR are average (F1), standard deviation (F2), mean square root (F3), and the maximum amplitude of FFT (F6).The purpose of introducing CRSR method is improving the performance of fault diagnosis. In other words, for a classifier, the diagnostic performance using selected feature set should not be less than that using the original feature set. The comparative models used in this study contain the classical ReliefF algorithm, the Mean Impact Value (MIV) algorithm, the Locally Linear Embedding (LLE) algorithm, and Kernel Principal Component Analysis (KPCA). In this study, the feature sets selected by different feature selection methods are used as inputs for the classifier based on Radial Basis Function (RBF) neural network. The ratio of training samples to test samples is 50%. The fault diagnosis accuracy of the hydraulic servo system is obtained by using the 10-fold cross validation method as shown in Table5.Table 5
Fault diagnosis results with different combinations of features.
Feature combination
Fault diagnosis accuracy of the 10-fold cross validation method (%)
F
1
+
F
2
+
F
3
+
F
6
94.62
95.17
94.29
94.40
93.81
94.11
93.54
95.12
95.01
93.98
F
1
+
F
2
+
F
3
+
F
5
+
F
6
94.48
93.61
93.01
93.77
94.15
95.25
93.99
94.58
94.62
93.71
F
1
+
F
2
+
F
3
+
F
5
+
F
6
+
F
7
+
F
8
92.04
93.62
93.11
93.25
91.37
94.54
93.27
92.32
93.29
93.41
F
1
+
F
3
+
F
4
+
F
5
93.02
93.57
92.15
92.89
92.46
94.19
93.32
93.55
92.53
92.77
KPCA1~KPCA4
93.08
93.78
93.07
93.57
92.17
93.04
92.80
92.42
93.48
93.36
LLE1~LLE4
94.10
93.59
94.51
94.22
93.98
94.10
93.99
93.44
93.98
94.78
F
1 ~F9
92.67
91.80
93.54
92.47
92.11
93.64
92.97
93.62
94.01
93.00In Table5, the first row (A1) represents the final feature set selected by CRSR; the second row (A2) represents the feature set selected by CRSR before the redundancy analysis; the third row (A3) represents the feature set selected by the classical ReliefF method; the fourth line (A4) represents the feature set selected by MIV; the fifth row (A5) and the sixth row (A6) are the high-dimensional feature sequences obtained by dimension reduction using KPCA and LLE, respectively, where the dimension reduction target is set to be same as the feature numbers determined by CRSR. The last row (A7) is the collection of original features. Taking the feature combinations mentioned above as input, the average and variance of fault diagnosis accuracy of hydraulic servo system calculated are shown in Figure 7, respectively.Figure 7
Fault diagnosis of hydraulic system under different combinations of features.According to the RBF network fault diagnostic results from Table5 and Figure 7, the following can be obtained.(
1
) Compared with using the original feature set directly, the feature selection methods are able to improve the precision of fault diagnosis based on RBF classifier, as the averages of the 10-fold cross validation from A1 to A7 are 94.405%, 94.217%, 93.022%, 93.045%, 93.077%, 94.069%, and 92.983%, respectively. The result indicates that the feature selection process plays a positive role in the classification task using high-dimensional data.(
2
) Compared with the original ReliefF method, the introduction of compressed random subspace method eliminates those features with unstable contribution in the process of subspace integration and improves the performance and efficiency of feature selection. Eventually, higher fault diagnosis accuracy can be achieved with fewer input features.(
3
) The variances calculated for A1 (with redundancy analysis) and A2 (without redundancy analysis) are 0.29 and 0.37, respectively. It can be seen that although the average diagnostic accuracy of them is close, the redundancy analysis can optimize the information repetition among the features and reduce the computational resource consumption.(
4
) Compared with the MIV-based feature selection method, CRSR has a great advantage in the diagnosis performance, which indicates that it has higher confidence in feature selection. Compared with KPCA and LLE, although the latter two (especially the LLE reduction algorithm) achieve high diagnostic accuracy as well, the variance of the 10-fold cross validation results for KPCA and LLE is higher than that of CRSR (0.29 and 0.31, resp.), which indicates that a certain degree of volatility exists in the results obtained by dimension reduction algorithms. In addition, due to the lack of clear interpretability and the existence of ambiguity in optimal target parameter setting, it is difficult to determine the best reduction method among the dimension reduction algorithms. Thus adaptive distance metrics based CRSR method has better applicability.To further illustrate the rationality of CRSR method, the diagnostic results were tested by rank-based nonparametric Kruskal-Wallis test, which is based on fault diagnosis accuracy as a basic indicator in Table5. The Kruskal-Wallis test was used to determine whether the different features sets have significant influence on the diagnostic performance. The significance threshold is set to 0.05, and the test results returned by Kruskal-Wallis function are shown in Figure 8. The p value returned by Kruskal-Wallis function is 0.00002 which is far less than 0.05. It indicates that the different combinations of features make significant impact on the fault diagnosis performance for hydraulic servo system.Figure 8
Nonparametric Kruskal-Wallis significance test results.Moreover, another improvement of CRSR is that the target optimization function is constructed based on the information entropy and the fuzzy theory. Compared with the distance measurement method used in the classical ReliefF model, the robustness of the feature selection process is improved in a complex environment. To illustrate the improvement of CRSR mentioned above, this study analyzes the original output signal and the output signal with 15 dB noise. Meanwhile, PCA is used to project two sets of the selected feature sequences into three-dimensional space. Furthermore, theK-means method is used to cluster the feature sets of different fault modes. The results are shown in Figure 9.Figure 9
Fault feature vector clustering results under different noise conditions.
(a)
CRSR cluster result without noise
(b)
CRSR cluster result with 15 dB noise
(c)
ReliefF cluster result without noise
(d)
ReliefF cluster result with 15 dB noiseIt can be seen from Figure9 that the feature qualities selected by CRSR and ReliefF are higher before the noise is added, and the feature sets corresponding to each failure mode can be clearly distinguished byK-means method. After adding 15 dB of noise, due to the influence of external disturbances, the data points of the amplifier gain fault and the sensor constant deviation fault are obviously confused in the clustering result of ReliefF, which indicates that the feature differentiation declines. However, in the result of CRSR method, spatial distribution boundaries between the four fault modes are still identifiable, which proves that the proposed objective function optimization method has good robustness against noise. Thus, CRSR method is a promising technique for feature selection and subsequent fault diagnosis of hydraulic servo systems.


## 4. Conclusion

This study presents a fault diagnosis scheme for hydraulic servo system using compressed random subspace based ReliefF (CRSR) method. Based on the feature selection structure of ReliefF, the proposed CRSR method employs feature integration rules in the compressed domain and substitutes information entropy and fuzzy membership for traditional distance measure index. The advantage of the proposed method lies in the ability of determining the feature set with the better generalization performance and the less resource consumption. As a data-driven method, CRSR could be practical and flexible in engineering.To demonstrate the effectiveness of the proposed CRSR method, validation data of three fault modes is generated through a hydraulic servo system joint simulation model. Comparing with existing feature reduction and feature selection methods, the result indicates that the feature selection process plays a positive role in the classification task using high-dimensional data, and CRSR based fault diagnosis method has higher average accuracy and smaller variance. Meanwhile, the compressed random subspace method can eliminate those features with unstable contribution in the process of subspace integration and improve the performance and efficiency of feature selection. Besides, due to the robustness and stability of the information entropy and fuzzy theory based objective function optimization, the result shows that CRSR method is more suitable for fault diagnosis problem under noisy conditions.



---

*Source: 8740989-2018-02-18.xml*