# A New Intelligent Fault Diagnosis Method of Rotating Machinery under Varying-Speed Conditions Using Infrared Thermography

Authors: Yongbo Li; Xianzhi Wang; Shubin Si; Xiaoqiang Du
Journal: Complexity
 (2019)

---

## Abstract

A novel systematic framework, infrared thermography- (IRT-) based method, for rotating machinery fault diagnosis under nonstationary running conditions is presented in this paper. In this framework, IRT technique is first applied to obtain the thermograph. Then, the fault features are extracted using bag-of-visual-word (BoVW) from the IRT images. In the end, support vector machine (SVM) is utilized to automatically identify the fault patterns of rotating machinery. The effectiveness of proposed method is evaluated using lab experimental signal of rotating machinery. The diagnosis results show that the IRT-based method has certain advantages in classification rotating machinery faults under nonstationary running conditions compared with the traditional vibration-based method.


---

## Body

## 1. Introduction

Monitoring the health condition, prognostics the possible faults can be of importance for the safety operation of rotating machinery. To achieve this purpose, vibration signal is the most commonly used signal resource and the most widely applied method [1–7]. The recent progress of the vibration-based fault diagnosis can be reviewed from Ref. [8–11]. However, the vibration-based method confronts challenges when the rotating machinery operates under varying-speed conditions [12–14] because the obtained vibration signal represents complex nonstationary characteristics, such as frequency modulation, amplitude modulation, and phase modulation, resulting in the difficulty of fault frequency detection. Moreover, it is not allowed to install a vibration transducer onto the machinery due to the cost, environment, and space restrictions [15]. To overcome such defects, the most achievable way is to find alternative signal source to accomplish the health monitor of rotating machinery.The rapid advanced technique developments of measurement and instrumentation bring convenience for the fault diagnosis. Nowadays, various of sensors have been developed in the fault diagnosis field, including acoustic sensors [9, 16], sound emission sensors [17], current transducers [18], and rotating encoders [15]. However, these sensors have their own limitations in the real applications [9]. Therefore, an alternative signal source which can avoid the common defects of the above mentioned sensors may result in a revolution for the health condition monitoring of mechanical equipment under nonstationary running conditions. Nowadays, the infrared thermography (IRT) technique raises increasing attention in the fault diagnostic field [9, 19–22]. IRT-based method is considered as a promising noninvasive technique to diagnose the various faults and guarantee the safety operation [23].IRT is a noncontact technique for temperature acquisition, which has been widely used in aerospace [24], medical [25], freeze-drying [26], cultural heritage [27], and energy saving [28, 29]. Recent researches have demonstrated that IRT images also contain rich diagnostic information [24, 30, 31]. Glowacz et al. utilized method of area selection of states (MoASoS) based on IRT images to identify different stator faults of the single-phase induction motor [32]. Janssens et al. adopted three features (standard deviation of the temperature, the Gini coefficient, and the moment of light) to extract fault features from the IRT images. Then the support vector machine (SVM) is used to automatically identify fault patterns of rolling bearings [33]. Younus Janssens et al. proposed an integrated method based on discrete wavelet transform, feature selection tool, and two classifiers using the IRT images [34].In comparison with the traditional vibration-based methods, the IRT-based fault diagnosis method has some main advantages. First, IRT-based method has the merits of easy-of-setup, noncontact, and nonintrusive. It will not cause surface damage to the subject when it works. Second, IRT-based method has the advantage of high-scalability. A high-definition IRT camera can synchronously monitor multiple objects. However, one accelerometer can barely monitor the entire large-scale equipment due to the limitation of long transition path. Third and most important, IRT is immune to the influence of nonstationary running operations. However, vibration-based method is difficult to deal with the vibration signal with heavy strong noise, especially when they are working under nonstationary running conditions. Furthermore, prior knowledge is free for IRT-based fault diagnosis method. When the rotating machinery operates with localized damage, it would excite the higher temperature generated by the friction. This would cause that the IRT images represent different characteristics; thereby, the IRT image-based method can be used to diagnose the various faults of rotating machinery.There are three stages in the IRT-based diagnostic method: data acquisition, feature extraction, and fault pattern identification. Among these, the key step of success is the fault feature extraction. In this paper, bag-of-visual-word (BoVW) is employed for fault feature extraction [35]. Compared with vibration-based method [36], BoVW preforms better in the fault feature extraction. In the fault pattern identification stage, a classifier is needed to recognize different fault patterns of rotating machinery. In this paper, a widely used multiclass identification method called support vector machine (SVM) is adopted to automatically identify various health conditions. SVM has been proved which has good performance in solving the small sample and nonlinear classification problem [37, 38]. Above all, a novel fault diagnosis method of rotating machinery based on IRT, BoVW, and SVM is proposed in this paper. The proposed method has been proved to be effective in diagnosing various faults of rotating machinery.This paper is arranged as follows: In Section2, the basic principle of BoVW is described in detail. The theory of SVM is recalled in Section 3. In Section 4, the roposed method is utilized to identify different fault types of the rotating machinery. Finally, Section 5 draws the conclusion. We note that a shorter conference version of this paper appeared in 2018 International Conference on Sensing, Diagnostics, Prognostics, and Control [39]. The initial conference paper did not address the problem of temperature fluctuation of IRT images. This manuscript addresses this issue and provides additional analysis on the classification accuracies trained by different percentage of samples.


## 2. Fault Feature Extraction Based on Bag-of-Visual-Words (BoVW)

BoVW is widely applied in image classification. It transforms an image into visual words and then uses a histogram to make the occurrences frequency of the visual words statistical. Such histogram is regarded as the features of a thermal image to distinguish different faults of rotating machinery. BoVW can be obtained by two steps as follows. First, adopt the SIFT method to obtain the visual words from the picked area of the thermographs. Then, the obtained characteristics can be encoded as a histogram by making the occurrences frequency of the visual words statistical. Figure1 displays the process of the BoVW algorithm.Figure 1
The flow diagram of the BoVW.


### 2.1. SIFT Feature Extraction

Since the whole IRT images of each condition are very similar, they are not sensitive to various faults of rotating machine. Only a few interest regions close to the test bearing in the IRT images are useful for fault diagnosis. To precisely extract the features from thermal images, SIFT was proposed to partition the interest points from the whole IRT images [40]. The detailed description of SIFT is given as follows.With the help of scale-space extreme in the difference of Gaussian function [41], it is effective to detect the stable locations of the interest points in scale space. The scale space Lp,σ is defined as the convolution of a variable-scale Gaussian function Gp,σ with the original image Ip, which can be expressed as(1)Lp,σ=Gp,σ∗Ipwhere ∗ represents the convolution. p=x,y represents a point in the IRT image.The Gaussian functionGp,σ can be written as(2)Gp,σ=12πσ2exp⁡-x2+y22σ2Via convolving the scale-space extreme usingDp,σ, we can separate the difference of two scales using a multiplicative index k, which can be written as(3)Dp,σ=Gp,kσ-Gp,σ∗Ip=Lp,kσ-Lp,kσwhere ∗ represents the convolution.The local extreme of the functionD can be detected through accurate localization of interest points. Following the method in [42], Taylor expansion of the scale-space function Dp,σ can be given by Eq.(4).(4)Dp,σ=D+∂DT∂pp+12pT∂2D∂2ppFollowing Eq.(4), the inherent fault information can be extracted as the SIFT descriptor. In real application, the above procedure can be conducted in the Open Source Computer Vision Library (OpenCV) environment. In the following study, we set the dimension of feature vectors as 128.


### 2.2. Feature Encoding and Histogram Generation

Before feature encoding, a representative of several similar interest points called vocabulary is generated usingk-means clustering [43]. First, a set of k clusters is learned by clustering the features extracted from each fault state into a specified vocabulary size. Then, the centers of the learned clusters are defined as the vocabulary. In this paper, a set of 128-dimensional vectors of SIFT feature x1,x2,⋯,xn is given. Through k-means clustering, the n SIFT vectors are partitioned into k different sets such that S=S1,S2,⋯,Sk with minimized intracluster squared summed error (SSE), which is formulated as(5)S=argminS∑i=1k∑x∈Six-μi2where μi represents the mean vector in Si.Note that the vocabulary sizek is set to 1000 for better discrimination capability. With this constructed vocabulary S, the extracted SIFT features are quantized as the label of the closest cluster [44]. Therefore, an image is abstracted as the frequency counts or histogram of the quantized SIFT features f1,f2,⋯,fi,⋯,fk, where fi denotes the frequency of ith visual word in the images.


## 2.1. SIFT Feature Extraction

Since the whole IRT images of each condition are very similar, they are not sensitive to various faults of rotating machine. Only a few interest regions close to the test bearing in the IRT images are useful for fault diagnosis. To precisely extract the features from thermal images, SIFT was proposed to partition the interest points from the whole IRT images [40]. The detailed description of SIFT is given as follows.With the help of scale-space extreme in the difference of Gaussian function [41], it is effective to detect the stable locations of the interest points in scale space. The scale space Lp,σ is defined as the convolution of a variable-scale Gaussian function Gp,σ with the original image Ip, which can be expressed as(1)Lp,σ=Gp,σ∗Ipwhere ∗ represents the convolution. p=x,y represents a point in the IRT image.The Gaussian functionGp,σ can be written as(2)Gp,σ=12πσ2exp⁡-x2+y22σ2Via convolving the scale-space extreme usingDp,σ, we can separate the difference of two scales using a multiplicative index k, which can be written as(3)Dp,σ=Gp,kσ-Gp,σ∗Ip=Lp,kσ-Lp,kσwhere ∗ represents the convolution.The local extreme of the functionD can be detected through accurate localization of interest points. Following the method in [42], Taylor expansion of the scale-space function Dp,σ can be given by Eq.(4).(4)Dp,σ=D+∂DT∂pp+12pT∂2D∂2ppFollowing Eq.(4), the inherent fault information can be extracted as the SIFT descriptor. In real application, the above procedure can be conducted in the Open Source Computer Vision Library (OpenCV) environment. In the following study, we set the dimension of feature vectors as 128.


## 2.2. Feature Encoding and Histogram Generation

Before feature encoding, a representative of several similar interest points called vocabulary is generated usingk-means clustering [43]. First, a set of k clusters is learned by clustering the features extracted from each fault state into a specified vocabulary size. Then, the centers of the learned clusters are defined as the vocabulary. In this paper, a set of 128-dimensional vectors of SIFT feature x1,x2,⋯,xn is given. Through k-means clustering, the n SIFT vectors are partitioned into k different sets such that S=S1,S2,⋯,Sk with minimized intracluster squared summed error (SSE), which is formulated as(5)S=argminS∑i=1k∑x∈Six-μi2where μi represents the mean vector in Si.Note that the vocabulary sizek is set to 1000 for better discrimination capability. With this constructed vocabulary S, the extracted SIFT features are quantized as the label of the closest cluster [44]. Therefore, an image is abstracted as the frequency counts or histogram of the quantized SIFT features f1,f2,⋯,fi,⋯,fk, where fi denotes the frequency of ith visual word in the images.


## 3. Support Vector Machine

Support vector machine (SVM) has been widely accepted in both academia and industry due to its good performance in solving the small sample and nonlinear classification problem [37, 38]. SVM is based on the principle of structural risk minimization, which is able to find the optimal separating hyperplane; thereby, the classification problem is transformed into linearly separable [37]. In this study, SVM is utilized to accomplish the pattern identification.For a group of sample setS=(xi,yi)i=1n∣xi∈RN,yi∈{-1,1},i=1,2,⋯,l, where xi is sample data and yi is sample category. The searching for the optimal separating hyperplane can be written into the following functions and constraints in Eq.(6).(6)min12w2s.t.yiwxi+b≥1where w denotes the weight vector and b represents the bias vector.In order to assure the higher classification accuracy, the optimization problem can be rewritten using a relaxing factorξi≥0i=1,2,⋯,n as expressed in Eq.(7).(7)min12w2+c∑i=1nξiξi≥0s.t.yiwxi+b≥1-ξic≥0i≥1,2,⋯,nwhere c denotes the penalty factor and b represents the bias vector.The contradiction between algorithm complexity and classification accuracy can be alleviated through adjusting the parameterc. As a result, the optimization problem can be further rewritten as(8)maxWa=∑i=1nai-12∑i,j=1naiaiyiyiKxixjs.t.∑i=1naiyii≥1,2,⋯,nwhere K(xixj)=φT(xi).φ(xi) and • is the inner product operation. Therefore, the final classification results can be estimated by a decision function f(x) as expressed in Eq.(9).(9)fx=sign∑i=1naiyiφTxi.φxi+b=sign∑i=1naiyiKxi,xj+bNote that the kernel function plays a crucial role in SVM; radial basis function (RBF) kernel is employed in this paper due to its universal application and good performance [45].(10)Kx,xi=exp⁡-rx-xi2where r>0, and d and r are the kernel parameters. It is worthy noticing that the parameters, penalty parameter C and the kernel parameter γ of SVM, are optimized by Genetic Algorithm (GA) algorithm for each multiclass SVM method. The ranges of C and γ are set as (0.1, 1000) and (0.001, 10) according to Ref. [46].


## 4. Experimental Validation

To show the effectiveness of the proposed method, we have conducted a series of experiments on the lab experiment called SpectraQuest Machinery Fault Simulator (MFS). The MFS (shown in Figures2(a) and 3(a)) consists of a drive motor, test bearing and rotor, and a data acquisition system. The test bearing is MB ER-12K (shown in Figure 2). To simulate different bearing fault, five fault types of bearing were used, including normal, ball fault (BF), outer race fault (ORF), inner race fault (IRF), and combination fault (CF). The unbalance state was realized by adding weights, as displayed in Figure 2(b). The setting parameters of the MFS are provided in Table 1. During the experiment, the thermal image is collected using two thermal cameras Hawk-1384, called Camera 1 and Camera 2, as shown in Figure 3(a). The layout of the thermal cameras is shown in Figure 3(b); we can find that Camera 1 focuses on the localized region close to the test bearing and Camera 2 is used to scan the whole rotating machine. The parameters setting of the thermal camera is listed in Table 2. The accelerometer was also mounted on the case of the bearing block to collect the vibration signals for comparison purpose. The sampling frequency was 12800Hz and the rotating speed was linear accelerated from 900 rpm to 3000 rpm. In the experiments, ten healthy conditions were designed, including Normal, BF, ORF, CF, IRF, unbalance fault (UN), outer race fault with unbalance fault (ORFUN), inner race fault with unbalance fault (IRFUN), combination fault with unbalance fault (CFUN), and ball fault with unbalance fault (BFUN). For each health condition, we collect the thermal images using two thermal cameras under varying-speed condition following the detailed steps in Algorithm 1.Table 1
Working parameters of the rotating machinery (Table1 is reproduced from the study by Li et al. (2018), under the Creative Commons Attribution License/public domain).
Bearing specs MB ER-12K  Number of rolling elements 8  Rolling element diameter 0.3125 inches  Pitch diameter 1.318 inches  Contact angle 0 °  Sampling frequency 12800Hz  Speed range 900RPM to 3000RPM  Angular acceleration 175r/s2Table 2
Configuration parameters setting of the thermal camera (Table2 is reproduced from the study by Li et al. (2018), under the Creative Commons Attribution License/public domain).
Configuration parameters Values  Producer of thermal camera Hawk, China  Image resolution 384×288  Frame rate 25fps  Temperature measurement range - 25 °C~260°C  Environmental temperature 18.9 °C  Thermal sensitivity 0.05 °C  Palette rainbow  Contrast 50  Brightness 50  Gain 2Algorithm 1:Steps for the thermal image acquisition.
Step1: Select health condition Step2: The initial temperature of test rig is set to the environment temperature (18.9°C in this paper) Step3: Heat up the test rig under the constant speed of 3000rpm Step4: When the highest temperature of test rig achieves a given temperature (60°C in this paper), use the thermal camera to acquire the thermographs under the varying-speed conditions (accelerated from 900rpm to 3000rpm) Step5: Cool down the test rig to the environment temperature Step6: Select another healthy condition and repeat the steps 1-5 until all the healthy condition collected.Figure 2
(a) The machinery fault simulator system; (b) the unbalance rotor and attaching weights; (c) the testing bearings with five health conditions.
(a) (b) (c)Figure 3
(a) The machinery fault simulator system; (b) the layout of the thermal camera (Figure3 is reproduced from the study by Li et al. (2018), under the Creative Commons Attribution License/public domain).
(a) (b)In this experiment, there are 100 samples in each healthy condition. Thus, there are 1000 samples in total. To show the superiority of the proposed IRT-based method, only 6% of samples were used in the training process. The remaining samples were used to test the performance of the proposed method. The detailed numbers of the training samples and testing samples are shown in Table3.Table 3
The numbers of lab experimental data.
Fault class Class label Number of training datasets Number of testing datasets BF 1 6 94  BFUN 2 6 94  CF 3 6 94  CFUN 4 6 94  IRF 5 6 94  IRFUN 6 6 94  Normal 7 6 94  UN 8 6 94  ORF 9 6 94  ORFUN 10 6 94The thermal images acquisition from Camera 1 and Camera 2 under ten health conditions is shown in Figures4 and 5, respectively. However, it is difficult to identify the healthy condition through direct observation of the thermographs. This can be attributed to the fact that the difference between the thermal images of each health condition is very small.Figure 4
Thermal images acquisition from Camera 1 under different health conditions: (a) ball fault. (b) ball fault and unbalance fault. (c) Combination bearing fault. (d) Combination bearing fault and unbalance fault. (e) Inner race fault. (f) Inner race fault and unbalance fault. (g) Normal condition. (h) Unbalance fault. (i) Outer race fault. (j) Outer race fault and unbalance fault (Figure4 is reproduced from the study by Li et al. (2018), under the Creative Commons Attribution License/public domain).
(a) (b) (c) (d) (e) (f) (g) (h) (i) (j)Figure 5
Thermal images acquisition from Camera 2 under different health conditions: (a) ball fault. (b) Ball fault and unbalance fault. (c) Combination bearing fault. (d) Combination bearing fault and unbalance fault. (e) Inner race fault. (f) Inner race fault and unbalance fault. (g) Normal condition. (h) Unbalance fault. (i) Outer race fault. (j) Outer race fault and unbalance fault (Figure5 is reproduced from the study by Li et al. (2018), under the Creative Commons Attribution License/public domain).
(a) (b) (c) (d) (e) (f) (g) (h) (i) (j)First, the SIFT algorithm is adopted to extract fault characteristics from IRT images. Then, we encode the features. In the end, the obtained BoVW features are taken as input of the SVM classifier to automatically identify the health conditions. In this paper, both of the IRT images acquisitions from Camera 1 and Camera 2 are utilized for the validation of IRT-based method. The classification results are shown in Figures6 and 7, respectively. From Figures 6 and 7, we can see that no testing samples are misclassified and the testing accuracy achieves 100% using the thermographs acquired from Camera 1, as shown in Figure 6. However, nine testing samples are misclassified using the images acquisition from Camera 2 with the final testing accuracy 99.04%. It indicates that Camera 1 is more sensitive to the various faults comparing with Camera 2. Meanwhile, the testing accuracies using thermal images from two cameras are both over 99.0%, which means the proposed IRT-based method can effectively classify the ten faults of rotating machinery under nonstationary running conditions.Figure 6
The classification results of the proposed method using thermal images from Camera 1.Figure 7
The classification results of the proposed method using thermal images from Camera 2.Furthermore, we investigate the classification results of the proposed IRT-based method trained by various percentages of samples. We run 20 times for each experiment to reduce the randomness. Then the averaging training and testing accuracies are obtained and their corresponding standard deviations are represented using the positive error bars. Figure8 illustrates the diagnosis results using the thermal images from two cameras. We can observe that the testing accuracy increases and its standard deviation decreased with the rise of percentage of training samples. In Figure 8(a), it is seen that the proposed method with Camera 1 classifies the ten health conditions of rotating machinery with testing accuracy 94.55% using only 2% of samples for training. The testing accuracy achieves 100% with 0 stand deviation using only 6% percentage of training samples from Camera 1. However, from Figure 8(b) it can be seen that the proposed method with Camera 2 reaches 100.00% requiring 20% of samples for training. This indicates that the proposed IRT-based method with Camera 1 performs better than that with Camera 2, which still has a high testing accuracy even in the situation of lacking training samples.Figure 8
The classification results of the proposed method trained by different percentage of samples: (a) thermal images from thermal Camera 1. (b) Thermal images from thermal Camera 2.
(a) (b)Meanwhile, we tested the testing accuracy of 100% using cross-validation (CV) method [47]. In this paper, 6-fold CV scheme is employed for experiment using Camera 1 when the training percentage reaches 6%. For experiment using Camera 2, 10-fold CV scheme is employed when the training percentage reaches 20%. For each CV scheme, the training dataset is randomly divided into several subsets. Each subset is verified on the SVM classifier that was trained using the other subsets. The process was repeated several times and the error rate of the SVM is calculated by averaging of the error rates obtained in each test fold. The obtained results are given in Table 4. We can find that the average training accuracy and its error rate of Camera 1 are 99.88% and 0.0010, respectively. For experiment using Camera 2, the corresponding average training accuracy and its error rate are 99.91% and 0.0012, respectively. This validates that our proposed method has good performance in recognizing various faults of rotating machinery.Table 4
The average training accuracy and error rate using cross-validation method.
Experiments Average training accuracy Error rate Camera 1 99.88% 0.0010  Camera 2 99.91% 0.0012To show the superiority of our proposed IRT-based fault diagnosis method, we compare the proposed method with reported vibration-based method [36]. Figure 9 displays the time domain waveforms of rotating machinery under varying-speed conditions. Seen from Figure 9, it is difficult to identify the healthy conditions from the waveforms. This may be attributed to the complex dynamical response caused by varying rotating speed [48].Figure 9
The waveforms of rotating machinery under varying-speed conditions: (a) ball fault. (b) Ball fault with unbalance fault. (c) Combination bearing fault. (d) Combination bearing fault with unbalance fault. (e) Inner race fault. (f) Inner race fault with unbalance fault. (g) Normal. (h) Unbalance fault. (i) Outer race fault. (j) Outer race fault with unbalance fault (Figure9 is reproduced from the study by Li et al. (2018), under the Creative Commons Attribution License/public domain).
(a) (b) (c) (d) (e) (f) (g) (h) (i) (j)Then, the wavelet energy combined with Shannon entropy is used to extract fault characteristics from the raw vibration signals and the SVM is used to recognize the ten health conditions. Note that the vibration-based method uses 33% of samples for training [36]. The classification result using the vibration-based method is displayed in Figure 10. It can be seen that the vibration-based method achieves testing accuracy 79.11% (530/670), which is much lower than the proposed IRT-based method. This indicates that the proposed method can obtain a higher testing accuracy with less training samples.Figure 10
Recognition results using the vibration-based method.To show the superiority of the proposed method in the fault feature extraction, 2-D and three-dimensional projections are used for visualizing with principal component analysis (PCA), as plotted in Figures11(a) and 11(b), respectively. From Figure 11 it can be seen that each group of samples is clustered and has distinct boundaries using IRT-based method. However, in Figure 12, the boundary of each group of samples is ambiguous using the vibration-based method.Figure 11
Projections of the IRT image features: (a) two-dimensional; (b) three-dimensional.Figure 12
Projections of the extracted features using vibration-based diagnostic method: (a) two-dimensional; (b) three-dimensional.


## 5. Conclusions

In this paper, the IRT-based method is developed to diagnose various faults of rotating machinery under varying-speed conditions. The IRT-based method has been proven to be having merits of noncontact, nonintrusive, and high sensitivity to various faults. The effectiveness of the IRT-based method is validated by recognizing ten fault types of rotating machinery faults. Compared with traditional vibration-based method, experimental results indicate that the proposed method immunizes the influence of varying speed and provides an alternative way for the health monitoring of rotating machinery under varying-speed conditions.There are two possible directions for future investigation. First, the combination of IRT images and vibration may be more effective. Second, this primary study only proved the effectiveness of the IRT-based method. But the calculation efficiency requires to be further improved.



---

*Source: 2619252-2019-08-19.xml*