# Ferrography Wear Particles Image Recognition Based on Extreme Learning Machine

Authors: Qiong Li; Tingting Zhao; Lingchao Zhang; Wenhui Sun; Xi Zhao
Journal: Journal of Electrical and Computer Engineering
 (2017)

---

## Abstract

The morphology of wear particles reflects the complex properties of wear processes involved in particle formation. Typically, the morphology of wear particles is evaluated qualitatively based on microscopy observations. This procedure relies upon the experts’ knowledge and, thus, is not always objective and cheap. With the rapid development of computer image processing technology, neural network based on traditional gradient training algorithm can be used to recognize them. However, the feedforward neural network based on traditional gradient training algorithms for image segmentation creates many issues, such as needing multiple iterations to converge and easy fall into local minimum, which restrict its development heavily. Recently, extreme learning machine (ELM) for single-hidden-layer feedforward neural networks (SLFN) has been attracting attentions for its faster learning speed and better generalization performance than those of traditional gradient-based learning algorithms. In this paper, we propose to employ ELM for ferrography wear particles image recognition. We extract the shape features, color features, and texture features of five typical kinds of wear particles as the input of the ELM classifier and set five types of wear particles as the output of the ELM classifier. Therefore, the novel ferrography wear particle classifier is founded based on ELM.


---

## Body

## 1. Introduction

Wear particle analysis for machine condition monitoring and fault diagnosis is not a new topic in tribology. Roylance et al. [1] pioneered computer wear pattern recognition research direction by establishing the computer aided visual engineering (CAVE) to extract the wear particles’ Fourier factor, roundness factor, and edge details. Then, they used these features to classify portion wear particles [2–7]. Hamblin and Stachowiak proposed the spike parameter to extract characteristics of abrasive particles via adequate description of wear debris boundary in multiscale [8]. Xu et al. [9] developed a set of wear particle image analysis system through the integration of neural network and expert system, which can realize the interactive wear particles image automatic recognition. Tian et al. [10] and Peng and Kirk [11] employed laser scanning confocal microscopy to acquire three-dimensional particle images. The scanning method provides surface information from the analysis of wear surface morphology, which facilitates better understanding of wear features and wear level. Stachowiak et al. [12] utilized an automated classification system to analyze and identify fatigue, abrasive, and adhesive wear particles. Such analysis and identification operations were fulfilled mainly based on the analyses of the area perimeter, elongation parameters, convexity, and surface texture. van Otterlo [13] distinguished different points on the curve of an object boundary by means of the proposed parameters, which might represent points with respect to a reference position in polar coordinate system. This approach provided a unified theoretical basis for analyzing shape similarity according to the proposed parametric contour representations. Laghari and Memon developed an automatic analysis system, KBWPAS, to reduce the dependence on domain experts [14]. Wang combined principal component analysis and gray relational analysis to optimize the parameters of features of wear particles [15]. Wang and Yin used wear particle shape, color, and surface texture parameters as input vectors and introduced radial basis function neural network to conduct automatic classification and identification on wear particle [16]. Gu applied support vector machine to ferrography wear particle pattern recognition and built wear particle classifier based on supper vector machine [17]. Yuan and Yan took surface texture and surface roughness as the important indicators of three-dimensional surface characteristic and adopted artificial intelligent neural network method to identify wear particle type, which efficiently improved accuracy of wear particle classification [18]. Luo et al. proposed to use principal component analysis to extract characteristic parameters of wear particles and then introduced BP neural network to conduct automatic classification on wear particles. This method optimized parameters firstly, which yielded better classification rate compared to traditional BP neural network [19]. Zhou et al. used the same method to optimize parameters of wear particle features and applied the improved genetic algorithms to the LS-SVM’s parameter optimization, thus improving wear particle recognition rate [20].At present, a lot of image classification algorithms have been proposed, among which the neural network based image classification algorithms are widely used. However, the traditional feedforward neural networks need many iterations to converge and are easy to fall into the local optimum in the procession of image classification, which seriously limit its development and application. On the other hand, the extreme learning machine as a new machine learning algorithm has become more and more popular due to its few adjustable parameters, fast learning speed, and good generalization performance. However, the effectiveness of the application of ELM to the field of image processing of iron spectrum abrasive particles is open to be investigated. In this paper, we present the method to apply the extreme learning machine to the recognition of wear particles.The rest of the paper is organized as follows. Section2 introduces principles and methods used in this paper. In Section 3, we describe the proposed method. In Section 4, we describe the experiments. Section 5 finalizes the paper with the conclusions.


## 2. Principle and Method

### 2.1. Characteristic Parameters of Wear Particles

Generally, characteristic parameters play important role in the final result of wear particles recognition. Although the obtained ferroscope images of wear particles are not three-dimensional, the size, texture, and color features are enough for typical wear particles recognition.Among the shape and size factors of wear particles, area, perimeter, equivalent circle diameter, circularity, aspect ratio, concavity, and so on are commonly used features to be extracted from the color images, which are captured via the microscope. These shape and size parameters can be further approximated by converting the sample color image into binary image with the values of 0 and 1.Besides the shape and size parameters, texture is also very important in image analysis. The texture is the description of the pixel distribution of a picture in gray space. Machines or equipment with a pair of relatively moving contact surfaces for long-time operation tends to produce wear particles. Usually, the surface texture provides statistical information of the wear state in the machinery operation process. Textural features are investigated by using the distribution function to analyze the gray level of the characteristic region of wear particles. Among the parameters of wear particles, two of the characteristic features are employed to distinguish the surface texture, namely, roughness and directivity. Gray level cooccurrence matrix (GLCM) [21] is commonly used to describe the surface textural factors of wear particles. The textural features, including energy, entropy, dependency, inertia moment, and stability, can be extracted by employing the GLCM. However, the connection between the obtained textural features and human vision perceptual system has not been established. Based on the psychological representation of human vision perception to the texture, Tamura et al. proposed the expression of the textural feature, Tamura textural feature [22], which can be used to extract six properties of the textural features, contrast ratio, roughness, direction level, regularity, linearity, and coarseness. Moreover, we have found that HoG gives a good simulation of the variation of the particle images [23]. The histogram of oriented gradient for the local region of the wear particle images can be calculated by using HoG.Color feature is another kind of significant properties to analyze wear particle images. The color factor is important for the identification of the red oxides, black oxides, and fatigued copper wear particles. By analyzing the color of the wear particles, mechanical abrasion degree can be predicted. The image commonly consisted of three-primary colors of red (R), green (G), and blue (B). In the RGB space, the three-color factors of R(i,j), G(i,j), and B(i,j) denote the color features of wear particles. After the images of wear particles are processed, the first-order and second-order statistical values (mean and variance) of R(i,j), G(i,j), and B(i,j) and the third-order color matrix of the HSV color space are extracted as the color characteristic parameters.


### 2.2. Extreme Learning Machine

The extreme learning machine (ELM) algorithm was originally proposed by Huang et al. [24–27]. The method is proved to be a universal approximator given enough hidden neurons [28]. It works as follows.Consider a set ofN distinct samples xi,ti with xi∈Rd and ti∈Rc. Then a single layer feedforward network with L hidden neurons is modeled as(1)∑i=1Lβiϕwixj+bi,j∈1,Nwith ϕ being an activation function, wi the input weights, bi the biases, and βi the output weights.In the case where the single layer feedforward network would perfectly approximate the data, the errors between the estimated outputsyi and the targets ti are zero, and the relation between inputs, weights, and targets is(2)∑i=1Lβiϕwixj+bi=ti,j∈1,Nwhich can be written compactly as Hβ=T, with(3)H=ϕw1x1+b1⋯ϕwLx1+bL⋮⋱⋮ϕw1xN+b1⋯ϕwLxN+bLβ=β1T⋯βLT,T=t1T⋯tNT.Finding the output weightsβ from the hidden layer outputs H and targets is a linear regression problem. In the general case of N≠d, a minimum L2-norm solution is given by the Moore-Penrose generalized inverse, or pseudoinverse, of the matrix H denoted as H† [29]. The training of ELM requires no iterations, and the most computational efficient part is the calculation of the pseudoinverse of the matrix HNsL, which makes ELM an extremely fast artificial neural networks method.


## 2.1. Characteristic Parameters of Wear Particles

Generally, characteristic parameters play important role in the final result of wear particles recognition. Although the obtained ferroscope images of wear particles are not three-dimensional, the size, texture, and color features are enough for typical wear particles recognition.Among the shape and size factors of wear particles, area, perimeter, equivalent circle diameter, circularity, aspect ratio, concavity, and so on are commonly used features to be extracted from the color images, which are captured via the microscope. These shape and size parameters can be further approximated by converting the sample color image into binary image with the values of 0 and 1.Besides the shape and size parameters, texture is also very important in image analysis. The texture is the description of the pixel distribution of a picture in gray space. Machines or equipment with a pair of relatively moving contact surfaces for long-time operation tends to produce wear particles. Usually, the surface texture provides statistical information of the wear state in the machinery operation process. Textural features are investigated by using the distribution function to analyze the gray level of the characteristic region of wear particles. Among the parameters of wear particles, two of the characteristic features are employed to distinguish the surface texture, namely, roughness and directivity. Gray level cooccurrence matrix (GLCM) [21] is commonly used to describe the surface textural factors of wear particles. The textural features, including energy, entropy, dependency, inertia moment, and stability, can be extracted by employing the GLCM. However, the connection between the obtained textural features and human vision perceptual system has not been established. Based on the psychological representation of human vision perception to the texture, Tamura et al. proposed the expression of the textural feature, Tamura textural feature [22], which can be used to extract six properties of the textural features, contrast ratio, roughness, direction level, regularity, linearity, and coarseness. Moreover, we have found that HoG gives a good simulation of the variation of the particle images [23]. The histogram of oriented gradient for the local region of the wear particle images can be calculated by using HoG.Color feature is another kind of significant properties to analyze wear particle images. The color factor is important for the identification of the red oxides, black oxides, and fatigued copper wear particles. By analyzing the color of the wear particles, mechanical abrasion degree can be predicted. The image commonly consisted of three-primary colors of red (R), green (G), and blue (B). In the RGB space, the three-color factors of R(i,j), G(i,j), and B(i,j) denote the color features of wear particles. After the images of wear particles are processed, the first-order and second-order statistical values (mean and variance) of R(i,j), G(i,j), and B(i,j) and the third-order color matrix of the HSV color space are extracted as the color characteristic parameters.


## 2.2. Extreme Learning Machine

The extreme learning machine (ELM) algorithm was originally proposed by Huang et al. [24–27]. The method is proved to be a universal approximator given enough hidden neurons [28]. It works as follows.Consider a set ofN distinct samples xi,ti with xi∈Rd and ti∈Rc. Then a single layer feedforward network with L hidden neurons is modeled as(1)∑i=1Lβiϕwixj+bi,j∈1,Nwith ϕ being an activation function, wi the input weights, bi the biases, and βi the output weights.In the case where the single layer feedforward network would perfectly approximate the data, the errors between the estimated outputsyi and the targets ti are zero, and the relation between inputs, weights, and targets is(2)∑i=1Lβiϕwixj+bi=ti,j∈1,Nwhich can be written compactly as Hβ=T, with(3)H=ϕw1x1+b1⋯ϕwLx1+bL⋮⋱⋮ϕw1xN+b1⋯ϕwLxN+bLβ=β1T⋯βLT,T=t1T⋯tNT.Finding the output weightsβ from the hidden layer outputs H and targets is a linear regression problem. In the general case of N≠d, a minimum L2-norm solution is given by the Moore-Penrose generalized inverse, or pseudoinverse, of the matrix H denoted as H† [29]. The training of ELM requires no iterations, and the most computational efficient part is the calculation of the pseudoinverse of the matrix HNsL, which makes ELM an extremely fast artificial neural networks method.


## 3. Proposed Method

Extreme learning machine (ELM) is a new machine learning algorithm. It is increasingly favored by many researchers due to its simple structure, fast learning speed, and good generalization performance. This part will introduce extreme learning machine (ELM) method into the wear particles image recognition. The specific flow chart is shown in Figure1Figure 1
Wear particles recognition based on ELMThe exact procedures are as follows: The first four steps are image preprocessing as shown in Figure2, which include the steps of 3D media filter, K-means clustering segmentation, region growing, morphology expansion, and erosion; the fifth step is feature extraction and the last step is recognition.Figure 2
Image preprocessing.Step 1 (the three-dimensional median filter [30]).
Apply the median filter to the three components of RGB color images; then use the relevant algorithms incorporating three components to get three-dimensional image of median filter. The change of the picture is shown in Figure3.Figure 3
The three-dimensional median filter.Step 2 (K-means clustering segmentation [31]).
K-means clustering is used to segment wear particles images in Lab color mode. The calculation of 3D color images needs more computer memory and is time consuming, but the characteristic information of 3D color images contributes to the wear particles recognition. Compared with three-dimensional color images, two-dimensional color images not only have quick computing speed, but also can be well segmented. We apply K-means clustering to the two-dimensional color images; then we can get the wear particle image that contains the color information. We choose the Lab color mode for its relatively small mutual association between three components. As (a,b) component represents the color change and L component represents the change of brightness in a Lab mode image, we use (a,b) component for K-means clustering. Firstly, the input of wear particle image is transformed from RGB space to Lab space. Randomly generate three initial clustering points for K-means clustering algorithm. At last, the cluster region with the most complete target region is selected as the target segmentation region.Step 3 (region growing [32]).
K-means clustering cannot extract the target area and just simply labeled it. After clustering, we use region growing method to extract the interested area and remove the grains around it.Step 4 (morphology expansion and erosion).
Corrosion and expansion are two important operations of mathematical morphology, which can make the segmentation target area closing, remove the holes in the target area, and make the segmentation region more even. The area that we are interested in extracted by region growing may have holds and rough edges. It requires using mathematical morphology method for further processing in order to get a complete wear particle image. The changes of the processed image are shown in Figure4.Figure 4
Changes of the image in the preprocessing.Step 5 (feature extraction).
Due to the fact that different wear particles have different characteristics as shown in Figure3, different characteristic parameters need to be extracted for recognition. The most obvious characteristic of red oxide is color, so we extract the first-order origin matrix, second-order center matrix, and third-order center matrix of H, S, and V components in the HSV color space as red oxide’s feature. Fatigue wear particles showed very similar morphological characteristics. The majority of fatigue wear particles are laminar particles. They had smooth flat surfaces and irregular contours. Based on the characteristics of fatigue wear particle, its HoG feature is extracted for classification. For the cutting wear particle, its shape feature is obvious. We get its area, perimeter, equivalent diameter, roundness, aspect ratio, and concavity which is due to the reflection of light and another important basis to recognize the spherical particles. This method uses the following characteristics of spherical particles to identify them: area, perimeter, equivalent diameter, roundness and slenderness ratio, concavity, energy, entropy, moment of inertia, local stability, R mean value, G mean value, and B average value as well as R, G, and B standard deviation. The surface of fatigue wear is relatively rough, characterized by severe galling, parallel scratches, or partial oxidation and irregular edge contour. As scratch is an important feature to recognize fatigue wears, we extracted its Tamura features.Step 6 (using extreme learning machine for recognition).
We use the extracted features to train the ELM classifier and then use the trained network for wear particle recognition.


## 4. Experiments

### 4.1. Experimental Database

In this paper, the experimental database was collected from Guangdong provincial Key Laboratory of Petrochemical Equipment Fault Diagnosis. Lubricating oil samples of petrochemical plant machinery were adopted to make ferrogram through using analytical ferrograph. Through a double light microscope, the morphological features and distribution were observed and the database was set up by shooting featured particles’ images trough CDD. The database consists of five typical ferrography abrasive particles, with totally 149 particles pictures. There are 33 red oxide images, 42 fatigue wear particles images, 29 cutting wear particles images, 18 spherical wear particles images, and 27 severe sliding wear particles images. The size of the images is 1024 × 768. Five kinds of wear particles are shown in Figure5.Figure 5
Five kinds of wear particles.


### 4.2. Results of Experiments

We conduct five experiments; each experiment only recognizes one of the five wear particles. Take the red oxide as an example; we extract color characteristics of all particles and use the ELM to classify them into two categories, red oxide and non-red oxide; thus we recognize the red oxide. According to this method, we recognize fatigue, cutting, and spherical and severe sliding wear particles.To test the performance of the proposed method, three sets of experiments had been carried out for one kind wear particle. 70%, 80%, and 90% of each type of wear particles images are used for training and the rest of them for test. The result of the experiments is shown in Table1.Table 1
Average recognition rate.
Training percentage (%)
Average recognition rate (%)
70
80
90
Red oxide
90.00
100
100
Fatigue
80.77
91.25
100
Cutting
77.78
88.33
100
Spherical
100
100
100
Severe sliding
85.00
100
100We compared the proposed method with two other classification methods, which are BP [33] and SVM [34]. We use the same features which we used to train the ELM to train the BP and SVM and then test them with the same samples. Table 2 shows that the recognition rates for training images and the average training time.Table 2
Average recognition rate of three classifiers (%).
classifier and training percentage (%)
BP
SVM
ELM
70
80
90
Average training time (s)
70
80
90
Average training time (s)
70
80
90
Average training time (s)
Red oxide
99.44
100
100
0.26
84.62
87.50
90.00
0.19
90.00
100
100
0.14
Fatigue
82.00
88.78
90
1.43
76.00
76.00
80.00
0.22
80.77
91.25
100
0.18
Cutting
85.00
92.00
95.00
0.19
87.50
90.00
90.00
0.09
77.78
88.33
100
0.07
Spherical
87.40
95.00
97.00
0.28
80.00
82.00
85.00
0.15
100
100
100
0.13
Severe sliding
83.00
88.00
94.05
0.21
78.95
88.89
91.57
0.11
85.00
100
100
0.07


### 4.3. Discussion

Table1 shows the results of the experiments. The recognition rate of red oxide, spherical wear particles, and severe sliding wear particles is great and can reach 100%. The reason why the recognition rate of cutting wear particles is not as good as the above three classifications is the shape features which we have chosen. For some short, thick, and big cutting wear particles, its aspect ratio is not obvious. The computer may mix the cutting wear particles with parts of fatigue wear particles and severe sliding wear particles so that it leads to the low recognition rate. However, for the fatigue wear particles, because the scratched parts of wear particles may not be obvious or not too many and Tamura features cannot represent it well, it causes the relatively low recognition rate.As we extract different features for different wear particles, the average training time of the five wear particles is different. we can see from Table2 that ELM has a good time efficiency. For the same kind of wear particle, the BP’s recognition rate is close to ELM, but its time efficiency is much lower than ELM. Among the three kinds of classifiers the SVM’s performance is the lowest on both recognition rate and time efficiency.


## 4.1. Experimental Database

In this paper, the experimental database was collected from Guangdong provincial Key Laboratory of Petrochemical Equipment Fault Diagnosis. Lubricating oil samples of petrochemical plant machinery were adopted to make ferrogram through using analytical ferrograph. Through a double light microscope, the morphological features and distribution were observed and the database was set up by shooting featured particles’ images trough CDD. The database consists of five typical ferrography abrasive particles, with totally 149 particles pictures. There are 33 red oxide images, 42 fatigue wear particles images, 29 cutting wear particles images, 18 spherical wear particles images, and 27 severe sliding wear particles images. The size of the images is 1024 × 768. Five kinds of wear particles are shown in Figure5.Figure 5
Five kinds of wear particles.


## 4.2. Results of Experiments

We conduct five experiments; each experiment only recognizes one of the five wear particles. Take the red oxide as an example; we extract color characteristics of all particles and use the ELM to classify them into two categories, red oxide and non-red oxide; thus we recognize the red oxide. According to this method, we recognize fatigue, cutting, and spherical and severe sliding wear particles.To test the performance of the proposed method, three sets of experiments had been carried out for one kind wear particle. 70%, 80%, and 90% of each type of wear particles images are used for training and the rest of them for test. The result of the experiments is shown in Table1.Table 1
Average recognition rate.
Training percentage (%)
Average recognition rate (%)
70
80
90
Red oxide
90.00
100
100
Fatigue
80.77
91.25
100
Cutting
77.78
88.33
100
Spherical
100
100
100
Severe sliding
85.00
100
100We compared the proposed method with two other classification methods, which are BP [33] and SVM [34]. We use the same features which we used to train the ELM to train the BP and SVM and then test them with the same samples. Table 2 shows that the recognition rates for training images and the average training time.Table 2
Average recognition rate of three classifiers (%).
classifier and training percentage (%)
BP
SVM
ELM
70
80
90
Average training time (s)
70
80
90
Average training time (s)
70
80
90
Average training time (s)
Red oxide
99.44
100
100
0.26
84.62
87.50
90.00
0.19
90.00
100
100
0.14
Fatigue
82.00
88.78
90
1.43
76.00
76.00
80.00
0.22
80.77
91.25
100
0.18
Cutting
85.00
92.00
95.00
0.19
87.50
90.00
90.00
0.09
77.78
88.33
100
0.07
Spherical
87.40
95.00
97.00
0.28
80.00
82.00
85.00
0.15
100
100
100
0.13
Severe sliding
83.00
88.00
94.05
0.21
78.95
88.89
91.57
0.11
85.00
100
100
0.07


## 4.3. Discussion

Table1 shows the results of the experiments. The recognition rate of red oxide, spherical wear particles, and severe sliding wear particles is great and can reach 100%. The reason why the recognition rate of cutting wear particles is not as good as the above three classifications is the shape features which we have chosen. For some short, thick, and big cutting wear particles, its aspect ratio is not obvious. The computer may mix the cutting wear particles with parts of fatigue wear particles and severe sliding wear particles so that it leads to the low recognition rate. However, for the fatigue wear particles, because the scratched parts of wear particles may not be obvious or not too many and Tamura features cannot represent it well, it causes the relatively low recognition rate.As we extract different features for different wear particles, the average training time of the five wear particles is different. we can see from Table2 that ELM has a good time efficiency. For the same kind of wear particle, the BP’s recognition rate is close to ELM, but its time efficiency is much lower than ELM. Among the three kinds of classifiers the SVM’s performance is the lowest on both recognition rate and time efficiency.


## 5. Conclusion

In this paper, ELM is introduced into the wear particles image recognition. Different from traditional method, each time only one of the five wear particles is recognized. This avoids the simultaneous extraction of various wear particle characteristics, resulting in redundancy features, and achieves better classification results. However, as shown from the experimental results, the recognition rate of fatigue and cutting wear particles are to be improved. Future work will include how to get robust wear particle features and improve the recognition rate based on ELM.



---

*Source: 3451358-2017-03-21.xml*