# Quantitative Nondestructive Testing of Wire Ropes Based on Features Fusion of Magnetic Image and Infrared Image

Authors: Shiliang Lu; Juwei Zhang
Journal: Shock and Vibration
 (2019)

---

## Abstract

Magnetic flux leakage (MFL) detection is one of the most widely used and best performing wire rope nondestructive testing (NDT) methods for more than a decade. However, the traditional MFL detection has the disadvantages of single source of information, low precision, easy to miss detection, and false detection. To solve these problems, we propose a method of fusion recognition of magnetic image features and infrared image features. A denoising algorithm based on Hilbert vibration decomposition (HVD) and wavelet transform is proposed to denoise the MFL signal, and the modulus maxima method is used to locate and segment the defect. An infrared image acquisition system was designed to collect the infrared image of the surface of the wire rope. Digital image processing techniques are used to segment infrared defect images. The features of the MFL image and the infrared image are extracted separately for fusion. The fusion feature is input into the nearest neighbor (NN) algorithm for quantitative identification, and the same data are input into the backpropagation (BP) neural network for comparison verification. The experimental results show that the fusion of MFL features and infrared features effectively improves the recognition rate of wire rope defects and reduces the recognition error.


---

## Body

## 1. Introduction

Wire ropes play an indispensable role in industrial production, commercial services, and high-tech industries in the modern world. The safety of the wire rope during these production processes is very important because it is usually related to the safety of life and property. Nondestructive testing (NDT) of wire ropes can identify various safety hazards in advance [1]. There are many methods for NDT of steel wire ropes. In GB/T5616 NDT application guidelines, they are divided into 6 categories and more than 70 types [2]. At present, the most commonly used methods at home and abroad include ultrasonic testing [3], radiation testing [4], eddy current testing [5], electromagnetic testing, and infrared testing. Since most of the steel wire ropes are made of high-carbon steel with good magnetic permeability, the electromagnetic detection method is very suitable for NDT of wire ropes and has become the most widely used and most mature method [6]. In addition to conventional detection methods, as technology advances, new detection methods continue to emerge. Infrared thermography (IRT) technology has attracted more and more attention because of its fast, intuitive, noncontact, and pollution-free advantages [7]. How to supplement and develop different kinds of methods to play their respective advantages and avoid their respective shortcomings is the focus of current research.The principle of magnetic flux leakage (MFL) testing is when the wire is magnetized in the axial direction, a magnetic field is generated inside the wire. If there is a defect on the surface of the wire rope, the magnetic field will leak through this defect to form a MFL. At this time, the magnetic sensitive element is used to collect the MFL information along the circumferential direction of the wire rope, and the broken wire position and quantity can be obtained after the analysis and processing [8].The first step in the MFL testing of wire ropes is magnetization. Magnetization is divided into coil magnetization and permanent magnet magnetization according to the excitation source [9]. The advantage of coil magnetization is that the magnitude of the magnetic field can be freely changed by changing the current. Okolo and Meydan [10] magnetizes the pipe to be tested by applying a pulse current on the excitation coil and then measures the leakage flux through a magnetic field sensor. However, the alternating magnetic field is prone to skin effect and eddy current, which is not conducive to subsequent MFL detection. Singh et al. [11] used two saddle coils to magnetize the wire rope and then uses a giant magnetoresistive (GMR) sensor that is inexpensive and easy to use to detect the defect leakage magnetic field. Park et al. [12] proposed a long-span bridge steel cable automatic monitoring system that uses strong permanent magnets to establish magnetic flux in the steel cable to be inspected and uses a Hall sensor to detect the MFL generated by the defect.Sensitive magnetic sensors are critical for MFL testing technology. Zhao [13] designed a MFL testing device based on a Hall array sensor. In view of the problem that the traditional induction coil collects data unevenly due to large span, four different structured printed circuit board (PCB) coils are designed to detect the leakage flux of different components. A spatial notch filter is proposed to filter out the strand noise of the MFL image. However, Hall sensors have lower sensitivity and stability than GMR sensors. Pham et al. [14] designed a MFL testing device based on a planar Hall magnetoresistive sensor, which is used to detect shallow defects on the lower surface of oil and gas pipelines due to its high sensitivity and low thermal drift. Xiucheng et al. [15] designed a circular sensor array based on a tunnel magnetoresistive (TMR) sensor to detect slight defects on the surface of the wire rope and judge the defect position based on the MFL information. But it only verifies the effect of the TMR sensor, and there is no signal for accurate identification.The processing of the detected MFL signal is crucial, which is related to the ability to accurately identify the location and number of defects. Zhang et al. [16] used a detection device consisting of 18 GMR sensor arrays and an ARM controller board to obtain the residual magnetic distribution of the wire rope. The system control board is composed of an encoding module, a signal amplifier module, an array control module, a data storage module, and a CPU module. The CPU core uses STM32F407. All system operations are controlled by the ARM9 core. The Hilbert–Huang transform is used to decompose the MFL signal, and then, each layer of the signal is subjected to wavelet decomposition based on compressed sensing. However, the compression sensing structure is complex, the amount of calculation is large, and the filtering of noise is not ideal. Zheng and Zhang [17] used an image processing method that converts the noise-reduced wire rope MFL signal into a grayscale image and then converts the grayscale image into a pseudo color image. The features such as the color moment of the defect image are extracted for recognition, and the recognition accuracy is better than the gray image. Liu et al. [18] proposed a method based on notch filtering combined with wavelet denoising to filter out the noise of the wire rope signal. Compared with traditional low-pass filtering and adaptive analysis, this method is more accurate and reliable.IRT is a promising technology for quantitative NDT of objects based on heat. Compared with other NDT methods, IRT has the advantages of noncontact data acquisition and sensitivity to near-surface defects of components [19]. Because the temperature difference between the defect and the intact surface is obvious, IRT is highly sensitive to small defects of the metal material. Eddazi and Belattar [20] used IRT and finite element analysis to perform NDT on the aluminum body of the conveyor, and the degree of corrosion of the fuselage was determined by analyzing the temperature difference between the heat flux value and the corrosion thickness.There are many ways to improve the effectiveness of defect recognition, including improvements in noise reduction algorithms and recognition networks. Increasing the diversity of information sources is also an extremely effective method. The fusion of multiple information can significantly reduce the rate of missed detection and false detection. At present, the infrared image fusion technology at home and abroad is basically the fusion of infrared image and visible light. Lu et al. [21] proposed a new infrared and visible image fusion method based on independent component analysis. This method extracts the main features of the infrared image into the visible light image, avoiding the loss of scene details and achieving a good fusion effect. Luo et al. [22] decompose the infrared image and the visible light image into high-frequency subbands and low-frequency subbands. The high-frequency subbands and low-frequency subbands of the different images are then fused separately. The method preferably retains the background information of the visible light and more prominently the infrared object.In order to solve the shortcomings of traditional MFL detection information, such as single source, low precision, and easy to be interfered, we propose a fusion recognition method of magnetic image features and infrared image features. In order to solve the problem of MFL signal noise, a denoising algorithm based on Hilbert vibration decomposition (HVD) and wavelet transform is proposed to denoise the MFL signal. The modulus maxima method is used to locate the defect and segment the image. In order to collect the infrared image of the wire rope surface, we designed an infrared image acquisition system. The infrared defect image is segmented using digital image processing techniques. The features of the MFL image and the infrared image are separately extracted for fusion. This paper adopts the feature-level fusion method. In order to verify whether the MFL feature is effective after fusion with the infrared feature, we input the fusion feature and the MFL defect feature into the nearest neighbor (NN) algorithm for quantitative identification. At the same time, in order to eliminate the influence of the algorithm on the results, we input the same data into the back propagation (BP) neural network for comparison verification. The experimental results show that the fusion of infrared image information and MFL signal effectively improves the recognition rate of wire rope defects and reduces the recognition error.


## 2. Data Collection

### 2.1. MFL Data Collection

The wire rope MFL signal acquisition device under the unsaturated magnetic excitation of the literature [23] is cited herein. As shown in Figure 1, the sensor array is used to collect the MFL information of the wire rope defect. The unsaturated magnetic excitation module is used to generate a uniform excitation field to magnetize the wire rope. The sensor array module is integrally provided with the unsaturated magnetic excitation module. The unsaturated magnetic excitation module is composed of a plurality of rod-shaped permanent magnets, and the permanent magnets are evenly arranged along the circumferential direction of the wire rope. All permanent magnets have the same magnetic pole direction. The sensor array consists of 18 highly sensitive GMR sensors. The sensor model is AAH002 analog GMR sensor. The linear output range for weak magnetic fields of the sensor is 0.6–3.0 Oe, the magnetic sensitivity range is 11–18 mV/V-Oe, and the voltage output range is 26.1–217.5 mV. The GMR sensors are evenly distributed along the circumference of the wire rope, with a linear distance of 9.94 mm between the center points of each sensor. Each GMR sensor has a distance of 11 mm from the surface of the wire rope. Since the sensitive surface of the GMR sensor is perpendicular to the surface of the wire rope, the acquired MFL component is a radial component. The collecting device moves at a constant speed along the axial direction of the wire rope, and the controller sequentially collects the MFL data of 18 channels. The encoder collects 1024 points per 0.31 m. The wire rope has a diameter of 28 mm, a structure of 6 × 36, and a length of 6 m.Figure 1
Schematic diagram of the MFL data acquisition device under unsaturated magnetic excitation.Figure2 shows the raw unfolded data for the wire rope MFL signal. The highest peak portion in the figure is not the defect signal but the magnetic pole portion generated during the excitation process.Figure 2
Raw data of MFL signal.


### 2.2. Infrared Image Acquisition

The wire rope infrared image acquisition system consists of two parts: wire rope heating device and infrared camera. Since the picture taken at normal temperature is not ideal, the wire must be heated before the infrared image of the wire rope is taken. The wire rope heating device is a cylindrical stainless steel sleeve which is covered with a heating belt. When it is necessary to heat the wire rope, the sleeve is placed on the wire rope for heating. If the temperature of the wire rope heating temperature is too high or too low, the color of the infrared image defect will be similar or even the same as the surrounding environment, which may result in the inability to accurately extract the defect. After repeated experiments, the heating temperature of the wire rope was set at 36–40°C, and the infrared image captured in this temperature range was of good quality.As shown in Figure3, the heating device heats the local position of the wire rope to a specified temperature and then photographs using an infrared camera. As shown in Figure 4, the infrared camera model is FLUKE TiX660.Figure 3
Schematic diagram of infrared image acquisition device.Figure 4
Infrared camera.Figure5 shows an infrared image of a wire rope captured by an infrared image acquisition system.Figure 5
Original infrared image.


## 2.1. MFL Data Collection

The wire rope MFL signal acquisition device under the unsaturated magnetic excitation of the literature [23] is cited herein. As shown in Figure 1, the sensor array is used to collect the MFL information of the wire rope defect. The unsaturated magnetic excitation module is used to generate a uniform excitation field to magnetize the wire rope. The sensor array module is integrally provided with the unsaturated magnetic excitation module. The unsaturated magnetic excitation module is composed of a plurality of rod-shaped permanent magnets, and the permanent magnets are evenly arranged along the circumferential direction of the wire rope. All permanent magnets have the same magnetic pole direction. The sensor array consists of 18 highly sensitive GMR sensors. The sensor model is AAH002 analog GMR sensor. The linear output range for weak magnetic fields of the sensor is 0.6–3.0 Oe, the magnetic sensitivity range is 11–18 mV/V-Oe, and the voltage output range is 26.1–217.5 mV. The GMR sensors are evenly distributed along the circumference of the wire rope, with a linear distance of 9.94 mm between the center points of each sensor. Each GMR sensor has a distance of 11 mm from the surface of the wire rope. Since the sensitive surface of the GMR sensor is perpendicular to the surface of the wire rope, the acquired MFL component is a radial component. The collecting device moves at a constant speed along the axial direction of the wire rope, and the controller sequentially collects the MFL data of 18 channels. The encoder collects 1024 points per 0.31 m. The wire rope has a diameter of 28 mm, a structure of 6 × 36, and a length of 6 m.Figure 1
Schematic diagram of the MFL data acquisition device under unsaturated magnetic excitation.Figure2 shows the raw unfolded data for the wire rope MFL signal. The highest peak portion in the figure is not the defect signal but the magnetic pole portion generated during the excitation process.Figure 2
Raw data of MFL signal.


## 2.2. Infrared Image Acquisition

The wire rope infrared image acquisition system consists of two parts: wire rope heating device and infrared camera. Since the picture taken at normal temperature is not ideal, the wire must be heated before the infrared image of the wire rope is taken. The wire rope heating device is a cylindrical stainless steel sleeve which is covered with a heating belt. When it is necessary to heat the wire rope, the sleeve is placed on the wire rope for heating. If the temperature of the wire rope heating temperature is too high or too low, the color of the infrared image defect will be similar or even the same as the surrounding environment, which may result in the inability to accurately extract the defect. After repeated experiments, the heating temperature of the wire rope was set at 36–40°C, and the infrared image captured in this temperature range was of good quality.As shown in Figure3, the heating device heats the local position of the wire rope to a specified temperature and then photographs using an infrared camera. As shown in Figure 4, the infrared camera model is FLUKE TiX660.Figure 3
Schematic diagram of infrared image acquisition device.Figure 4
Infrared camera.Figure5 shows an infrared image of a wire rope captured by an infrared image acquisition system.Figure 5
Original infrared image.


## 3. MFL Data Processing

The MFL signal collected by the MFL detecting device contains a large amount of noise before being processed. It includes the noise caused by the uneven excitation, the influence of the earth’s magnetic field, the wave noise caused by the spiral structure of the wire rope, the influence of the lift distance, and the jitter of the wire rope. The quality of noise processing will greatly affect the quantitative identification in the later stage. If noise is not filtered out, the defect signal will be submerged in the noise. Eventually, the defect will not be recognized. In order to remove the influence of noise in the original signal, the Hilbert vibration decomposition (HVD) and wavelet transform are used to denoise the signal.


### 3.1. Wavelet Transform

Wavelets are widely used in signal analysis, such as signal filtering, signal-to-noise separation, and multiscale edge detection. Especially for nonstationary signals, wavelet transform is an extremely ideal tool [24]. The Mallat algorithm is a fast algorithm for wavelet transform. The algorithm can decompose the signal step by step into a high-frequency part and a low-frequency part. The algorithm decomposition and reconstruction formula is as follows:(1)cj+1,k=∑mh0m−2kcj,m,(2)dj+1,k=∑mh1m−2kcj,m,(3)cj−1,m=∑kcj,kh0m−2k+∑kdj,kh1m−2k.Among them, (1) and (2) are the wavelet decomposition formulas, and (3) is the wavelet reconstruction formula. h0k is the low-pass filter coefficient. h1k is the high-pass filter coefficient.


### 3.2. Hilbert Vibration Decomposition

HVD is a method for analyzing nonstationary signals proposed by Fledman [25], which was named for its initial application to the analysis of mechanical vibration signals. Although both HVD and Hilbert–Huang transform (HHT) are based on the Hilbert transform, HVD avoids the complex empirical mode decomposition (EMD) process in HHT and is very suitable for analyzing multicomponent stationary signals with fast decomposition speed and high precision.The method steps are as follows:(1)
First, the original signal is parsed using a Hilbert transform to obtain an analytical signal [26].For any continuous nonstationary time signal, the Hilbert transform is(4)yt=1π∫−∞+∞xτt−τdτ,wherey(t) is the signal after x(t) Hilbert transform and t is the time.x(t) and y(t) form a conjugate complex pair to obtain an analytical signal:(5)zt=xt+jyt=atejφt,where(6)at=x2t+y2t,φt=arctanytxt,wherea(t) is the instantaneous amplitude and φ(t) is the instantaneous phase.The instantaneous frequencyf(t) can be expressed as(7)ft=12π×dφtdt.(2)
The instantaneous frequency of the largest amplitude component of the original signal is obtained by low-pass filtering, and the instantaneous amplitude and the initial phase of the component are obtained by synchronous detection.The multicomponent nonstationary signalx(t) can be expressed as(8)xt=∑l=1Malcos2πflt+θl=∑l=1Malcosφl,whereM is the number of components of x(t), and al, fl, θl, and φl are the instantaneous amplitude, instantaneous frequency, initial phase, and phase of the first oscillation component, respectively.The analytical signal ofx(t) after Hilbert transform is(9)zt=∑l=1Malejφlt.The instantaneous amplitudea(t) is(10)at=∑m=1Mλmt1/2,where(11)λmt=∑n=1Mamancosφm−φn.The instantaneous frequencyf(t) can be expressed as(12)ft=flt+∑m=2Mfmt−fltλmta2t.f(t) consists of two parts: the instantaneous frequency flt of the largest component of the amplitude and the high oscillation frequency. In practical applications, low-pass filtering can be used to filter out the high oscillation frequency, leaving only the instantaneous frequency flt.(3)
The estimated instantaneous frequency is taken as the reference frequencyfγ, and the corresponding instantaneous amplitude aγ and initial phase θγ are estimated by synchronous detection. Construct two orthogonal signals and multiply the initial oscillating signal:(13)z1t=12aγcosθγ+cos4πfγt+θγ.Quadrature phase output is as follows:(14)z2t=12aγt−sinθγ+sin4πfγt+θγ.Bothz1t and z2t consist of two parts, which can be filtered out by low-pass filtering to obtain the following two forms:(15)z1¯t=12aγtcosθγ,z2¯t=−12aγtsinθγ.The instantaneous valueaγt and the initial phase θγ are(16)aγt=2z1¯t2+z2¯t2,θγ=−arctanz2¯tz1¯t.(4)
The time-frequency information of each component is adaptively detected by iterative operation. According to the above steps, the oscillation component with the largest amplitude is extracted:(17)xγt=aγcos2πfγt+θγ.Subtracting the currently detected amplitude, maximum oscillation componentxγt from the initial signal is obtained:(18)Xrt=Xr−1t−xγt,r=1,2,…,wherer is the number of iterations, Xrt is the signal residual value of the detected oscillating component after the rth iteration operation, and X0t is the original oscillating signal. The time-frequency information of the other components is sequentially obtained by an iterative operation, and the iteration is stopped when the standard deviation of the residual value Xrt is <0.001.


### 3.3. Algorithm Description

The selection of the wavelet basis function and the threshold value in wavelet denoising is uncertain, and when the noise signal is close to the defect signal, it is easy to filter out the useful information.The oscillation components after HVD are stationary signals. We decompose the components containing the defect information with the wavelet soft threshold and delete the components that do not contain the defect information. Set the number of decomposition layersr = 5. If r > 5, the number of decomposition layers is excessive. The algorithm flow is as follows:Step 1.
The signalxi of the i-th channel is decomposed using Hilbert vibration decomposition.(1)
The Hilbert transform is performed on the signalxi to obtain an analytical signal zi.(2)
The instantaneous amplitudeai and the instantaneous frequency fi of the maximum oscillation component are obtained according to equations (10) and (12).(3)
Using the obtained instantaneous frequencyfi as the reference frequency, the corresponding instantaneous amplitude aγ and initial phase θγ are estimated by synchronous detection.(4)
Theaγ and θγ obtained by the above steps extract the oscillation component having the largest amplitude and iteratively obtain all the oscillation components. The normalized standard deviation of equation (18) is taken as the iteration termination condition σ, and the iteration is terminated when σ<0.001.Step 2.
Perform wavelet soft threshold denoising on the oscillatory component containing useful information.(1)
The wavelet component is decomposed using the Mallat algorithm. Set the parameters related to wavelet decomposition: DaubechiesV (db5) wavelet, the number of decomposition layersk = 8.(2)
The low-frequency coefficients are cleared, and soft threshold quantization processing is performed using the extreme value threshold (minimax) for the high-frequency coefficients at each scale.(3)
The processed wavelet coefficients are subjected to one-dimensional wavelet reconstruction to obtain a filtered oscillation component.Step 3.
Remove the oscillating component without useful information.Step 4.
The noise-reduced data are obtained by superimposing all the filtered components.
Figures6(a) and 6(c) show single-channel original MFL data. Figures 6(b) and 6(d) show single-channel data filtered by the above algorithm.
The MFL raw signal of 18 channels is denoised by the above algorithm. Figures7 and 8, respectively, show the expanded data of MFL signal before and after denoising.Figure 6
(a, c) Single-channel raw data. (b, d) Data after denoising.
(a)
(b)
(c)
(d)Figure 7
Expanded data of MFL signal.Figure 8
Expanded data after denoising.


### 3.4. Grayscale Normalization and Defect Segmentation

The MFL signal after noise reduction is still too complicated, which is not conducive to defect segmentation and feature extraction. This paper converts the MFL signal into a grayscale image and divides the defect.


#### 3.4.1. Circumferential Interpolation

Since the acquisition device uses an 18-channel giant magnetoresistive sensor array, the resolution of the data expansion map is only 18, which is much lower than the axial resolution. In order to obtain a more intuitive image, the cubic interpolation method is used to perform circumferential interpolation on the filtered MFL data, and the circumferential resolution is increased from 18 to 300. Figure9 is a graph of MFL data after interpolation.Figure 9
Graph of MFL data after interpolation.


#### 3.4.2. Grayscale Normalization

Grayscale normalization is the process of converting a MFL signal into a grayscale image. Use the following formula to normalize its magnitude between 0 and 255 grayscale values:(19)dati,j=datai,j800+0.5×255.Figure10 is an image normalized by MFL data.Figure 10
Grayscale image of MFL data.


#### 3.4.3. Defect Location and Segmentation

After normalizing the MFL signal to a grayscale image, the modulus maxima method is used to locate and segment the defects. The data are first thresholded, and then, the maximum value of the defect location data is calculated. Taking the maximum value found as the center, 150 pixel points are intercepted in the axial direction, and a defect gray scale of 300 × 300 is obtained.


## 3.1. Wavelet Transform

Wavelets are widely used in signal analysis, such as signal filtering, signal-to-noise separation, and multiscale edge detection. Especially for nonstationary signals, wavelet transform is an extremely ideal tool [24]. The Mallat algorithm is a fast algorithm for wavelet transform. The algorithm can decompose the signal step by step into a high-frequency part and a low-frequency part. The algorithm decomposition and reconstruction formula is as follows:(1)cj+1,k=∑mh0m−2kcj,m,(2)dj+1,k=∑mh1m−2kcj,m,(3)cj−1,m=∑kcj,kh0m−2k+∑kdj,kh1m−2k.Among them, (1) and (2) are the wavelet decomposition formulas, and (3) is the wavelet reconstruction formula. h0k is the low-pass filter coefficient. h1k is the high-pass filter coefficient.


## 3.2. Hilbert Vibration Decomposition

HVD is a method for analyzing nonstationary signals proposed by Fledman [25], which was named for its initial application to the analysis of mechanical vibration signals. Although both HVD and Hilbert–Huang transform (HHT) are based on the Hilbert transform, HVD avoids the complex empirical mode decomposition (EMD) process in HHT and is very suitable for analyzing multicomponent stationary signals with fast decomposition speed and high precision.The method steps are as follows:(1)
First, the original signal is parsed using a Hilbert transform to obtain an analytical signal [26].For any continuous nonstationary time signal, the Hilbert transform is(4)yt=1π∫−∞+∞xτt−τdτ,wherey(t) is the signal after x(t) Hilbert transform and t is the time.x(t) and y(t) form a conjugate complex pair to obtain an analytical signal:(5)zt=xt+jyt=atejφt,where(6)at=x2t+y2t,φt=arctanytxt,wherea(t) is the instantaneous amplitude and φ(t) is the instantaneous phase.The instantaneous frequencyf(t) can be expressed as(7)ft=12π×dφtdt.(2)
The instantaneous frequency of the largest amplitude component of the original signal is obtained by low-pass filtering, and the instantaneous amplitude and the initial phase of the component are obtained by synchronous detection.The multicomponent nonstationary signalx(t) can be expressed as(8)xt=∑l=1Malcos2πflt+θl=∑l=1Malcosφl,whereM is the number of components of x(t), and al, fl, θl, and φl are the instantaneous amplitude, instantaneous frequency, initial phase, and phase of the first oscillation component, respectively.The analytical signal ofx(t) after Hilbert transform is(9)zt=∑l=1Malejφlt.The instantaneous amplitudea(t) is(10)at=∑m=1Mλmt1/2,where(11)λmt=∑n=1Mamancosφm−φn.The instantaneous frequencyf(t) can be expressed as(12)ft=flt+∑m=2Mfmt−fltλmta2t.f(t) consists of two parts: the instantaneous frequency flt of the largest component of the amplitude and the high oscillation frequency. In practical applications, low-pass filtering can be used to filter out the high oscillation frequency, leaving only the instantaneous frequency flt.(3)
The estimated instantaneous frequency is taken as the reference frequencyfγ, and the corresponding instantaneous amplitude aγ and initial phase θγ are estimated by synchronous detection. Construct two orthogonal signals and multiply the initial oscillating signal:(13)z1t=12aγcosθγ+cos4πfγt+θγ.Quadrature phase output is as follows:(14)z2t=12aγt−sinθγ+sin4πfγt+θγ.Bothz1t and z2t consist of two parts, which can be filtered out by low-pass filtering to obtain the following two forms:(15)z1¯t=12aγtcosθγ,z2¯t=−12aγtsinθγ.The instantaneous valueaγt and the initial phase θγ are(16)aγt=2z1¯t2+z2¯t2,θγ=−arctanz2¯tz1¯t.(4)
The time-frequency information of each component is adaptively detected by iterative operation. According to the above steps, the oscillation component with the largest amplitude is extracted:(17)xγt=aγcos2πfγt+θγ.Subtracting the currently detected amplitude, maximum oscillation componentxγt from the initial signal is obtained:(18)Xrt=Xr−1t−xγt,r=1,2,…,wherer is the number of iterations, Xrt is the signal residual value of the detected oscillating component after the rth iteration operation, and X0t is the original oscillating signal. The time-frequency information of the other components is sequentially obtained by an iterative operation, and the iteration is stopped when the standard deviation of the residual value Xrt is <0.001.


## 3.3. Algorithm Description

The selection of the wavelet basis function and the threshold value in wavelet denoising is uncertain, and when the noise signal is close to the defect signal, it is easy to filter out the useful information.The oscillation components after HVD are stationary signals. We decompose the components containing the defect information with the wavelet soft threshold and delete the components that do not contain the defect information. Set the number of decomposition layersr = 5. If r > 5, the number of decomposition layers is excessive. The algorithm flow is as follows:Step 1.
The signalxi of the i-th channel is decomposed using Hilbert vibration decomposition.(1)
The Hilbert transform is performed on the signalxi to obtain an analytical signal zi.(2)
The instantaneous amplitudeai and the instantaneous frequency fi of the maximum oscillation component are obtained according to equations (10) and (12).(3)
Using the obtained instantaneous frequencyfi as the reference frequency, the corresponding instantaneous amplitude aγ and initial phase θγ are estimated by synchronous detection.(4)
Theaγ and θγ obtained by the above steps extract the oscillation component having the largest amplitude and iteratively obtain all the oscillation components. The normalized standard deviation of equation (18) is taken as the iteration termination condition σ, and the iteration is terminated when σ<0.001.Step 2.
Perform wavelet soft threshold denoising on the oscillatory component containing useful information.(1)
The wavelet component is decomposed using the Mallat algorithm. Set the parameters related to wavelet decomposition: DaubechiesV (db5) wavelet, the number of decomposition layersk = 8.(2)
The low-frequency coefficients are cleared, and soft threshold quantization processing is performed using the extreme value threshold (minimax) for the high-frequency coefficients at each scale.(3)
The processed wavelet coefficients are subjected to one-dimensional wavelet reconstruction to obtain a filtered oscillation component.Step 3.
Remove the oscillating component without useful information.Step 4.
The noise-reduced data are obtained by superimposing all the filtered components.
Figures6(a) and 6(c) show single-channel original MFL data. Figures 6(b) and 6(d) show single-channel data filtered by the above algorithm.
The MFL raw signal of 18 channels is denoised by the above algorithm. Figures7 and 8, respectively, show the expanded data of MFL signal before and after denoising.Figure 6
(a, c) Single-channel raw data. (b, d) Data after denoising.
(a)
(b)
(c)
(d)Figure 7
Expanded data of MFL signal.Figure 8
Expanded data after denoising.


## 3.4. Grayscale Normalization and Defect Segmentation

The MFL signal after noise reduction is still too complicated, which is not conducive to defect segmentation and feature extraction. This paper converts the MFL signal into a grayscale image and divides the defect.


### 3.4.1. Circumferential Interpolation

Since the acquisition device uses an 18-channel giant magnetoresistive sensor array, the resolution of the data expansion map is only 18, which is much lower than the axial resolution. In order to obtain a more intuitive image, the cubic interpolation method is used to perform circumferential interpolation on the filtered MFL data, and the circumferential resolution is increased from 18 to 300. Figure9 is a graph of MFL data after interpolation.Figure 9
Graph of MFL data after interpolation.


### 3.4.2. Grayscale Normalization

Grayscale normalization is the process of converting a MFL signal into a grayscale image. Use the following formula to normalize its magnitude between 0 and 255 grayscale values:(19)dati,j=datai,j800+0.5×255.Figure10 is an image normalized by MFL data.Figure 10
Grayscale image of MFL data.


### 3.4.3. Defect Location and Segmentation

After normalizing the MFL signal to a grayscale image, the modulus maxima method is used to locate and segment the defects. The data are first thresholded, and then, the maximum value of the defect location data is calculated. Taking the maximum value found as the center, 150 pixel points are intercepted in the axial direction, and a defect gray scale of 300 × 300 is obtained.


## 3.4.1. Circumferential Interpolation

Since the acquisition device uses an 18-channel giant magnetoresistive sensor array, the resolution of the data expansion map is only 18, which is much lower than the axial resolution. In order to obtain a more intuitive image, the cubic interpolation method is used to perform circumferential interpolation on the filtered MFL data, and the circumferential resolution is increased from 18 to 300. Figure9 is a graph of MFL data after interpolation.Figure 9
Graph of MFL data after interpolation.


## 3.4.2. Grayscale Normalization

Grayscale normalization is the process of converting a MFL signal into a grayscale image. Use the following formula to normalize its magnitude between 0 and 255 grayscale values:(19)dati,j=datai,j800+0.5×255.Figure10 is an image normalized by MFL data.Figure 10
Grayscale image of MFL data.


## 3.4.3. Defect Location and Segmentation

After normalizing the MFL signal to a grayscale image, the modulus maxima method is used to locate and segment the defects. The data are first thresholded, and then, the maximum value of the defect location data is calculated. Taking the maximum value found as the center, 150 pixel points are intercepted in the axial direction, and a defect gray scale of 300 × 300 is obtained.


## 4. Infrared Image Processing

This article uses active infrared detection. It is characterized by heating an object to be measured with an external heat source before performing the test. When the object to be tested loses thermal equilibrium, the defective position causes an abnormality in the surface temperature of the object to be tested.In order to proceed to the next step, it is necessary to extract the defective area of the infrared image. Since the broken wire of the wire rope is a heat insulating-type defect, the temperature of the defect is significantly higher than the normal connection of the steel wire. IRT is greatly affected by external factors, such as oil stains, dross, and uneven heating. Because there is a large amount of oil between the wire rope strands, and the oil substances heat up faster than the metal substances under the external temperature, the oil temperature between the strands is also higher than the normal position of the steel ropes.


### 4.1. RGB Image Segmentation

The Matlab image processing toolbox treats color images as RGB images. The RGB image is anM ∗ N ∗ 3 array of color pixels, and each color pixel can find the corresponding three components of red, green, and blue in a specific space [27].Segmenting a colored area using RGB color vectors is simple and effective. Using the colored area of interest as a color sample point, an “average” color estimate is obtained. This average color is represented by the RGB column vectorm. We need to classify each RGB pixel in the image to determine if it is similar to m. Let z denote any pixel in the RGB space. If the distance between z and m is less than the specified threshold T, z is similar to m; otherwise, z is discarded.The Euclidean distance formula betweenz and m is(20)Dz,m=z−m=z−mTz−m1/2=zR−mR2+zG−mG2+zB−mB21/2,where z−m is the norm of the argument and the subscripts R, G, and B represent the RGB components of the vectors m and z. The points in the image that satisfy different conditions are coded; the point of Dz,m≤T is defined as black, and the point of Dz,m≥T is defined as white.


### 4.2. Algorithm Description

The infrared image processing steps are as follows:Step 1: Use the RGB channel to segment the region of interest (ROI) of the infrared image. The ROI is selected from the red to white range.Step 2: The extracted ROI is a binary image and contains a large amount of noise in addition to the defective portion. The area method is used to select a suitable threshold to filter out large areas of oil pollution noise and small area of residual noise.Step 3: The extracted defect portion has a rough edge and contains internal voids. Use open and close operations to fill holes and smooth edges.Figure11 shows the results after infrared image processing.Figure 11
Infrared image processing: (a, e) Original infrared image; (b, f) ROI area binary map; (c, g) extracted defect portion; (d, h) morphology of defect part after processing.
(a)
(b)
(c)
(d)
(e)
(f)
(g)
(h)


## 4.1. RGB Image Segmentation

The Matlab image processing toolbox treats color images as RGB images. The RGB image is anM ∗ N ∗ 3 array of color pixels, and each color pixel can find the corresponding three components of red, green, and blue in a specific space [27].Segmenting a colored area using RGB color vectors is simple and effective. Using the colored area of interest as a color sample point, an “average” color estimate is obtained. This average color is represented by the RGB column vectorm. We need to classify each RGB pixel in the image to determine if it is similar to m. Let z denote any pixel in the RGB space. If the distance between z and m is less than the specified threshold T, z is similar to m; otherwise, z is discarded.The Euclidean distance formula betweenz and m is(20)Dz,m=z−m=z−mTz−m1/2=zR−mR2+zG−mG2+zB−mB21/2,where z−m is the norm of the argument and the subscripts R, G, and B represent the RGB components of the vectors m and z. The points in the image that satisfy different conditions are coded; the point of Dz,m≤T is defined as black, and the point of Dz,m≥T is defined as white.


## 4.2. Algorithm Description

The infrared image processing steps are as follows:Step 1: Use the RGB channel to segment the region of interest (ROI) of the infrared image. The ROI is selected from the red to white range.Step 2: The extracted ROI is a binary image and contains a large amount of noise in addition to the defective portion. The area method is used to select a suitable threshold to filter out large areas of oil pollution noise and small area of residual noise.Step 3: The extracted defect portion has a rough edge and contains internal voids. Use open and close operations to fill holes and smooth edges.Figure11 shows the results after infrared image processing.Figure 11
Infrared image processing: (a, e) Original infrared image; (b, f) ROI area binary map; (c, g) extracted defect portion; (d, h) morphology of defect part after processing.
(a)
(b)
(c)
(d)
(e)
(f)
(g)
(h)


## 5. Feature Extraction

The features of the MFL defect image and the infrared defect image obtained in the foregoing are, respectively, extracted and fused. Figure12 shows several broken wires photographs, MFL defect grayscale images, and infrared defect images. In this paper, the morphological features of the defect image are extracted including area, squareness, and elongation; and the seventh-order invariant moments have a total of 10 features as quantitatively identified feature vectors.Figure 12
Photos (above), MFL grayscale images (middle), and infrared binary images (below) of 6 different defects: (a) one broken wire; (b) two broken wires; (c) three broken wires; (d) four broken wires; (e) five broken wires; (f) seven broken wires.
(a)
(b)
(c)
(d)
(e)
(f)


### 5.1. Morphological Features

The morphological features of objects are also one of the important features of image recognition. Morphological features include area, squareness, and elongation.The areaS of the area is defined as the number of pixels in the area:(21)S=∑x,y∈QG,where Q is defined as the defect area.The squarenessR is defined as the ratio of the area to the product of length and width(22)R=AH×W,where H is the length of the region and W is the width of the region.The elongationE is defined as the ratio of the minimum to the maximum of the length and width of the region:(23)E=minH,WmaxH,W.


### 5.2. Invariant Moment Feature

Invariant moments are an average statistical description that is not sensitive to translation, scaling, mirroring, and rotation. It is an important feature of defects [28].The two-dimensionalp+q moment of a digital image fx,y of size M ∗ N is defined as(24)mpq=∑x=0M−1∑y=0N−1xpyqfx,y.Thep+q step center distance is defined as(25)μpq=∑x=0M−1∑y=0N−1x−x¯py−y¯qfx,y,where p=0,1,2,…, q=0,1,2,…,(26)x¯=m10m00,y¯=m01m00,The normalized center distance represented byηpq is defined as(27)ηpq=μpqμ00γ.In the above formula,(28)γ=p+q2+1,where p+q=2,3,…The set of 7 two-dimensional invariant moments can be derived from the above formula:(29)ϕ1=η20+η02,ϕ2=η20−η022+4η112,ϕ3=η30−3η122+3η21−η032,ϕ4=η30+η122+η21+η032,ϕ5=η30−3η12η30+η12η30+η122−3η21+η032+3η21−η03η21+η033η30+η122−η21+η032,ϕ6=η20−η02η30+η122−η21+η032+4η11η30+η12η21+η03,ϕ7=3η21−η03η30+η12η30+η122−3η21+η032+3η21−η03η21+η033η30+η122−η21+η032.


### 5.3. Feature-Level Fusion

This paper adopts the feature-level fusion method. Feature-level fusion is a fusion between pixel-level fusion and decision-level fusion. For the multisource image of the same target, the feature information of the image is extracted separately to form the feature vector, which more accurately reflects the essential features of the target and improves the defect classification and target description accuracy.The extracted magnetic image features and infrared image features are simultaneously input to the neural network hereinafter.


## 5.1. Morphological Features

The morphological features of objects are also one of the important features of image recognition. Morphological features include area, squareness, and elongation.The areaS of the area is defined as the number of pixels in the area:(21)S=∑x,y∈QG,where Q is defined as the defect area.The squarenessR is defined as the ratio of the area to the product of length and width(22)R=AH×W,where H is the length of the region and W is the width of the region.The elongationE is defined as the ratio of the minimum to the maximum of the length and width of the region:(23)E=minH,WmaxH,W.


## 5.2. Invariant Moment Feature

Invariant moments are an average statistical description that is not sensitive to translation, scaling, mirroring, and rotation. It is an important feature of defects [28].The two-dimensionalp+q moment of a digital image fx,y of size M ∗ N is defined as(24)mpq=∑x=0M−1∑y=0N−1xpyqfx,y.Thep+q step center distance is defined as(25)μpq=∑x=0M−1∑y=0N−1x−x¯py−y¯qfx,y,where p=0,1,2,…, q=0,1,2,…,(26)x¯=m10m00,y¯=m01m00,The normalized center distance represented byηpq is defined as(27)ηpq=μpqμ00γ.In the above formula,(28)γ=p+q2+1,where p+q=2,3,…The set of 7 two-dimensional invariant moments can be derived from the above formula:(29)ϕ1=η20+η02,ϕ2=η20−η022+4η112,ϕ3=η30−3η122+3η21−η032,ϕ4=η30+η122+η21+η032,ϕ5=η30−3η12η30+η12η30+η122−3η21+η032+3η21−η03η21+η033η30+η122−η21+η032,ϕ6=η20−η02η30+η122−η21+η032+4η11η30+η12η21+η03,ϕ7=3η21−η03η30+η12η30+η122−3η21+η032+3η21−η03η21+η033η30+η122−η21+η032.


## 5.3. Feature-Level Fusion

This paper adopts the feature-level fusion method. Feature-level fusion is a fusion between pixel-level fusion and decision-level fusion. For the multisource image of the same target, the feature information of the image is extracted separately to form the feature vector, which more accurately reflects the essential features of the target and improves the defect classification and target description accuracy.The extracted magnetic image features and infrared image features are simultaneously input to the neural network hereinafter.


## 6. Quantitative Identification

### 6.1. Nearest Neighbor Algorithm

Thek-nearest neighbor (kNN) classification algorithm is one of the simplest methods in data mining classification technology. Its working mechanism is for a given test sample, find the k training samples closest to it based on the distance metric and then based on the k “neighbors” information to make predictions. When k = 1, the algorithm is the 1-nearest neighbor algorithm, referred to as the nearest neighbor (NN) algorithm. NN is essentially different from kNN. Usually, kNN uses the voting method in the classification task, which is to select the category label that appears the most in the k samples as the prediction result. The NN is a two-category problem, which is to select the training sample closest to the test sample as the result [29].The NN algorithm is actually a lazy learning algorithm. Such algorithms save the samples during the training phase and then process the samples after receiving the test samples. The training time is zero. Since the NN algorithm mainly relies on the surrounding neighboring samples, rather than relying on the discriminant domain method to determine the category, the NN algorithm is more suitable for cross-class domains or overlapping sample sets than other methods.In the experiments quantitatively identified in this paper, the wire rope is 28 mm in diameter and 6∗ 36 in structure. The wire rope has six types of broken wires: one wire, two wires, three wires, four wires, five wires, and seven wires. The dimension of the defect is 15 mm. A total of 187 defective samples were obtained in this experiment. A total of 147 samples were randomly selected from 187 samples as training samples, and the remaining 40 samples were used as test samples.The MFL defect feature and the infrared defect feature are fusion and input into the NN algorithm. Figure13 shows the recognition rate of the fusion feature. When the recognition error is allowed to be 0.93%, the recognition success rate is 97.87%, and the maximum recognition error is not more than 1.39%. The percentage of broken wire refers to the percentage of wire rope broken wire to the total number of wire ropes. When the percentage error is less than the maximum allowable identification error, it can be considered as the correct result.Figure 13
Recognition rate of fusion feature.Figure14 shows a recognition rate for inputting only MFL defects. When the recognition error is allowed to be 0.93%, the recognition success rate is 94.12%, and the maximum recognition error is not more than 1.39%.Figure 14
Recognition rate of MFL defect feature.As shown in the figure, the recognition rate of the fusion feature is increased from 94.12% to 97.87% compared with the magnetic image feature recognition rate.


### 6.2. BP Neural Network

This paper uses the BP neural network model in [17] for comparative verification. Fusion features and magnetic image features are input into the BP neural network.Figure15 shows the fusion feature recognition rate of different hidden layer nodes in the BP network. When the number of hidden layer nodes is set to 12, the recognition rate is the highest. When the recognition error is 0.93%, the recognition success rate is 98.7%, and the maximum recognition error is less than 1.39%.Figure 15
Fusion feature recognition rate of BP network different nodes: the hidden layer has (a) 11 nodes, (b) 12 nodes, (c) 15 nodes, and (d) 18 nodes.
(a)
(b)
(c)
(d)Figure16 shows the MFL defect feature recognition rate of different hidden layer nodes in the BP network. When the number of hidden layer nodes is set to 14, the recognition rate is the highest, and when the recognition error is 0.93%, the recognition success rate is 92.21%, and the maximum recognition error is not more than 1.39%.Figure 16
MFL defect feature recognition rate of BP network different nodes: the hidden layer has (a) 10 nodes, (b) 12 nodes, (c) 14 nodes, and (d) 17 nodes.
(a)
(b)
(c)
(d)As shown in the figure, the recognition rate of the fusion feature is increased from 92.21% to 98.7% compared with the magnetic image feature recognition rate.


### 6.3. Result

This section verifies that the fusion of magnetic image features and infrared features is effective. Whether using NN algorithm or BP neural network for quantitative identification, the recognition rate of fusion features is higher than the recognition rate of magnetic image features.


## 6.1. Nearest Neighbor Algorithm

Thek-nearest neighbor (kNN) classification algorithm is one of the simplest methods in data mining classification technology. Its working mechanism is for a given test sample, find the k training samples closest to it based on the distance metric and then based on the k “neighbors” information to make predictions. When k = 1, the algorithm is the 1-nearest neighbor algorithm, referred to as the nearest neighbor (NN) algorithm. NN is essentially different from kNN. Usually, kNN uses the voting method in the classification task, which is to select the category label that appears the most in the k samples as the prediction result. The NN is a two-category problem, which is to select the training sample closest to the test sample as the result [29].The NN algorithm is actually a lazy learning algorithm. Such algorithms save the samples during the training phase and then process the samples after receiving the test samples. The training time is zero. Since the NN algorithm mainly relies on the surrounding neighboring samples, rather than relying on the discriminant domain method to determine the category, the NN algorithm is more suitable for cross-class domains or overlapping sample sets than other methods.In the experiments quantitatively identified in this paper, the wire rope is 28 mm in diameter and 6∗ 36 in structure. The wire rope has six types of broken wires: one wire, two wires, three wires, four wires, five wires, and seven wires. The dimension of the defect is 15 mm. A total of 187 defective samples were obtained in this experiment. A total of 147 samples were randomly selected from 187 samples as training samples, and the remaining 40 samples were used as test samples.The MFL defect feature and the infrared defect feature are fusion and input into the NN algorithm. Figure13 shows the recognition rate of the fusion feature. When the recognition error is allowed to be 0.93%, the recognition success rate is 97.87%, and the maximum recognition error is not more than 1.39%. The percentage of broken wire refers to the percentage of wire rope broken wire to the total number of wire ropes. When the percentage error is less than the maximum allowable identification error, it can be considered as the correct result.Figure 13
Recognition rate of fusion feature.Figure14 shows a recognition rate for inputting only MFL defects. When the recognition error is allowed to be 0.93%, the recognition success rate is 94.12%, and the maximum recognition error is not more than 1.39%.Figure 14
Recognition rate of MFL defect feature.As shown in the figure, the recognition rate of the fusion feature is increased from 94.12% to 97.87% compared with the magnetic image feature recognition rate.


## 6.2. BP Neural Network

This paper uses the BP neural network model in [17] for comparative verification. Fusion features and magnetic image features are input into the BP neural network.Figure15 shows the fusion feature recognition rate of different hidden layer nodes in the BP network. When the number of hidden layer nodes is set to 12, the recognition rate is the highest. When the recognition error is 0.93%, the recognition success rate is 98.7%, and the maximum recognition error is less than 1.39%.Figure 15
Fusion feature recognition rate of BP network different nodes: the hidden layer has (a) 11 nodes, (b) 12 nodes, (c) 15 nodes, and (d) 18 nodes.
(a)
(b)
(c)
(d)Figure16 shows the MFL defect feature recognition rate of different hidden layer nodes in the BP network. When the number of hidden layer nodes is set to 14, the recognition rate is the highest, and when the recognition error is 0.93%, the recognition success rate is 92.21%, and the maximum recognition error is not more than 1.39%.Figure 16
MFL defect feature recognition rate of BP network different nodes: the hidden layer has (a) 10 nodes, (b) 12 nodes, (c) 14 nodes, and (d) 17 nodes.
(a)
(b)
(c)
(d)As shown in the figure, the recognition rate of the fusion feature is increased from 92.21% to 98.7% compared with the magnetic image feature recognition rate.


## 6.3. Result

This section verifies that the fusion of magnetic image features and infrared features is effective. Whether using NN algorithm or BP neural network for quantitative identification, the recognition rate of fusion features is higher than the recognition rate of magnetic image features.


## 7. Conclusion

In order to solve the problem that the wire rope defect information source is single and the recognition accuracy is not high, this paper proposes a method of fusion recognition of magnetic image features and infrared image features. We have collected the wire rope MFL signal under nonsaturated magnetic excitation. In order to filter out a large amount of noise in the MFL signal, we have proposed a denoising algorithm based on HVD and wavelet transform to filter the MFL signal. The gray scale of the MFL data after noise reduction is normalized, and the defect portion is positioned and segmented by the modulus maximum value method. We have designed a wire rope infrared image acquisition system that was used to acquire infrared images of wire rope surface defects. We take the RGB image segmentation of the acquired infrared image to obtain the ROI and use the area method to obtain the binary image of the defective portion. The morphological features and the seventh-order invariant moment features of the MFL image and the infrared defect image are extracted separately. To verify the validity of the fusion of features, we have designed two sets of quantitative recognition experiments. The MFL defect feature and the fusion feature are, respectively, used as input to the NN algorithm. At the same time, in order to avoid the influence of the recognition algorithm, we input the same data into the BP neural network for comparison. The experimental results show that the recognition rate of the fusion feature is significantly higher than that of the MFL defect feature, whether it is the NN algorithm or the BP neural network.Our experiments verify that the fusion of infrared image information and MFL signals is effective in improving the recognition rate of wire rope defects. Future research work will focus on the improvement of the wire rope infrared image acquisition system and the optimization of the image fusion method.



---

*Source: 2041401-2019-11-18.xml*