# Pipeline Leak Aperture Recognition Based on Wavelet Packet Analysis and a Deep Belief Network with ICR

Authors: Xianming Lang; Zhiyong Hu; Ping Li; Yan Li; Jiangtao Cao; Hong Ren
Journal: Wireless Communications and Mobile Computing
 (2018)

---

## Abstract

The leakage aperture cannot be easily identified, when an oil pipeline has small leaks. To address this issue, a leak aperture recognition method based on wavelet packet analysis (WPA) and a deep belief network (DBN) with independent component regression (ICR) is proposed. WPA is used to remove the noise in the collected sound velocity of the ultrasonic signal. Next, the denoised sound velocity of the ultrasonic signal is input into the deep belief network with independent component regression (DBNICR) to recognize different leak apertures. Because the optimization of the weights of the DBN with the gradient leads to a local optimum and a slow learning rate, ICR is used to replace the gradient fine-tuning method in conventional DBN for improving the classification accuracy, and a Lyapunov function is constructed to prove the convergence of the DBNICR learning process. By analyzing the acquired ultrasonic sound velocity of different leak apertures, the results show that the proposed method can quickly and effectively identify different leakage apertures.


---

## Body

## 1. Introduction

Because of aging pipelines, corrosion, or welding defects, small leaks and slow leaks occur frequently; such leaks represent risks to the environment and can cause financial losses [1–4]. The pressure drop produced by a small leakage is low and difficult to detect; however, small leak and slow leak are main forms of leakage of long distance oil pipelines during the service period. Determining how to identify the small leakage aperture of the pipeline in time has become a popular topic of study in the management of the integrity of a pipeline [5, 6]. Thus, it is important to estimate the leakage aperture, to assist in the development of the pipeline repair plan and in the evaluation of the leakage area.In a Kneser liquid, the transmission speed of an ultrasonic wave changes as the liquid pressure changes at a definite temperature [7]. In this study, the sound velocity of an ultrasonic signal of a pipeline is used to identify different leakage apertures. In practical engineering works, the general signals collected by data acquisition equipment include noises in the external environment, and the purpose of signal denoising is to distinguish the high-frequency signals from the interference caused by the high-frequency noise and to remove the high-frequency interference signal, as well as retain the useful information in the signal. The weak signal must be guaranteed to not be filtered out in the process of signal analysis, whereas the original signal can be reduced as much as possible. Wavelet packet analysis (WPA) presents a powerful ability for denoising a signal from the measured signals [8]. However, this method requires selection of a suitable wavelet basis function to achieve better denoising effect. When the measured signals are decomposed using empirical mode decomposition (EMD), because of the influence of the end effect, it is very difficult to accurately reconstruct the signal [9]. The local mean decomposition (LMD) method is more relaxed than EMD in terms of decomposition conditions, as the end effect is reduced and the over envelope phenomenon is avoided in the decomposition process [6, 10]. However, the end effect still affects the signal reconstruction. Recently, variable mode decomposition (VMD) has been proposed as an adaptive signal decomposition method, which is an entirely nonrecursive signal decomposition method. VMD not only has a good separation effect of signal and noise, but can also effectively suppress modal aliasing; nevertheless, the end effect also affects the signal reconstruction [11, 12].Theoretically, a deep belief network (DBN) [13] is composed of multiple restricted Boltzmann machines (RBMs), in contrast with a shallow learning model, such as ANN and SVM. Recently, the DBN is becoming a useful tool for classification [14, 15]. The most significant difference between deep learning methods and shallow learning methods is that the former can present features from the original feature set automatically, instead of selecting feature manually. Two types of optimization methods of DBN are the adjustment of the depth structure and the optimization of the related parameters [16, 17]. The supervised learning algorithm of DBN is based on the backpropagation algorithm with a gradient; as a result, the weight adjustment can easily fall into a local optimum and slow the learning speed, thereby affecting the classification accuracy. Determining how to select the optimal weights and avoid the gradient of the fine adjustment method is the key to improve the classification accuracy. To overcome these difficulties, it is necessary to find a learning algorithm without the gradient. Independent component regression (ICR) is a layer-to-layer supervised parameter regression model without the gradient [18, 19]. Therefore, a weight optimization method based on DBNICR is proposed, to avoid the local optimum brought by the gradient algorithm, and the classification accuracy of DBN is further improved.The rest of this paper is organized as follows. Section2 introduces a method of aperture identification. In Section 3, the experimental results for pipeline are analyzed and discussed. Finally, Section 4 draws the main conclusions.


## 2. Leak Apertures Identification Method

In this section, WPA andDBNICR are first introduced, and then, a combination of WPA and DBNICR is proposed for different leakage apertures identification of a pipeline. Afterwards, an illustration is provided of the aperture identification process based on the time domain sound velocity of ultrasonic signal through WPA and DBNICR.


### 2.1. Principle of Wavelet Packet Analysis-Based Denoising

WPA [20–22] is an effective time-frequency analysis technique for nonstationary signals, multiresolution analysis of the wavelet packet transform decomposes signals into low frequency signals and high-frequency signals. Furthermore, the process continues to decompose the following layers until reaching the preset level; as a result, the wavelet packet analysis has obviously better ability to achieve accurate local analysis. Moreover, wavelet packet analysis has better characteristics for further segmentation and refinement of the frequency band broadened with the increase of scale. Therefore, WPA is a precise analysis method with the characteristics of broad high-frequency bandwidth and narrow low frequency bandwidth. When the signal is decomposed by wavelet packets, a variety of wavelet basis functions can be adopted. For example, the signal S is decomposed by wavelet packets using three-layer decomposition [23], as shown in Figure 1.Figure 1
Wavelet packet decomposition of the signal S.The signalS is decomposed by the wavelet packet decomposition tree and can be represented as(1)S=A1+AAD3+DAD3+DD2


### 2.2. Deep Belief Networks with Independent Component Regression

First, the independent component regression (ICR) is presented. Next, DBN with ICR and its learning algorithm are presented.


#### 2.2.1. Independent Component Regression

Independent component analysis (ICA) [24] is a statistical and computational technique for revealing the hidden factors of signals; the goal of ICA is to decompose the observed data linearly into a statistical independent component. Denote the process observation samples as xi,yii=1,2,⋯,N; the samples include independent variables xi∈Rc×1 and dependent variable yi∈Rp×1. After the independent components have been estimated from ICA, the ICR model can be obtained.According to the ICR algorithm, it is assumed that the measured process variablesxi∈Rc×1 can be expressed as linear combinations of r(≤c) unknown independent components si∈Rr×1; the ICR model is given by(2)xi=Aisi+eiwhereAi∈Rc×r is the mixing matrix and ei∈Rc×1 is the residual vector.Thus, the independent components can be estimated as(3)s^i=WixiwhereWi∈Rr×c is the separating matrix.(4)yi=QiTs^iThus, the linear regression matrix can be expressed as shown:(5)Qi=s^iTs^i-1s^iTyiTherefore, the desired model betweenxi and yi can be obtained by the ICR model, which is given as (6)yi=QiTWixi=TICRixiwhereQiTWi=TICRi.


#### 2.2.2. Deep Belief networks with Independent Component Regression Learning Process

Please refer to the drawing [25–27] for more information on DBN. ICR not only is a method of regression, but is also regarded as a supervised learning algorithm. DBN with ICR supervised fine-tuning starts from the classifier layer, and an alternative ICR method is repeatedly used to model every two hidden layers from the top layer to the bottom layer. Therefore, the ICR algorithm can be used to replace the gradient-based supervised learning; the goal of DBNICR is to overcome the low-accuracy and time-consuming nature of DBN.For the classifier layer and the last hidden layer, the detailed training steps are as follows:(a) First, the state of the last hidden layer is extracted as the independent variable, and the classifier layer is used as the dependent variable.It is assumed that the classifier matrix isp dimensional; the last hidden layer state matrix is q dimension; the N sample observation matrices are as follows:(7)Ys=ys11⋯ys1p⋮⋱⋮ysN1⋯ysNp,HL=hL11⋯hL1q⋮⋱⋮hLN1⋯hLNqwhereYs is from the N samples and HL is the last hidden layer state matrix, obtained by the hidden layer of the last RBM.(b) Independent componentss^L are extracted from the observation sample matrix HL.(8)s^L=WLHL(c) The linear regressionQL can be performed on s^L and the output matrix Ys.(9)QL=s^LTs^-1s^LTYs(d) Therefore, the ICR model can be expressed by(10)Y^s=QLTs^L=QLTWLHLwhere the output weight matrix optimized by the ICR model is described by(11)Wout=QLTWLNote that the number of the independent componentscL is very important for the accuracy of ICR. (12)errorcL=Ys-Y^swherecL is determined by minimizing the absolute error of prediction; the corresponding cL is optimal.Thus, the ICR fine-tuning process is completed between the output layer and the last hidden layer. Next, ICR is repeatedly conducted in every two hidden layers, starting formhL-1 and hL-2 to h1 and xinput. As a result, the remaining weight matrices optimized by ICR are W=WoutWLWL-1⋯W2Wvisible.The contrastive divergence (CD) algorithm is used to train each RBM from down layer to top layer first, and the fixed weights (initialization weights) after unsupervised training are determined. Next, the actual output is used to build the layer-layer ICR model and fine tune the initialization weights layer by layer; in the process of establishing the ICR model, the independent variable comes from the state variable of the RBM after the unsupervised training is completed.The architecture of theDBNICR algorithm is shown in Figure 2.Figure 2
Architecture ofDBNICR.


### 2.3. Convergence Analysis

For the proposedDBNICR, weight parameters are crucial to the convergence of DBNICR. Thus, a theoretical proof for the convergence of DBNICR is described in this section. According to the learning process of DBNICR, the whole dynamics transmission error can be described in [28](13)E˙error=y˙-y^˙=-Eerror+Fxvisible,W--F′xvisible,WwhereEerror(0)=0, and F(·) and F′(·) are the ideal deep belief network architecture function and the obtained deep belief network architecture function of DBNICR for the training samples, respectively. W=WoutWLWL-1⋯W2Wvisible is the final weight derived from DBNICR, and W- is the ideal weight of DBNICR for the training samples.According to [29], assuming that(14)Fxvisible,W--F′xvisible,W≤Efwhere· is the Euclidean distance, Ef>0. Because F(·) is bounded, according to the architectural of deep belief networks, F′(·) is also bounded. Therefore, this assumption is achievable.Theorem 1.
Considering a stable system by (13) and (14), if DBNICR is used to recognize the different leak apertures, then the classification error Eerror can converge to a finite vector: Eerror<Em (Em is a small positive number). Moreover, with the increase of samples and time, Eerror is uniformly and ultimately bounded, and it approaches 0 when the training samples are adequate.Proof.
Given the Lyapunov function(15)VEerrort=12EerroerTtEerrort
The derivative ofVEerror(t) is described as (16)V˙Eerrort=EerrorTtE˙errort=EerrorTt-Eerrort+Fxinputt,W--F′xinputt,W=-EerrorTt2+EerrorTtF′xinputt,W-Fxinputt,W-≤-EerrorTt2+EerrortEm=-EerrortEerrort-Em
To further analyze (16), it is discussed according to the following two cases:
( 1 ) If Eerror(t)≥Em, then V˙Eerror(t)≤0, and V˙Eerror(t) is negative semidefinite.
( 2 ) If Eerror(t)<Em, then V˙Eerror(t)>0, and V˙Eerror(t) is positive definite. However, as time goes on or the number of samples increase, there must be two types of situations:
E e r r o r ( t ) = E m or Eerror(t)>Em; both situations are just same as that of item (1).
Thus,V˙Eerror(t) is negative semidefinite.
The training error sumAE is expressed by(17)AE=∑i=1NEerrorTtEerrort
According to the Lyapunov stability theorem,(18)limt→∞AE=0
wheret→∞ indicates that the number of the training samples is enough large and that all the training samples have been put into DBNICR.
Therefore, the convergence ofDBNICR with respect to weight parameters is guaranteed theoretically.


### 2.4. Leak Apertures Recognition Based on WPA andDBNICR

It has been shown that a deep belief network can achieve lower error rate compared to traditional methods in fault patterns classification [30]. Hence, using the advantages of both WPA and DBNICR, we propose a hybrid leakage aperture identification method. The sound velocity of ultrasonic signals can be denoised using WPA. These signals corresponding to different leakage apertures may be different; however, it is hard to differentiate the different apertures through pattern recognition without feature extraction based on prior knowledge.Therefore,DBNICR is applied to identify the different leakage apertures via signals after WPA denosing. The schematic of the proposed method is shown in Figure 3.Figure 3
Schematic of the method to recognize pipeline leakage apertures.


## 2.1. Principle of Wavelet Packet Analysis-Based Denoising

WPA [20–22] is an effective time-frequency analysis technique for nonstationary signals, multiresolution analysis of the wavelet packet transform decomposes signals into low frequency signals and high-frequency signals. Furthermore, the process continues to decompose the following layers until reaching the preset level; as a result, the wavelet packet analysis has obviously better ability to achieve accurate local analysis. Moreover, wavelet packet analysis has better characteristics for further segmentation and refinement of the frequency band broadened with the increase of scale. Therefore, WPA is a precise analysis method with the characteristics of broad high-frequency bandwidth and narrow low frequency bandwidth. When the signal is decomposed by wavelet packets, a variety of wavelet basis functions can be adopted. For example, the signal S is decomposed by wavelet packets using three-layer decomposition [23], as shown in Figure 1.Figure 1
Wavelet packet decomposition of the signal S.The signalS is decomposed by the wavelet packet decomposition tree and can be represented as(1)S=A1+AAD3+DAD3+DD2


## 2.2. Deep Belief Networks with Independent Component Regression

First, the independent component regression (ICR) is presented. Next, DBN with ICR and its learning algorithm are presented.


### 2.2.1. Independent Component Regression

Independent component analysis (ICA) [24] is a statistical and computational technique for revealing the hidden factors of signals; the goal of ICA is to decompose the observed data linearly into a statistical independent component. Denote the process observation samples as xi,yii=1,2,⋯,N; the samples include independent variables xi∈Rc×1 and dependent variable yi∈Rp×1. After the independent components have been estimated from ICA, the ICR model can be obtained.According to the ICR algorithm, it is assumed that the measured process variablesxi∈Rc×1 can be expressed as linear combinations of r(≤c) unknown independent components si∈Rr×1; the ICR model is given by(2)xi=Aisi+eiwhereAi∈Rc×r is the mixing matrix and ei∈Rc×1 is the residual vector.Thus, the independent components can be estimated as(3)s^i=WixiwhereWi∈Rr×c is the separating matrix.(4)yi=QiTs^iThus, the linear regression matrix can be expressed as shown:(5)Qi=s^iTs^i-1s^iTyiTherefore, the desired model betweenxi and yi can be obtained by the ICR model, which is given as (6)yi=QiTWixi=TICRixiwhereQiTWi=TICRi.


### 2.2.2. Deep Belief networks with Independent Component Regression Learning Process

Please refer to the drawing [25–27] for more information on DBN. ICR not only is a method of regression, but is also regarded as a supervised learning algorithm. DBN with ICR supervised fine-tuning starts from the classifier layer, and an alternative ICR method is repeatedly used to model every two hidden layers from the top layer to the bottom layer. Therefore, the ICR algorithm can be used to replace the gradient-based supervised learning; the goal of DBNICR is to overcome the low-accuracy and time-consuming nature of DBN.For the classifier layer and the last hidden layer, the detailed training steps are as follows:(a) First, the state of the last hidden layer is extracted as the independent variable, and the classifier layer is used as the dependent variable.It is assumed that the classifier matrix isp dimensional; the last hidden layer state matrix is q dimension; the N sample observation matrices are as follows:(7)Ys=ys11⋯ys1p⋮⋱⋮ysN1⋯ysNp,HL=hL11⋯hL1q⋮⋱⋮hLN1⋯hLNqwhereYs is from the N samples and HL is the last hidden layer state matrix, obtained by the hidden layer of the last RBM.(b) Independent componentss^L are extracted from the observation sample matrix HL.(8)s^L=WLHL(c) The linear regressionQL can be performed on s^L and the output matrix Ys.(9)QL=s^LTs^-1s^LTYs(d) Therefore, the ICR model can be expressed by(10)Y^s=QLTs^L=QLTWLHLwhere the output weight matrix optimized by the ICR model is described by(11)Wout=QLTWLNote that the number of the independent componentscL is very important for the accuracy of ICR. (12)errorcL=Ys-Y^swherecL is determined by minimizing the absolute error of prediction; the corresponding cL is optimal.Thus, the ICR fine-tuning process is completed between the output layer and the last hidden layer. Next, ICR is repeatedly conducted in every two hidden layers, starting formhL-1 and hL-2 to h1 and xinput. As a result, the remaining weight matrices optimized by ICR are W=WoutWLWL-1⋯W2Wvisible.The contrastive divergence (CD) algorithm is used to train each RBM from down layer to top layer first, and the fixed weights (initialization weights) after unsupervised training are determined. Next, the actual output is used to build the layer-layer ICR model and fine tune the initialization weights layer by layer; in the process of establishing the ICR model, the independent variable comes from the state variable of the RBM after the unsupervised training is completed.The architecture of theDBNICR algorithm is shown in Figure 2.Figure 2
Architecture ofDBNICR.


## 2.2.1. Independent Component Regression

Independent component analysis (ICA) [24] is a statistical and computational technique for revealing the hidden factors of signals; the goal of ICA is to decompose the observed data linearly into a statistical independent component. Denote the process observation samples as xi,yii=1,2,⋯,N; the samples include independent variables xi∈Rc×1 and dependent variable yi∈Rp×1. After the independent components have been estimated from ICA, the ICR model can be obtained.According to the ICR algorithm, it is assumed that the measured process variablesxi∈Rc×1 can be expressed as linear combinations of r(≤c) unknown independent components si∈Rr×1; the ICR model is given by(2)xi=Aisi+eiwhereAi∈Rc×r is the mixing matrix and ei∈Rc×1 is the residual vector.Thus, the independent components can be estimated as(3)s^i=WixiwhereWi∈Rr×c is the separating matrix.(4)yi=QiTs^iThus, the linear regression matrix can be expressed as shown:(5)Qi=s^iTs^i-1s^iTyiTherefore, the desired model betweenxi and yi can be obtained by the ICR model, which is given as (6)yi=QiTWixi=TICRixiwhereQiTWi=TICRi.


## 2.2.2. Deep Belief networks with Independent Component Regression Learning Process

Please refer to the drawing [25–27] for more information on DBN. ICR not only is a method of regression, but is also regarded as a supervised learning algorithm. DBN with ICR supervised fine-tuning starts from the classifier layer, and an alternative ICR method is repeatedly used to model every two hidden layers from the top layer to the bottom layer. Therefore, the ICR algorithm can be used to replace the gradient-based supervised learning; the goal of DBNICR is to overcome the low-accuracy and time-consuming nature of DBN.For the classifier layer and the last hidden layer, the detailed training steps are as follows:(a) First, the state of the last hidden layer is extracted as the independent variable, and the classifier layer is used as the dependent variable.It is assumed that the classifier matrix isp dimensional; the last hidden layer state matrix is q dimension; the N sample observation matrices are as follows:(7)Ys=ys11⋯ys1p⋮⋱⋮ysN1⋯ysNp,HL=hL11⋯hL1q⋮⋱⋮hLN1⋯hLNqwhereYs is from the N samples and HL is the last hidden layer state matrix, obtained by the hidden layer of the last RBM.(b) Independent componentss^L are extracted from the observation sample matrix HL.(8)s^L=WLHL(c) The linear regressionQL can be performed on s^L and the output matrix Ys.(9)QL=s^LTs^-1s^LTYs(d) Therefore, the ICR model can be expressed by(10)Y^s=QLTs^L=QLTWLHLwhere the output weight matrix optimized by the ICR model is described by(11)Wout=QLTWLNote that the number of the independent componentscL is very important for the accuracy of ICR. (12)errorcL=Ys-Y^swherecL is determined by minimizing the absolute error of prediction; the corresponding cL is optimal.Thus, the ICR fine-tuning process is completed between the output layer and the last hidden layer. Next, ICR is repeatedly conducted in every two hidden layers, starting formhL-1 and hL-2 to h1 and xinput. As a result, the remaining weight matrices optimized by ICR are W=WoutWLWL-1⋯W2Wvisible.The contrastive divergence (CD) algorithm is used to train each RBM from down layer to top layer first, and the fixed weights (initialization weights) after unsupervised training are determined. Next, the actual output is used to build the layer-layer ICR model and fine tune the initialization weights layer by layer; in the process of establishing the ICR model, the independent variable comes from the state variable of the RBM after the unsupervised training is completed.The architecture of theDBNICR algorithm is shown in Figure 2.Figure 2
Architecture ofDBNICR.


## 2.3. Convergence Analysis

For the proposedDBNICR, weight parameters are crucial to the convergence of DBNICR. Thus, a theoretical proof for the convergence of DBNICR is described in this section. According to the learning process of DBNICR, the whole dynamics transmission error can be described in [28](13)E˙error=y˙-y^˙=-Eerror+Fxvisible,W--F′xvisible,WwhereEerror(0)=0, and F(·) and F′(·) are the ideal deep belief network architecture function and the obtained deep belief network architecture function of DBNICR for the training samples, respectively. W=WoutWLWL-1⋯W2Wvisible is the final weight derived from DBNICR, and W- is the ideal weight of DBNICR for the training samples.According to [29], assuming that(14)Fxvisible,W--F′xvisible,W≤Efwhere· is the Euclidean distance, Ef>0. Because F(·) is bounded, according to the architectural of deep belief networks, F′(·) is also bounded. Therefore, this assumption is achievable.Theorem 1.
Considering a stable system by (13) and (14), if DBNICR is used to recognize the different leak apertures, then the classification error Eerror can converge to a finite vector: Eerror<Em (Em is a small positive number). Moreover, with the increase of samples and time, Eerror is uniformly and ultimately bounded, and it approaches 0 when the training samples are adequate.Proof.
Given the Lyapunov function(15)VEerrort=12EerroerTtEerrort
The derivative ofVEerror(t) is described as (16)V˙Eerrort=EerrorTtE˙errort=EerrorTt-Eerrort+Fxinputt,W--F′xinputt,W=-EerrorTt2+EerrorTtF′xinputt,W-Fxinputt,W-≤-EerrorTt2+EerrortEm=-EerrortEerrort-Em
To further analyze (16), it is discussed according to the following two cases:
( 1 ) If Eerror(t)≥Em, then V˙Eerror(t)≤0, and V˙Eerror(t) is negative semidefinite.
( 2 ) If Eerror(t)<Em, then V˙Eerror(t)>0, and V˙Eerror(t) is positive definite. However, as time goes on or the number of samples increase, there must be two types of situations:
E e r r o r ( t ) = E m or Eerror(t)>Em; both situations are just same as that of item (1).
Thus,V˙Eerror(t) is negative semidefinite.
The training error sumAE is expressed by(17)AE=∑i=1NEerrorTtEerrort
According to the Lyapunov stability theorem,(18)limt→∞AE=0
wheret→∞ indicates that the number of the training samples is enough large and that all the training samples have been put into DBNICR.
Therefore, the convergence ofDBNICR with respect to weight parameters is guaranteed theoretically.


## 2.4. Leak Apertures Recognition Based on WPA andDBNICR

It has been shown that a deep belief network can achieve lower error rate compared to traditional methods in fault patterns classification [30]. Hence, using the advantages of both WPA and DBNICR, we propose a hybrid leakage aperture identification method. The sound velocity of ultrasonic signals can be denoised using WPA. These signals corresponding to different leakage apertures may be different; however, it is hard to differentiate the different apertures through pattern recognition without feature extraction based on prior knowledge.Therefore,DBNICR is applied to identify the different leakage apertures via signals after WPA denosing. The schematic of the proposed method is shown in Figure 3.Figure 3
Schematic of the method to recognize pipeline leakage apertures.


## 3. Field Experiment and Analysis

Acquired sound velocities of the ultrasonic signals are first processed by WPA and then classified byDBNICR. In addition, a comparative study between the proposed and existing pipeline leakage aperture methods is performed.


### 3.1. Field Experiment

Because of safety and cost problems, water instead of oil was used to simulate pipeline leaks in the experiment. According to the leakage experimental protocols for examining the aperture of different sizes, as shown in Figure4, the sound velocities of upstream or downstream ultrasonic signals were chosen to perform the test. The length of the pipeline segment is 2,800 m with an inner diameter of 50 mm, and the leakage apertures are 4 mm, 6 mm, 10 mm, and 15 mm. The operation conditions were as follows: (a) water was transported at 12 m3/h; (b) different apertures of valves were installed to emulate leaks; (c) the leakage flow and the energy were added by the upstream pump (the lift of the pump was 120 m); (d) leaked water was stored by a tank. The experiment apparatus is shown in Figure 5. The WPA and DBNICR algorithms are tested in MATLAB environment. We acquired a database from ultrasonic equipment by National Instruments DAQ-9184, at a sampling rate of 100 Hz. All the methods are implemented in MATLAB R2014a on a PC with an Intel Pentium processor (2.90 GHz) and 6 GB RAM.Figure 4
Different leak apertures.Figure 5
Experiment apparatus.


### 3.2. Sound Velocity of Ultrasonic Processing and Aperture Recognition

The sound velocities of the ultrasonic signals from different apertures were acquired by an ultrasonic sensor at the end of the pipeline; these velocities were used as the database to verify the proposed algorithm. The acquired signals are shown in Figure6.Figure 6
Ultrasonic signals under different leakage apertures.According to the measured sound velocities of the ultrasonic signals, they are decomposed by WPA of best tree with db3 in the three-layer model. Using the best tree structure, the sound velocity of the ultrasonic signal is reconstructed. To validate denoised signals, which are reconstructed signals obtained via the wavelet packet, the results of the signal denoised with VMD, EMD, and LMD are as shown in Figure7.Figure 7
Pipeline inlet sound velocities of the ultrasonic signals of the 4 mm leakage aperture.Figure7 shows that the wavelet packet analysis presents a powerful ability for denoising the signal from the measured sound velocities of the ultrasonic signal, compared with other methods, such as EMD, LMD, and VMD. When the measured sound velocities of the ultrasonic signals are reconstructed with EMD, because of the influence of end effect, it is very difficult to accurately remove the noise. The LMD method is more relaxed than EMD method in terms of decomposition conditions as the end effect is reduced and the over envelope phenomenon is avoided in the reconstruction process. However, the end effect still affects the extraction for removing the noise. VMD can adaptively extract the intrinsic modes of original sound velocities of the ultrasonic signal, but the end effect also affects the reconstruction signal.Therefore, WPA is used to denoise the sound velocities of the ultrasonic signals at the end of the pipeline; the sound velocities of the ultrasonic signals of normal condition and for leakage apertures of 4 mm, 6 mm, 10 mm, and 15 mm are collected by ultrasonic equipment. Five cases are created, and 1,000 samples from each case are chosen for training theDBNICR and the DBN. Thus, 5,000 samples of sound velocities of the ultrasonic signals are selected. Moreover, 4,000 samples are chosen as training samples, and the others are used as the testing samples.In this simulation, the denoised sound velocities of the ultrasonic signals are the inputs of theDBNICR and DBN, and the five feature vectors (absolute mean value, effective value, kurtosis, plus factor, and peak value factor) are input into the least squares twin support vector machine (LSTSVM) [31], the least squares support vector machine (LSSVM), the support vector machine (SVM), and the back propagation neural network (BPNN). The feature vectors of 4,000 groups (800 of each condition) are chosen as training samples and the others are used as testing samples. This paper adopts the “One-against-All” algorithm (OAA) of multiclassification.We follow the experience in the architecture selection of DBN: the number of neurons of the next layer is smaller than that of the previous layer so that the process of the DBN can be a feature extraction process. In this study, the three hidden layers ofDBNICR are selected by simulation. Moreover, to determine the number of neurons of the hidden layer, a trail-and-error approach is chosen to demonstrate the relation between the number of neurons in the hidden layer and the classification error. The relation curve between the number of neurons in the hidden layer and the classification error is shown in Figure 8. Figure 8 reveals that the best number of neurons in hidden layer 3 is 65, and the corresponding root mean square error (RMSE) is 0.01; the RMSE is given as(19)RMSE=1N∑i=1Ny^i-yi2Figure 8
Relation curve between the number of neurons in the hidden layer and the classification error.whereyi is the desired value, y^i is the output value of DBNICR, and N is the number of testing samples.According to the above analysis, the architecture ofDBNICR and and DBN is selected as 400-200-100-65-5. To improve the accuracy of fine-tuning, the number of independent components must be set as c=c4c3c3c1; the trail-and-error approach is used to determine the optimal number of independent components, and the relation curve between number of independent components and the classification error is shown in Figure 9.Figure 9
Relation curve between the number of independent components and the classification error.Figure9 reveals that the best numbers of independent components are c4=4c3=3c3=2c1=2, and the corresponding classifier error is 0.01. Thus, the DBNICR is constructed with the optimal number of independent components. After unsupervised learning and supervised learning, we use DBNICR to recognize different leakage apertures. To efficiently demonstrate the proposed method of DBNICR, the input and output for DBN are the same as those for DBNICR; moreover, 4,000 samples results using LSTSVM, LSSVM, SVM, and BPNN are obtained for comparison with the simulation. Practically, DBN is the traditional DBN.The DBN’s weights are initialized randomly, the biases are initialized to zero, the maximum iterations are 1, the batch size is 1, and the learning rate is 0.4. In LSTSVM, we set the slack variablea1=a2 to 0.01, and the kernel parameter σ1=σ2 to 1. The OAA method is used to accomplish multi- classification. We employ LS-SVMlab to implement the multiclassifier of LSSVM, where the slack variable α is 1 and the kernel parameter σ is 1. We also employ Libsvm to implement the multiclassifier of SVM, where the slack variable α is 10 and the kernel parameter σ is 0.1. This paper chooses a three-layer BPNN for which the middle layer node number is 100, using 100 iterations, the learning rate of 0.5, and the minimum error of 1×10-5. A comparison of the testing results with LSTSVM, LSSVM, SVM and BPNN is shown in Table 1.Table 1
Simulation results of different methods in different leakage apertures.
methods Neurons number Recognition accuracy (%) Average running-time(s) Training time Testing time D B N I C R 400-200-100-65-5 98.98 42.32 10.25 DBN 400-200-100-65-5 98.7 58.25 15.63 LSTSVM - 98.58 16.23 4.11 LSSVM - 98.42 20.61 6.75 SVM - 98.31 22.15 7.04 BPNN 400-100-5 98.91 15.13 4.41The experimental results are listed in Table1. Table 1 reveals that DBNICR has the best results in terms of average training time. Note that, although the accuracy of BPNN is as good as that of DBN, the time domain signals of sound velocities of the ultrasonic signals without feature selection by experience were input into DBN and DBNICR. Table 1 also reveals that the LSTSVM improves the accuracy and the average running time than LSSVM and SVM; the accuracy of LSSVM is the same as that of SVM, but the average running time of LSSVM is less than SVM. Compared with DBN, DBNICR has improved in recognition accuracy of different leakage apertures and program running time, primarily because of the fine-tuning of ICR of DBNICR.


### 3.3. Results and Analysis

The classification rate is calculated as the ratio of the number of correctly classified test samples to the total number of test samples. The proposed method and the LSTSVM-based, LSSVM-based, SVM-based, and BPNN-based methods are used to identify the different leakage apertures. 5,000 trials are performed, where 80% of samples are randomly selected for training and other samples are used for testing.In the simulation with the proposed method, the average accuracies of testing are 98.98%; i.e., all the different apertures are effectively recognized. Using the LSTSVM-based, LSSVM-based, SVM-based, and BPNN-based methods, however, the average testing accuracy is 98.58%, 98.42%, 98.1%, and 98.91%, respectively. This result implies that the proposed method obtains higher recognition accuracies and shows better robustness than the other methods in distinguishing the different apertures.( 1 ) DBN can process the sound velocities of ultrasonic signals of a pipeline in the time domain to recognize the different leakage apertures directly, without feature extraction and feature selection by prior knowledge. Thus, the intelligence of leak detection and leak aperture recognition is enhanced.( 2 ) In this paper, we have not studied the leak location, and the architecture selection of DBN. At present, the architecture of deep learning networks including the number of hidden layers and the number of neurons in each hidden layer is selected through empirical or experimental methods. Thus, this approach requires much work and may affect the accuracy or speed by architecture selection. The architecture adaptive selection is still a difficult problem to solve for deep neural networks; we will study this problem in the future.


## 3.1. Field Experiment

Because of safety and cost problems, water instead of oil was used to simulate pipeline leaks in the experiment. According to the leakage experimental protocols for examining the aperture of different sizes, as shown in Figure4, the sound velocities of upstream or downstream ultrasonic signals were chosen to perform the test. The length of the pipeline segment is 2,800 m with an inner diameter of 50 mm, and the leakage apertures are 4 mm, 6 mm, 10 mm, and 15 mm. The operation conditions were as follows: (a) water was transported at 12 m3/h; (b) different apertures of valves were installed to emulate leaks; (c) the leakage flow and the energy were added by the upstream pump (the lift of the pump was 120 m); (d) leaked water was stored by a tank. The experiment apparatus is shown in Figure 5. The WPA and DBNICR algorithms are tested in MATLAB environment. We acquired a database from ultrasonic equipment by National Instruments DAQ-9184, at a sampling rate of 100 Hz. All the methods are implemented in MATLAB R2014a on a PC with an Intel Pentium processor (2.90 GHz) and 6 GB RAM.Figure 4
Different leak apertures.Figure 5
Experiment apparatus.


## 3.2. Sound Velocity of Ultrasonic Processing and Aperture Recognition

The sound velocities of the ultrasonic signals from different apertures were acquired by an ultrasonic sensor at the end of the pipeline; these velocities were used as the database to verify the proposed algorithm. The acquired signals are shown in Figure6.Figure 6
Ultrasonic signals under different leakage apertures.According to the measured sound velocities of the ultrasonic signals, they are decomposed by WPA of best tree with db3 in the three-layer model. Using the best tree structure, the sound velocity of the ultrasonic signal is reconstructed. To validate denoised signals, which are reconstructed signals obtained via the wavelet packet, the results of the signal denoised with VMD, EMD, and LMD are as shown in Figure7.Figure 7
Pipeline inlet sound velocities of the ultrasonic signals of the 4 mm leakage aperture.Figure7 shows that the wavelet packet analysis presents a powerful ability for denoising the signal from the measured sound velocities of the ultrasonic signal, compared with other methods, such as EMD, LMD, and VMD. When the measured sound velocities of the ultrasonic signals are reconstructed with EMD, because of the influence of end effect, it is very difficult to accurately remove the noise. The LMD method is more relaxed than EMD method in terms of decomposition conditions as the end effect is reduced and the over envelope phenomenon is avoided in the reconstruction process. However, the end effect still affects the extraction for removing the noise. VMD can adaptively extract the intrinsic modes of original sound velocities of the ultrasonic signal, but the end effect also affects the reconstruction signal.Therefore, WPA is used to denoise the sound velocities of the ultrasonic signals at the end of the pipeline; the sound velocities of the ultrasonic signals of normal condition and for leakage apertures of 4 mm, 6 mm, 10 mm, and 15 mm are collected by ultrasonic equipment. Five cases are created, and 1,000 samples from each case are chosen for training theDBNICR and the DBN. Thus, 5,000 samples of sound velocities of the ultrasonic signals are selected. Moreover, 4,000 samples are chosen as training samples, and the others are used as the testing samples.In this simulation, the denoised sound velocities of the ultrasonic signals are the inputs of theDBNICR and DBN, and the five feature vectors (absolute mean value, effective value, kurtosis, plus factor, and peak value factor) are input into the least squares twin support vector machine (LSTSVM) [31], the least squares support vector machine (LSSVM), the support vector machine (SVM), and the back propagation neural network (BPNN). The feature vectors of 4,000 groups (800 of each condition) are chosen as training samples and the others are used as testing samples. This paper adopts the “One-against-All” algorithm (OAA) of multiclassification.We follow the experience in the architecture selection of DBN: the number of neurons of the next layer is smaller than that of the previous layer so that the process of the DBN can be a feature extraction process. In this study, the three hidden layers ofDBNICR are selected by simulation. Moreover, to determine the number of neurons of the hidden layer, a trail-and-error approach is chosen to demonstrate the relation between the number of neurons in the hidden layer and the classification error. The relation curve between the number of neurons in the hidden layer and the classification error is shown in Figure 8. Figure 8 reveals that the best number of neurons in hidden layer 3 is 65, and the corresponding root mean square error (RMSE) is 0.01; the RMSE is given as(19)RMSE=1N∑i=1Ny^i-yi2Figure 8
Relation curve between the number of neurons in the hidden layer and the classification error.whereyi is the desired value, y^i is the output value of DBNICR, and N is the number of testing samples.According to the above analysis, the architecture ofDBNICR and and DBN is selected as 400-200-100-65-5. To improve the accuracy of fine-tuning, the number of independent components must be set as c=c4c3c3c1; the trail-and-error approach is used to determine the optimal number of independent components, and the relation curve between number of independent components and the classification error is shown in Figure 9.Figure 9
Relation curve between the number of independent components and the classification error.Figure9 reveals that the best numbers of independent components are c4=4c3=3c3=2c1=2, and the corresponding classifier error is 0.01. Thus, the DBNICR is constructed with the optimal number of independent components. After unsupervised learning and supervised learning, we use DBNICR to recognize different leakage apertures. To efficiently demonstrate the proposed method of DBNICR, the input and output for DBN are the same as those for DBNICR; moreover, 4,000 samples results using LSTSVM, LSSVM, SVM, and BPNN are obtained for comparison with the simulation. Practically, DBN is the traditional DBN.The DBN’s weights are initialized randomly, the biases are initialized to zero, the maximum iterations are 1, the batch size is 1, and the learning rate is 0.4. In LSTSVM, we set the slack variablea1=a2 to 0.01, and the kernel parameter σ1=σ2 to 1. The OAA method is used to accomplish multi- classification. We employ LS-SVMlab to implement the multiclassifier of LSSVM, where the slack variable α is 1 and the kernel parameter σ is 1. We also employ Libsvm to implement the multiclassifier of SVM, where the slack variable α is 10 and the kernel parameter σ is 0.1. This paper chooses a three-layer BPNN for which the middle layer node number is 100, using 100 iterations, the learning rate of 0.5, and the minimum error of 1×10-5. A comparison of the testing results with LSTSVM, LSSVM, SVM and BPNN is shown in Table 1.Table 1
Simulation results of different methods in different leakage apertures.
methods Neurons number Recognition accuracy (%) Average running-time(s) Training time Testing time D B N I C R 400-200-100-65-5 98.98 42.32 10.25 DBN 400-200-100-65-5 98.7 58.25 15.63 LSTSVM - 98.58 16.23 4.11 LSSVM - 98.42 20.61 6.75 SVM - 98.31 22.15 7.04 BPNN 400-100-5 98.91 15.13 4.41The experimental results are listed in Table1. Table 1 reveals that DBNICR has the best results in terms of average training time. Note that, although the accuracy of BPNN is as good as that of DBN, the time domain signals of sound velocities of the ultrasonic signals without feature selection by experience were input into DBN and DBNICR. Table 1 also reveals that the LSTSVM improves the accuracy and the average running time than LSSVM and SVM; the accuracy of LSSVM is the same as that of SVM, but the average running time of LSSVM is less than SVM. Compared with DBN, DBNICR has improved in recognition accuracy of different leakage apertures and program running time, primarily because of the fine-tuning of ICR of DBNICR.


## 3.3. Results and Analysis

The classification rate is calculated as the ratio of the number of correctly classified test samples to the total number of test samples. The proposed method and the LSTSVM-based, LSSVM-based, SVM-based, and BPNN-based methods are used to identify the different leakage apertures. 5,000 trials are performed, where 80% of samples are randomly selected for training and other samples are used for testing.In the simulation with the proposed method, the average accuracies of testing are 98.98%; i.e., all the different apertures are effectively recognized. Using the LSTSVM-based, LSSVM-based, SVM-based, and BPNN-based methods, however, the average testing accuracy is 98.58%, 98.42%, 98.1%, and 98.91%, respectively. This result implies that the proposed method obtains higher recognition accuracies and shows better robustness than the other methods in distinguishing the different apertures.( 1 ) DBN can process the sound velocities of ultrasonic signals of a pipeline in the time domain to recognize the different leakage apertures directly, without feature extraction and feature selection by prior knowledge. Thus, the intelligence of leak detection and leak aperture recognition is enhanced.( 2 ) In this paper, we have not studied the leak location, and the architecture selection of DBN. At present, the architecture of deep learning networks including the number of hidden layers and the number of neurons in each hidden layer is selected through empirical or experimental methods. Thus, this approach requires much work and may affect the accuracy or speed by architecture selection. The architecture adaptive selection is still a difficult problem to solve for deep neural networks; we will study this problem in the future.


## 4. Conclusions

In this paper, the method for leak aperture recognition of pipeline based on WPA andDBNICR was proposed. To effectively extract more valuable leakage information, WPA was applied to refine the measured sound velocity of the ultrasonic signal to design an original set. To achieve the desirable performance of leak aperture recognition and remove the requirement for manual feature selection, DBNICR was used as classifier. To investigate the effectiveness of the proposed method, it was tested on sound velocity of the ultrasonic data of an experimental pipeline to recognize the different leak apertures. The results showed that the proposed method can reliably recognize the different leakage apertures.



---

*Source: 6934825-2018-08-16.xml*