# Reliability Assessment of CNC Machining Center Based on Weibull Neural Network

Authors: Zhaojun Yang; Chao Chen; Jili Wang; Guofa Li
Journal: Mathematical Problems in Engineering
 (2015)

---

## Abstract

CNC machining centers, as the key device in modern manufacturing industry, are complicated electrohydraulic products. The reliability is the most important index of CNC machining centers. However, simple life distributions hardly reflect the true law of complex system reliability with many kinds of failure mechanisms. Due to Weibull model’s versatility and relative simplicity and artificial neural networks’ (ANNs) high capability of approximating, they are widely used in reliability engineering and elsewhere. Considering the advantages of these two models, this paper defined a novel model: Weibull neural network (WNN). WNN inherits the hierarchical structure from ANNs which include three layers, namely, input layer, hidden layer, and output layer. Based on more than 3000 h field test data of CNC machining centers, WNN has been successfully applied in comprehensive operation data analysis. The results show that WNN has good approximation ability and generalization performance in reliability assessment of CNC machining centers.


---

## Body

## 1. Introduction

Common life distributions, like normal distribution, lognormal distribution, and Weibull distribution, usually are simple for system reliability modeling [1, 2]. However, CNC machining centers are complex repairable systems in which reliability distribution could not be responded precisely by these simple life distribution models. Mixture distribution has been used popularly during the development process of modern statistics. The application of mixture distribution could trace back to the late 19th century, while Weibull mixture distribution started in 1950s [3–5]. At present, the most common Weibull mixture distribution is twofold Weibull distribution [6, 7]. Multifold Weibull mixture distribution has been seldom used so far. There are two reasons for this: (
1
) it is hard to estimate large number of parameters, and (
2
) its bad generalization performance makes it difficult to avoid overfitting.With the rapid development of computer technology, artificial neural networks (ANNs), as machine learning model with powerful nonlinear approximation ability, have been developed and get wide applications [8–10]. It is often used to deal with the nonlinear relationship between input and output of complex system [11]. However, ANNs easily bring overfitting phenomenon which is a hot topic and attracts many researchers [12, 13]. Improving the generalization performance of artificial neural networks is a key point to solve overfitting problem.In this paper, Weibull neural network (WNN) is defined based on some advantages of Weibull mixture distribution and artificial neural networks. In this network, hierarchical structure of radical-basis function network (RBF), which has simple structure and powerful nonlinear approximation performance [14, 15], is adopted. RBF was proposed by Moody and Darken [16, 17] with three layers, namely, input layer, hidden layer, and output layer. The input layer is a series of source nodes that connects the networks to reliability data of CNC machining centers. The hidden layer applies a finite Weibull mixture distribution model connecting the input layer and the output layer. The output layer is the probability density of the data. Finite Weibull mixture distribution [7] is applied as hidden layer nodes function (HLNF). Wide application and multiple distribution curve shape are the main characteristics of finite Weibull mixture distribution which suits not only the life distributions of electronic products, but also the life distributions of mechanical parts.This paper will be focused on WNN’s two key issues: to develop an efficient learning method and to improve the generalization performance of WNN. And the rest of the paper is organized as follows: a definition of Weibull neural network (WNN) is given in Section2 with the introduction of basic characteristics of artificial neural networks and Weibull mixture distribution. Section 3 presents the learning process of Weibull neural network (WNN). Section 4 offers field test data of CNC machining centers and applies them into comprehensive simulations and reliability assessment by WNN. For comparison, authors also analyze the data by two-parameter Weibull distribution (TPWD). Finally, conclusions are given in Section 5.


## 2. Weibull Neural Network

### 2.1. Artificial Neural Networks

Artificial neural networks (ANNs) [18] are the abstraction and simulation of certain basic characteristics of biological neural networks. As complex nonlinear approximation mathematical models, ANNs rely on the complexity of network structure by adjusting the internal connections between nodes and then achieve the purpose of training and learn any complex nonlinear relationship with strong robustness and fault tolerance [19].Hierarchical structure is the most common structure of ANNs as BP neural network [20], RBF neural network [14, 15], and so on. Hierarchical structure of ANNs can be divided into several layers by function, such as the input layer, an intermediate layer (also called hidden layer), and the output layer. Each layer is connected in order, as shown in Figure 1. The input layer is responsible for receiving input information from the outside and transfers the information to the neurons of hidden layer. A neutron is an information processing unit which is the fundamental of neural networks. Neuron model constituted different transformation functions with various information processing abilities. The hidden layer is internal information processing layer of neural network, responsible for information conversion. According to required information processing capacity, the hidden layer may be designed as one or more layers. The final one, output layer, supplies the response of neural network to the activation pattern (signal) which is applied to the input layer.Figure 1
Hierarchical structure of artificial neural networks (ANNs).Under the external stimulus of input samples, neural network continuously changes connection weights of network as well as topology structure, so that the output of network is close to desired output. The above process is called learning progress of neural network. In this progress, adjustments and changes of connection weights need to follow certain rules called learning rule.


### 2.2. Weibull Mixture Distribution

Weibull distribution, including two-parameter type and three-parameter type, is the most common life distribution, which is originally proposed by Swedish physicist Waloddi Weibull, for studying the life of components [21]. In practical application, Weibull distribution is often used as the basic model of more complex distributions, such as Weibull mixture distribution [7], Weibull competing risk distribution [22], Weibull parallel distribution, and Weibull segmentation distribution [23]. Among them, Weibull mixture distribution is the most widely used.In many cases, a sample population may be composed of two or more subsamples. Because of the difference of design methods, raw materials, manufacturing processes, and other aspects of reasons, products may follow different life distributions in different conditions. If the sample population is composed ofn subsamples, corresponding subsample cumulative distribution function is expressed as F
1
(
t
)
,
F
2
(
t
)
,
…
,
F
n
(
t
), respectively, probability density is f
1
(
t
)
,
f
2
(
t
)
,
…
,
f
n
t, and corresponding mixture weight of subsample is indicated as α
1
,
α
2
,
…
,
α
n, respectively; then the cumulative mixture distribution function of sample population is shown below:(1)
F
t
=
α
1
F
1
t
+
α
2
F
2
t
+
⋯
+
α
n
F
n
t
.Corresponding probability density function for the mixture distribution is shown below:(2)
f
t
=
α
1
f
1
t
+
α
2
f
2
t
+
⋯
+
α
n
f
n
t
.The general form of Weibull mixture distribution is (2), which is called n-fold Weibull mixture distribution. n-fold refers to any n distinct subsamples. Mixture weights should satisfy the equation: ∑
i
=
1
n
α
i
=
1 and 0
<
α
i
<
1.IfF
i
(
t
) is two-parameter Weibull cumulative distribution with shape parameter β
i and scale parameter η
i or three-parameter Weibull cumulative distribution with shape parameter β
i, scale parameter η
i, and positional parameter ζ
i, two-parameter Weibull cumulative distribution is shown below:(3)
F
i
t
=
1
-
exp
⁡
-
t
η
i
β
i
.Three-parameter Weibull cumulative distribution is shown below:(4)
F
i
t
=
1
-
exp
⁡
-
t
-
ζ
i
η
i
β
i
.The most important feature of mixture Weibull distributions is the diversity in shape. Taking twofold mixture Weibull distribution with two parameters, for example, there are four basic types of density functions shape, as shown in Figure2. Therefore, it is significant to research on the nonlinear approximation performance of mixture Weibull distribution.Figure 2
Basic types of density functions of twofold mixture Weibull distribution with two parameters.


### 2.3. Weibull Neural Network Model

In practical application, the mixture Weibull distributions have two significant limitations: difficult to precisely estimate parameters and hard to choose a suitable folds number. However, the folds affect generalization performance of mixture Weibull distributions seriously. Low folds lead to underfitting; however, high folds result in overfitting. To improve the generalization performance of mixture Weibull distributions, Weibull neural network is proposed in this paper. Weibull neural network is a kind of mixture distribution but is different from traditional mixture Weibull distributions in structure and learning process.The hierarchical structure of Weibull neural network is identical to radical-basis function network (RBF), including the three layers shown in Figure3. The input layer is made up of source nodes that connect the network to reliability data. The hidden layer connects the input layer and the hidden layer and finite Weibull mixture distribution is used as hidden layer function in the network. The output layer is the probability density of the data. The connection between input layer and hidden layer is the probability, namely, that each input data and the hidden layer node are connected in a certain probability. There is a linear weighted connection between the hidden layer and output layer.Figure 3
Hierarchical structure of Weibull neural network (WNN).In the hierarchical structure of Weibull neural network (WNN), the input of network is expressed asT
=
[
t
1
,
t
2
,
…
,
t
N
], where N is the number of input nodes. In this paper, the hidden layer nodes function (HLNF) is finite Weibull mixture distribution expressed as φ
k
(
t
). The output of Weibull neutral network (WNN) is expressed as Y
=
[
y
1
,
y
2
,
…
,
y
Q
], where Q is the number of output data. Connection probability between input layer and hidden layer is expressed as p
j
k which means the probability of the jth data sampled by the kth node. The process is random sampling with replacement, so the value of connection probability p
j
k is 1
/
N. The connection weight between hidden layer and output layer is expressed as ω
k
l which means the weight of the kth node to the lth output y. According to the characteristic of mixture distribution, ω
k
l needs to satisfy(5)
∑
k
=
1
M
ω
k
l
=
1
,where M is the number of the hidden layer nodes. As every node has the same weight, ω
k
l
=
1
/
M.As shown in Figure3, the input layer achieves nonlinear mapping from input data T to hidden layer nodes function (HLNF) φ
k
(
t
), while the output layer achieves linear mapping from hidden layer nodes function (HLNF) φ
k
(
t
) to output data Y. The mathematical model is shown below:(6)
y
l
=
∑
k
=
1
M
ω
k
l
φ
k
t
k
=
1,2
,
…
,
M
.According to the finite Weibull mixture distributionf
i
(
t
) shown in (7), the hidden layer nodes function (HLNF) is calculated by (8). Consider(7)
f
i
t
=
β
i
η
i
t
η
i
β
i
-
1
exp
⁡
-
t
η
i
β
i
,
(8)
φ
k
t
=
α
1
f
1
t
+
⋯
+
α
n
f
n
t
,where n is the folds number of mixture Weibull distributions, i
=
1,2
,
…
n.


## 2.1. Artificial Neural Networks

Artificial neural networks (ANNs) [18] are the abstraction and simulation of certain basic characteristics of biological neural networks. As complex nonlinear approximation mathematical models, ANNs rely on the complexity of network structure by adjusting the internal connections between nodes and then achieve the purpose of training and learn any complex nonlinear relationship with strong robustness and fault tolerance [19].Hierarchical structure is the most common structure of ANNs as BP neural network [20], RBF neural network [14, 15], and so on. Hierarchical structure of ANNs can be divided into several layers by function, such as the input layer, an intermediate layer (also called hidden layer), and the output layer. Each layer is connected in order, as shown in Figure 1. The input layer is responsible for receiving input information from the outside and transfers the information to the neurons of hidden layer. A neutron is an information processing unit which is the fundamental of neural networks. Neuron model constituted different transformation functions with various information processing abilities. The hidden layer is internal information processing layer of neural network, responsible for information conversion. According to required information processing capacity, the hidden layer may be designed as one or more layers. The final one, output layer, supplies the response of neural network to the activation pattern (signal) which is applied to the input layer.Figure 1
Hierarchical structure of artificial neural networks (ANNs).Under the external stimulus of input samples, neural network continuously changes connection weights of network as well as topology structure, so that the output of network is close to desired output. The above process is called learning progress of neural network. In this progress, adjustments and changes of connection weights need to follow certain rules called learning rule.


## 2.2. Weibull Mixture Distribution

Weibull distribution, including two-parameter type and three-parameter type, is the most common life distribution, which is originally proposed by Swedish physicist Waloddi Weibull, for studying the life of components [21]. In practical application, Weibull distribution is often used as the basic model of more complex distributions, such as Weibull mixture distribution [7], Weibull competing risk distribution [22], Weibull parallel distribution, and Weibull segmentation distribution [23]. Among them, Weibull mixture distribution is the most widely used.In many cases, a sample population may be composed of two or more subsamples. Because of the difference of design methods, raw materials, manufacturing processes, and other aspects of reasons, products may follow different life distributions in different conditions. If the sample population is composed ofn subsamples, corresponding subsample cumulative distribution function is expressed as F
1
(
t
)
,
F
2
(
t
)
,
…
,
F
n
(
t
), respectively, probability density is f
1
(
t
)
,
f
2
(
t
)
,
…
,
f
n
t, and corresponding mixture weight of subsample is indicated as α
1
,
α
2
,
…
,
α
n, respectively; then the cumulative mixture distribution function of sample population is shown below:(1)
F
t
=
α
1
F
1
t
+
α
2
F
2
t
+
⋯
+
α
n
F
n
t
.Corresponding probability density function for the mixture distribution is shown below:(2)
f
t
=
α
1
f
1
t
+
α
2
f
2
t
+
⋯
+
α
n
f
n
t
.The general form of Weibull mixture distribution is (2), which is called n-fold Weibull mixture distribution. n-fold refers to any n distinct subsamples. Mixture weights should satisfy the equation: ∑
i
=
1
n
α
i
=
1 and 0
<
α
i
<
1.IfF
i
(
t
) is two-parameter Weibull cumulative distribution with shape parameter β
i and scale parameter η
i or three-parameter Weibull cumulative distribution with shape parameter β
i, scale parameter η
i, and positional parameter ζ
i, two-parameter Weibull cumulative distribution is shown below:(3)
F
i
t
=
1
-
exp
⁡
-
t
η
i
β
i
.Three-parameter Weibull cumulative distribution is shown below:(4)
F
i
t
=
1
-
exp
⁡
-
t
-
ζ
i
η
i
β
i
.The most important feature of mixture Weibull distributions is the diversity in shape. Taking twofold mixture Weibull distribution with two parameters, for example, there are four basic types of density functions shape, as shown in Figure2. Therefore, it is significant to research on the nonlinear approximation performance of mixture Weibull distribution.Figure 2
Basic types of density functions of twofold mixture Weibull distribution with two parameters.


## 2.3. Weibull Neural Network Model

In practical application, the mixture Weibull distributions have two significant limitations: difficult to precisely estimate parameters and hard to choose a suitable folds number. However, the folds affect generalization performance of mixture Weibull distributions seriously. Low folds lead to underfitting; however, high folds result in overfitting. To improve the generalization performance of mixture Weibull distributions, Weibull neural network is proposed in this paper. Weibull neural network is a kind of mixture distribution but is different from traditional mixture Weibull distributions in structure and learning process.The hierarchical structure of Weibull neural network is identical to radical-basis function network (RBF), including the three layers shown in Figure3. The input layer is made up of source nodes that connect the network to reliability data. The hidden layer connects the input layer and the hidden layer and finite Weibull mixture distribution is used as hidden layer function in the network. The output layer is the probability density of the data. The connection between input layer and hidden layer is the probability, namely, that each input data and the hidden layer node are connected in a certain probability. There is a linear weighted connection between the hidden layer and output layer.Figure 3
Hierarchical structure of Weibull neural network (WNN).In the hierarchical structure of Weibull neural network (WNN), the input of network is expressed asT
=
[
t
1
,
t
2
,
…
,
t
N
], where N is the number of input nodes. In this paper, the hidden layer nodes function (HLNF) is finite Weibull mixture distribution expressed as φ
k
(
t
). The output of Weibull neutral network (WNN) is expressed as Y
=
[
y
1
,
y
2
,
…
,
y
Q
], where Q is the number of output data. Connection probability between input layer and hidden layer is expressed as p
j
k which means the probability of the jth data sampled by the kth node. The process is random sampling with replacement, so the value of connection probability p
j
k is 1
/
N. The connection weight between hidden layer and output layer is expressed as ω
k
l which means the weight of the kth node to the lth output y. According to the characteristic of mixture distribution, ω
k
l needs to satisfy(5)
∑
k
=
1
M
ω
k
l
=
1
,where M is the number of the hidden layer nodes. As every node has the same weight, ω
k
l
=
1
/
M.As shown in Figure3, the input layer achieves nonlinear mapping from input data T to hidden layer nodes function (HLNF) φ
k
(
t
), while the output layer achieves linear mapping from hidden layer nodes function (HLNF) φ
k
(
t
) to output data Y. The mathematical model is shown below:(6)
y
l
=
∑
k
=
1
M
ω
k
l
φ
k
t
k
=
1,2
,
…
,
M
.According to the finite Weibull mixture distributionf
i
(
t
) shown in (7), the hidden layer nodes function (HLNF) is calculated by (8). Consider(7)
f
i
t
=
β
i
η
i
t
η
i
β
i
-
1
exp
⁡
-
t
η
i
β
i
,
(8)
φ
k
t
=
α
1
f
1
t
+
⋯
+
α
n
f
n
t
,where n is the folds number of mixture Weibull distributions, i
=
1,2
,
…
n.


## 3. Parameter Estimation of Weibull Neural Network (WNN)

As shown above, in order to define the hidden layer, we need to find the hidden layer nodes function (HLNF)φ
(
t
) and make sure of the number of hidden layer nodes. Then Weibull neural network (WNN) can be finally proposed. This process is called parameter estimation of WNN which could be divided into three steps. They are parameter estimation of HLNF, selection of HLNF, and determining the number of hidden layer nodes. The following is the detailed process.


### 3.1. Parameter Estimation of HLNF

Expectation maximization (EM) algorithm is widely used in mixture distribution to estimate parameters [24]. Because of the independence of learning process of neuron function, EM algorithm can be used to estimate parameters of neuron function separately [25]. EM algorithm is an iterative algorithm based on maximum likelihood estimation, each iteration is divided into two steps, namely, expectation step (E-step) and maximization step (M-step). E-step is to calculate expectations of likelihood function and estimate parameters; M-step is to maximize the expectations. According to input data, the core idea of EM algorithm is to estimate parameters by the iterations of expectations. The whole EM algorithm steps are as follows.Step 1.
Initialize parameters: meansμ
^
1
0
,
μ
^
2
0
,
…
,
μ
^
n
0, variances σ
^
1
2
0
,
σ
^
2
2
0
,
…
,
σ
^
n
2
0, and mixture weights ω
^
1
0
,
ω
^
2
0
,
…
,
ω
^
n
0.Step 2.
E-step: calculate responsivityγ
j according to (9)
γ
j
l
+
1
=
ω
k
l
f
t
j
∑
k
=
1
n
ω
k
l
f
t
j
j
=
1,2
,
…
,
N
.Step 3.
M-step: calculate weightω
k, mean μ
^
k, and variance σ
^
2
k according to (10)
μ
^
k
l
+
1
=
∑
j
=
1
N
γ
j
k
l
t
j
∑
j
=
1
N
γ
j
k
l
,
σ
^
k
2
l
+
1
=
∑
j
=
1
N
γ
j
k
l
t
j
-
μ
^
k
2
∑
k
=
1
M
γ
j
k
k
,
ω
k
l
+
1
=
∑
j
=
1
N
γ
j
k
l
N
,
j
=
1,2
,
…
,
N
;
k
=
1,2
,
…
,
M
.Step 4.
Repeat Steps 2 and 3 until the convergence of maximum likelihood function value is realized.In Step 3,μ
^
k, σ
^
k
2, β
^
k, and η
^
k meet (11) in EM algorithm. However the following is a transcendental equation which is hard to calculate analytical solutions:(11)
μ
=
γ
+
η
Γ
1
+
1
β
,
σ
2
=
η
2
Γ
1
+
2
β
-
Γ
2
1
+
1
β
.In each iteration of EM algorithm, letλ
=
μ
/
σ. When γ
=
0, then the following exists:(12)
λ
=
Γ
1
+
1
/
β
Γ
1
+
2
/
β
-
Γ
2
1
+
1
/
β
0.5
.According to (12), there is a monotonic relationship between λ and β. Therefore, RBF interpolation can be used to establish the mapping relationship between λ and β. Then β can be calculated based on λ. Then, according to the following, η can be easily gotten:(13)
η
=
μ
Γ
1
+
1
/
β
.


### 3.2. Selection of HLNF

Selection of HLNF has a great impact on the generalization performance of neutral network and the efficiency of learning process. Single Weibull distribution as neuron function may lead to underfitting problem. What is more, multifolds Weibull mixture distribution as neuron function may lead to overfitting phenomenon. It is necessary to design an algorithm to select the most appropriate HLNF. Based on the idea of random sampling, this research chooses the value of maximum likelihood function as the index to select the most appropriate finite Weibull mixture distribution as hidden layer nodes function (HLNF) by multiple sampling. This selection algorithm is as follows.Step 1.
Based on the original sample dataS, use the bootstrap methodology [26] to generate N groups of training samples X
=
[
X
1
,
X
2
,
…
,
X
N
] and N groups of testing samples X
′
=
[
X
1
′
,
X
2
′
,
…
,
X
N
′
].Step 2.
For thep groups of training samples X, use the EM algorithm to estimate parameters of finite Weibull mixture distributions which are from 1 to M folds, separately getting mixture weight ω
j
k
o, shape parameter β
j
k
o, and scale parameter η
j
k
o, j
=
1,2
,
…
,
N,  k
=
1,2
,
…
,
M,  o
=
1,2
,
…
,
k.Step 3.
According to the parameters of training samples in Step 2, maximum likelihood function valuesl
j
k
o of N groups testing samples can be calculated by (14), where Y
i is obtained by formulas (6) (7), and (8). Maximum likelihood function values form an evaluation matrix L whose size is N
×
N
×
M:(14)
l
=
ln
⁡
∏
i
=
1
N
Y
i
.Step 4.
According to (15), mean evaluation value P
o of N groups testing samples corresponding to o-folds Weibull mixture distribution can be gotten, and select the maximum evaluation value corresponding to o-folds Weibull mixture distribution. Then o-folds Weibull mixture distribution would be the most appropriate finite Weibull mixture distribution which is the hidden layer nodes function (HLNF) for the original sample data S:(15)
P
o
=
∑
j
=
1
N
∑
k
=
1
M
l
j
k
o
.


### 3.3. Determining the Number of Hidden Layer Nodes

The number of nodes in the hidden layer is associated with not only the function between input and output, but also sample size, random noise, and so forth. Generally small amount of nodes causes poor recognition performance and fitting performance. However large numbers of nodes easily result in random noise and poor recognition performance. Therefore, choosing an appropriate nodes in hidden layer is critical for improving generalization performance of network.Sampling study is helpful to weaken the influence of random noise. Therefore, as the number of nodes in hidden layer dynamically increases, the shape of density function of sample data tends to be stable in the learning process of WNN. For the learning process, similarity coefficient is defined to determine the stopping condition.When the number of nodes in hidden layer isk and k
+
1, define the density function as f
k
(
x
) and f
k
+
1
(
x
), respectively. Then the similarity coefficient (SC) between f
k
(
x
) and f
k
+
1
(
x
) is defined as(16)
S
C
=
∫
0
+
∞
min
⁡
f
k
x
,
f
k
+
1
x
d
x
∫
0
+
∞
max
⁡
f
k
x
,
f
k
+
1
x
d
x
.As is shown in Figure4, the ratio between intersection and union of coverage area by f
k
(
x
) and f
k
+
1
(
x
) is the similarity coefficient (SC). The interval of theoretical value of similarity coefficient is (
0,1
]. The larger SC means the higher similarity between f
k
(
x
) and f
k
+
1
(
x
). When SC = 1, it means that two density functions are identical.Figure 4
Schematic diagram of similarity coefficient.By combining HLNF, bootstrap algorithm, and EM algorithm, the specific algorithm to determine the hidden layer nodes is given as follows.Step 1.
InitializeS
C
c which is the stopping condition of the algorithm.Step 2.
Using Bootstrap algorithm to generate a group of initial training sample dataX
1, estimate parameters θ
1 of HLNF corresponding to sample data X
1 and θ
1
=
(
ω
1
,
η
1
,
β
1
); set k
=
1.Step 3.
Using Bootstrap algorithm to generatek
+
1th group of training sample data X
k
+
1, estimate parameters θ
k
+
1 of HLNF corresponding to sample data X
k
+
1, and θ
k
+
1
=
(
ω
k
+
1
,
η
k
+
1
,
β
k
+
1
). Then the WNN model can be gotten: (17)
f
k
+
1
x
=
1
k
+
1
∑
q
=
1
k
+
1
p
x
;
θ
q
,where p
(
x
;
θ
q
) is the probability density function of qth hidden layer node.Step 4.
According to (16), similarity coefficient S
C
k between k and k
+
1 layers WNN can be gotten to judge whether it meets the stopping condition: S
C
k
>
S
C
c. If S
C
k does not satisfy the condition, go to Step 3; if S
C
k satisfies the condition, end the training.Finally the WNN is defined. And the reliability assessment process is shown in Figure5. In the process, at the 3rd step, when parameters of WNN are gotten, the probability density on WNN is also gotten.Figure 5
Flow chart of reliability assessment based on WNN.


## 3.1. Parameter Estimation of HLNF

Expectation maximization (EM) algorithm is widely used in mixture distribution to estimate parameters [24]. Because of the independence of learning process of neuron function, EM algorithm can be used to estimate parameters of neuron function separately [25]. EM algorithm is an iterative algorithm based on maximum likelihood estimation, each iteration is divided into two steps, namely, expectation step (E-step) and maximization step (M-step). E-step is to calculate expectations of likelihood function and estimate parameters; M-step is to maximize the expectations. According to input data, the core idea of EM algorithm is to estimate parameters by the iterations of expectations. The whole EM algorithm steps are as follows.Step 1.
Initialize parameters: meansμ
^
1
0
,
μ
^
2
0
,
…
,
μ
^
n
0, variances σ
^
1
2
0
,
σ
^
2
2
0
,
…
,
σ
^
n
2
0, and mixture weights ω
^
1
0
,
ω
^
2
0
,
…
,
ω
^
n
0.Step 2.
E-step: calculate responsivityγ
j according to (9)
γ
j
l
+
1
=
ω
k
l
f
t
j
∑
k
=
1
n
ω
k
l
f
t
j
j
=
1,2
,
…
,
N
.Step 3.
M-step: calculate weightω
k, mean μ
^
k, and variance σ
^
2
k according to (10)
μ
^
k
l
+
1
=
∑
j
=
1
N
γ
j
k
l
t
j
∑
j
=
1
N
γ
j
k
l
,
σ
^
k
2
l
+
1
=
∑
j
=
1
N
γ
j
k
l
t
j
-
μ
^
k
2
∑
k
=
1
M
γ
j
k
k
,
ω
k
l
+
1
=
∑
j
=
1
N
γ
j
k
l
N
,
j
=
1,2
,
…
,
N
;
k
=
1,2
,
…
,
M
.Step 4.
Repeat Steps 2 and 3 until the convergence of maximum likelihood function value is realized.In Step 3,μ
^
k, σ
^
k
2, β
^
k, and η
^
k meet (11) in EM algorithm. However the following is a transcendental equation which is hard to calculate analytical solutions:(11)
μ
=
γ
+
η
Γ
1
+
1
β
,
σ
2
=
η
2
Γ
1
+
2
β
-
Γ
2
1
+
1
β
.In each iteration of EM algorithm, letλ
=
μ
/
σ. When γ
=
0, then the following exists:(12)
λ
=
Γ
1
+
1
/
β
Γ
1
+
2
/
β
-
Γ
2
1
+
1
/
β
0.5
.According to (12), there is a monotonic relationship between λ and β. Therefore, RBF interpolation can be used to establish the mapping relationship between λ and β. Then β can be calculated based on λ. Then, according to the following, η can be easily gotten:(13)
η
=
μ
Γ
1
+
1
/
β
.


## 3.2. Selection of HLNF

Selection of HLNF has a great impact on the generalization performance of neutral network and the efficiency of learning process. Single Weibull distribution as neuron function may lead to underfitting problem. What is more, multifolds Weibull mixture distribution as neuron function may lead to overfitting phenomenon. It is necessary to design an algorithm to select the most appropriate HLNF. Based on the idea of random sampling, this research chooses the value of maximum likelihood function as the index to select the most appropriate finite Weibull mixture distribution as hidden layer nodes function (HLNF) by multiple sampling. This selection algorithm is as follows.Step 1.
Based on the original sample dataS, use the bootstrap methodology [26] to generate N groups of training samples X
=
[
X
1
,
X
2
,
…
,
X
N
] and N groups of testing samples X
′
=
[
X
1
′
,
X
2
′
,
…
,
X
N
′
].Step 2.
For thep groups of training samples X, use the EM algorithm to estimate parameters of finite Weibull mixture distributions which are from 1 to M folds, separately getting mixture weight ω
j
k
o, shape parameter β
j
k
o, and scale parameter η
j
k
o, j
=
1,2
,
…
,
N,  k
=
1,2
,
…
,
M,  o
=
1,2
,
…
,
k.Step 3.
According to the parameters of training samples in Step 2, maximum likelihood function valuesl
j
k
o of N groups testing samples can be calculated by (14), where Y
i is obtained by formulas (6) (7), and (8). Maximum likelihood function values form an evaluation matrix L whose size is N
×
N
×
M:(14)
l
=
ln
⁡
∏
i
=
1
N
Y
i
.Step 4.
According to (15), mean evaluation value P
o of N groups testing samples corresponding to o-folds Weibull mixture distribution can be gotten, and select the maximum evaluation value corresponding to o-folds Weibull mixture distribution. Then o-folds Weibull mixture distribution would be the most appropriate finite Weibull mixture distribution which is the hidden layer nodes function (HLNF) for the original sample data S:(15)
P
o
=
∑
j
=
1
N
∑
k
=
1
M
l
j
k
o
.


## 3.3. Determining the Number of Hidden Layer Nodes

The number of nodes in the hidden layer is associated with not only the function between input and output, but also sample size, random noise, and so forth. Generally small amount of nodes causes poor recognition performance and fitting performance. However large numbers of nodes easily result in random noise and poor recognition performance. Therefore, choosing an appropriate nodes in hidden layer is critical for improving generalization performance of network.Sampling study is helpful to weaken the influence of random noise. Therefore, as the number of nodes in hidden layer dynamically increases, the shape of density function of sample data tends to be stable in the learning process of WNN. For the learning process, similarity coefficient is defined to determine the stopping condition.When the number of nodes in hidden layer isk and k
+
1, define the density function as f
k
(
x
) and f
k
+
1
(
x
), respectively. Then the similarity coefficient (SC) between f
k
(
x
) and f
k
+
1
(
x
) is defined as(16)
S
C
=
∫
0
+
∞
min
⁡
f
k
x
,
f
k
+
1
x
d
x
∫
0
+
∞
max
⁡
f
k
x
,
f
k
+
1
x
d
x
.As is shown in Figure4, the ratio between intersection and union of coverage area by f
k
(
x
) and f
k
+
1
(
x
) is the similarity coefficient (SC). The interval of theoretical value of similarity coefficient is (
0,1
]. The larger SC means the higher similarity between f
k
(
x
) and f
k
+
1
(
x
). When SC = 1, it means that two density functions are identical.Figure 4
Schematic diagram of similarity coefficient.By combining HLNF, bootstrap algorithm, and EM algorithm, the specific algorithm to determine the hidden layer nodes is given as follows.Step 1.
InitializeS
C
c which is the stopping condition of the algorithm.Step 2.
Using Bootstrap algorithm to generate a group of initial training sample dataX
1, estimate parameters θ
1 of HLNF corresponding to sample data X
1 and θ
1
=
(
ω
1
,
η
1
,
β
1
); set k
=
1.Step 3.
Using Bootstrap algorithm to generatek
+
1th group of training sample data X
k
+
1, estimate parameters θ
k
+
1 of HLNF corresponding to sample data X
k
+
1, and θ
k
+
1
=
(
ω
k
+
1
,
η
k
+
1
,
β
k
+
1
). Then the WNN model can be gotten: (17)
f
k
+
1
x
=
1
k
+
1
∑
q
=
1
k
+
1
p
x
;
θ
q
,where p
(
x
;
θ
q
) is the probability density function of qth hidden layer node.Step 4.
According to (16), similarity coefficient S
C
k between k and k
+
1 layers WNN can be gotten to judge whether it meets the stopping condition: S
C
k
>
S
C
c. If S
C
k does not satisfy the condition, go to Step 3; if S
C
k satisfies the condition, end the training.Finally the WNN is defined. And the reliability assessment process is shown in Figure5. In the process, at the 3rd step, when parameters of WNN are gotten, the probability density on WNN is also gotten.Figure 5
Flow chart of reliability assessment based on WNN.


## 4. Data Collection and Analysis

In order to validate the WNN model, we collected 23 CNC machining centers’ field test data, almost more than 3000 h running time for each. After data preprocessing, the time between failures obtained within the operation time is listed in Table1.Table 1
Time between failures of 23 CNC machining centers.
No.
Time between failures
1
1008
251.5
369.3
402.5
119.3
402.5
193.3
2
637.3
723.2
185.8
274
183.4
246.7
334.3
3
613.5
178.8
826.3
125.9
391.9
1088.2
4
1044
5
916
383.2
724.7
208.7
509.8
6
1039.5
162.5
426.5
598.3
150
807
7
720.5
154
112.7
573.8
841.3
693.8
8
1283.5
446.5
512.8
9
721.5
1232.7
10
662.2
373.5
516.5
115.7
462.8
135.5
475.8
316.5
11
938.2
171.7
343.5
884.2
208.8
209.5
149
175.2
12
1238.5
13
2243
132.7
694.7
14
1537.2
15
912
1707
16
1359.8
303.7
582.2
17
998.7
448.7
765.2
18
2954.2
19
1597.3
356.8
1249.7
20
2377
983.3
21
2725
612
22
2523
23
2784For the failure data in Table1, according to Section 3.2, twofold Weibull mixture distribution is selected as neuron function. Similarity coefficient S
C
c
=
0.99 is set to be the stopping condition in learning process of Weibull neural network. According to S
C
k
>
S
C
c, the number of hidden layer nodes is calculated as 51. Follow the steps in Section 3.1, estimate the parameters for hidden layer nodes function (HLNF) φ
(
t
), and get mixture weights α, shape parameter β, and scale parameter η of 51 nodes that are shown in Table 2. Substitute those three parameters into (6), (7), and (8) to calculate probability density Y which is the blue line in Figure 6.Table 2
Parameters estimation results of WNN.
Nodes
Parameters
α
β
η
1
0.044068
0.955932
9.033967
1.551175
2637.021
693.5101
2
0.959583
0.040417
1.455573
24.94987
667.7464
2845.028
3
0.082716
0.917284
10.44422
1.39166
2747.245
653.5588
4
0.068718
0.931282
14.46845
1.367156
2747.849
537.2695
5
0.94974
0.05026
1.306634
24.94987
723.8985
2466.681
6
0.056699
0.943301
10.93411
1.494548
2760.401
663.9032
7
0.025545
0.974455
13.8031
1.403081
2557.15
624.5496
8
0.085692
0.914308
13.55691
1.369662
2819.825
579.1172
9
0.107066
0.892934
19.14584
1.43281
2736.786
654.4722
10
0.068432
0.931568
13.75564
1.550829
2724.443
634.1239
11
0.936825
0.063175
1.331295
13.47746
604.2012
2860.957
12
0.055034
0.944966
13.10611
1.355398
2797.873
606.5653
13
0.896614
0.103386
1.347853
16.53756
671.6885
2674.685
14
0.041398
0.958602
16.30011
1.426415
2814.403
684.1883
15
0.086068
0.913932
11.68258
1.546744
2662.409
713.4144
16
0.053959
0.946041
12.07066
1.454367
2669.093
679.6944
17
0.066304
0.933696
17.7975
1.531203
2817.255
722.0759
18
0.932007
0.067993
1.664349
15.259
694.9488
2750.57
19
0.119909
0.880091
16.52835
1.397195
2816.441
706.3912
20
0.08975
0.91025
16.62012
1.656576
2582.263
597.3034
21
0.941526
0.058474
1.561535
24.89791
716.9525
2864.898
22
0.960506
0.039494
1.735449
24.94987
674.279
2433.25
23
0.106106
0.893894
13.30659
1.350209
2830.795
590.5806
24
0.119571
0.880429
13.3864
1.459669
2582.189
577.6149
25
0.922893
0.077107
1.451593
13.29719
620.3416
2750.541
26
0.96321
0.03679
1.448205
12.71502
649.7289
2833.593
27
0.947078
0.052922
1.34193
24.94987
698.0294
2464.589
28
0.945748
0.054252
1.35939
13.40081
617.7688
2778.687
29
0.079954
0.920046
11.68818
1.854535
2766.994
667.8148
30
0.960552
0.039448
1.693323
17.84738
672.5292
2629.48
31
0.627252
0.372748
1.576105
2.480788
923.7622
283.3751
32
0.053352
0.946648
17.31282
1.544846
2786.179
719.6491
33
0.961311
0.038689
1.683207
12.15247
686.9369
2510.049
34
0.881935
0.118065
1.508832
12.78487
651.8541
2706.579
35
0.039565
0.960435
18.62358
1.588752
2760.565
645.8836
36
0.921309
0.078691
1.503808
22.29027
678.9718
2808.833
37
0.958295
0.041705
1.535188
16.46958
743.197
2781.517
38
0.931281
0.068719
1.631859
13.85231
652.3391
2772.066
39
0.912698
0.087302
1.593501
14.62023
696.2038
2602.976
40
0.350292
0.649708
2.550637
1.42347
284.9288
958.2812
41
0.787888
0.212112
1.515084
1.854923
554.2234
1567.009
42
0.980145
0.019855
1.586655
24.94987
678.2233
2932.822
43
0.063178
0.936822
13.20558
1.394374
2851.574
645.8096
44
0.028453
0.971547
11.44262
1.525982
2573.366
627.3912
45
0.87827
0.12173
1.284063
5.906063
753.368
177.2281
46
0.058714
0.941286
10.43322
1.472725
2659.111
612.2541
47
0.922711
0.077289
1.62935
15.13635
690.0196
2786.844
48
0.95051
0.04949
1.543645
24.94987
650.0357
2715.602
49
0.96071
0.03929
1.499167
22.50363
630.3924
2831.961
50
0.922835
0.077165
1.420721
11.26682
618.792
2585.095
51
0.072285
0.927715
13.29679
1.309585
2727.193
631.4372Figure 6
The probability density curves on WNN and TPWD.Distribution law of time between failures of 23 CNC machining centers is modeled by WNN. For comparison, general two-parameter Weibull distribution (TPWD) is used to analysis the same data. And the probability density curves on two methods are shown in Figure6. The blue curve is the probability density on WNN and the red one is on TPWD. The probability density function curves of the data are continuous and derivable in this case, presenting good generalization performance which inherits Weibull method.Different from the only one peak of the probability density curve on TPWD, the blue one has another peak around 1500 h which means the CNC machining center has more probability to be a failure after running about 2500 h. The probability density curve on WNN reveals more accurate information of the distribution law of time between failures than that on TPWD. Two-peak curve by WNN approximates actual condition. In other words, WNN has better approximation ability than TPWD in distribution modeling of life data.Mean time between failures (MTBF) describes the expected time between two failures for a repairable system [27]. MTBF is the major index of the reliability of CNC machining centers; MTBF based on point estimation and MTBF based on interval estimation are the most famous methods for MTBF. To deeply estimate WNN and TPWD, these two MTBF are calculated based on the above two methods. The two MTBF of WNN are calculated by the method in [28–30]. The comparison results are shown in Table 3.Table 3
MTBF of CNC machining centers.
Model
MTBF based on pointestimation (h)
MTBF based on interval estimation (h)
Lower limit
Upper limit
WNN
751.42
628.4559
874.3859
TPWD
737.1336
635.7854
858.1939The results show that MTBF based on point estimation under WNN is larger than that under TPWD by 14.2864 h. And MTBF based on interval estimation, under the same confidence level, are much different: the lower limit of WNN is lower than that of TPBD, and the upper limit of WNN is higher than that of TPBD by 16.192 h. What is more, the confidence interval of MTBF based on interval estimation of WNN is wider than the result of TPBD by over 23.5 h equivalent to 10.58% of TPBD. The big differences of MTBF based on interval estimation and point estimation further show that WNN and TPBD reveal the different distribution laws of time between failures of CNC machining centers. And there is a big error for TPBD in reliability assessment of CNC machining centers. Because WNN has better approximation ability than TPWD in distribution modeling of life data of CNC machining centers, combined with the above comparison, WNN is more close to the actual distribution law of time between failures.


## 5. Conclusions

The basic idea discussed in this paper is the study on applications of Weibull neural network for complex system reliability assessment. General reliability model easily results in overfitting or underfitting in reliability modeling process. The poor generalization performance of general reliability model could not reflect the actual life distribution law of reliability data. For the above problem, through analyzing the characteristics of artificial neural networks and Weibull mixture distribution, the authors propose Weibull neural network (WNN) for system reliability modeling. A common structure of neural network, hierarchical structure, is adopted in Weibull neural network. And three-step learning process of Weibull neural network is proposed in this paper. In the learning process, using interpolation method to solve transcendental equation significantly improves the computational efficiency of EM algorithm. Finally, a practical application case is presented. WNN is used for analyzing distribution law of time between failures of certain type of CNC machining centers. The probability density function curve of the data is continuous and derivable in the case, presenting good generalization performance. For further comparison, authors introduce two-parameter Weibull distribution (TPWD) to calculate MTBF based on point estimation and MTBF based on interval estimation. The result of the case indicates that Weibull neural network (WNN) has better approximation ability than TPWD in distribution modeling of life data. And Weibull neural network (WNN) could be popularized and applied on reliability assessment for complex systems.



---

*Source: 292197-2015-11-16.xml*