# Vibration Signal Classification Using Stochastic Configuration Networks Ensemble
## 2 Stochastic Configuration Networks  
As a feedforward neural network, SCNs [25] introduce a supervision mechanism to configure the weights of new nodes in the hidden layer, and the stochastic configuration algorithm can ensure the universal approximation ability of the built model. The SCN with a single hidden layer is shown in Figure 1. Given the input data \(x\in R^{m}\), then the modeling process of the SCN framework can be described as follows.  
Given a target function \(f:R^{d}\to R^{m}\), it is assumed that we have already built a SCN model with \(L-1\) hidden nodes. The output is  
\[f_{L-1}(x)=\sum_{l=1}^{L-1}\beta_{l}g_{l}(w_{l}^{T}x+b_{l}),L=1,2,... \ (f_{0}=0) \tag{1}\]  
where \(w_{l}\in R^{d}\) is the weight parameter, \(b_{l}\in R\) is the bias, \(\beta_{l}=[\beta_{l,1},\beta_{l,2},...,\beta_{l,m}]\) is the weight of the output layer, and \(g\) is the activation function. Then, the residual error is calculated  
\[e_{L-1}=f-f_{L-1}=[e_{L-1,1},...,e_{L-1,m}]. \tag{2}\]  
If the error does not reach the stop condition, a new node should be added, and the new weights \(w_{L},b_{L}\) are configured to calculate the hidden output  
\[h_{L}(x)=g_{L}(w_{L}^{T}x+b_{L}). \tag{3}\]  
Denote a set of variables \(\xi_{L,q},q=1,2,...,m\) as follows:  
\[\xi_{L,q}=\frac{(e_{L-1,q}(x)^{T}\cdot h_{L}(x))^{2}}{h_{L}(x)^{T} \cdot h_{L}(x)}-(1-r-\mu_{L})e_{L-1,q}(x)^{T}e_{L-1,q}(x), \tag{4}\]  
Figure 1: The SCN structure with a single hidden layer.  
where \(0<r<1\), \(\mu_{L}\) is a real sequence with \(0<\mu_{L}\leq 1-r,\lim_{L\rightarrow\infty}\mu_{L}=0\). Based on the the universal approximation property of SCNs in [25], the SCN model is built under the constraint condition  
\[\xi_{L,q}\geq 0,\,q=1,2,...,m. \tag{5}\]  
Besides, \(\beta_{L}\) is calculated by  
\[\beta_{L,q}=\frac{e_{L-1,q}(x)^{T}\cdot h_{L}(x)}{h_{L}(x)^{T}\cdot h_{L}(x)}, \,q=1,2,...,m, \tag{6}\]  
and the current output is  
\[f_{L}=f_{L-1}+\beta_{L}g_{L}(w_{L}^{T}x+b_{L}). \tag{7}\]  
In the stochastic configure learning algorithm, the hidden layer weights \(w_{l},l=1,2,...,L\) are firstly taken from a certain distribution (e.g., uniform distribution or Gaussian distribution), and the bias \(b_{l}\) can be calculated by  
\[b_{l}=-w_{l}^{T}x^{\star},\,l=1,2,...,L, \tag{8}\]  
where \(x^{\star}\) is randomly taken from the training dataset [35]. Then, the supervision mechanism (5) is employed to select proper weight values and add new hidden nodes. The output weight \(\beta\) can also be calculated by solving the following least square problem  
\[\beta^{\star}=[\beta_{1}^{\star},\beta_{2}^{\star},...,\beta_{L}^{\star}]= \arg\min_{\beta}||f-\sum_{l=1}^{L}\beta_{l}h_{l}||_{2}^{2}. \tag{9}\]  
Similarly, for the 2 DSCN model, i.e., the input is matrix data \(x\in R^{d1\times d2}\), the target function is \(f:R^{d1\times d2}\to R^{m}\). The 2 DSCN model is also built under the supervised mechanism (5) with the configured parameters, and the output of the hidden node is  
\[h_{l}(x)=g_{l}(u_{l}^{T}xv_{l}+b_{l}),\,l=1,2,...,L, \tag{10}\]  
where \(u_{l}\in R^{d1}\), \(v_{l}\in R^{d2}\) are the hidden layer weights, \(b_{l}\) is the bias, and \(\beta_{l}\) is the output weight. Then, the model output is  
\[f_{L}(x)=\sum_{l=1}^{L}\beta_{l}h_{l}. \tag{11}\]  
Like \(w_{l},b_{l}\) in (7), the weights \(u_{l},v_{l}\) are randomly assigned and the bias is calculated via \(b_{l}=-u_{l}^{T}x^{\star}v_{l}\) with randomly sampled data \(x^{\star}\). After successfully adding \(L\) hidden nodes, the weight \(\beta\) can be estimated by (7) or another two classical regression models:  
\[\text{Ridge}:\arg\min_{\beta}||f-\sum_{l=1}^{L}\beta_{l}h_{l}||_{2}^{2}+\alpha ||\beta||_{2}^{2}, \tag{12}\]  
\[\text{Lasso}:\arg\min_{\beta}||f-\sum_{l=1}^{L}\beta_{l}h_{l}||_{2}^{2}+\alpha ||\beta||_{1}. \tag{13}\]