# Distributed Kernel Extreme Learning Machines for Aircraft Engine Failure Diagnostics
## 3 Distributed Kernel Extreme Learning Machines
### Distributed Framework  
The framework of the proposed DKELMs algorithm is illustrated by Figure 2, including feature samples input and output, distributed subnetworks, fusion module and estimated output. In the distributed subnetworks, \(l\) learning machines are used to process the input features, and the fusion module can mix the outputs of \(l\) subnetworks. The entire training dataset can be represented as \(\{\mathbf{x}_{i},t_{i}\}_{i}^{n}\), and divide it into \(l\) subsets. The training samples number of the \(p_{th}\) subset is represented as \(n_{p}\), and we have \(\sum\limits_{p=1}^{l}n_{p}=n\). The \(p_{th}\) subset of training samples \(\{\tilde{\mathbf{x}}_{i},\tilde{t}_{i}\}_{1}^{n_{p}}\) is used to construct the \(p_{th}\) subnetwork DKELM \(p\) in Figure 2. For the \(p_{th}\) subnetwork, the hidden nodes number is \(n_{p}\), and the portion kernel matrix \(\tilde{K}_{p}\) is defined as follows  
\[\tilde{K}_{p}:\tilde{K}_{i,j}=k(\tilde{\mathbf{x}}_{i},\mathbf{x}_{j}),i=1,\cdots,n_{p },j=1,\cdots,n \tag{10}\]  
For the inversion calculation of the non-square matrix \(\tilde{K}_{p}\), the generalized inverse theory should be used, and taking no account of regular terms, the kernel output weights can be represented as \(\tilde{\mathbf{x}}_{p}=\tilde{K}_{p}^{\intercal}\mathbf{t}=(\tilde{K}_{p}\tilde{K}_{p} ^{\intercal})^{-1}\tilde{K}_{p}\mathbf{t}\). According to the ridge regression theory, adding a small positive deviation to the diagonal value of \(\tilde{K}_{p}\tilde{K}_{p}^{\intercal}\) can improve the stability of the solution. Then, the model output of subnetwork DKELM \(p\) can be expressed as follows  
\[\tilde{\mathbf{x}}_{p}=(I/C+\tilde{K}_{p}\tilde{K}_{p}^{\intercal})^{-1}\tilde{K} _{p}\mathbf{t} \tag{11}\]  
\[f_{p}(\mathbf{x})=\left[k(\mathbf{x},\tilde{\mathbf{x}}_{1})\cdots k(\mathbf{x},\tilde{\mathbf{x} }_{n_{p}})\right]\tilde{\mathbf{x}}_{p} \tag{12}\]  
Figure 1: Network structure of the original Kernel Extreme Learning Machines (KELM) algorithm.  
The proposed DKELMs algorithm can replace the inversion of a \(n\times n\) matrix by inversions of \(n_{p}\times n_{p}\) matrix in \(l\) times. When it comes to largescale dataset, the computation complexity can be largely cut down. Given the kernel function and a training dataset \(\{\mathbf{x}_{i},t_{i}\}_{i}^{n}\), the \(p_{th}\) subnetwork DKELM \(p\) can be constructed as follows  
1. Divide \(\{\mathbf{x}_{i},t_{i}\}_{i}^{n}\) into \(l\) subsets, and pick the \(p_{th}\) subset \(\{\mathbf{\tilde{x}}_{i},\tilde{t}_{i}\}_{1}^{n_{p}}\).
2. Calculate the rectangular kernel matrix \(\tilde{K}_{p}\in R^{n_{p}}\times R^{n_{l}}\).
3. Calculate the kernel output weights \(\mathbf{\tilde{x}}_{p}\) as Formula (11).
4. Figure out the output of the subnetwork DKELM \(p\) for new input vector \(\mathbf{x}\) according to Formula (12).  
In the fusion layer, output information of \(l\) parallel subnetworks should be taken into account comprehensively and come up with more excellent results. Therefore, a likelihood probability distribution-based regression fusion method is constructed for the regression problem, and for the classification problem, a classification fusion method based on DS evidence theory is designed.