# Local Coupled Extreme Learning Machine Based on Particle Swarm Optimization
## 2 Theoretical Background  
### Local Coupled Extreme Learning Machine  
The ELM learning algorithm is a simple, fast and efficient method. For further improving the generalization performance of the ELM, the LC-ELM learning algorithm was proposed by [PERSON] [10] in which the efficiency of LC-ELM in terms of classification and regression benchmark problems was investigated. In the LC-ELM, due to the utilization of the fuzzy membership function \(F(\cdot)\) and the similarity relation \(S(x,d_{i})\), the complexity of the weight searching space was reduced and the generalization performance was correspondingly improved in terms of the simple neural networks structure. The mathematical formulation of the LC-ELM is presented as follows:  
For \(M\) arbitrary distinct examples \((\mathbf{x}_{i},\mathbf{t}_{i})\), where \(\mathbf{x}_{i}=[x_{i1},x_{i2},\cdots x_{ip}]\in R^{p}\) is the input and \(\mathbf{t}_{i}=[t_{i1},t_{i1},\cdots t_{iq}]\in R^{q}\) is the expected output, \(i=1,\ldots,M\). The output of the hidden layer neurons \(g(\mathbf{w}_{i}\mathbf{x}_{j}+b_{i})\) for the ELM is modified with the help of fuzzy membership function as \(g(\mathbf{w}_{i}\cdot\mathbf{x}_{j}+b_{i})F(S(\mathbf{x}_{j},\mathbf{d}_{i}))\). Therefore, the network output of the LC-ELM with \(N\) hidden neurons are mathematically modeled by  
\[f(\mathbf{x}_{j})=\sum_{i=1}^{N}\beta_{i}g(\mathbf{w}_{i}\cdot\mathbf{x}_{j}+b_ {i})F(S(\mathbf{x}_{j},\mathbf{d}_{i})),i=1,\ldots,N,j=1,\ldots,M \tag{1}\]  
where \(g(\cdot)\) denotes the activation function of the ELM, which can not only be sigmoid functions, however also other functions such as \(\sin\), \(\cos\), cubic, etc. \(\beta_{i}\) denotes the weight vector connecting the \(i\)th hidden neuron and the output neurons, \(\mathbf{w}_{i}\) is the weight vector connecting the \(i\)th hidden neuron and the input neurons. \(b_{i}\) is the bias of the \(i\)th hidden neuron. \(\mathbf{d}_{i}\) is the address of the \(i\)th hidden node. In the LC-ELM learning algorithm, the similarity relation \(S(\mathbf{x},\mathbf{d}_{i})\) is the distance between the input \(\mathbf{x}\) and the \(i\)th hidden node with address \(\mathbf{d}_{i}\). Various forms of fuzzy membership functions \(F(\cdot)\), such as Gaussian function, sigmoid function and reverse sigmoid function [21; 22], are utilized. In addition, the underlying radius parameter \(r\) is kept in \(F(\cdot)\) for adjusting the width of the activation area, which is also an optimized parameter, to the same as the address parameter \(\mathbf{d}\). Combining the structure of the LCFNN with the learning mechanism of the ELM, the LC-ELM also is a three step learning algorithm and the parameters (input weights \(\mathbf{w}\) and biases \(b\) between the input layer and hidden layer, the address \(\mathbf{d}\) of the hidden neurons) of the networks are assigned randomly, which is the same as the ELM [10]. The standard LC-ELM learning algorithm can approximate these \(M\) examples with zero error means \(\sum_{j=1}^{N}\bigl{|}\bigl{|}\sigma_{j}-t_{j}\bigr{|}\bigr{|}=0\), where \(\mathbf{o}_{j}\) is the actual output of the LC-ELM. i.e., the corresponding relation is defined by  
\[f(\mathbf{x}_{i})=\mathbf{t}_{j},j=1,\ldots,M \tag{2}\]  
The above \(M\) equations can be written compactly as a linear system:  
\[\mathbf{H}\beta=\mathbf{T},j=1,\ldots M \tag{3}\]  
where \(\mathbf{H}\) is the output matrix of the hidden layer and can be expressed as  
\[\mathbf{H}=\left\{\mathbf{h}_{ji}=g(\mathbf{w}_{i}\mathbf{x}_{j}+b_{i})F(S( \mathbf{x},\mathbf{d}_{i}))\right\}i=1,\ldots,N,j=1,..,M. \tag{4}\]  
in the above Equation (4), \(\mathbf{h}_{ji}=g(\mathbf{w}_{i}\mathbf{x}_{j}+b_{i})\) denotes the output of the \(i\)th hidden neuron with respect to \(\mathbf{x}_{j}\). \(\beta=\left[\beta_{1},\ldots,\beta_{N}\right]_{N\times q}^{T}\) is the matrix of the output weights and \(\beta_{i}\) denotes the weight vector connecting the \(i\)th hidden node and the output layer.