## 2 Extreme Learning Machine Improved by Genetic Algorithm
### Extreme Learning Machine Neural Network  
The structure of the neural network of an extreme learning machine is shown in Figure 3, which consists of an input layer, a hidden layer and an output layer. Suppose the input layer has \(n\) input variables, the hidden layer has \(K\) neurons, the output layer has \(m\) output variables and the input and output matrices of the training dataset have \(N\) samples. Then, the input and output of ELM are  
\[\mathbf{X}= \begin{bmatrix}x_{11}&x_{12}&\cdots&x_{1N}\\ x_{21}&x_{22}&\cdots&x_{2N}\\ \vdots&\vdots&\ddots&\vdots\\ x_{n1}&x_{n2}&\cdots&x_{nN}\end{bmatrix} ,\ \mathbf{Y}= \begin{bmatrix}y_{11}&y_{12}&\cdots&y_{1N}\\ y_{21}&y_{22}&\cdots&y_{2N}\\ \vdots&\vdots&\ddots&\vdots\\ y_{m1}&y_{m2}&\cdots&y_{mN}\end{bmatrix}  \tag{1}\]  
Assume that the connection weight matrix \(\mathbf{W}\) between the input and hidden layers is  
\[\mathbf{W}= \begin{bmatrix}w_{11}&w_{12}&\cdots&w_{1n}\\ w_{21}&w_{22}&\cdots&w_{2n}\\ \vdots&\vdots&\ddots&\vdots\\ w_{K1}&w_{K2}&\cdots&w_{Kn}\end{bmatrix}  \tag{2}\]  
where \(w_{ji}\) represents the weight between the \(j\)-th neuron of the hidden layer and the \(i\)-th neuron of the input layer. Suppose the connection weight matrix \(\mathbf{\beta}\) of the output and hidden layers is  
\[\mathbf{\beta}= \begin{bmatrix}\beta_{11}&\beta_{12}&\cdots&\beta_{1m}\\ \beta_{21}&\beta_{22}&\cdots&\beta_{2m}\\ \vdots&\vdots&\ddots&\vdots\\ \beta_{K1}&\beta_{K2}&\cdots&\beta_{Km}\end{bmatrix}  \tag{3}\]  
where \(\beta_{ji}\) represents the weight between the \(j\)-th neuron of the hidden layer and the \(i\)-th neuron of the output layer. In this case, the output of the ELM is \(\mathbf{Z}=[z_{1},z_{2},\cdots,z_{N}]\)  
Figure 3: Illustration of single-layer feedforward neural network. \[z_{j}= \begin{bmatrix}z_{1j}\\ z_{2j}\\ \vdots\\ z_{mj}\end{bmatrix} = \begin{bmatrix}\sum\limits_{i=1}^{K}\beta_{i1}\varphi\big{(} \mathbf{w}_{i}x_{j}+b_{i}\big{)}\\ \sum\limits_{i=1}^{K}\beta_{i2}\varphi\big{(}\mathbf{w}_{i}x_{j}+b_{i}\big{)}\\ \vdots\\ \sum\limits_{i=1}^{K}\beta_{im}\varphi\big{(}\mathbf{w}_{i}x_{j}+b_{i}\big{)} \end{bmatrix}  \tag{4}\]  
in which \(\mathbf{w}_{i}=[w_{i1},w_{i2},\cdots,w_{in}]\), \(\mathbf{x}_{j}= \begin{bmatrix}w_{1j},w_{2j},\cdots,w_{nj}\end{bmatrix} ^{\mathsf{T}}\), and \(j=\)1, 2, \(\cdots\), \(N\). Moreover, \(\mathbf{b}= \begin{bmatrix}b_{1}&b_{2}&\cdots&b_{K}\end{bmatrix} ^{\mathsf{T}}\) indicates the neuron offset of the hidden layer. In addition, the superscript T indicates the transpose operation. Equation (4) can be rearranged into  
\[\mathbf{H}\mathbf{\beta}=\mathbf{Z}^{\mathsf{T}} \tag{5}\]  
in which \(\mathbf{H}\) is the output matrix of the hidden layer and expressed as  
\[\mathbf{H}= \begin{bmatrix}\varphi(\mathbf{w}_{1}\!\cdot\!\mathbf{x}_{1}+b_{1})&\varphi (\mathbf{w}_{2}\!\cdot\!\mathbf{x}_{1}+b_{2})&\cdots&\varphi(\mathbf{w}_{K}\!\cdot\!\mathbf{x} _{1}+b_{K})\\ \varphi(\mathbf{w}_{1}\!\cdot\!\mathbf{x}_{2}+b_{1})&\varphi(\mathbf{w}_{2}\!\cdot\!\mathbf{x} _{2}+b_{2})&\cdots&\varphi(\mathbf{w}_{K}\!\cdot\!\mathbf{x}_{2}+b_{K})\\ \vdots&\vdots&\ddots&\vdots\\ \varphi(\mathbf{w}_{1}\!\cdot\!\mathbf{x}_{N}+b_{1})&\varphi(\mathbf{w}_{2}\!\cdot\!\mathbf{x} _{N}+b_{2})&\cdots&\varphi(\mathbf{w}_{K}\!\cdot\!\mathbf{x}_{N}+b_{K})\end{bmatrix}  \tag{6}\]  
The ideal case is that the output of the network \(\mathbf{Z}^{\mathsf{T}}\) is identical to \(\mathbf{Y}^{\mathsf{T}}\), but in practice, the constraint condition is that \(\|\mathbf{Y}^{\mathsf{T}}-\mathbf{Z}^{\mathsf{T}}\|<\varepsilon\) (\(\varepsilon\) is an arbitrary positive value). Then, the weight matrix \(\mathbf{\beta}\) between the hidden and output layers is determined as  
\[\min_{\mathbf{\beta}}\|\mathbf{H}\mathbf{\beta}-\mathbf{Y}^{\mathsf{T}}\| \tag{7}\]  
Equation (7) can be effectively solved by a Moore-Penrose inverse according to the least-squares criterion, and the solution of \(\mathbf{\beta}\) is  
\[\mathbf{\beta}=\mathbf{H}^{+}\mathbf{Y}^{\mathsf{T}} \tag{8}\]  
in which the superscript \(+\) represents the pseudoinverse operator.