# Fingerprint Classification Based on Multilayer Extreme Learning Machines
## 3 Theoretical Foundations
### Extreme Learning Machines
#### 3.2.2 Multilayer ELM
Due to the ELM-AE's output weights, it is possible to transform features from the entity space back to the input data space. Across all ELM-AE representations, the output weights can be expressed in the following form:  
\[\gamma= \begin{cases}(H^{T}H)^{-1}H^{T}X&M>N\\ H^{T}(H^{T}H)^{-1}X&M\leq N\end{cases}  \tag{6}\]  
Figure 2: Representative structure of a multilayer ELM. where \(H=[h_{1},\ldots,h_{M}]\) represents the outputs of the hidden layer of the ELM-AE, denotes the operator associated with the generalized Moore-Penrose inverse of a matrix, and \(X=[x1,\ldots,x_{M}]\) corresponds to both input and output data of the ELM-AE (unsupervised training. Additionally, \(N\) denotes the number of hidden nodes. The output weights of the ELM-AE are specifically designed to effectively extract features (or entities) from the input data by leveraging singular values. The parameter of regularization (C) is excluded in ELM-AEs for simplicity. ```
Dataset \(X=\{x_{1},x_{2},\ldots,x_{M}\}\), where \(x_{i}\in\mathbb{R}^{n\times j}\), number of hidden neurons \(d\), activation function \(f(\cdot)\)  Extracted features from the autoencoder
1: Initialize random weights \(c=\{c_{1},c_{2},\ldots,c_{M}\}\) and biases \(b=\{b_{1},b_{2},\ldots,b_{M}\}\) such that \(c^{T}c=I\) and \(b^{T}b=1\)
2: For each sample \(x_{i}\), calculate the hidden layer output: \[h_{i}=f(cx_{i}+b)\]
3: Construct the hidden layer output matrix \(H=[h_{1},h_{2},\ldots,h_{M}]\)
4: Calculate the output weights \(\gamma\) by solving the linear system: \[\gamma= \begin{cases}(H^{T}H)^{-1}H^{T}X&\text{if }M>N\\ H^{T}(H^{T}H)^{-1}X&\text{if }M\leq N\end{cases} \]
5: Return the learned features \(H\). ```  
**Algorithm 2** Unsupervised Feature Extraction Phase (Autoencoder)  
The ELM-AE is stacked layer by layer following a hierarchical structure. Prior to supervised least squares optimization, the weights of each M-ELM hidden layer are initialized using the ELM-AE, which performs unsupervised learning at the layer level, excluding random entity mapping. Mathematically, the output of each hidden layer is written as follows:  
\[H^{i}=f[(\gamma^{j})^{T}H^{i-1}], \tag{7}\]  
where \(H^{i}\) represents the output matrix of the \(i\)-th hidden layer (for \(i-1=0\) is the input layer, and the inputs to the M-ELM are represented as \(H^{i-1}\)). Once the features of the previous hidden layer have been computed, the weights and hyperparameters (including  
Figure 3: General architecture of an ELM-AE. The colors indicate the type of neuron, following the same scheme as the original ELM. the activation function and the number of hidden neurons) for the current hidden layer are determined. Within this multilayer learning framework, the AE functions as a feature extractor, utilizing the encoded results to approximate the original inputs while minimizing reconstruction errors. The output of the connections between the final hidden layer and the output node (\(t_{i}\in\mathbb{R}^{N\times m},i=1,\ldots,N\)) can be determined analytically by solving a linear system, similar to the standard ELM approach. **Algorithm 3** Supervised Classifier Learning Phase (Standard ELM)  
**Extracted features \(H\) from AE, labels \(T\), number of hidden neurons \(N\), activation function \(f(\cdot)\) Trained classifier output weights \(\gamma_{ELM}\) 1: Initialize random weights and biases for the ELM hidden layer. 2: Compute the hidden layer output matrix \(H_{ELM}\) using the extracted features \(H\):**  
\[H_{ELM}=f(W\cdot H+b)\]  
**where \(W\) are the weights and \(b\) are the biases for the hidden layer. 3: Solve for the output weights \(\gamma_{ELM}\) by minimizing the least squares error:**  
\[\gamma_{ELM}=(H_{ELM}^{T}H_{ELM})^{-1}H_{ELM}^{T}T\]  
**4: Return the trained classifier output weights \(\gamma_{ELM}\). **  
**Algorithm 3** Supervised Classifier Learning Phase (Standard ELM)