# Fingerprint Classification Based on Multilayer Extreme Learning Machines
## 3 Theoretical Foundations
### Extreme Learning Machines  
#### 3.2.1 Original ELM  
The concept of an ELM was proposed for the training of single-layer feedforward neural networks (SLFNs) in 2006 [17]. ELMs have gained significant popularity due to their ease of implementation. [PERSON] et al. [17] demonstrated that ELMs outperform gradient-based artificial neural networks and SVMs in terms of prediction performance for both classification and regression tasks, offering the unquestionable advantage of negligible computation time, even when executed on standard commercial computers [13].  
Given a training dataset containing \(M\) samples, a basic ELM maps inputs (data samples) to outputs (labels) using a single hidden layer comprising \(N\) nodes. The ELM can be expressed as follows [13]:  
\[H\gamma=T, \tag{1}\]  
\[H=\begin{bmatrix}f(c_{1}\cdot x_{1}+b_{1})&\cdots&f(c_{M}\cdot x_{1}+b_{N})\\ \vdots&f(c_{j}\cdot x_{i}+b_{j})&\vdots\\ f(c_{1}\cdot x_{M}+b_{1})&\cdots&f(c_{M}\cdot x_{M}+b_{N})\end{bmatrix},\]  
\[\gamma=\begin{bmatrix}\gamma_{1}^{T},&\cdots,&\gamma_{N}^{T}\end{bmatrix}^{t},\]  
\[T=\begin{bmatrix}t_{1}^{T},&\cdots,&t_{M}^{T}\end{bmatrix}^{t},\]  
where \(H\) represents the output matrix of the hidden layer, \(\gamma\) denotes the matrix of output weights between the hidden layer and the output layer, and \(T\) corresponds to the label matrix. The function \(f(\cdot)\) is the activation function, which is a nonlinear piecewise continuous function, such as a sigmoid function. The \(c_{j}\) vector is the input weight connecting the input node and the \(j\)-th hidden node, and \(x_{i}\in\mathbb{R}^{n}\) represents the \(i\)-th input data point, where \(n\) is the dimensionality of the input layer. The \(b_{j}\) parameter is the bias of the \(j\)-th hidden node, and \(\gamma_{j}\) denotes the output weight vector connecting the \(j\)-th hidden neuron to the output nodes. Finally, \(t_{i}\in\mathbb{R}^{m}\) is the \(m\)-dimensional target vector corresponding to \(x_{i}\).  
Additionally, \(c_{j}\) and \(b_{j}\) are derived from any continuous probability distribution, such as the uniform distribution, reducing the need for manual intervention. This simplification affects the \(c_{j}\cdot x_{i}\) term, which is calculated as part of the mapping process. The original structure of the ELM is illustrated in Figure 1, where all parameters, including inputs and outputs, are detailed for clarity and comprehension.  
Figure 1: General architecture of an original ELM.  
Moreover, the least squares solution with a minimum norm can be calculated analytically through the generalized Moore-Penrose inverse of \(H\) as follows [13]:  
\[\gamma=\begin{cases}\big{(}H^{T}H+I/C\big{)}^{-1}H^{T}T&M>N\\ H^{T}\big{(}H^{T}H+I/C\big{)}^{-1}T&M\leq N^{\prime}\end{cases} \tag{2}\]  
where \(I\) is an identity matrix and \(C\) is a regularization parameter (\(\in\mathbb{R}^{+}\)). The dimensions of \(I\) depend on the ratio between \(N\) and \(M\), and \(C\) is added to balance the training error and the norm of the output weights, avoiding overfitting. To simplify the analysis, the second term, \(I/C\), is omitted from the equation. This removes the regularization factor, allowing the use of a standard ELM and thereby reducing the model's complexity. To summarize the ELM training process, its algorithm is presented below (Algorithm 1) [13].  
```
1:the training set \(\phi=\{(x_{i},t_{i})\mid i=1,\dots,M\}\), set the hyperparameters, including the activation function \(f(\cdot)\), and the number of hidden neurons \(N\).
2:Set the input weights \(c_{i}\) and biases \(b_{i}\) randomly.
3:Compute the output of the hidden layer matrix \(H\).
4:Set the output weights \(\gamma\).
```  
**Algorithm 1** ELM Training Algorithm