# Fingerprint Classification through Standard and Weighted Extreme Learning Machines
## 3 Background
### Extreme Learning Machines  
ELM results in an algorithm for single hidden layer feedforward neural networks (SLFNs), massively popular for its fast learning speed and excellent performance in generalization. [PERSON] et al. [17] have shown that ELM outperforms gradient-based artificial neural networks and support vector machines in terms of prediction performance. Given a training set with \(L\) samples, the basic ELM maps inputs (data samples) and outputs (labels) by employing a single hidden layer composed by \(N\) nodes. As mathematical representation [7; 48]:  
\[ \begin{bmatrix}g(w_{1}\cdot x_{1}+b_{1})&\cdots&g(w_{L}\cdot x_{1}+b_{N})\\ \vdots&g(w_{j}\cdot x_{i}+b_{j})&\vdots\\ g(w_{1}\cdot x_{L}+b_{1})&\cdots&g(w_{L}\cdot x_{L}+b_{N})\end{bmatrix}\begin{bmatrix} \beta_{1}^{T}\\ \vdots\\ \beta_{N}^{T}\end{bmatrix}  = \begin{bmatrix}t_{1}^{T}\\ \vdots\\ t_{L}^{T}\end{bmatrix} , \tag{1}\]  
where \(H\) is the hidden layer output matrix, \(\beta\) denotes the output weights matrix between the hidden layer and output layer, \(T\) represents the target output results of the output layer, \(g(\cdot)\) refers to a non-linear piecewise continuous function, such as the sigmoid function, \(w_{j}\) is the input weight vector between the input node and \(j\)th hidden node, \(x_{i}\in\mathbb{R}^{n}\) refers to the \(i\)th input data where \(n\) means the dimension of the input layer, \(b_{j}\) represents the bias of the \(j\)th hidden node, \(\beta_{j}\) denotes the output weight vector between the \(j\)th hidden neuron and output nodes, and \(t_{i}\in\mathbb{R}^{m}\) is the m-dimensional target vector originated by \(x_{i}\). Furthermore, \(w_{j}\) and \(b_{j}\) result from any continuous probability distribution, such as the rectangular distribution; the human intervention consequently decreases. To conclude, the term \(w_{j}\cdot x_{i}\) comes to be the inner product of \(w_{j}\) and \(x_{i}\). For clarification purposes, the structure of the traditional ELM is shown in Figure 2 where all layers are identified in detail. The least square solution with minimal norm can be analytically calculated through the Moore-Penrose generalized inverse of \(H\) as follows [16; 17]:  
\[\beta=\left\{ \begin{array}{rl}(H^{T}H+I/C)^{-1}H^{T}T&\text{if }L>N\\ H^{T}(HH^{T}+I/C)^{-1}T&\text{otherwise}\end{array} ,\right. \tag{2}\]  
with \(I\) and \(C\) being a unit matrix and regularization parameter \(\in\mathbb{R}^{+}\), respectively. The \(I\) dimensions depend on the relationship between \(N\) and \(L\), and \(C\) is added in order to balance the training error and the norm of output weights, by avoiding the over-fitting. Like the rest of conventional learning algorithms, the learning capability of the original ELM can be affected by the class distribution [19]. It provides superior performance in the case of balanced datasets; however, the unbalanced classification can be difficult. To this end, samples with high training errors must be related to small weights and vice-versa in the ELM algorithm [20]. According to the Karush-Kuhn-Tucker theorem, the solution to \(\beta\) acquires the form of [10]:  
\[\beta=\left\{ \begin{array}{rl}(H^{T}WH+I/C)^{-1}H^{T}WT&\text{if }L>N\\ H^{T}(WHH^{T}+I/C)^{-1}WT&\text{otherwise}\end{array} ,\right. \tag{3}\]  
Figure 2: General architecture of a single hidden layer feedforward neural network (SLFN) with the extreme learning machine (ELM) algorithm. where \(W\) denotes the misclassification cost matrix. It is a \(L\times L\) diagonal matrix according to the class distribution as follows [19]:  
\[\text{Weighted ELM1:}\quad W_{ii}=1/N(t_{i}), \tag{4}\]  
\[\text{Weighted ELM2:}\quad W_{ii}=\left\{ \begin{array}{cl}0.618/N(t_{i})& \text{ if }N(t_{i})>\text{mean}[N(t_{i})]\\ 1/N(t_{i})&\text{ otherwise}\end{array} ,\right. \tag{5}\]  
where \(N(t_{i})\) refers to the number of samples in the class \(t_{i}\). In the weighted ELM1 (W-ELM1), the unbalanced datasets reach a cardinal balance.