## 4 Program Text Classification
### Extreme Learning Machine Classifier  
Extreme learning machine (ELM) is a supervised machine learning algorithm proposed in [20]. ELMs originate from such algorithms as neural networks with random weights (NNRW) [43] and random vector functional link networks (RVFL) [44], and have been successfully applied to various real-world problems [22; 23; 24; 26]. An ELM is a computationallyinexpensive algorithm in comparison with neural networks trained with backpropagation or with evolutionary algorithms [45]. The structure of the ELM adopted for classification is shown in Figure 4. The weights of connections among neurons in ELM can be modeled as two matrices, \(\mathbf{\alpha}\) and \(\mathbf{\beta}\). Cells of the \(\mathbf{\alpha}\) matrix represent weights of connections among \(n\) input neurons \(h_{1},h_{2},\ldots,h_{n}\) and \(d\) hidden neurons \(k_{1},k_{2},\ldots,k_{d}\), so \(\mathbf{\alpha}\in\mathbb{R}^{d\times n}\). Cells of the \(\mathbf{\alpha}\) matrix are initialized randomly, as well as hidden layer biases \(b_{1},b_{2},\ldots,b_{d}\) (see Figure 4). The matrix \(\mathbf{\beta}\in\mathbb{R}^{d\times m}\) containing the weights of connections among \(d\) hidden neurons \(k_{1},k_{2},\ldots,k_{d}\) and \(m\) output neurons \(o_{1},o_{2},\ldots,o_{m}\) is computed according to:  
\[\mathbf{\beta}=\mathbf{H}^{\intercal}\mathbf{Y}_{L}, \tag{1}\]  
where \(\mathbf{Y}_{L}\in\mathbb{R}^{s\times m}\) denotes a matrix containing one-hot encoded class labels for \(s\) objects from the training set \(X_{L}\) that is represented as a matrix \(\mathbf{X}_{L}\in\mathbb{R}^{s\times n}\), \(m\) denotes the dimensionality of \(s\) one-hot encoded class labels, \(m\) also equals to class count, \(n\) denotes the dimensionality of input vectors, \(n\) equals to input neurons count in ELM. \(\mathbf{H}^{\intercal}\) denotes the Moore-Penrose pseudoinverse [21] of the hidden layer output matrix \(\mathbf{H}\), the \(\mathbf{H}^{\intercal}\) matrix belongs to \(\mathbb{R}^{d\times s}\), where \(s\) denotes objects count in \(X_{L}\), \(d\) denotes hidden neuron count, the hidden layer output matrix pseudoinverse \(\mathbf{H}^{\intercal}\in\mathbb{R}^{d\times s}\) is computed as:  
\[\mathbf{H}^{\intercal}=\Big{(}\mathbf{H}^{\intercal  
well; \(\sigma\) is an infinitely differentiable [20] activation function that is applied to every cell of the hidden layer output matrix, sigmoid works best [45] in ELMs.  
The sigmoid activation function is given by the following equation:  
\[\sigma(x)=\frac{1}{1-e^{-x\gamma}} \tag{4}\]  
where \(x\) denotes the cell of the \(\mathbf{H}\in\mathbb{R}^{s\times d}\) matrix before activation. Input neuron count \(s\) depends on the considered classification or regression problem, hidden layer neuron count \(d\) is the hyperparameter of ELM, and output neuron count \(m\) is either set to 1 if a regression problem is considered, or is equal to the count of classes in a multi-class classification problem, where each class label is one-hot encoded. The training process in ELM is described in Algorithm 3. ```
Input:\(\mathbf{X}_{L}\in\mathbb{R}^{s\times n}\)-matrix of \(s\) rows encoding \(n\)-dimensional input vectors, \(\mathbf{Y}_{L}\in\mathbb{R}^{s\times m}\)-matrix of \(s\) rows encoding \(m\)-dimensional output vectors, \(\gamma\in\mathbb{R}\)-regularization parameter, \(d\in\mathbb{N}\)-hidden neuron count, \(\sigma\)-hidden layer activation function. 1. Initialize the weights \(\boldsymbol{\alpha}\in\mathbb{R}^{d\times n}\) with uniformly distributed random numbers. 2. Initialize the biases \(\overset{\rightarrow}{b}\in\mathbb{R}^{d}\) with uniformly distributed random numbers. 3. Compute \(\mathbf{b}\in\mathbb{R}^{s\times d}\) by cloning \(\overset{\rightarrow}{b}\in\mathbb{R}^{d}\)\(s\) times. 4. Compute hidden layer output matrix \(\mathbf{H}=\sigma\big{(}\mathbf{X}_{L}\boldsymbol{\alpha}^{T}+\mathbf{b}\big{)}\). 5. Initialize the identity matrix \(\mathbf{I}\in\mathbb{R}^{d\times d}\). 6.