# BELMKN: Bayesian Extreme Learning Machines Kohonen Network
## 2 Methodology
### Partitional Clustering Using the Kohonen Network  
The third phase in BELMKN involves partitional clustering of the data using Kohonen Network (KN). The input to this network is the output obtained from ELM-based feature learning (\(E\)) with BIC-based cluster prediction (\(n_{c}\)). The Kohonen Network consists of two layers, namely, the input and the output layers. The number of output layer neurons (\(n_{o}\)) is the number of clusters (\(n_{c}\)) that is determined by BIC. The weight matrix between the input and the output layers, \(W_{k}eR^{\beta\times n_{c}}\), in which \(\beta\) is the number of input neurons. Further, the weights are calculated using the discriminant function value that is used as the basis for competition using Euclidean distance as a distance metric given by,\[d(j)=\sqrt{\sum\limits_{i=1}^{n_{i}}{(y_{i}-w_{ij})}} \tag{8}\]  
in which \(j\) is the dimension ranging 1 to \(n_{o}\). The winning neuron is the one that closely matches with the input, i.e., the neuron for which the discriminant function value is minimum [17]. We calculate the value of neighborhood function using,  
\[h_{ci}(t)=\alpha(t)\exp\left(\frac{-d_{ij}{}^{2}}{2\sigma(t)}\right) \tag{9}\]  
in which \(t\) is the iteration number, \(\alpha(t)\) is the learning rate at iteration \(t\) given by \(\alpha(t)=\alpha_{0}\exp\left(\frac{-t}{T_{1}}\right)\), and \(\sigma(t)\) is the spread of the data points in consideration given by \(\sigma(t)=\sigma_{0}\exp\left(\frac{-t}{T_{2}}\right)\) in which \(T_{1}\) and \(T_{2}\) are time constants [29]. Finally, we update the weights of the winning neuron and the neighboring neurons using [17],  
\[\Delta w_{ij}=h_{ci}(t)(y_{i}-w_{ij}) \tag{10}\]  
The performance of partitional clustering is calculated using the clustering accuracy. The clustering accuracy (CA) is given by [30],  
\[\text{CA}=\frac{\sum\limits_{i=1}^{n_{c}}{a_{i}}}{N} \tag{11}\]  
in which \(a_{i}\) is the number of correctly clustered samples with respect to the class labels, and \(N\) is the total number of samples in the dataset. ```
Input: Input: Input the normalized dataset (\(X_{i}\)) and randomly initialize weights (\(W\)) between input and hidden layers Output: Output the feature learning (\(E\)) obtained from ELM network, the number of clusters computed using BIC (\(n_{c}\)), the clustering centers (\(C_{no}\)), and clustering accuracy obtained from the KN Beginning:
1. Obtain the hidden layer activations using Equation (1)
2. Construct the graph Laplacian (\(L\)) using Equations (3) and (4) from the normalized input data
3. Calculate the eigenvectors \(v_{j}\) from the eigenvalues \(\gamma_{j}\) using Equation (5)
4. Choose the eigenvectors \(\delta=[\widetilde{v}_{2},\widetilde{v}_{3},\ldots,\widetilde{v}_{\beta+1}]\) in which \(\widetilde{v}_{j}=\frac{v_{j}}{\|Hv_{j}\|},j=2,3,\ldots,\beta+1\), corresponding to the (2 to \(\beta+1\)) eigenvalues. 5. Obtain \(\delta\) in which columns are normalized eigenvectors
6. Compute feature learning (\(E\)) using Equation (6)
7. Calculate BIC to obtain the number of clusters (\(n_{c}\)) using Equation (7)
8. The input to the KN will be feature learning (\(E\)) and the number of clusters (\(n_{c}\))
9. Repeat steps 10 and 11 until there is no change in topology
10. Compute the winning neuron index using Equation (8)
11. Update the weights of winning neuron and its neighbors using Equations (9) and (10)
12. Assign the cluster numbers for each sample with the weights of the KN using minimum distance criteria
13. Evaluate the clustering accuracy using Equation (11) End
```  
**Pseudo code:** A high-level description of BELMKN  
```
Input: Input: Input the normalized dataset (\(X_{i}\)) and randomly initialize weights (\(W\)) between input and hidden layers Output:

Output the feature learning (\(E\)) obtained from ELM network, the number of clusters computed using BIC (\(n_{c}\)), the clustering centers (\(C_{no}\)), and clustering accuracy obtained from the KN

**Beginning:**
1. Obtain the hidden layer activations using Equation (1)
2.