# BELMKN: Bayesian Extreme Learning Machines Kohonen Network
## 2 Methodology
### Partitional Clustering Using the Kohonen Network
Hence, prediction accuracy is 80% in the case of ELM-based feature learning with BIC, whereas it is 60% when BIC is applied directly to the dataset. As a result, by using ELM-BIC we are able to obtain the exact number of clusters as given by the dataset. ### Effect of Parameter Settings

In BELMKN, it is observed that the performance of clustering and convergence rate of the ELM network depends on the number of hidden neurons. ELM requires a sufficient number of hidden neurons to capture the nonlinearity, as it contains only one hidden layer. The number of hidden neurons varied in multiples of 10 from 10 to 150 for each dataset. The variation of the clustering accuracy with the number of hidden neurons for the Iris dataset is shown in \[h_{ci}(t)=\alpha(t)\exp\left(\frac{-d_{ij}{}^{2}}{2\sigma(t)}\right) \tag{9}\]0 It is observed that with 10 hidden neurons, clustering accuracy is less, as class 2 and class 3 samples overlap. This leads to underfitting. Although it is very difficult to determine the exact number of hidden neurons for a given dataset, we can get an approximate estimate by empirically trying different values. It is observed that for Iris Dataset, with 120 hidden neurons, the maximum accuracy is achieved. With further increase in the hidden neurons, the clustering accuracy is observed to decrease due to overfitting. \begin{table}
\begin{tabular}{c c c c c} \hline \hline
**Sl. No.** & **Dataset** & **Number of Samples** & **Input Dimension** & **Number of Clusters** \\ \hline
1 & Cancer & 569 & 30 & 2 \\
2 & Dermatology & 366 & 34 & 6 \\
3 & _E. coli_ & 327 & 7 & 5 \\
4 & Glass & 214 & 9 & 6 \\
5 & Heart & 270 & 13 & 2 \\
6 & Horse & 364 & 27 & 3 \\
7 & Iris & 150 & 4 & 3 \\
8 & Thyroid & 215 & 5 & 3 \\
9 & Vehicle & 846 & 18 & 4 \\
10 & Wine & 178 & 13 & 3 \\ \hline \hline \end{tabular}
\end{table} 
Table 1: Properties of the dataset. \begin{table}
\begin{tabular}{c c c c c c c c c c} \hline \hline
**Dataset** & **Cancer** & **Dermatology** & _E. coli_ & **Glass** & **Heart** & **Horse** & **Iris** & **Thyroid** & **Vehicle** & **Wine** \\ \hline Actual Clusters & 2 & 6 & 5 & 6 & 2 & 3 & 3 & 3 & 4 & 3 \\ \hline BIC cluster predicted & & & & & & & & & \\ on original dataset & 3 & 6 & 4 & 3 & 2 & 2 & 3 & 3 & 4 & 3 \\ \hline BIC cluster predicted & & & & & & & & & \\ using ELM & 3 & 6 & 5 & 6 & 3 & 3 & 3 & 3 & 4 & 3 \\ \hline \hline \end{tabular}
\end{table} 
Table 2: Cluster predicted using BIC for actual data and ELM-based feature learning data. In comparison with the US-ELM [28], in which 97% clustering accuracy was obtained for 1000 hidden neurons, the proposed BELMKN framework uses only 120 hidden neurons to obtain 97% clustering accuracy. Hence, we can observe that BELMKN provides a significant improvement by reducing the number of hidden neurons, which saves computational time. The other ELM parameter values used are tradeoff parameter \(\lambda=\{10^{-4},10^{-3},\ldots,1,10^{1},\)\(10^{2},\ldots,10^{4}\}\), sigma \(\sigma=\{0,10^{0},10^{1},10^{2},\ldots\}\). The parameters \(\lambda\) and \(\sigma\) are varied as in the above sequences empirically for each of the 10 datasets. It is observed that with negative orders of magnitude of the hyperparameter \(\lambda\), a smoother fit is obtained. On increasing the value of \(\lambda\), overfitting is observed. On increasing the value of \(\sigma\), which signifies the number of nearest neighbours, overfitting is also observed. For the Iris dataset, by assigning \(\sigma=50\), the effect of \(\lambda\) on the clustering accuracy is presented in Figure 6. With \(\lambda=10^{-4}\), the clustering accuracy is maximum, which is 97%. With \(\lambda=10^{-3}\), the accuracy is observed to decrease to 85%. On further increase in the value of \(\lambda\), the accuracy decreases from 85% to 75%. Hence, it is essential to select the optimal values of the parameters to achieve higher accuracy. Similarly, the parameter setting is done for the remaining datasets. Figure 5: Clustering accuracy versus number of hidden neurons.