## 4 Non-Ensemble-Based Methods
### K-Nearest Neighbor  
The basic working procedure of kNN is to find a cluster (subset) of \(k\) instances which is the nearest to the predicted sample in the underlying training space, thus showing the independence from the structure of the data. The set of parameters [64] that needs to be tuned to improve the performance of kNN is \(k\), _'weights'_, and _'leaf\({}_{size}\)'_. The distance between two points is calculated using the Euclidean distance function. The class having majority \(k\)-nearest neighbors is the predicted class for the new predicted sample. \(K\) is the key parameter to be tuned (after exhaustive grid search) to obtain a satisfactory result. Other important parameters are _'weights'_ and _'leaf\({}_{size}\)'_, which are weights of two types, namely uniform and distance; for a better prediction accuracy, the uniform weight is  
\begin{table}
\begin{tabular}{p{56.9 pt} p{113.8 pt} p{113.8 pt} p{113.8 pt}} \hline \hline
**Algorithm** & **Tuned Parameter** & **Search Range** & **Best P** & **Dataset** \\ \hline Gradient boosting & Number of estimators & [100; 150; 200; 250; 300; 400; 500; 600; 800; 100; 1200] & 200 & Soybean, Satimage, Opt\_Digits, Glass, LED\_Domain, Ecoli, Dermatology, Bridges, Breast\_Tissue, Human\_Activity\_Recognition \\ \hline  & Learning rate & [0.01; 0.02; 0.5; 0.1; 0.2; 0.25; 0.3; 0.4; 0.5] & 0.1 & \\ \hline  & Min\_samples\_split & [2; 3; 4; 5; 6; 8; 10; 15] & 3 & \\ \hline  & Max\_tree\_depth & [3; 4; 5; 6; 7; 8; 9; 10; 12; 15] & 8 & \\ \hline Random forest & Number of estimators & [50, 120; 150; 200; 220; 250; 300; 350; 600; 700] & 200 & \\ \hline  & Max\_number of features & [’log2’; ‘sqrt’; ‘all’] & all \\ \hline  & Min\_samples\_leaf & [1; 2; 3; 4; 5; 6; 7; 8; 9; 10] & 2 & \\ \hline Decision tree & ‘max\_features’: & [’auto’, ‘sqrt’, ‘log2’, None ] & None \\ \hline  & criterion & ‘gini’, ‘entropy’ & gini \\ \hline  & max\_depth & 1, 20, 2 & \\ \hline  & splitter & ‘best’, ‘random’ & best \\ \hline  & ‘min\_samples\_split’: & [2; 5; 10], & 5 & \\ \hline  & ‘min\_samples\_leaf’: & [1, 2, 4, 10], & 1 & \\ \hline K-NN & n\_neighbors’: & np.arrange (1, 15), & 5 & \\ \hline  & weights’: & [’uniform’, ‘distance’], & Uniform \\ \hline  & leaf\_size’ & [1, 3, 5] & 5 & \\ \hline  & & & 2 \(\times\)\(10^{-15},2\)\(\times\)\(10^{-13}\), & \\ RBF support vector machine & gamma & & 2 \(\times\)\(10^{-11},2\)\(\times\)\(10^{-9},2\)\(\times\)\(10^{-7}\), & \\  & & 2 \(\times\)\(10^{-5},2\)\(\times\)\(10^{3},2\)\(\times\)\(10^{-1}\), & 200 & \\  & & 2 \(\times\)\(10^{4},2\)\(\times\)\(10^{31},0.1,0.01\), & 0.001, 1, 10, 50, 100, 200, 500 & \\ \hline  & C & 2 \(\times\)\(10^{-1},2\)\(\times\)\(10^{1},2\)\(\times\)\(10^{3}\), & 0.01 & \\  & & 2 \(\times\)\(10^{5},0.1,0.01\), 1, 10, 100 & & \\ \hline  & Decision\_function\_shape & [O-vs-O, O-vs-R] & O-vs-R & \\ \hline  & Class\_weight & Uniform, balanced & uniform & \\ \hline Logistic regression & penalty & [2; 6, 8, 12, 15, 16] & 12 & \\ \hline  & solver & [’newton-cg’, ‘lbfgs’, ‘liblinear’, ‘sag’, ‘saga’, ‘saga’] & ‘newton-cg’ & ‘saga’ and ‘sag’ will be used for a larger dataset \\ \hline \hline \end{tabular}
\end{table}
Table 4: Parameters Tuning of the Selected Classifiers for the 11 Datasets.  
considered for the multi-class classification problem, and the '\(leaf_{size}\)' parameter is a key indicator for the speedy construction, query and memory requirement to store the tree.