## 3 Experiments and Results
### Study of the Applied Classifiers
#### 3.1.5 \(k\)-NN Classifier  
Neighbors-based classification is a type of instance-based learning, which does not construct a general internal model, but simply stores instances of the training data [37]. Classification is computed from a majority vote of the nearest neighbors of each point. A query point is assigned the data class, which has the most representatives within the nearest neighbors of the particular point [37]. \(k\)-nearest neighbors (\(k\)-NN) algorithm is the most common technique to implement nearest neighbors classification in where \(k\) is an integer value. However, the optimal choice of the value of \(k\) is highly data-dependent [37]. Various metrics are evaluated to determine the distance to categorize query points in a training dataset [40; 41; 42]. As an instance, a \(gx\)-by-\(h\) data sample \(X\), which can be treated as \(gx\) (1-by-\(h\)) row vectors and a \(gy\)-by-\(h\) data sample \(Y\), which can be treated as \(gy\) (1-by-\(h\)) row vectors; different distance metrics between \(x_{m}\) and \(y_{n}\) are defined as follows. * Euclidean Distance \[R_{eu}^{2}=(x_{m}-y_{n})(x_{m}-y_{n})^{\prime}\] (17)
* Standardized Euclidean Distance \[R_{seu}^{2}=(x_{m}-y_{n})V^{-1}(x_{m}-y_{n})^{\prime}\] (18) here \(V\) is the \(h\)-by-\(h\) diagonal matrix. * Mahalanobis Distance \[R_{ma}^{2}=(x_{m}-y_{n})C^{-1}(x_{m}-y_{n})^{\prime}\] (19) here \(C\) is the covariance matrix. * City Block Distance \[R_{ct}=\sum_{k=1}^{h}|x_{mk}-y_{nk}|\] (20)
* Minkowski Distance \[R_{mi}=\sqrt[p]{\sum_{k=1}^{h}|x_{mk}-y_{nk}|^{p}}\] (21)
* Chebychev Distance \[R_{ch}=max_{k}|x_{mk}-y_{nk}|\] (22)* Cosine Distance \[R_{cos}=1-\frac{x_{m}y_{n}^{\prime}}{\sqrt{(x_{m}x_{m}^{\prime})(y_{n}y_{n}^{ \prime})}}\] (23)
* Correlation Distance \[R_{cor}=1-\frac{(x_{m}-\mu_{x})(y_{n}-\mu_{y})^{\prime}}{\sqrt{(x_{m}-\mu_{x})( x_{m}-\mu_{x})^{\prime}}\sqrt{(y_{n}-\mu_{y})(y_{n}-\mu_{y})^{\prime}}}\] (24) here \(\mu_{x}=\frac{1}{h}\sum\limits_{k}x_{mk}\) and \(\mu_{y}=\frac{1}{h}\sum\limits_{k}y_{nk}\). * Hamming Distance \[R_{ham}=\#(x_{mk}\
eq y_{nk})/h\] (25)
* Jaccard Distance \[R_{jac}=\frac{\#[(x_{mk}\
eq y_{nk})\cap((x_{mk}\
eq 0)\cup(y_{nk}\
eq 0))]}{ \#[(x_{mk}\
eq 0)\cup(y_{nk}\
eq 0)]}\] (26)
* Spearman Distance \[R_{seu}^{2}=(x_{m}-y_{n})V^{-1}(x_{m}-y_{n})^{\prime}\]0 (27) here \(r_{m}\) and \(r_{n}\) are the coordinate-wise rank vectors of \(x_{m}\) and \(y_{n}\), respectively. \(\mu_{rm}=\frac{1}{h}\sum\limits_{k}r_{mk}\) and \(\mu_{m}=\frac{1}{h}\sum\limits_{k}r_{nk}\). Exhaustive search algorithm is applied to find \(k\)-nearest neighbors if the following criteria are satisfied. * Number of columns in \(X\) is more than 10. * \(X\) is sparse. * The distance matrix is any of--\(R_{eu}\), \(R_{ma}\), \(R_{cos}\), \(R_{cor}\), \(R_{sp}\), \(R_{ham}\) and \(R_{jac}\). _kd_-tree search algorithm is applied to find \(k\)-nearest neighbors if the following criteria are satisfied. * Number of columns in \(X\) is less than 10. * \(X\) is not sparse. * The distance matrix is any of--\(R_{eu}\), \(R_{ct}\), \(R_{mi}\) and \(R_{ch}\). The detailed analysis of \(k\)-NN and other nearest neighbors algorithms are reported in [40; 41; 42]. In general, classification model parameters are determined by fitting obtained data to the predicted model response. The order is an index representing the flexibility of a model- higher the order, more flexible the model would be. With an increased order, the propensity of fitting the observed data more accurately increases. On the other hand, flexibility contradicts with the certainty in the estimates and this uncertainty is measured by a higher value of variance error. However, a model with low order yields to greater system computational errors, referred to as bias errors. Therefore, it is required to maintain a tradeoff between bias error and variance error. In this research work, MATLAB(r) library functions are used to implement the supervised learning algorithms, which optimize bias-variance tradeoff characterization and avoid overfitting of the training dataset.