# Classification of Retail Products: From Probabilistic Ranking to Neural Networks
## 6 Nearest-Neighbors Model
#### Optimal \(k\) and Distance for KNN and FKNN  
The key to success in KNN and FKNN are finding the optimal value of \(k\) and the distance function. Usually, the \(k\) parameter is chosen empirically depending on each problem, different numbers of nearest neighbors are tested, and the parameter with the best performance (accuracy) is chosen to define the classifier. Choosing the optimal \(k\) is almost impossible for a variety of problems since the performance of a KNN classifier varies significantly when changing \(k\) and the change of the _distance_ metric used [49].  
Therefore, the decision about the _distance_ measure also plays a vital role in determining the final result of the classification. Although Euclidean distance is the most widely used distance metric in NN-based classifiers, we have applied a variety of _distance_ metrics in MDD-DS. We call the \"best distance metric\" the metric which allows KNN, FKNN to classify test samples with the highest accuracy, recall, precision, and F-score. The definition of the distances considered in this paper (Spearman, Cosine, Correlation, Euclidean, Cityblock, Chebychev, Hamming, Jaccard, and Seuclidean) are described in Table 11, included as an Appendix A.  
To determine the optimal value of \(k\) and _distance_ for Nearest-Neighbors Models KNN and FKNN, we deploy the pseudocode in Algorithm 2 to try different combinations in two loops (odd \(k\) values initialized from \([1\)_to_\(30]\) and the different _distances_ used) over the reduced dimension MDD-DS for 600 principal components. This value was selected taking into account that, with 600 principal components, we obtain a value around 85% of retained variance, as a criteria to choose the appropriate number of principle components.  
In addition, we have perceived that the results do not significantly improve if the number of principal components increase. Since we pursue obtaining the optimal pair of \(k\) and _distance_ not only for classification per se (_Top_1_) but also for _Top_2_ and _Top_3_, the pseudocode obtains the accuracy for all the combinations, as it is shown in Figure 3.  
The optimal type of the _distance_ is Spearman of three approaches of KNN and FKNN. Table 7 shows that the optimal value of \(k\) and accuracy result for each approach according to Figure 3.  
\begin{table}
\begin{tabular}{c c c c c} \hline \hline  & \multicolumn{2}{c}{**KNN**} & \multicolumn{2}{c}{**FKNN**} \\  & \(k\) & _Accuracy_ & \(k\) & _Accuracy_ \\ \hline _Top_1_ & 1 & 0.8367 & 1 & 0.8446 \\ _Top_2_ & 3 & 0.8874 & 13 & 0.9153 \\ _Top_3_ & 7 & 0.9141 & 23 & 0.9413 \\ \hline \hline \end{tabular}
\end{table}
Table 7: Optimal parameters of _KNN, FKNN_ for each approach.  
Figure 3: Accuracy of KNN and FKNN for the three approaches: _Top_1_, _Top_2_ and _Top_3_.  
```
1:procedureTuning_Parameters(Train_PCA600, Test_PCA600)
2:\(Dist[]\) = ['Spearman', 'Cosine', 'Correlation', Euclidean', 'Cityblock', 'Chebychev', 'Hamming', 'Jaccard', 'Seuclidean']
3: Label C1 \(\triangleright\) First, Variety
4: Label C2 \(\triangleright\) Second Variety
5: Label C3 \(\triangleright\) Third Variety
6: i \(\gets\) 1
7:forK \(\gets\) 1 : 30 by 2 do
8:forJ \(\gets\) 1 : 9 do
9:\(M\)\(\leftarrow\) build_model(Train_PCA600, \(k\), \(Dist[J]\))
10:\([\)C1,C2,C3\(]\)\(\leftarrow\) Predict(\(M\), Test_PCA600)
11:endfor
12:\(i\gets i+1\)
13:endfor
14:\(Top\_1\)\(\leftarrow\) C1
15:\(Top\_2\)\(\leftarrow\) C1 & C2
16:\(Top\_3\)\(\leftarrow\) C1 & C2 & C3
17:returnTop_1,Top_2,Top_3
18:endprocedure
```  
**Algorithm 2** Tuning parameter for KNN and FKNN