## 2 Classification Algorithms Theory  
### K-Nearest-Neighbors  
K-Nearest-Neighbors or KNN is a simple classification algorithm first proposed by [PERSON] and [PERSON] [32] in 1951 and further developed by [PERSON] [33] in 1967. This algorithm stores all the input data with its corresponding labels and classifies a new observation based on similarity [34]. This classification is done based on the labels of its nearest neighbors. The KNN algorithm has been used on several Industry 4.0 framework applications, such as cybersecurity [35], aircraft's useful life prediction [36], fault classification [37], nephropathy prediction in children [38], intrusion detection systems [39], etc. In order to understand better how KNN works, let us take a brief example. Data about the height and the weight of men and women were gathered and are presented in Figure 1a. The blue points represent data that were labeled as men and the red as women. Then, suppose that we add a new individual to the dataset. We are told that this new individual is a man; however, we only introduce their height and weight to the algorithm to see if it is capable of classifying them correctly. This is shown in Figure 1b, where the new individual is represented as a star. Figure 1: (**a**) Men and women distribution according to height and weight. (**b**) New observation added to the dataset. To do the classification, we need to search for its nearest neighbors and their corresponding labels so that we can classify this new point depending on the majority of votes from its neighbors. To determine which of the data points are closest to the new observation, we have to measure the distance between data points. There are several types of functions that can be used to obtain the distance between points, such as cosine similarity measure, Minkowsky, correlation, Chi-squared, and Euclidean distance [40]. From all of these, the most widely used function is the Euclidean distance, which is the measure of the straight line between two points and, for two dimensions, is obtained as:  
\[d(x,y)=\sqrt{\sum_{i=1}^{n}(x_{i}-y_{i})^{2}} \tag{1}\]  
where \(d\) represents the distance between two points, and \(x\) and \(y\) represent the point coordinates. Once we obtain the distance between the new observation and the other points, we can choose the closest neighbors to proceed with the classification. However, an important question arises here: how many nearest neighbors should we consider? Let us take Figure 2 as an example. In Figure 2, the new observation is being classified by its \(K\) nearest neighbors, where \(K\) has two different values: \(K=3\) and \(K=7\). For \(K=3\), the closest neighbors are the ones inside the green circle, and for \(K=7\), the closest neighbors are inside the purple circle. It can be seen that, if \(K=3\), two of those three neighbors are women, so the new observation would be classified as a woman. However, if \(K=7\), most of the closest neighbors are men, and then the new individual would be classified as a man. So, for the classification of this new individual, \(K=7\) is better than \(K=3\) because it classified it as a man, which was its real gender. Nevertheless, this does not mean that \(K=7\) is the best overall value for the dataset; more new observations must be classified with different \(K\) values to find the value with the best performance. As we can see, the resulting classification from the KNN algorithm strongly depends on the chosen number of nearest neighbors. Thus, the algorithm must be tested with different values of \(K\) until we find its most optimum value for the dataset that we are working with. This is usually the most time-demanding task when talking about the KNN algorithm. Once the optimum value of \(K\) is found, the training time is usually low, which is one of the main advantages when compared to other classification methods. Furthermore, another advantage of KNN is that it is very simple to implement and interpret, as was seen during the previous example.