# Lean Manufacturing Soft Sensors for Automotive Industries
## 2 Methodology
### Soft Sensor Modeling
#### 2.2.2 K-Nearest Neighbours (KNN)  
The KNN algorithm works on the premise \"birds of a feather flock together\". Translated in terms of a dataset, this premise suggests that data points having similar attributes are located closer to each other as compared to data points having dissimilar characteristics. KNN classifies a query based on its relative distances from a specified number of neighbouring data points (K) closest to the query. KNN sorts the K-nearest neighbouring data points in the ascending order of their distances from the query point and classifies the query point as per the mode of the K-nearest data point labels. Figure 4 shows a pictorial representation of KNN classification.  
The classification accuracy of the KNN algorithm depends on the appropriate selection of the K value as per the dataset in question. Increasing the K value improves prediction accuracy to a certain extent due to averaging/majority voting. Beyond a certain limit, the  
\begin{table}
\begin{tabular}{l l l l l l} \hline \hline
**Model Hyperparameters** & **Narrow ANN** & **Medium ANN** & **Wide ANN** & **Bilayered ANN** & **Trilayered ANN** \\ \hline Preset Neural Network & Narrow & Medium & Wide & Bilayered & Trilayered \\ Number of fully connected layers & 1 & 1 & 1 & 2 & 3 \\ Layer Size & 10 & 25 & 100 & 10, 10 & 10, 10, 10 \\ Activation Function & ReLU & ReLU & ReLU & ReLU & ReLU \\ Iteration Limit & 1000 & 1000 & 1000 & 1000 & 1000 \\ Regularization strength (Lambda) & 0 & 0 & 0 & 0 & 0 \\ Standardized data & Yes & Yes & Yes & Yes & Yes \\ \hline \hline \end{tabular}
\end{table}
Table 4: ANN classification hyperparameters.  
Figure 4: KNN Classification.  
prediction errors start rising because the algorithm starts considering data points from the neighbouring clusters of dissimilar characteristics/labels. In a nutshell, the KNN algorithm methodology is simple, versatile, and easy to implement. It can be used for regression, search, and classification tasks. The main drawback of this algorithm is its inability to quickly solve problems involving large datasets. Table 5 shows the classification hyperparameters of the six KNN architectures explored in the present work. The number of K neighbours is ten in all cases except for the fine and cosine KNNs, which have 1 and 100 neighbours, respectively. The distance weights were kept equal and flexibilities-LM data were standardized for all architectures. The fine, medium, and coarse KNNs employed the Euclidean distance metric given by:  
\[E_{d}(x,y)=\sqrt{\sum_{j=1}^{n}(y_{j}-x_{j})^{2}} \tag{2}\]  
The cosine distance metric used in the cosine KNN is given by:  
\[\cos(\theta)=\frac{\vec{x}\cdot\vec{y}}{\|x\|\|y\|}=\frac{\sum_{i=1}^{n}(x_{i} y_{i})}{\sqrt{\sum_{i=1}^{n}x_{i}^{2}}\sqrt{\sum_{i=1}^{n}y_{i}^{2}}} \tag{3}\]  
The Minkowski distance metric was used in the cubic KNN, expressed as follows:  
\[M_{d}(x,y)=(\sum_{j=1}^{n}|y_{j}-x_{j}|^{p})^{1/p} \tag{4}\]  
The inverse squared distance used in the weighted KNN is calculated as follows:  
\[I_{d}(x,y)=\frac{1}{\sum_{j=1}^{n}|y_{j}-x_{j}|^{2})} \tag{5}\]  
where \(x\) and \(y\) represent inputs and corresponding outputs, respectively. \(\vec{x}\) and \(\vec{y}\) are the respective input and output vectors. \(\theta\) is the angle between the input and output vectors. \(n\) is the total number of samples in the dataset and \(p\) is a real number typically set between 1 and 2.