# Detection of Bovine Mastitis in Raw Milk, Using a Low-Cost NIR Spectrometer and k-NN Algorithm
## 2 Materials and Methods
### Data Partition  
When a machine learning model is developed, its performance has to be evaluated on a different dataset than the one used for training [41]. Otherwise, it will not be possible to measure the performance of the model on real data. Initially, a single dataset is usually split into two subsets: training and test. The first is used for training the model, while the latter is the one used to evaluate the model with no previously seen data. This common schema becomes nearly impossible in a situation where there are few captured data due to the expensiveness of the measures. In order to minimize the impact of this division and maximize the reliability of the results, several techniques have been developed over the years, such as the so-called leave-one-out technique [42]. In this technique, a test is carried out for each of the samples in the dataset. For example, in a dataset of n samples, n tests have to be carried out. For each one of these n tests, one sample is taken out of the data set as a test, while the remaining n \(-\) 1 samples will be used as a training set for the model. Once the training is completed, the resulting model is tested with the test set. This training-test procedure is repeated as many times as the number of samples in the dataset [43], which maximizes the samples used in the training while the model with unseen data is evaluated.  
Something that may be highlighted is the fact that, in this particular case, one example was collected from each cow over each quarter. Even though some experts argue that the mastitis degree is independent of the quarter, a partition of the data based on the cow instead of the sample was used. This approach was called leave-one-cow-out because all samples from each cow were used as a test dataset.  
Additionally, using the same dataset not only to measure the performance of the model but to choose the values for the hyperparameters could result in an overfitted model depending on the size of the dataset [44]. To deal with this situation, a technique known as nested cross-validation was used. This technique generates a series of sub-set divisions for training, validation, and testing that go on to be used by a pair of nested loops. The inner loop is aimed at selecting the best hyperparameters of the trained models based on the best result with the validation set. On the other hand, the outer loop is responsible for estimating the generalization of the model. To do that, the outer loop calculates the average and the standard deviation of the results obtained using totally independent samples of milk (test set) after the hyperparameters have been selected.  
In this work, we combined both ideas: nested cross-validation and leave-one-cow-out. The result was an approach we named as nested-leave-one-cow-out cross-validation, for which the general schema of the approach is shown in Figure 1. According to this schema, each iteration of the outer loop reserves 7 cows for testing; the remaining 60 cows are passed to the inner loop to perform a leave-cow-out cross-validation. The result is that the outer loop will have a value for the test for each one of the models developed in each iteration of the inner loop.