## 3 Results
### Machine Learning Principles
#### 3.1.4 Credibility. Algorithm Evaluation  
Among the most common methods used for analysing the performance of the algorithm by splitting the initial labelled data set are the holdout, the random sampling, the cross-validation, and the bootstrap methods.  
The holdout method splits data into training and testing sets being mutually exclusive subsets. The input provides the training set and the output is tested in the testing set [52]. This procedure provokes that this method needs datasets with a large amount of data. In practice, it is common to use one third of the data for testing and two-thirds for training.  
The random sampling method, also called repeated holdout method, is similar to the holdout method. It repeats the holdout method several times with the end of improving the accuracy of the algorithm [53; 54]. Similarly to the holdout method, large data sets are also required in this method.  
Both holdout and random sampling methods are simple procedures for evaluating the performance of an algorithm, but they raise an important issue. Statistically speaking, the sample size that is required by both method procedures must be as large as possible to obtain a good accuracy of the algorithm. This fact is due if a database is small, it is feasible that the sample selected for training could not be representative of the dataset. However, as we said before, it is not always possible to have a large database [55]. If a data amount is limited and small, other evaluation methods must be applied to analyse the performance of the algorithm.  
One of them is cross-validation method, where a dataset is split into a number of partitions. Each partition is split by holdout method and used for testing and the remainder is used for training. There are different types of cross-validation methods. Some of them are K-fold cross-validation, Leave-one-out (LOO) cross-validation, and Leave-p-out (LPO) cross-validation. In K-fold cross-validation, dataset is split exactly into \(k\) partitions. The cross-validation process is then repeated \(k\) times (the folds). This procedure is repeated as often as the number of partitions. Thus, each partition is used once for testing [56]. In general, \(k\) remains an unfixed parameter, but 10-fold (\(k=10\)) cross-validation is commonly used. In LOO cross-validation, the size of the training set is fixed as \(n-1\), where \(n\) is the database size. In this way, each data point is successively \"left out\" from the sample and used for validation. In LPO cross-validation, the size of the training set is fixed as \(n-p\), with \(p\in\{1,2,\ldots n-1\}\). Thus, every possible subset of \(p\) data is successively \"left out\" of the sample and used for validation [57].  
Generally, the prediction error of cross-validation methods are nearly unbiased, but can be highly variable. Thus, the bootstrap method emerged as smoother version of cross-validation methods [58].  
The previous methods considered a sampling without replacement, i.e., when data were selected from the dataset to form the training or testing sets, it cannot be chosen to form the other set. On the other hand, the bootstrap method is based on the statistical procedure of sampling with replacement. In this method, the idea is to sample the data set with replacement to form the training set. Thus, when an input is chosen to form the training set, it is placed again into the entire data set and it can be selected again [59]. The bootstrap is a quite good procedure for estimating the error for very small data sets. However, the training set can represent a special and artificial situation since an input can be selected an unlimited number of times.  
Once we know the basic principles of ML, DL can be understood. Next Subsection details the basic DL principles.