## 2 Preliminaries  
### Principal Component Analysis  
Data dimensionality reduction techniques are divided into feature selection and feature extraction techniques. Feature selection techniques, such as random forest or grid search algorithms, select a subset of the original features in order to reduce the complexity and computational efficiency of the model. Conversely, feature extraction techniques extract information from the original features set and create a new features subspace. While feature selection techniques aim to select the most significant features, discarding the less significant ones from the set of original features, feature extraction techniques construct a new reduced set of features, starting from the existing ones, able to synthesize most of the information contained in the original set of features. The use of feature selection techniques is preferable when the explainability of the model and the semantic meaning of the features are required; feature extraction techniques are used to reduce the model complexity, improving its predictive performance. PCA is one of the most used feature extraction techniques in data analysis. Its strong point is to be able to reduce the dimensionality of the data, while preserving their information content. Let D be the original dataset with s features \(\mathrm{X_{1}},\ldots,\mathrm{X_{s}}\) and N instances. The \(\mathrm{i}\)_th_ instance is characterized by a vector (\(\mathrm{x_{i1}}\), \(\mathrm{x_{i2}}\), \(\ldots,\mathrm{x_{is}}\))T where \(\mathrm{x_{ij}}\) is the value of the \(\mathrm{i}\)_th_ instance in correspondence to the \(\mathrm{i}\)_th_ feature. Let \(\mathrm{m_{j}}\) be the mean value of the \(\mathrm{i}\)_th_ feature, given by:  
\[\mathrm{m_{j}}=\frac{1}{\mathrm{N}}\sum_{i=1}^{\mathrm{N}}\mathrm{x_{ij}} \mathrm{j}=1,\ldots,\mathrm{s} \tag{1}\]  
and \(\mathrm{st_{j}}\) the standard deviation is given by:  
\[\mathrm{st_{j}}=\sqrt{\frac{1}{\mathrm{N}}\sum_{i=1}^{\mathrm{N}}\left(\mathrm{ z_{ij}}-\mathrm{m_{j}}\right)^{2}}\mathrm{j}=1,\ldots,\mathrm{s} \tag{2}\]  
To remember this definition, we can break it down into eight steps:  
1. The aim of this phase is to standardize the range of the initial variables so that each one of them contributes equally to the analysis. For each instance \(\mathrm{x_{ij}}\) its normalized value is computed, given by: \[\mathrm{z_{ij}}=\frac{\mathrm{x_{ij}}-\mathrm{m_{j}}}{\mathrm{s_{j}}}\mathrm{ i}=1,\ldots,\mathrm{N}\mathrm{\hskip 28.452756 pt}\mathrm{j}=1,\ldots, \mathrm{s}\] (3)
2. The relationships among features are analyzed by computing the symmetric covariance matrix \(\mathrm{C}=Z^{\mathrm{T}}Z\), where \(Z^{\mathrm{T}}\) is the transpose of the normalized matrix \(Z\). The components of \(\mathrm{C}\) are given by \[\mathrm{C_{jk}}=\frac{1}{\mathrm{N}-1}\sum_{i=1}^{\mathrm{N}}\mathrm{z_{ji}} \mathrm{z_{ik}}\mathrm{j},\mathrm{\hskip 28.452756 pt}\mathrm{j},\mathrm{ \hskip 28.452756 pt}\mathrm{k}=1,\ldots,\mathrm{s}\] (4)
3. In this phase, the s eigenvalues and the s eigenvectors of the covariance matrix are extracted. The eigendecomposition of \(\mathrm{C}\) is where we decompose \(\mathrm{C}\) into \(\mathrm{VDV}^{-1}\), where \(\mathrm{V}\) is the matrix of eigenvectors and \(\mathrm{D}\) is a diagonal matrix in which the diagonal components are the eigenvalues \(\lambda_{\mathrm{i}}\) i =1,...,s and the other elements are equal to 0. 4. The eigenvalues on the diagonal of \(\mathrm{D}\) will be associated with the corresponding column in \(\mathrm{P}\)--that is, the first element of \(\mathrm{D}\) is \(\lambda_{1}\) and the corresponding eigenvector is the first column of \(\mathrm{P}\). This holds for all elements in \(\mathrm{D}\) and their corresponding eigenvectors in \(\mathrm{P}\). We will always be able to calculate \(\mathrm{PDP}^{-1}\) in this fashion. 5.