# NMR in Metabolomics: From Conventional Statistics to Machine Learning and Neural Network Approaches
## 2 Conventional Approaches
### Supervised Methods
#### 2.2.2 Principal Component Regression (PCR) and Partial Least Squares (PLS)  
It is well known that a linear model can be written as \(Y=X\beta+\epsilon\), in which \(Y\) represents the response variable (it can be a single variable or even a matrix), and \(X\) represents the design matrix having variables along its columns and observations along its rows; \(\beta\) corresponds to the coefficients vector (or matrix) and \(\epsilon\) represents the random error vector (or matrix). For a small number of variables and a high number of observations, it is commonly adopted for \(\beta\) the ordinary least square solution (\((X^{T}X)^{-1}\,X^{T}Y\)). In the opposite case, where it is not possible to evaluate the inverse of the singular matrix \((X^{T}X)\), other solutions have to be considered [60]. One of them is the principal component regression (PCR) that makes use of the first PCs achieved by running PCA to fit the linear regression model instead of using all original variables. However, often, there is not a good correlation between these PCs and the response variables \(Y\). Alternatively, the partial least squares (PLS) regression method is more efficient [61]. In the latter case, one has to determine the most suitable number of components to maintain, and then PLS evaluates a linear regression model by employing the projection of predicted and observed variables to a new space according to the following relations:  
\[Y=UQ^{\prime}+F \tag{4}\] \[X=TP^{\prime}+E \tag{5}\]  
where \(T\) and \(U\), analogously to PCA, correspond to \(X\) and \(Y\) scores and are matrices constituted by latent variables; at the same time, \(P^{\prime}\) and \(Q^{\prime}\) correspond to \(X\) and \(Y\) loadings, representing the weight matrices of the linear combinations; \(E\) and \(F\) represent all that is not possible to explain by using latent variables. Each of them, being expressed as a linear combination of \(X\) and \(Y\), can be rewritten in terms of weight factors as \(t=Xw\) and \(u=Yc\), where \(t\) and \(u\) are two latent variables and \(w\) and \(c\) are the corresponding weight vectors. Indeed, PLS evaluates that set of \(X\) variables that is able to explain the majority of the changes in \(Y\) variables. Therefore, PLS, by using orthogonal conditions, evaluates those latent variables \(t\) and \(u\), whose covariance is maximal. Ultimately, there are some substantial differences between the PCA and PCR-PLS approaches. In fact, as already mentioned, PCA pertains to unsupervised methods, whereas PCR and PLS pertain to supervised approaches. Moreover, as already mentioned, PCR takes advantage of the first PCs obtained from the PCA, using them as predictors for fitting the regression of a latent variable. Hence, PCA is able to explain just the \(X\) variance, whereas PLS allows achieving a multi-dimensional route in the \(X\) space, indicating the maximum variance route in the \(Y\) space. In other words, in PCR, the principal components become the new (unrelated) variables of the regression, which thus becomes more easily resolvable. Otherwise, in PLS, the \(Y\) variables are decomposed into principal components too, while those of \(X\) are rotated along the direction of maximum correlation with respect to the principal components of \(Y\). Therefore, the purpose of PLS is to determine latent variables similar to the principal components that maximize the variance of both matrices. We also mention the partial least squares discriminant analysis, or PLS-DA, which is an alternative when the dependent variables are categorical. Discriminant analysis is a classification algorithm which adds the dimension reduction part to it. PLS-DA allows the employment of predictive and descriptive algorithms other than for discriminative variable choice (see Figure 8a). PLS-DA is executed on NMR spectra for different aims, including food authentication and diseases classification in medical diagnostics [62; 63; 64; 65]. However, a more comprehensive variant of PLS is the orthogonal PLS (OPLS) method.