# Getting over High-Dimensionality: How Multidimensional Projection Methods Can Assist Data Science
## 4 Multidimensional Projection Approaches and Domains
### Principal Component Analysis  
Principal Component Analysis (PCA) was introduced in the earlier 1900s. It was probably the first dimensionality reduction technique and has been well-known and used [47]. PCA maps data from a high dimensional space to a low one by encountering orthogonal linear combinations that better represent original data variability. These combinations are named principal components. The process is performed so that the first component is related to the highest data variance direction; the second component is related to the second higher variance, orthogonal to the previous component, and so on.  
One can obtain the components \(a_{1},a_{2},\ldots,a_{p}\) by calculating the covariance of the \(m\) attributes of a dataset \(X\) (Section 3.3). Being \(x_{i}\) and \(x_{j}\) two attributes of \(X\) and \(\tilde{x_{i}}\), and \(\tilde{x_{j}}\) their means, the covariance between them is defined as  
\[cov(x_{i},x_{j})=\frac{1}{n-1}\sum_{k=1}^{n}(x_{ik}-\tilde{x_{i}})(x_{jk}- \tilde{x_{j}}) \tag{8}\]  
With this formulation, it is possible to build a square matrix \(C_{m\times m}\) with each position representing the covariance between each data attribute pair. Following, spectral decomposition can be applied to encounter eigenvectors and eigenvalues by writing \(C\) as:  
\[C=U\Lambda U^{T}, \tag{9}\]  
such that \(\Lambda=diag(\lambda_{1},\lambda_{2},\ldots,\lambda_{m})\), with \(\lambda_{1}\geq\lambda_{2}\geq\cdots\geq\lambda_{m}\), is a diagonal matrix with the eigenvalues of \(C\) and \(U_{m\times m}\) is the orthogonal matrix with its eigenvectors. Finally, the principal components that represent the \(p\)-dimensional space are generated by the application of eigenvectors in the data matrix:  
\[A=X\times[u_{1},u_{2},\ldots,u_{p}], \tag{10}\]  
with each \(u_{i}\) being the columns of the \(U\) matrix.  
It is possible to obtain the number of principal components equal to the number of original data dimensions. However, the number \(p\) of dimensions should be sufficiently large to represent the initial space without significant information loss. It is important to observe that the first components generated by PCA can absorb the more significant part of data variance. This characteristic makes PCA a suitable approach to identifying data trends and patterns [48], beyond weeding out part of data noise and catching data variability in a few dimensions.  
Due to numerical and practical questions, PCA is implemented by Singular Value Decomposition (SVD), which can be similarly interpreted as the eigendecomposition aforementioned. The process centers on \(X\) columns, subtracting each value by the respective column mean. SVD decomposes the centered matrix with  
\[B=USV^{T}, \tag{11}\]  
being \(U\) columns left singular vectors, \(V\) columns right singular vectors, and \(S\) a diagonal matrix with the singular values. Then, the \(p\)-dimensional space is created with \(A=US\)[49]. In this case, covariance was not calculated, but the centralized data force the meaning of singular vectors and values to be similar to eigendecomposition results.  
As a projection technique, PCA preserves more global structures than the local ones, owing to the preservation of the large variances that appear in orthogonal principal axes [7; 15]. Nevertheless, it is not capable of well-representing datasets formed by nonlinear relations among attributes [31]. Besides, PCA is unsuitable for representing data with several groups that have distinct variances since, for visualization purposes, just 2 or 3 first components are used [31].