2-PCA vs 3-PCA. Does Having 3 Principal Components... | by Arun Prasad | Towards Data Science

Principal Component Analysis is used to reduce the number of dimensions (features) in a dataset. Say, we have 10 dimensions/features in our dataset. It's not a smart way to just drop 7–8 features based on an assumption (or a wild guess). One method to tackle this problem could be a feature selection technique (say, Chi-Squared). Feature Selection Techniques help us select the most important ‘k’ features corresponding to the target feature. But again, this will not be an overall representation of all the features (instead, it would be just a few features selected from ‘n’ number of features).
To tackle this problem, we have the Principal Component Analysis (PCA) Dimensionality reduction technique. The principal components we shall have would be an overall representation of all the features in the dataset. In this way, information loss can be minimised.
Now that we have understood the importance of PCA, we shall dive deeper to check if having 3 principal components and plotting them in a 3-Dimensional graph is more fruitful than having a 2 component PCA and a 2-Dimensional graph?
To keep things simple, I am not showing the dataset I have used, instead directly showing the three principal components which I obtained by using the PCA technique (with n=3) on my dataset.
The Principal Components are as follows       Principal component 1  Principal component 2  Principal component 30                  -0.593113               0.877027               1.7824261                  -0.582788               0.926408               0.2358752                  -0.593406               0.885406               1.0263933                  -0.590781               0.913825               1.0575994                  -0.557457               1.003013               0.7719945                  -0.590091               0.892999               1.7547616                  -0.532054               1.071892               0.6143117                  -0.600141               0.873119               1.3740948                  -0.604967               0.862305               1.6899379                  -0.590725               0.910980               1.15160010                 -0.571536               0.955599               1.26903511                 -0.612102               0.921839               0.60181012                 -0.638983               1.028594              -0.65870113                 -0.538442               1.069557              -0.31585814                 -0.545885               1.042621              -0.43917715                 -0.213216              -1.068956               1.62484716                 -0.585643               0.958585               0.56633817                 -0.528046               1.081252               0.81483618                 -0.586254               1.015987               0.56042719                 -0.588309               0.893676               1.74815520                 -0.545498               1.042484               0.04420721                 -0.525951               1.099272              -0.23981522                 -0.545391               1.036316               0.24982923                 -0.553054               1.015097               1.36101524                 -0.549921               1.043496              -0.02877225                 -0.541706               1.044087               0.50017326                 -0.580348               0.927584               0.69396127                 -0.565069               0.983091              -0.06093128                 -0.569777               0.958700               0.70642729                 -0.572816               0.956895               0.263877...                      ...                    ...                    ...38991               4.154215               0.411276               2.07402338992              -0.405145              -0.943906              -0.97919938993              -0.294493              -0.724124               0.18259038994               0.946546              -0.426986              -2.31542738995               3.755563               0.385886              -2.34363838996               1.362237              -0.325860               0.13543638997               3.085934               0.127931               0.23375838998               5.803189               0.739613              -0.59875038999               4.126051               0.320192              -1.14289239000               5.285017               0.732141              -2.11322839001               4.235777               0.488489              -2.42217439002               5.337110               0.681343              -1.86533939003               2.046786              -0.255520               0.51248539004               6.575937               1.052580               0.41412839005              -0.549162              -0.773501              -0.45389939006              -0.489873              -0.747580              -2.54333239007              -0.403737              -0.943530              -0.97913339008               5.632072               0.761396              -1.41929839009               1.532861              -0.252514              -1.25933139010               0.283742              -0.684903              -0.72987839011              13.207580               2.725922               0.23752339012              -0.558653              -0.799741              -0.61854639013              -0.734531              -0.815442               0.75588239014              -0.563215              -0.746943              -1.34988139015              -0.600636              -0.858194              -1.31946439016               1.536320              -0.299190              -0.16450239017              -0.553627              -0.785324              -0.57675939018              -0.554424              -0.957772              -0.89779539019              -0.744299              -0.842957               0.63108839020              -0.558107              -0.957167              -1.901427
You can see that I have 39020 tuples. Let us also look at the variance ratios for the three Principal Components.
The Variance Ratios are[0.24679625 0.16479195 0.15822251]
Let us imagine that you are used to only 2-Component PCA and you haven’t got the 3rd Component in your analysis. Which now means in our scenario, you have only the first two principal components. Okay! Let us plot the first two principal components.
Consider finding clusters within the dataset is our primary goal, then we can clearly see from the above plot (Figure 1) that there are no obvious clusters that are occurring. Which means, 2-Component PCA has now failed to fulfil our expectations here.
Okay, can the Three-Component PCA fulfil it? Why doubt it, let us check it out by plotting all the three components now.
But again I am not able to find any clusters as such in the plot. What we shall do now is try to rotate the plots along all the three axes and try to find some insight in them.
Voila! From the above plots (Figure 3), I am able to come to the assertion that there are two possible clusters within my dataset. Suppose that my dataset is a survey of school students and primarily talks about them using laptops and mobile phones, then clearly I can see that there are two separate groups of students and we can now proceed to further analysis with this assertion that one large group of students and one small group of students are present in the dataset and the former or the later is more comfortable with either laptops or mobile phones.
Okay, why could the Three-component PCA give us more insights than that of the Two-Component PCA? Check out the Variance Ratios which we had obtained. When considering only the first two components, the ratios add up to ~0.4 (40%). Meaning, the first two components together only represent 40% of the overall data. But when we add up all the three ratios, it is around ~0.55 (55%). A visibly higher percentage overall representation of the dataset features.
To conclude, when the overall sum of the two variance ratios is extremely low and the plot doesn’t give us any insight, then using a 3rd Principal Component is mostly very fruitful. When even 3-PCA doesn’t give us insights then considering other Dimensionality reduction techniques is a better option. In one another analysis of mine, using only the very first principal component (0.99 variance ratio), meaningful insights could be obtained from the plot. (Link: https://towardsdatascience.com/an-insight-into-performing-fundamental-data-analysis-9c67c62e0766 ). Hence the variance ratios do play a very vital role in determining the number of principal components which we shall use for plotting.
Thanks for your time! If you liked the story, do share it with your friends.
You can follow me on LinkedIn. Thank you once again!