Feature Variation Explanation. Employing PCA in Scikit-Learn | by Maryam Kargar | Towards Data Science

As a multivariate ordination technique, principal component analysis (PCA) can be carried out on dependent variables in a multivariate dataset to explore relationships between them. This results in displaying the relative positions of data points in fewer dimensions while retaining as much information as possible [2].
In this article, I will apply PCA on the auto-mpg dataset (derived from GitHub) using Scikit-learn library. The article is divided into three sections of data preprocessing, PCA and principal component visualization.
We first need to import the required libraries for the dataset preprocessing:
import tensorflow as tffrom tensorflow import kerasimport numpy as npimport pandas as pdimport seaborn as sns!pip install -q git+https://github.com/tensorflow/docsimport tensorflow_docs as tfdocsimport tensorflow_docs.plotsimport tensorflow_docs.modeling
We can then import the data:
datapath = keras.utils.get_file(“auto-mpg.data”, “http://archive.ics.uci.edu/ml/machine-learning-databases/auto-mpg/auto-mpg.data")datapath
and change the title of the columns:
columnTitles = [‘MPG’,’Cylinders’,’Displacement’,’Horsepower’,’Weight’, ‘Acceleration’, ‘Model Year’, ‘Origin’]rawData = pd.read_csv(dataPath, names=columnTitles, na_values = “?”, comment=’\t’, sep=” “, skipinitialspace=True)
Here is how the data looks:
data = rawData.copy()data.head()
We then need to clean the data:
data.isna().sum()
data = data.dropna()
The next step will be defining the features, separating them out from the response variable and standardizing them as the input for PCA:
from sklearn.preprocessing import StandardScalerdf = pd.DataFrame(data)features = [‘Cylinders’, ‘Displacement’, ‘Horsepower’, ‘Weight’, ‘Acceleration’]X = df.loc[:, features].valuesy = df.loc[:,[‘MPG’]].valuesX = StandardScaler().fit_transform(X)
In PCA, we first need to know how many components are required to explain at least 90% of our feature variation:
from sklearn.decomposition import PCApca = PCA().fit(X)plt.plot(np.cumsum(pca.explained_variance_ratio_))plt.xlabel(‘number of components’)plt.ylabel(‘cumulative explained variance’)
In this case study, two components were chosen as the optimum number of components. We can then start to conduct PCA:
from sklearn.decomposition import PCApca = PCA(n_components=2)principalComponents = pca.fit_transform(X)
As indicated below, in total, the two components explained around 95% of the feature variation of the dataset:
pca.explained_variance_ratio_array([0.81437196, 0.13877225])
Given the response variable value (MPG in the current dataset), the two principal components can be visualized as below :
plt.scatter(principalComponents[:, 0], principalComponents[:, 1],c=data.MPG, edgecolor=’none’, alpha=0.5,cmap=plt.cm.get_cmap(‘Spectral’, 10))plt.xlabel(‘Principal component 1’)plt.ylabel(‘Pricipal component 2’)plt.colorbar()
Principal component analysis (PCA) reduces the dimensionality of large datasets, increases the data interpretability while minimizes the information loss. It does so by creating new uncorrelated components that successively maximize variance [3].
[1]https://commons.wikimedia.org/wiki/File:PCA_vs_Linear_Autoencoder.png
[2] C. Syms. Principal Components Analysis. (2008), Encyclopedia Of Ecology, 2940–2949. doi: 10.1016/b978–008045405–4.00538–3.
[3] T. I. Jolliffe, and J. Cadima. “Principal Component Analysis: a Review and Recent Developments.” (2016), Philosophical Transactions of the Royal Society A: Mathematical, Physical and Engineering Sciences, vol. 374, no. 2065, p. 2015–2020, doi:10.1098/rsta.2015.0202.