Principal Component Analysis with Python - GeeksforGeeks

10 Mar, 2022
Principal Component Analysis is basically a statistical procedure to convert a set of observations of possibly correlated variables into a set of values of linearly uncorrelated variables. Each of the principal components is chosen in such a way so that it would describe most of them still available variance and all these principal components are orthogonal to each other. In all principal components first principal component has a maximum variance. 
Uses of PCA: 
It is used to find inter-relation between variables in the data.
It is used to interpret and visualize data.
The number of variables is decreasing it makes further analysis simpler.
It’s often used to visualize genetic distance and relatedness between populations.
These are basically performed on a square symmetric matrix. It can be a pure sums of squares and cross-products matrix or Covariance matrix or Correlation matrix. A correlation matrix is used if the individual variance differs much. 
Objectives of PCA: 
It is basically a non-dependent procedure in which it reduces attribute space from a large number of variables to a smaller number of factors.
PCA is basically a dimension reduction process but there is no guarantee that the dimension is interpretable.
The main task in this PCA is to select a subset of variables from a larger set, based on which original variables have the highest correlation with the principal amount.
Principal Axis Method: PCA basically searches a linear combination of variables so that we can extract maximum variance from the variables. Once this process completes it removes it and searches for another linear combination that gives an explanation about the maximum proportion of remaining variance which basically leads to orthogonal factors. In this method, we analyze total variance.
Eigenvector: It is a non-zero vector that stays parallel after matrix multiplication. Let’s suppose x is an eigenvector of dimension r of matrix M with dimension r*r if Mx and x are parallel. Then we need to solve Mx=Ax where both x and A are unknown to get eigenvector and eigenvalues. Under Eigen-Vectors we can say that Principal components show both common and unique variance of the variable. Basically, it is variance focused approach seeking to reproduce total variance and correlation with all components. The principal components are basically the linear combinations of the original variables weighted by their contribution to explain the variance in a particular orthogonal dimension.
Eigen Values: It is basically known as characteristic roots. It basically measures the variance in all variables which is accounted for by that factor. The ratio of eigenvalues is the ratio of explanatory importance of the factors with respect to the variables. If the factor is low then it is contributing less to the explanation of variables. In simple words, it measures the amount of variance in the total given database accounted by the factor. We can calculate the factor’s eigenvalue as the sum of its squared factor loading for all the variables. 
Now, Let’s understand Principal Component Analysis with Python.To get the dataset used in the implementation, click here.Step 1: Importing the libraries 
Python
# importing required librariesimport numpy as npimport matplotlib.pyplot as pltimport pandas as pd
 Step 2: Importing the data setImport the dataset and distributing the dataset into X and y components for data analysis.
Python
# importing or loading the datasetdataset = pd.read_csv('wine.csv') # distributing the dataset into two components X and YX = dataset.iloc[:, 0:13].valuesy = dataset.iloc[:, 13].values
 Step 3: Splitting the dataset into the Training set and Test set 
Python
# Splitting the X and Y into the# Training set and Testing setfrom sklearn.model_selection import train_test_split X_train, X_test, y_train, y_test = train_test_split(X, y, test_size = 0.2, random_state = 0)
 Step 4: Feature ScalingDoing the pre-processing part on training and testing set such as fitting the Standard scale.
Python
# performing preprocessing partfrom sklearn.preprocessing import StandardScalersc = StandardScaler() X_train = sc.fit_transform(X_train)X_test = sc.transform(X_test)
 Step 5: Applying PCA functionApplying the PCA function into the training and testing set for analysis. 
Python
# Applying PCA function on training# and testing set of X componentfrom sklearn.decomposition import PCA pca = PCA(n_components = 2) X_train = pca.fit_transform(X_train)X_test = pca.transform(X_test) explained_variance = pca.explained_variance_ratio_
 Step 6: Fitting Logistic Regression To the training set
Python
# Fitting Logistic Regression To the training setfrom sklearn.linear_model import LogisticRegression  classifier = LogisticRegression(random_state = 0)classifier.fit(X_train, y_train)
 Step 7: Predicting the test set result 
Python
# Predicting the test set result using# predict function under LogisticRegressiony_pred = classifier.predict(X_test)
 Step 8: Making the confusion matrix 
Python
# making confusion matrix between#  test set of Y and predicted value.from sklearn.metrics import confusion_matrix cm = confusion_matrix(y_test, y_pred)
 Step 9: Predicting the training set result
Python
# Predicting the training set# result through scatter plotfrom matplotlib.colors import ListedColormap X_set, y_set = X_train, y_trainX1, X2 = np.meshgrid(np.arange(start = X_set[:, 0].min() - 1,                     stop = X_set[:, 0].max() + 1, step = 0.01),                     np.arange(start = X_set[:, 1].min() - 1,                     stop = X_set[:, 1].max() + 1, step = 0.01)) plt.contourf(X1, X2, classifier.predict(np.array([X1.ravel(),             X2.ravel()]).T).reshape(X1.shape), alpha = 0.75,             cmap = ListedColormap(('yellow', 'white', 'aquamarine'))) plt.xlim(X1.min(), X1.max())plt.ylim(X2.min(), X2.max()) for i, j in enumerate(np.unique(y_set)):    plt.scatter(X_set[y_set == j, 0], X_set[y_set == j, 1],                c = ListedColormap(('red', 'green', 'blue'))(i), label = j) plt.title('Logistic Regression (Training set)')plt.xlabel('PC1') # for Xlabelplt.ylabel('PC2') # for Ylabelplt.legend() # to show legend # show scatter plotplt.show()
  Step 10: Visualizing the Test set results
Python
# Visualising the Test set results through scatter plotfrom matplotlib.colors import ListedColormap X_set, y_set = X_test, y_test X1, X2 = np.meshgrid(np.arange(start = X_set[:, 0].min() - 1,                     stop = X_set[:, 0].max() + 1, step = 0.01),                     np.arange(start = X_set[:, 1].min() - 1,                     stop = X_set[:, 1].max() + 1, step = 0.01)) plt.contourf(X1, X2, classifier.predict(np.array([X1.ravel(),             X2.ravel()]).T).reshape(X1.shape), alpha = 0.75,             cmap = ListedColormap(('yellow', 'white', 'aquamarine'))) plt.xlim(X1.min(), X1.max())plt.ylim(X2.min(), X2.max()) for i, j in enumerate(np.unique(y_set)):    plt.scatter(X_set[y_set == j, 0], X_set[y_set == j, 1],                c = ListedColormap(('red', 'green', 'blue'))(i), label = j) # title for scatter plotplt.title('Logistic Regression (Test set)')plt.xlabel('PC1') # for Xlabelplt.ylabel('PC2') # for Ylabelplt.legend() # show scatter plotplt.show()
surindertarika1234
as5853535
23603vaibhav2021
shashwatsonu21
Picked
Advanced Computer Subject
Machine Learning
Python
Machine Learning
Writing code in comment?
Please use ide.geeksforgeeks.org,
generate link and share the link here.
Comments
Old Comments
Decision Tree
Copying Files to and from Docker Containers
System Design Tutorial
Python | Decision tree implementation
Decision Tree Introduction with example
Agents in Artificial Intelligence
Decision Tree
Python | Decision tree implementation
Search Algorithms in AI
Difference between Informed and Uninformed Search in AI