Covariance Matrix Visualization Using Seaborn’s Heatmap Plot
By: Benjamin Obi Tayo Ph.D.
Tags: Seaborn, Feature Selection, Data Science, Data Visualization, Machine Learning

Covariance Matrix Visualization Using Seaborn’s Heatmap Plot

Image by Benjamin O. Tayo

I. Introduction

Before implementing a machine learning algorithm, it is necessary to select only relevant features in the training dataset. The process of transforming a dataset in order to select only relevant features necessary for training is called dimensionality reduction. Feature selection and dimensionality reduction are important because of three main reasons:

Prevents Overfitting: A high-dimensional dataset having too many features can sometimes lead to overfitting (model captures both real and random effects). Simplicity: An over-complex model having too many features can be hard to interpret especially when features are correlated with each other. Computational Efficiency: A model trained on a lower-dimensional dataset is computationally efficient (execution of algorithm requires less computational time).

The covariance matrix is a very simple, efficient, and reliable method for feature selection and dimensionality reduction. This tutorial will illustrate how the covariance matrix plot can be created and visualized using the seaborn library.

We will illustrate the process of feature selection and dimensionality reduction with the covariance matrix plot using the cruise ship dataset cruise_ship_info.csv. Suppose we want to build a regression model to predict cruise ship crew size based on the following features: [‘age’, ‘tonnage’, ‘passengers’, ‘length’, ‘cabins’, ‘passenger_density’]. Our model can be expressed as:

where X is the feature matrix, and w the weights to be learned during training. The question we would like to address is the following:

Out of the 6 features [‘age’, ‘tonnage’, ‘passengers’, ‘length’, ‘cabins’, ‘passenger_density’], which of these are the most important? We can answer this question by creating and visualizing the covariance matrix.

II. Creating and Visualizing a Covariance Matrix Using Seaborn Library

Import Necessary Libraries

import numpy as np import pandas as pd import matplotlib.pyplot as plt import seaborn as sns

Read dataset and display columns

df = pd.read_csv("cruise_ship_info.csv") df.head()

Calculate the covariance matrix

cols = ['Age', 'Tonnage', 'passengers', 'length', 'cabins','passenger_density','crew'] from sklearn.preprocessing import StandardScaler stdsc = StandardScaler() X_std = stdsc.fit_transform(df[cols].iloc[:,range(0,7)].values cov_mat =np.cov(X_std.T)

Generate a heatmap for visualizing the covariance matrix

plt.figure(figsize=(10,10)) sns.set(font_scale=1.5) hm = sns.heatmap(cov_mat,

cbar=True,

annot=True,

square=True,

fmt='.2f',

annot_kws={'size': 12},

yticklabels=cols,

xticklabels=cols) plt.title('Covariance matrix showing correlation coefficients') plt.tight_layout() plt.show()

Feature selection using covariance matrix plot

From the covariance matrix plot above, we see that the “crew” variable correlates strongly (correlation coefficient ≥ 0.6) with 4 predictor variables: “Tonnage”, “passengers”, “length, and “cabins”.

cols_selected = ['Tonnage', 'passengers', 'length', 'cabins','crew'] df[cols_selected].head()

Define your features and target variables

X = df[cols_selected].iloc[:,0:4].values # features matrix y = df[cols_selected]['crew'].values # target variable

The features matrix and the target variable obtained above can then be used for model building.

III. Summary and Conclusion

In summary, we’ve shown how a covariance matrix can be created and visualized using seaborn’s heatmap tool. Using the visualization of the covariance matrix, feature selection was used to reduced the original dimension from 6 to 4.

Other advanced methods for feature selection and dimensionality reduction are Principal Component Analysis (PCA), Linear Discriminant Analysis (LDA), Lasso Regression, and Ridge Regression. Find out more by clicking on the following links:

Machine Learning: Dimensionality Reduction via Principal Component Analysis

Machine Learning: Dimensionality Reduction via Linear Discriminant Analysis

The dataset and jupyter notebook file for this article can be downloaded from this repository: https://github.com/bot13956/ML_Model_for_Predicting_Ships_Crew_Size.