The Fastest Way to Visualize Correlation in Python | by Roman Orac | Towards Data Science

As a Data Scientist, I use correlation frequently to calculate and visualize relationships between features.
I used to start by importing matplotlib and seaborn packages, which render a good-looking plot. But it’s cumbersome to import both packages just to visualize the correlation when starting with an empty Jupyter Notebook.
I decided to do a bit of research, because of the frequency I use this command,— there has to be a better (quicker) way of visualizing correlation in Python. Voila, and there is...
By reading this short tutorial, you’ll learn a quicker way to calculate and visualize correlation with pandas.
Here are few links that might interest you:
- Complete your Python analyses 10x faster with Mito [Product]- Free skill tests for Data Scientists & ML Engineers [Test]- All New Self-Driving Car Engineer Nanodegree [Course]
Would you like to read more such articles? If so, you can support me by clicking on any links above. Some of them are affiliate links, but you don’t need to buy anything.
Let’s say we have the following DataFrame (it just has 4 columns with random numbers):
import numpy as npimport pandas as pddf = pd.DataFrame(np.random.randint(0, 100, size=(15, 4)), columns=list("ABCD"))
Calculating and visualizing correlation is as simple as (no other third party packages required):
df.corr().style.background_gradient(cmap="Blues")
Before we delve deeper into other types of correlations, I have to admit, I never used any other than Pearson correlation in practice. Regardless, It’s good to know that others do exist and pandas supports a few of them out of the box.
Let me know in the comments if you’ve used any other than Pearson correlation in practice and what was the use case.
By default, pandas calculates Pearson correlation, which is a measure of linear correlation between two sets of data.
Pandas also supports:
Kendall correlation — use it with df.corr(‘kendall’)
Spearman correlation — use it with df.corr(‘spearman’)
From minitab:
Spearman correlation is often used to evaluate relationships involving ordinal variables. For example, you might use a Spearman correlation to evaluate whether the order in which employees complete a test exercise is related to the number of months they have been employed.
From Stats Direct:
Spearman’s rank correlation is satisfactory for testing a null hypothesis of independence between two variables but it is difficult to interpret when the null hypothesis is rejected. Kendall’s rank correlation improves upon this by reflecting the strength of the dependence between the variables being compared.
As expected, Spearman correlation produces different values than Pearson:
df.corr('spearman').style.background_gradient(cmap="Blues")
To visualize correlation without using other packages is a neat trick, which makes practicing Data Science slightly more enjoyable.
One such command might not seem much, but adding 10 such tricks in your toolbox can make a big difference.
Applying style.background_gradient to correlation results is just the tip of the iceberg. This concept is useful for much, some of the things I already described in:
towardsdatascience.com
Follow me on Twitter, where I regularly tweet about Data Science and Machine Learning.