Heatmap Basics with Seaborn. A guide for how to create heatmaps with... | by Thiago Carvalho | Towards Data Science

The idea is straightforward, replace numbers with colors.
Now, this visualization style has come a long way from simple color-coded tables. It became widely used with geospatial data. Its commonly applied for describing the density or intensity of variables, visualize patterns, variance, and even anomalies.
With so many applications, this elementary method deserves some attention. This article will go through the basics of heatmaps and see how to create them using Matplotlib and Seaborn.
We’ll use Pandas and Numpy to help us with data wrangling.
import pandas as pdimport matplotlib.pyplot as pltimport seaborn as sbimport numpy as np
The dataset for this example is a time series of foreign exchange rates per U.S. dollar.
Instead of the usual line chart to represent the values over time, I want to visualize this data with a color-coded table, having the months as columns and the years as rows.
I’ll try sketching both the line chart and the heatmap to understand how this will look.
Line charts would be more effective in displaying the data; it’s easier to compare how higher a point is in the line than to distinguish colors.
Heatmaps will have a higher impact as they are not the conventional way of displaying this sort of data. They’ll lose some accuracy, especially in this case, since we’ll need to aggregate the values in months. But overall, they would still be able to display patterns and summarize the periods in our data.
Let’s read the dataset and rearrange the data according to the sketch.
# read filedf = pd.read_csv('data/Foreign_Exchange_Rates.csv',                  usecols=[1,7], names=['DATE', 'CAD_USD'],                  skiprows=1, index_col=0, parse_dates=[0])
For this example, we’ll use the columns 1 and 7, which are the ‘Time Serie’ and ‘CANADA — CANADIAN DOLLAR/US$’.
Let’s rename those columns to ‘DATE’ and ‘CAD_USD’, and since we’re passing our headers, we also need to skip the first row.
We also need to parse the first column, so the values are in a DateTime format, and we’ll define the date as our index.
Let’s make sure all our values are numbers, and remove the empty rows as well.
df['CAD_USD'] = pd.to_numeric(df.CAD_USD, errors='coerce')df.dropna(inplace=True)
We need to aggregate those values by month. Let’s create separate columns for month and year, then we group the new columns and get the mean.
# create a copy of the dataframe, and add columns for month and yeardf_m = df.copy()df_m['month'] = [i.month for i in df_m.index]df_m['year'] = [i.year for i in df_m.index]# group by month and year, get the averagedf_m = df_m.groupby(['month', 'year']).mean()
All that’s left to do is unstack the indexes, and we’ll have our table.
df_m = df_m.unstack(level=0)
Everything is in place. Now we can use Seaborn’s .heatmap and plot our first chart.
fig, ax = plt.subplots(figsize=(11, 9))sb.heatmap(df_m)plt.show()
Alright, there’s lots to do before this visualization is ready.
The colors are the most critical part of our chart, and the colormap is a bit overcomplicated. We don’t need that; instead, we could use a sequential cmap with only two colors.
We can also make the limits of the colormap explicit by defining vmin and vmax. Pandas .min and .max can help us figure out what are the best values for those.
fig, ax = plt.subplots(figsize=(11, 9))# plot heatmapsb.heatmap(df_m, cmap="Blues", vmin= 0.9, vmax=1.65,           linewidth=0.3, cbar_kws={"shrink": .8})plt.show()
There are lots of other arguments to be explored with .heatmap.
For example linewidth defines the size of the line between the boxes, and we can even pass arguments directly to the color bar with cbar_kws.
The colors look good, and now we can move our attention to the ticks. I don’t think CAD_USD-1 is the right name for January. Let’s replace them with some friendlier text.
Moving the ticks to the top of the chart would improve the visualization and make it look more like a table. We can also eliminate the x and y labels since the values in our axis are pretty self-explaining, and the title would also make them redundant.
# figurefig, ax = plt.subplots(figsize=(11, 9))# plot heatmapsb.heatmap(df_m, cmap="Blues", vmin= 0.9, vmax=1.65, square=True,           linewidth=0.3, cbar_kws={"shrink": .8})# xticksax.xaxis.tick_top()xticks_labels = ['Jan', 'Feb', 'Mar', 'Apr', 'May', 'Jun',                'Jul', 'Aug', 'Sep', 'Oct', 'Nov', 'Dec']plt.xticks(np.arange(12) + .5, labels=xticks_labels)# axis labelsplt.xlabel('')plt.ylabel('')# titletitle = 'monthly Average exchange rate\nValue of one USD in CAD\n'.upper()plt.title(title, loc='left')plt.show()
There’s one last argument I passed to the heatmap, which is square. That will make the cells of our matrix in a square shape regardless of the size of the figure.
Overall, it looks good. We can see that the U.S. dollar was almost 50% higher than the Canadian in the early 2000s, which started changing around 2003. This lower dollar was sustained until late 2014, with some variation during the financial crisis of 2008.
By 2015 it had stabilized around 1.20~1.40, with relatively slight variation in the monthly averages until 2019, the end of our records.
For the following example, I’ll go through a correlation matrix to see some more functions of Seaborn’s heatmap.
The dataset is a sample of 80 different kinds of cereal, and I want to look at their compositions.
To build the correlation matrix, we can use Pandas .corr().
# read datasetdf = pd.read_csv('data/cereal.csv')# get correlationsdf_corr = df.corr()# irrelevant fieldsfields = ['rating', 'shelf', 'cups', 'weight']# drop rowsdf_corr.drop(fields, inplace=True)# drop colsdf_corr.drop(fields, axis=1, inplace=True)
There’s lots of redundancy in a correlation matrix; the upper triangle of the table has the same information as the lower.
Luckily we can use masks with Seaborn’s heatmap, and Numpy has the functions to build one.
np.ones_like(df_corr, dtype=np.bool)
Numpy .ones_like can create a matrix of booleans with the same shape as our data frame, while .triu will return only the upper triangle of that matrix.
mask = np.triu(np.ones_like(df_corr, dtype=np.bool))
The mask can help, but there are still two empty cells in our matrix.
There’s nothing wrong with it. Those values can add to the symmetry of our plot — That is to say, it’s easier to know two lists are the same if they start and end with the same values.
If like me, you’re bothered with that, you can filter those out when plotting.
fig, ax = plt.subplots(figsize=(10, 8))# maskmask = np.triu(np.ones_like(df_corr, dtype=np.bool))# adjust mask and dfmask = mask[1:, :-1]corr = df_corr.iloc[1:,:-1].copy()# plot heatmapsb.heatmap(corr, mask=mask, annot=True, fmt=".2f", cmap='Blues',           vmin=-1, vmax=1, cbar_kws={"shrink": .8})# yticksplt.yticks(rotation=0)plt.show()
Cool, the only thing not mentioned was the annotations. We can set those with the parameter annot and we can pass a formatting function to it with fmt.
We still need a title, and the ticks would look better with an upper case, but that’s not the priority yet.
Correlations range from -1 to 1, so they have two directions, and in this case, a diverging palette works better than a sequential one.
Seaborn has an efficient method for that, called .diverging_palette, it serves to build the colormaps we need with one color on each side, converging to another color in the center.
That method uses HUSL colors, so you need hue, saturation, and lightness. I used hsluv.org to select the colors of this chart.
fig, ax = plt.subplots(figsize=(12, 10))# maskmask = np.triu(np.ones_like(df_corr, dtype=np.bool))# adjust mask and dfmask = mask[1:, :-1]corr = df_corr.iloc[1:,:-1].copy()# color mapcmap = sb.diverging_palette(0, 230, 90, 60, as_cmap=True)# plot heatmapsb.heatmap(corr, mask=mask, annot=True, fmt=".2f",            linewidths=5, cmap=cmap, vmin=-1, vmax=1,            cbar_kws={"shrink": .8}, square=True)# ticksyticks = [i.upper() for i in corr.index]xticks = [i.upper() for i in corr.columns]plt.yticks(plt.yticks()[0], labels=yticks, rotation=0)plt.xticks(plt.xticks()[0], labels=xticks)# titletitle = 'CORRELATION MATRIX\nSAMPLED CEREALS COMPOSITION\n'plt.title(title, loc='left', fontsize=18)plt.show()
Pretty cool, we built a beautiful visualization for the correlation matrix. Now it’s easier to see the most significant correlation coefficients, like Fiber and Potassium, for example.
Usually, after a correlation matrix, we get a better look at the variables with a strong relationship.
In this case, we don’t have too much data to look into, so a scatter plot would be enough to start investigating those variables.
The problem with scatter plots is that they tend to become hard to read with too much data, as the points start to overlap. That’s when heatmaps get back in the scene to visualize density.
fig, ax = plt.subplots(1, figsize=(12,8))sb.kdeplot(df.potass, df.fiber, cmap='Blues',           shade=True, shade_lowest=False, clip=(-1,300))plt.scatter(df.potass, df.fiber, color='orangered')
If you’re interested in learning more about KDE, I suggest you get a look at Matthew Conlen’s article about this topic.
Well, we explored most of the basics in heatmaps and looked at how they can increase complexity with color maps, bars, masks, and density estimations.
Thanks for reading my article. I hope you enjoyed it.
Here you can find more tutorials on DataViz with Python.
Resources:Seaborn Example — Correlation Matrix;Seaborn Choosing Color Palletes;Seaborn Diverging Palette;Seaborn Kernel Density Estimation Plot;