Beautiful correlation plots in R — a new approach | by Stefan Haring | Towards Data Science

Everyone working with data knows that beautiful and explanatory visualization is key. After all, it's much easier to tell a story with a chart than it is with a plain table. This is especially important when you’re creating reports and dashboards whose aim it is to give your users and clients a quick overview over sometimes very complex and big datasets.
One type of data that is not trivial to visualize in an explanatory way is a correlation matrix. In this post, we are going to take a look at transforming a correlation matrix into a beautiful, interactive and very descriptive chart using R and the plotly library.
Update (2020–10–04): I had to replace some of the plotly linked charts with static images because they were not displayed properly on mobile.
In our example, we are going to use the mtcars dataset to calculate the correlation between 6 variables.
data <- mtcars[, c(1, 3:7)]corrdata <- cor(data)
This gives us the correlation matrix that we are going to work with.
Now while all the information is there, it is not particularly easy to digest all the information in one go. Enter charts, specifically heatmaps.
As a starting point, base R provides us with the heatmap() function that lets us visualize the data at least a little bit better.
While this is a first step in the right direction, this chart is still not very descriptive and, on top of that, it is not interactive! Ideally, we want to include our final product in a nice Shiny dashboard and enable our users and clients to interact with it.
Plotly.js is a JavaScript Graphing Library that is built on top of d3.js and stack.gl that allows users to easily create interactive charts. It is free and open source, and luckily for us, an R implementation exists!
This is again an improvement. Our correlation matrix is now displayed as an interactive chart and we have a colorbar indicating the strength of the correlation.
However, when taking just a quick glance at the chart, what jumps out? Are you able to identify the strongest and weakest correlations immediately? Probably not! And there is also lots of unnecessary data displayed. By definition, a correlation matrix is symmetric and therefore contains each correlation twice. Additionally, the correlation of a variable with itself is always 1 so there is no need to have that in our chart.
Now take a look at the following chart and try to answer the same questions.
Much better! The chart is clean, we can immediately spot the strongest and weakest correlations, all the unnecessary data has been removed and it is still interactive and ready to be displayed as part of a beautiful dashboard!
To achieve this we’ve used a scatter plot and made the size of the squares dependant on the absolute value of the correlations.
How can you create such a chart (with a little effort) yourself? Let’s take a look!
The first thing we need to do is to transform our data. In order to create a scatter plot suitable for our needs, all we need is a grid. For the correlation matrix, the x and y values would correspond to the variable names, but all we really need are equally spaced numeric values to create the grid. Our transformation converts our correlation matrix into a data frame with 3 columns: the x and y coordinates of the grid as well as the relevant correlations.
#Store our variable names for later usex_labels <- colnames(corrdata)y_labels <- rownames(corrdata)#Change the variable names to numeric for the gridcolnames(corrdata) <- 1:ncol(corrdata)rownames(corrdata) <- nrow(corrdata):1#Melt the data into the desired formatplotdata <- melt(corrdata)
You might wonder why the numeric values for the rownames are reversed in the code above. This is to ensure that the resulting plot has the main diagonal of the correlation plot going from the top left to the bottom right corner (unlike in our base R and base plotly examples above).
As a result, we get a data frame looking like this:
We can plot it with the following code:
fig <- plot_ly(data = plotdata, width = 500, height = 500)fig <- fig %>% add_trace(x = ~Var2, y = ~Var1, type = “scatter”,   mode = “markers”, color = ~value, symbol = I(“square”))
This is a good start, we have our grid set up correctly and our markers are coloured according to the correlations of our data. Admittedly, we can’t really see them properly and they all have the same size. We will tackle this next.
#Adding the size variable & scaling itplotdata$size <-(abs(plotdata$value))scaling <- 500 / ncol(corrdata) / 2plotdata$size <- plotdata$size * scaling
First, we define a size variable to be the absolute value of the correlations. To properly size the squares we need to scale them up otherwise we would just have little dots that won’t tell us much. Afterwards, we can add the size to the markers.
fig <- plot_ly(data = plotdata, width = 500, height = 500)fig <- fig %>% add_trace(x = ~Var2, y = ~Var1, type = "scatter", mode = "markers", color = ~value, marker = list(size = ~size, opacity = 1), symbol = I("square"))
One step closer! The base functionality is now there, our squares are scaled correctly with the correlation and together with the colouring enable us to identify high/low correlation pairs at a glimpse.
We will perform some cleanup next. We will correctly name our variables, remove all gridlines and remove the axis titles. To achieve this, we will set up custom axis lists. We will also center the colorbar.
xAx1 <- list(showgrid = FALSE, showline = FALSE, zeroline = FALSE, tickvals = colnames(corrdata), ticktext = x_labels, title = FALSE)yAx1 <- list(autoaxis = FALSE, showgrid = FALSE, showline = FALSE, zeroline = FALSE, tickvals = rownames(corrdata), ticktext = y_labels, title = FALSE)fig <- plot_ly(data = plotdata, width = 500, height = 500)fig <- fig %>% add_trace(x = ~Var2, y = ~Var1, type = “scatter”, mode = “markers”, color = ~value, marker = list(size = ~size, opacity = 1), symbol = I(“square”))fig <- fig %>% layout(xaxis = xAx1, yaxis = yAx1)fig <- fig %>% colorbar(title = “”, limits = c(-1,1), x = 1.1, y = 0.75)
We’ve already mentioned before that there is a lot of duplicated and unnecessary data displayed in a correlation matrix, due to it being symmetric. We can therefore remove all entries above and including the main diagonal (since all entries in the main diagonal are 1 by definition) in our plot. The easiest way to do this is to just set these values to NA in the original correlation matrix before we apply the transformation. Since this will lead to the first row and last column of our chart being empty, we can remove those as well.
#do this before the transformation!corrdata[upper.tri(corrdata, diag = TRUE)] <- NAcorrdata <- corrdata[-1, -ncol(corrdata)]
Plotting our chart again yields the following:
Almost there! The last step is to add the gridlines back in, give our plot a nice background and fix info that is displayed when hovering over the squares.
To add the grid, we will add a second trace to our plot so that we are able to have a second set of x and y axes. We will make this trace invisible so that nothing interferes with our correlation squares. Since we used unit values for placing our initial grid, we need to shift those by 0.5 to create the gridlines. We also need to make sure that our axes are plotted on the same range, otherwise everything gets shifted and messy. It sounds complicated but it is really straightforward.
Since we have covered quite a lot to get this far, below is the full code to produce our final plot.
After this quite lengthy description on how to create prettier charts displaying correlations we have finally arrived at our desired output. Hopefully, this post will allow you to create amazing, interactive plots that deliver insights into correlations quickly.
Please make sure to let me know if you have any feedback or suggestions for improving what I have described in this post!
For those interested, I have made the full code including more features available as an R package called correally.
Added functionality includes:
automatic rescaling depending on plot size
coloring options including Hex colors, RColorBrewer and viridis
auto formatting of the background, fonts and grids to fit different shiny themes
animations of correlation changes over time (in development)
Also, make sure to check out my post about 3 easy tricks to improve your plotly charts to further enhance what we’ve covered here!