10 tips to improve your plotting. Because in real-life data science... | by Gonzalo Ferreiro Volpi | Towards Data Science

I have to be honest with you: when I was studying Data Science I completely underestimated the importance of plotting. It’s true, everything was a mess back then: in between learning Python from scratch, getting my head around all the possible algorithms and understanding the math behind everything, plotting was the very last of my problems.
And why shouldn’t it be? We were always plotting the same kind of things. You know: pairplots, distplots, qqplots...those charts you use when visualizing the data is the only way of understanding it. Very useful charts. But very generic and default charts too. So copying and pasting a bunch of code became my best friend those days.
For my projects, the deliverable was always a model. Hopefully with some reasonable score thanks to hours and hours of cleaning and feature engineering. I was the only person involved in my projects and my professors already knew everything about that data since they gave it to me. So who would I be plotting for? Myself? C’mon...unnecessary! Right? I knew better than no one what was I trying to achieve in each step. I needed to explain nothing to nobody.
But apart from all this, let’s be honest, plotting it’s not fancy at all. Anyone can plot. My 60 years old father can pull some charts out of the hat just by using Excel. And if everybody can do it then, of course, that’s the definition of not being fancy. Cause here my friends, we’re doing Data Science and Machine Learning. And most of the people can’t even understand what that is. That’s why we’re all so cool and sexy, as Harvard Business Review magazine said.
The problem is peeps -if you haven’t found out yet by my excessive irony, that this is not how real life works. And I believe that was probably the biggest failure in my Data Science immersive: not weighting up enough the importance of explainability and interpretability. You might be a genius, but if you are not able to explain to a third party how and why you’re getting those wonderful predictions, then you might have nothing. For example, at Ravelin Technology we offer machine learning-based solutions for fraud prevention. Imagine telling a client, that we’re blocking X% of the transactions because the machine learning model says so, but without having any idea at all about why is doing it. Surely not very appealing for any e-commerce out there trying to maximize conversion and sells, right? Imagine now this same kind of situation in some other sensitive domains as healthcare...disaster would be just around the corner.
Now, apart from business-related problems, and even aside from a legal point of view or from the fact that maybe your business only cares about predictions -no matter how you get them, understanding how an algorithm is actually working can help. Not only to better explain outputs to customers but also to better align the activities of data scientists and analysts.
So it comes up that in the real world, the picture is completely different from the one I had when I was working in my academic data science projects: I’m never the only person involved in a project and my workmates and/or clients usually don’t know much about the data I’m using. So who would I be plotting for now? Does it still sound unnecessary? Probably not. And being able to explain your thinking process to people is a key part of any data-related job. That’s why copying and pasting charts is not enough and charts personalization becomes very important.
In what’s left of this post I’ll like to share with you 10basic, intermediate and advanced tools I’ve found very useful in real life when it comes to plotting for explaining things about your data.
The libraries I’ll be referencing in the following lines we’ll be:
Seaborn | From: import seaborn as sns
Matplotlib | From: matplotlib.pyplot as plt
Additionally, if you want, you can set up a style and your favourite format like:
plt.style.use(‘fivethirtyeight’)%config InlineBackend.figure_format = ‘retina’%matplotlib inline
Having said that, let’s jump straight to the tools:
There will be opportunities in which you’ll want to plot several things within one chart, and we’ll cover that in brief. But some other times, you’ll want to throw different charts in the same row or column, complementing each other and/or showing different pieces of information.
For this, we’re going to see a very basic but essential tool: subplots. How to use it? Very simple. A chart in matplotlib is a construction using:
A figure: the background or canvas for drawing our charts
Axes: our chart or charts
Usually, these things are set up automatically on the background of our code, but if we want to draw several plots, we only need to create our figure and axes objects in the following way:
fig, ax = plt.subplots(ncols=number_of_cols, nrows=number_of_rows, figsize=(x,y))
So for example, if we set up ncols = 1 and nrows = 2, we’ll be creating a figure of size x,y with only 2 charts, distributed in two different rows. The only thing left would be to specify the order for the different plots using the ‘ax’ parameter and starting from 0. For example:
sns.scatterplot(x=horizontal_data_1, y=vertical_data_1, ax=ax[0]);sns.scatterplot(x=horizontal_data_2, y=vertical_data_2, ax=ax[1]);
This might appear unnecessary, or not very helpful, but you cannot imagine how many times you can be asked what’s on the X/Y axis if your chart is somehow confusing, or who is seeing it is not very familiar with the data. Following our previous example with two plots, if we want to set up a specific name for our axis, we’ll have to use the following code lines:
ax[0].set(xlabel=’My X label’, ylabel=’My Y label’)ax[1].set(xlabel=’My second X label’, ylabel=’My second and very creative Y label’)
Another basic but key tool if we’re presenting our data to a third party is using titles, and it works in a very similar way as our previous axis labelling point:
ax[0].title.set_text(‘This title has to be very clear and explicative’)ax[1].title.set_text(‘And this title has to explain what’s different in this chart’)
More often than note, just having the y-scale to the right or left of our chart is not going to be very clear by itself. Whether it is because all values are very close in between them or because precision is very important for the thing being analyzed. Whatever the case, annotating the value on the plot can be very useful to add clarification and self-explanation to it.
Suppose now we’re using subplots, so we have several charts, and one of them is a Seaborn’s barplot in the position ax[0]. In this case, the code for getting the annotation on each bar is a bit more complex but very easy to implement:
for p in ax[0].patches:ax[0].annotate(“%.2f” % p.get_height(), (p.get_x() + p.get_width() / 2., p.get_height()),ha=’center’, va=’center’, fontsize=12, color=’white’, xytext=(0, -10), textcoords=’offset points’)
For each ‘patch’ or bar in our chart, the code till the ‘ha’ parameter gets the location, height and width of the bar, to put the value annotation in the right position. In a similar way, we can also specify alignment, fontsize and colour of the annotation, while the ‘xytext’ parameter indicates whether we want to move our annotation in some x or y direction. In our example above, we would be moving the text down on the y axis.
In some cases, over a period of time or a range of values, we might have measured a different kind of objects. For example, suppose that over 6 months we measure the weight of dogs and cats. At the end of the period, we want to plot the weight of each animal, but differencing in between dogs and cats using blue and red respectively. For that, in most of the traditional plots, we can use the parameter ‘hue’ to provide a list of colours for our elements.
Take the following example:
weight = [5,4,8,2,6,2]month = [‘febrero’,’enero’,’abril’,’junio’,’marzo’,’mayo’]animal_type = [‘dog’,’cat’,’cat’,’dog’,’dog’,’dog’]hue = [‘blue’,’red’,’red’,’blue’,’blue’,’blue’]sns.scatterplot(x=month, y=weight, hue=hue);
Using the same example from above, we could want also to add into the chart the size of the weighted animal, using a scale from 1 to 5. A good option for adding this extra metric into the plot would be modifying the size of the scatterplot, assigning the sizes to a new additional vector through the ‘size’ parameter and using ‘sizes’ to adjust the relationship in between them:
size = [2,3,5,1,4,1]sns.scatterplot(x=month, y=weight, hue=hue, size=size, sizes=(50,300));
And by the way, if as in the chart above, the legend is making the plot harder to read instead of helping, you can always remove it setting the ‘legend’ parameter equal False ;)
In many cases in real life, having data above or below a certain threshold might be a signal of problems or a warning for errors. If we want to clearly show this on a plot, we can add a line using the following command:
ax[0].axvline(32,0,c=’r’)
Where:
ax[0] will be the chart in which we want to insert the line
32 is going to be the value in which the line is going to be drawn
And c=’r’ draws in colour red
If we are working with subplots, it is as simple as adding an axvline to the corresponding axe as in the example above. However, if we are not using subplots, we should do the following:
g = sns.scatterplot(x=month, y=weight, hue=hue, legend=False)g.axvline(2,c=’r’)plt.show()
This might be the simplest, but also one of the most helpful tips.
Sometimes we just need to add more information to our chat, and there’s no other way around it than adding a new metric on the right y-axis of our plot:
ax2 = ax[0].twinx()
Now you can add any chart you want pointing the ‘ax’ parameter to ‘ax2’
sns.lineplot(x=month, y=average_animal_weight, ax=ax2)
Mind that this example assumes again you’re working with subplots. If you’re not, you should follow the same kind of logic as in the previous point:
g = sns.scatterplot(x=month, y=weight, hue=hue, legend=False)g.axvline(2,c=’r’)ax2 = g.twinx()sns.lineplot(x=month, y=average_animal_weight, ax=ax2, c=’y’)plt.show()
Mind that for this to work, you should set up always the same data for x-axis in both charts. Otherwise, they’re not going to match.
Overlapping charts on the same axis is easy: we just need to write the code for all the plots we want, and afterwards, we can simply call ‘plt.show()’ for all of them to be drawn together:
a = [1,2,3,4,5]b = [4,5,6,2,2]c = [2,5,6,2,1]sns.lineplot(x=a, y=b, c=’r’)sns.lineplot(x=a, y=c, c=’b’)plt.show()
However, sometimes overlapping can result in confusion, so we might want to personalize our chart a beat for making eat clearer.
For example, suppose you want to overlap in the same figure the height distribution of two different samples you took: one from your workmates and another from your local basketball team. It would be nice to personalize things such as the colour of both plots and add a legend specifying which is which, right? Well, easy peasy:
Setting up the ‘colour’ label we can set up a specific colour for each one. Be aware that sometimes this parameter can change to simply ‘c’
And with the ‘label’ parameter we can specify any text for them to appear by simply calling after ‘x.legend()’
Take the following example:
g = sns.distplot(workmates_height, color=’b’, label=’Workmates’)sns.distplot(basketball_team, color=’r’, ax=g, label=’Basket team’)g.legend()plt.show()
Finally, a very specific tool if you like using barplots, you might have faced the problem of not having your bars in the order you’d like them to be. In that case, there’s a simple fix for that, passing a list with the specific order you want to the ‘order’ parameter:
a=[‘second’,’first’,’third’]b=[15,10,20]sns.barplot(x=a,y=b,order=[‘first’,’second’,’third’]);
Plotting is a world by itself, and in my experience, the best world of improving your skills is by practising. But I hope these tools and tips help you when the real-world Data Science hits you as it happened to me :)
Finally, don’t forget to check out some of my last articles, like 6 amateur mistakes I’ve made working with train-test splits or Web scraping in 5 minutes. All of them available in my profile. Also, if you want to receive my latest articles directly on your email, just subscribe to my newsletter :)
See you around!