Visualizing Intersections and Overlaps with Python
By: Thiago Carvalho
Tags: Data Visualization, Python, Matplotlib, Data Science, Editors Pick

Venn Diagrams

Let’s start with a simple and very familiar solution, Venn diagrams. I’ll use Matplotlib-Venn for this task.

import pandas as pd

import numpy as np

import matplotlib.pyplot as plt

from matplotlib_venn import venn3, venn3_circles

from matplotlib_venn import venn2, venn2_circles

Now let’s load the dataset and prepare the data we want to analyze.

The question we’ll check is, “Which of these best describes your role as a data visualizer in the past year?”.

The answers to this question are distributed in 6 columns, one for each response.

If the respondent selected that answer, the field will have a text. If not, it’ll be empty.

We’ll convert that data to 6 lists containing the indexes of the users that selected each response.

df = pd.read_csv('data/2020/DataVizCensus2020-AnonymizedResponses.csv') nm = 'Which of these best describes your role as a data visualizer in the past year?'

d1 = df[~df[nm].isnull()].index.tolist() # independent

d2 = df[~df[nm+'_1'].isnull()].index.tolist() # organization

d3 = df[~df[nm+'_2'].isnull()].index.tolist() # hobby

d4 = df[~df[nm+'_3'].isnull()].index.tolist() # student

d5 = df[~df[nm+'_4'].isnull()].index.tolist() # teacher

d6 = df[~df[nm+'_5'].isnull()].index.tolist() # passive income

Venn diagrams are straightforward to use and understand.

We need to pass the sets with the key/ids we’ll analyze. If it’s an intersection of two sets, we use Venn2; if it's three sets, we use Venn3.

venn2([set(d1), set(d2)])

plt.show()

Venn Diagram — Image by the author

Great! With Venn Diagrams, we can clearly display that 201 respondents selected A and didn’t select B, 974 selected B and didn’t select A, and 157 selected both.

We can even customize some aspects of the chart.

venn2([set(d1), set(d2)],

set_colors=('#3E64AF', '#3EAF5D'),

set_labels = ('Freelance

Consultant



Independent contractor',

'Position in an organization

with some data

viz job responsibilities'),

alpha=0.75) venn2_circles([set(d1), set(d2)], lw=0.7) plt.show()

Venn Diagram — Image by the author

venn3([set(d1), set(d2), set(d5)],

set_colors=('#3E64AF', '#3EAF5D', '#D74E3B'),

set_labels = ('Freelance

Consultant



Independent contractor',

'Position in an organization

with some data

viz job responsibilities',

'Academic

Teacher'),

alpha=0.75) venn3_circles([set(d1), set(d2), set(d5)], lw=0.7)



plt.show()

Venn Diagram — Image by the author

That’s great, but what if we want to display the overlaps of more than 3 sets?

Well, there are a couple of possibilities. We could use multiple diagrams, for example.

labels = ['Freelance

Consultant

Independent contractor',

'Position in an organization

with some data viz

job responsibilities',

'Non-compensated

data visualization hobbyist',

'Student',

'Academic/Teacher',

'Passive income from

data visualization

related products']

c = ('#3E64AF', '#3EAF5D') # subplot indexes

txt_indexes = [1, 7, 13, 19, 25]

title_indexes = [2, 9, 16, 23, 30]

plot_indexes = [8, 14, 20, 26, 15, 21, 27, 22, 28, 29] # combinations of sets

title_sets = [[set(d1), set(d2)], [set(d2), set(d3)],

[set(d3), set(d4)], [set(d4), set(d5)],

[set(d5), set(d6)]] plot_sets = [[set(d1), set(d3)], [set(d1), set(d4)],

[set(d1), set(d5)], [set(d1), set(d6)],

[set(d2), set(d4)], [set(d2), set(d5)],

[set(d2), set(d6)], [set(d3), set(d5)],

[set(d3), set(d6)], [set(d4), set(d6)]] fig, ax = plt.subplots(1, figsize=(16,16)) # plot texts

for idx, txt_idx in enumerate(txt_indexes):

plt.subplot(6, 6, txt_idx)

plt.text(0.5,0.5,

labels[idx+1],

ha='center', va='center', color='#1F764B')

plt.axis('off') # plot top plots (the ones with a title)

for idx, title_idx in enumerate(title_indexes):

plt.subplot(6, 6, title_idx)

venn2(title_sets[idx], set_colors=c, set_labels = (' ', ' '))

plt.title(labels[idx], fontsize=10, color='#1F4576') # plot the rest of the diagrams

for idx, plot_idx in enumerate(plot_indexes):

plt.subplot(6, 6, plot_idx)

venn2(plot_sets[idx], set_colors=c, set_labels = (' ', ' ')) plt.savefig('venn_matrix.png')

Venn Diagram Matrix — Image by the author

That’s ok, but it didn’t really solve the problem. We can’t tell if there’s someone who selected all answers, nor can we tell the intersection of three sets.

What about a Venn with four circles?

Four circles — Image by the author

Here is where things start to get complicated.

In the above image, there is no intersection for only blue and green. To solve that, we can use ellipses instead of circles.

I’ll use PyVenn for the next example.

from venn import venn sets = {

labels[0]: set(d1),

labels[1]: set(d2),

labels[2]: set(d3),

labels[3]: set(d4)

}

fig, ax = plt.subplots(1, figsize=(16,12))

venn(sets, ax=ax)

plt.legend(labels[:-2], ncol=6)

Venn Diagram — Image by the author

Alright, there it is!

But, we lost a critical encoding in our diagram — the size.

The blue (807) is smaller than the yellow (62), which doesn’t help much in visualizing the data. We can use the legends and the labels to figure what is what, but using a table would be clearer than this.

There are a few implementations of area proportional Venn diagrams that can handle more than three sets, but I couldn’t find any in Python.