Visualize GitHub Social Network with PyGraphistry | by Khuyen Tran | Towards Data Science

Have you ever wondered how GitHub developers with different roles are connected? For example, if a person is a machine learning developer, what percentage of people he/she follows are web developers? If a person is a web developer, what percentage of people he/she follows are machine learning developers?
In this article, we will use the data that consists of a large social network of GitHub developers to answer exactly that. We will also use PyGraphistry to visualize how different types of developers are connected to each other.
We will analyze a large social network of GitHub developers using the GitHub Social Network in Kaggle. Nodes are developers who have starred at least 10 repositories. If a person is a machine learning developer, ml_target=1 , otherwise ml_target=0 .
Edges are mutual follower relationships between these developers.
Let’s start with finding the percentage of web developers — P(Web Developer) and machine learning developers — P(ML Developer) in the network.
74% of developers in this network are web developers.
What is the percentage of mixed connections (the connection between a web developer and a machine learning developer) in the network?
Start with joining the edges table and the nodes table:
Next, calculate the percentage of mixed connections in the network — P(ML developer connects with web developer).
0.1546558340224842
The percentage of the same connections is almost 5.5 times higher than the percentage of the mixed connections.
Another interesting question to analyze is: If a person is a web developer, what percentage of his/her connections are web developers, and what percentage of his/her connections are machine learning developers?
Let’s use Bayes’ theorem to answer that question.
towardsdatascience.com
From Bayes’ theorem, we know that:
We already know P(ML developer connects with web developer) and P(Web Developer) from previous calculations. Let’s use those percentages to find P(Connect to an ML developer|Web Developer):
0.20852347708049263
Now, finding P(Connect to a Web Developer|Web Developer) is easy:
This means if a person is a web developer:
the percentage that his connections are web developers is 71.95%
the percentage that his connections are ML developers is 28.05%.
These percentages are very similar to the percentage of ML developers and web developers in the network (0.258 and 0.741).
Similarly, we can find P(Connect to a Web developer|ML Developer):
0.5986779897985065
and P(Connect to an ML developer|ML Developer):
0.4103
Now, we know that if a person is an ML developer:
the percentage that his/her connections are web developers is 59.87%
the percentage that his/her connections are ML developers is 41.03%.
Since the percentage of ML developers in the network is only 25.8%, it is surprising to see that 41.03% of an ML developer’s connections are ML developers. This means that ML developers are more likely to follow one another than follow web developers.
Let’s try to confirm the conclusion in the previous section by visualizing the GitHub network using PyGraphistry.
PyGraphistry is a Python library to visualize large graphs. Since we will work with a large graph in this article, PyGraphistry is a perfect tool for that.
To install PyGraphistry, type:
pip install pygraphistry
To use PyGraphistry, start with creating a free account at graphistry.com, then register with your new account:
Next, specify the nodes and edges of the graph. We also use encode_point_icon to show different icons for different roles:
Plot the graph:
Now you should see something like below in your notebook!
You can interact with the plot above here. The larger a node is, the more nodes it is connected to.
Hovering to a specific node will show you the nodes that it connects to.
Clicking a node will give you the information about that node.
The current plot is beautiful but is not very informative. Since we want to know how developers of different roles are connected to one another, let’s color the nodes based on the role of each developer.
Machine learning developers are colored silver, and web developers are colored maroon.
You can interact with the plot above here. You might need to refresh if the color doesn’t show up as you expected.
We can see that developers of the same roles tend to cluster together.
Who are the developers that get the most number of connected nodes? We can find that out by clicking the Data Table icon at the top of the screen.
Once the table opens, sort the table by Degree (how many nodes a node is connected to), then click the point that you are interested in, which will be highlighted in the plot like below:
From the table, we can see that most of the popular developers are web developers. This is not surprising since there are many more web developers than ML developers.
A community is a subset of nodes that are densely connected to each other and loosely connected to the nodes in other communities in the same graph.
Let’s try to detect the communities in our graph using the Louvain method. Since we are dealing with a large graph, we will use cuGraph to speed up the code.
In the table above, partition is the community a node (or vertex) belongs to.
Let’s find out how many communities there are:
array([ 0,  1,  2,  3,  4,  5,  6,  7,  8,  9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21], dtype=int32)
There are a total of 22 communities. Let’s visualize how these communities look like. Start with joining the nodes table and the parts table to get the names of the nodes and their partitions.
Then we use faker to automatically create different colors for different partitions.
towardsdatascience.com
And you should see something like below!
You can interact with the graph above here. We can see that nodes that are in the same community (have the same color) tend to cluster together.
What communities do most of the web developers belong to? We can figure that out by selecting only the nodes that are web developers. To filter nodes, click the Filter button at the top of the screen.
And you should see only the nodes of web developers:
It seems like web developers belong to a diverse number of communities. Let’s see which communities ML developers belong to:
The graph consists of only ML developers:
Interesting! Most of the ML developers belong to the community colored pink. This means that, unlike web developers, ML developers tend to be in the same community. In other words, ML developers are more densely connected to one another.
What is the percentage of ML developers in each community? Let’s use a grouped bar chart to figure that out:
It seems like there is a small percentage of ML developers in most of the communities. This means most communities are quite diverse.
Congratulations! You have just learned how to analyze a social network using Bayesian statistics and PyGraphistry. I hope this article will give you the motivation and the knowledge needed to analyze the network around you. You might be surprised by what you can find from a graph.
Feel free to play and fork the source code of this article in this GitHub repo:
github.com
I like to write about basic data science concepts and play with different data science tools. You could connect with me on LinkedIn and Twitter.
Star this repo if you want to check out the codes for all of the articles I have written. Follow me on Medium to stay informed with my latest data science articles like these:
towardsdatascience.com
towardsdatascience.com
pub.towardsai.net
khuyentran1476.medium.com
FEMI K J. (2021–07–25). GitHub Social Network, Version 1. Retrieved 2021–11–22 from https://www.kaggle.com/femikj/github-social-network. Public license.