Reading and visualizing data from Jira by python | by Sergei Dmitriev | Towards Data Science

Jira is one of the widespread software application for bug/ issue tracking and agile project management. One of the entity of this tool is an issue, which can represent a bug, a task, an order or something else (it depends on in what way an organization uses this application):
There is a lot communications information in issues, for example, authors, assignees, descriptions, comments, statuses. This may be interesting in terms of finding patterns, potential problems, improving business processes and increasing efficiency of using Jira or simply taking ‘bird’s-eye view’ how communication flows in Jira.
The article covers the following:
Reading information from Jira by using jira-python library.
Visualizing data by using python’s networkx library and graph visualization tool Gephi.
No, we are not going to consider Big Data tools. Nevertheless, we don’t know in advance how much data in Jira (maybe somebody really meets large volumes of data) and it’s a good practice to process data chunk by chunk.
Jira-python library allows us to easily communicate with Jira APIs. After installing this library, we import the following object and exception as well as create instance of the imported object:
where:
url — a Jira’s server address,
username and password — username and password of user on behalf of communication with Jira will be conducted. It’s reasonable to use so-called technical user, which does not represent a real person and was created only for system to system communication.
After creating instance jira, we can read data by using its issue (for reading single issue) or search_issues (for extracting several issues) methods. For example, let’s find issues belonging to projects ‘Project1’ and ‘Project2’ and which were created during the last 365 days:
where jql — Jira Query Language finds issues corresponding conditions given in a query. jira_search has type jira.client.ResultList — a list of issue resources matching a JQL, for example:
[<JIRA Issue: key=’Key1–12000', id=’102007'>,<JIRA Issue: key=’Key2–12001', id=’102008'>,...]
Iterating over this ResultList, issue’s fields can be extracted. As an example in the code above, we read status name of the first issue.
Maximum number of issue resources in ResultList Jira’s server returns is limited by number of results configured in a server. As a rule, we don’t know how many issues we will read from Jira’s server. Moreover, we would like to avoid possible overload Jira’s server by reading a big piece of data at once. So, it’s a good way to read data iteratively, chunk by chunk. In this case, we can avoid mentioned problems and not think about number of issues in Jira.
‘While’ loop may be used for iterative reading data. Before giving example, suppose we want to read unknown number of issues belonging projects ‘Project1’ and ‘Project2’ and created during the last 365 days. We are going to read certain fields: issue’s type, creation date and time, resolution date and time, reporter, assignee, status. Let’s assume that all reading data we would like to save into ‘csv’ file.
Here is code for reading and saving data:
In the code above, we read 100 issues at once every iteration and after reading necessary fields write this data to ‘csv’ file. For reading certain portion of issues we defined parameters in search_issues method:
startAt — index of the first issue to return,
maxResults — maximum number of issues to return.
By using search_issues method we can read not only fields, but also, for example, changelog. For doing this, parameter expand='changelog' must be passed. Changelog is interesting in terms of statuses or assignees history. Suppose, we would like to know assignees history for a given issue (who made change, previous assignee, new assignee, change date and time):
Ok, let’s move to the next step — visualizing data. Properly visualized data can point to interesting features.
To make read data to tell a story, we need visualization or more accurately we are going to build a directed graph represented connections between reporters and assignees. For doing this, we read created before ‘csv’ file, once again, chunk by chunk for saving computational resources:
For building the graph we use networkx library. In line 25 of the code above, we defined nodes, edges and weights (which represent frequency of connection between nodes). Optionally (lines 32–46), I created the color attribute for nodes, showing belonging to department units. The last line of the code saves our graph to ‘graphml’ format (other formats).
The next step is very interesting and intriguing — graph visualization. I prefer to use Gephi. So after loading saved graph in this tool and making some adjustments, we produce something like this:
What features can we see at this graph? Some of them:
size of the nodes — the bigger, the more incoming or/ and outcoming edges,
size of the edges (in terms of width) — the bigger, the more frequent of this connection.
Maybe employees representing the big nodes and edges need something for increasing efficiency of their work. Nevertheless, all this cases need further investigation.
One more thing: there is a date variable (lines 27–30) in the previous batch of code. This variable is date when a node appeared in data from Jira. So, using it, we can represent our graph as the dynamic one:
Things represented above can help to build further steps: calculate statistics (how many comments assignees create, how much time employees spend on communication and so on). Diving deeper, we can, for example, make semantic or morphological analysis of description/ comments. There is no limitation for your desired goals!