Django, Pandas, and Chart.js for a quick dashboard | by Shane Gary | Towards Data Science

Today we will talk about how to use Django, Pandas, and Chart.js together to throw up a quick dashboard. This will not be a Django tutorial, and I am taking some shortcuts here. The goal is to show you how to start using these tools together to quickly display some data. This is best applied when you need to get something up fast to visualize a single table with a few different charts. It could also be a starting point for a more robust django and chart.js site.
Why Pandas? It is my experience that doing a bunch of pandas wrangling behind the scenes on a web server while someone is waiting on a website to respond is generally not an good strategy. There is certainly a way to do all of this from within Django directly from the database. The right database queries are almost always going to be faster and a better long-term answer for a dynamic production website. However, there are times when you have gotten to a good data exploration stage in a Jupyter notebook and you just want to throw something together to show what you have found so far.
Why Django? Well, we are really talking about just two minutes of work to have Django up, running, authenticating, and talking to the database server of your choice. Django is a great choice for this project when you can get the data exported into a single table. Flask would probably be a better choice if you don’t need authentication and want to access the data from an existing database.
Why Chart.js? I have found that if you want to deploy a bunch of different interactive charts by modifying a few variables Chart.js is amazing. After I hacked my way through the first chart, I found that the rest of them pretty much just worked with slight modification.
The code is here. Let’s dig in.
For the tutorial I grabbed this dataset. It is not the best dataset for dynamic charts, but it was easy.
I am running this on a Debian server and my goal is just to document what was done. Again, this is not a Django or a python tutorial. I am going to speed through this section. Feel free to skip it but I thought it would be good to include for reference or so someone could reproduce this directly.
pip install django pandasdjango-admin startproject django_chartscd django_chartspython manage.py migratepython manage.py createsuperuserpython manage.py startapp datacd data mkdir templatescd ..python manage.py runserver
Optionally palettable is a fantastic for color palettes. However, the code will work without it.
pip install palettable
Based on the dataset I created the following model but this should be modified for your data.
Make sure to update the database after you create this from the terminal:
python manage.py makemigrationspython manage.py migrate
I created a file with a method to load the csv file downloaded for Kaggle into the database using pandas and Django:
The easiest way to run this is just to import it and run it in your view. We will delete this later but this is a good way to make sure the site is setup. So lets create the view but for now it just needs to run this function.
Next, create a blank file named base/dashboard.html and create a data/urls.py file
Edit django_charts/urls.py to add the data urls
Update the django_charts/settings.py to include ‘data’ in INSTALLED_APPS:
Let’s make sure we can see purchase in the admin site by adding this to data/admin.py
Now you should be able to go to the dashboard page in a browser. Only do this once and then check on the admin site to see if you see the records. There should be 1000 records if you used the same dataset.
OK. Sorry about the speed run but now we can start from the same place.
I have a base file that will be extended.
This might seem like a lot but this is a good basic template. I tend to put 4 code blocks in my base template.
{% block title %}{% endblock %}{% block custom_css %}{% endblock %}{% block page_content %}{% endblock %}{% block js_scripts %}{% endblock %}
The names are probably self-explanatory but the this gives me a way to add a title, css, body element, or javascript from any page. Admittedly, I mostly only use the page_content and js_scripts blocks.
You will notice that I have included bootstrap CDN here. You really don’t need popper.js for this project. You could do the project without bootstrap actually but you would need to change some of dashboard.html.
I have also included the chart.js cdn. This you will need for sure.
I wrap any scripts I write into a $(document).ready in the base so that any manipulations won’t happen until the page is ready.
Next here is dashboard.html which is the page for our charts:
Basically, we are creating a bootstrap container, then a row, then a column. Inside the column I have a card-deck. The card-deck will create evenly sized cards. I then loop through each chart and place a card inside the card-deck.
The card-deck will try and cram everything on the same row. The forloop.counters are used to detect the size of the screen and the wrap the card deck accordingly. This is my modification from this example. However, in practice you will find that somehow chart.js ignores your canvas size settings. This is one area that I wish chart.js would reconsider. HTML’s main function here is layout and I am not sure how to stop chart.js for ignoring the html layout.
You will see my 2 main loops. I insert the html elements chart by chart in a card. Then later in the js code block I implement the js script chart by chart. This is a very simple way to get a responsive column (for mobile) or grid of charts. In my most recent case, I simply wanted a column of charts to show up on a mobile device.
I think it might be easiest to look at views.py file first to get an idea of the flow of things. You will notice I deleted the one time method to load the data.
We will be using a TemplateView. This is a very simple view to add things to. I tend to name my html files something I will be able to guess so I identify which template I am using with template_name. The only method we need to extend in this case is get_context_data. This is the method Django uses in class based views to get the data to be presented.
We pull the objects we want and create a dataframe. Because chart.js generally does not play well with dates, I go ahead the convert the dates to strings after the dataframe is created. Then we just add each chart we want to the chart’s context. This allows us to loop through charts in the html code. For reference each chart is a dictionary that contains an html and a js entry. The html is just a simple canvas tag with the ID and the js does most of the work. I covered the loops in the last html section.
The rest of this is in a file data/charts.py. I do this so it is easily portable to another project. You could just put this in views. I really should clean this up and package it, but the goal here is quick, temporary charts. I will include the entire file at the end but it is best to walk through some of the functions first and then get into the Chart class. All of the following functions are in the data/charts.py file.
You can read through the comments to understand all the options. But this function takes in a Django model and returns all the records. You can include some filters after all of the name arguments. So for example:
would limit the result to the city of Mandalay. Include and exclude fields work the same way Django typically uses these. If you do not include anything it will pull in all the fields from the model. It is important to note that excludes are processed after includes. So, if you include and exclude a column it will not show up. I use pandas apply to convert the dates based on the input.
This does not actually do anything in this example. However, I used a dataclass for Chart and this is a way to set default options for all charts. I reuse the code and often modify the function when want to set a default option for all charts.
If you wanted to not bother labeling each chart this function sets a random ID that is used by js to modify the associated html element. This ID is also used to name the JS function and variable to make debugging is easier. Because of that you need to avoid a chart ID with disallowed js characters (ex. -).
get_colors is used to set the initial colors. If you pass in a palette it will use that instead. This is mainly written to take advantage of any palettable imports. However, if you do not pass anything in and don’t import something from palletable it will randomly generate colors.
get_random_colors is used to generate colors in the case where no colors were passed in, imported or when you have more colors than values. It starts with the colors you pass in and them adds them at random until you reach the desired number.
These are all the functions we needed before we get into the class.
So, the following is the Chart class. I will start with just the class and then get into each function within the class.
I used a dataclass here. These are relatively recent additions to Python and for quick and easy classes they often are a good choice. Setting the initial values can look a bit strange but it helps to ensure you do not use a mutable object for instances. It really is not difficult but does lead to writing some little functions to create initial values.
from_lists is how I populate all the chart object with data, labels, and colors. After checking to make sure we have enough colors and adding some random colors, it builds what chart.js calls the datasets. Each stack needs its only little dictionary appended here. If there is only one value, we want to append the entire color palette so each bar is a different color. Labels are assigned directly. from_lists is how you could bypass pandas and create a chart from a database query.
from_dataframe is what helps us use a few lines of code to manipulate just about any dataframe to be able to easily feed directly into from_lists for chart.js. We use the pandas pivot_table method to create a pivot table based on the input. That pivot table can the directly dump out the lists we need for the chart. If you are doing a ton of charts from a single dataframe this is very fast.
OK.... I know. This is the one function that almost and maybe should have prevented me from publishing this article until I broke it up. However, it is easy to see what is going on here. We need to form our initial elements dictionary that contains our label, datasets, and options. These are the basic charts.js requirements. Once that happens the code uses the passed in type to set additional chart options. I think you will find that this might be the only function you need to add to for most if not all of the chart.js charts. This is where, for example, we need to tell chart.js to stack an axis for the stacked bar chart. You will see ‘begin at zero’ for a few of them because chart.js sometimes defaults to starting closer to the lowest value. Personally I think the default should be zero. Often it is as easy as the three at the end that really just need a chart type defined.
I write functions here to handle the html and javascript code. Typically I would create actual html and js templates. It would be easy to do but this is more portable.
It doesn’t get more simple than this. Just create the canvas for js to update. This is what the html loop places in the html template for each chart.
get_js just generates the js code. You can see that most of the work is already done in the code above (get_elements). This grabs the chart element in the dom (the canvas) and adds the chart.
Finally I just defined a function that returns both the html and js as a dictionary.
You could basically just copy chart.py into your project and use it like I did in my views.py file. You need to make sure your base html file imports the chart.js cdn. Bootstrap is obviously optional but does make it easy. Happy dashboarding!!!
Here is a complete class definition for reference: