Which library should I use for my Python dashboard?
By: Antoine Hue
Tags: Data Visualization, Data Science, Plotly, Matplotlib, Bokeh

How can I interact with my graph?

You are very pleased with the chart but it feels so static, there is not even a zoom!

Interactivity is so many different things. It starts with common operations like zoom and pan. The next step is synchronized graphs: zoom and pan are applied simultaneously on several charts that share an axis. You might also be interested in synchronous selection on two graphs, also called brushing (example in Bokeh¹⁹). Matplolib has such interactivity for all render engines but within a notebook²⁰. There is however a solution based on Matplotlib, mpld3 is handling this and might provide all you need. However, the trend is to use newer libraries like Plotly or Bokeh that have zoom and pan in notebooks “out of the box”.

Then come dynamic annotations. They span from hover information when the mouse is located on a marker to line plot highlights. Regarding hover, whatever the used library (Matplotlib with Mpld3 and plugins, Plotly, Bokeh) it means attaching an HTML document div to each marker, and probably also some CSS.

Bokeh plot of a UMAP projection of the MNIST digits (784 pixels) in the 2D feature plan with hover information containing the original image © the author

More complex interactions are related to filtering or querying the data. It can be close to the zoom function when the filter is modifying a range (e.g.: daily / weekly / monthly selector for a time series), or a selector on the series of facets, or even more complex associations. Selectors are available in Plotly as Custom Controls²¹ and in Bokeh as widgets²².

The common plotting libraries provide basic capabilities for interactivity up to the creation of some widget, but, as for advanced layouts, I would suggest to directly switch to Plotly Dash which is more versatile.

Is the rendering of my dataviz dashboard fluid?

The more complex the dashboard or the larger the data, the longer it takes to process, thus the longer to render. It may be ok to wait for a few seconds to get a view of a static plot. It is no longer ok to wait more than 1 second when the graph is interactive with widgets and controls.

If fluidity is gone, you have four main solutions, with increasing complexity:

Simplify the dashboard with fewer plots, fewer controls. That may be ok but then you should think why such complexity was needed first?

Simplify the data, that shows less data or with less granularity. That may provide a good tradeoff between features and accuracy. It is leading to the next solution…

Offline pre-processing of data to pre-assemble the data shown in the dashboard. That probably means storing new series or new tables, leading eventually to other issues linked to data management. In the end, you will do a lot more data engineering and will probably reach a dead-end with two much data, too many tables. The solution to the dead-end is even more data engineering with…

Online processing in dedicated servers and the design of an API.

A server with an API is not the first thing you were thinking but you end up dealing with this software project sooner than you think. It is also better to anticipate it than delay until there is no other solution and project deadlines are coming fast.

Defining an API involves often several teams and skills: data scientists to define the why, data engineers to define the what, and infrastructure engineers to define the how, including performance, security, persistence, and integrity.

Plotly Dash allows for an initial API since it is based on the Flask framework. See the Dash documentation on integration with a Flask app which is defining a custom Flask route, that could serve data, i.e. an API²³. There is still no proper API access management, including authentication. On that latter aspect, Dash is very limited²⁴.

While developing the dataviz, I feel like I am fast/slow/don’t know

How much effort will it take you to develop and publish?

Some tools are effective, they deliver the expected result, but are not efficient, getting to the result takes a large amount of time. For example, d3.js is known as a very powerful and complete data-visualization and charting library but at the same time, it requires dealing with many things that are by default available in libraries with higher abstraction.

Productivity is coming not only with using the right level of abstraction, but also an API that is easy to use and well documented. I would say that none of the surveyed Python charting libraries are easy to master.

Matplotlib’s API is quite complex when dealing with axes, formats (of labels), it is not always consistent and quite redundant. As an example, see the above comment on “`plt.subplot()`”. That’s not the only example, for example, there is a sub-routine “`plt.xlabel()`” that is equivalent to the method “`ax.set_xlabel()`” on the Axes object.

Plotly’s API is not better, first, you must choose between two API sets: the Express set that is quite simple but limited and mostly targeted at dataframes, and the Graphical Object set that is more complete and complementing the Express set, but does not have some nice high-level features that are in Express. Then you will have to deal with the Plotly documentation that is, to me, really difficult to get through. Searching with either the Plotly website internal or a Web search engine seldom leads you to the API you are looking for. And you may have to deal with the documentation of the underneath data model in JSON.

Bokeh API is probably leaner and better documented but has some weird things like two separate instructions to plot a line chart and associated markers.

I really need a nice and slick web app, should I be afraid of it?

Your dashboard is successful and will be deployed as a product internally to your organization, available to clients, or even directly exposed on the Internet.

As a data scientist, you are missing skills to handle that and get the help of software specialists. However, you are asked what is the effort or scope of this development. This highly depends on the path to production, which framework is used there, and on the framework that you have been using until now.

Getting to production with (part of) current framework

Plotly standalone graphics can be exported as static HTML. Bokeh provides some schemes to embed it²⁵. Matplotlib with Mpld3 has an HTML output²⁷. However, this solution is targeting illustrations rather than dashboards.

Plotly Dash may in some cases go up to production as the main Web app or an embedded widget²³. As said earlier in such setup you will need to inspect the security of your system before jumping to online production. Regarding security, as a data designer, you mainly need to check that you are exposing only the wanted data and not more.

Network and geographical interactive graph showing the incoming subway traffic in Paris © Data-Publica

Getting to production with a single page application

Today, most of the Web applications we use are based on a pattern called single page application (SPA): the application is loaded once in the Web browser and then interacts with the server through some asynchronous API without reloading the Web page. This is what all of us now expect from a nice Web application.

SPA has two separate components: the browser side application in Javascript with a framework like Angular or React.js, and the server-side application or service that may get written on many frameworks and language: Java, C#, PHP, Javascript…, and even Python.

Dash is already doing part of it. In fact, Dash is using one of the leading browser-side framework, React.js, and on the server-side is based on Flask and Python. But as said above you may reach some limits of Dash.

Besides the transition through Dash, Plotly and Bokeh have another advantage: they are also available in Javascript as Plotly JS (and a React.js wrapper wrapper²⁶), Bokeh JS. In fact, the Python version of Plotly is a wrapper around the Javascript. This implies that given some plots or dashboards in Python based on Plotly or Dash or Bokeh, most of the concepts and chart properties can be reused in the equivalent Javascript implementation.

Conclusion

In this post, we have brushed the path for a data-visualization dashboard from experiments within notebooks, up to production. We have seen that the traditional plotting library, Matplotlib, still has strong features and is usually the default backend for specialized libraries like NetworkX and the Pandas Dataframe. But Matplolib is also lagging on some aspects like integration and interactivity. Learning another framework may be a good investment and will help you going forward up to production.

Two alternative frameworks are presented: Plotly and Bokeh. Both bring value as they are more modern than Matplotlib. Both of them have a leading advantage when it comes to bringing the dashboard to production: they are based on Javascript plotting frameworks and most of the plots Python code can be translated directly in the Javascript equivalent.

Plotly has another advantage on the go-to production path: it is integrated with Dash, a framework to develop a simple dashboard as single-page applications while sticking to Python. Required Javascript, including React components, and server API is generated smoothly by Dash.

We have also seen that, as a data scientist or data-visualization designer, you should anticipate requirements like interactivity, and their implications that may lead to the development of an API to serve data.

In “Plotly Dash or React.js + Plotly.js”, a side by side comparison is showing what is required to port a Dash application to a Javascript application in React.js with Plotly.js for the dataviz, and served by a Web API in Python.