GraphQL, Grafana and Dash
By: Uditha Maduranga
Tags: Grafana, Data Visualization, Dash, Data Science, GraphQL

If you’re a person who is interested in Data Science, Data Manipulation or Data Visualization this article is the right one for you. I’m sure you have heard of the names that I have used for my topic above. In this article I would be first going through each of these in some detail and later there will be the comparison.

GraphQL, Grafana and Dash

From the above three except for GraphQL other two are used for data visualization. Today’s data visualization tools go beyond the standard charts and graphs used in Microsoft Excel spreadsheets, displaying data in more sophisticated ways such as infographics, dials and gauges, geographic maps, sparklines, heat maps, and detailed bar, pie and scatter plots. The images may include interactive capabilities, enabling users to manipulate them or drill into the data for querying and analysis. Indicators designed to alert users when data has been updated or predefined conditions occur can also be included. Data visualization has become the de facto standard for modern business intelligence.

In order to visualize the data in anyway first you have to extract data from some sort of source. It can be a system of sensors, API, database etc. It also involves understanding of the different features of data and to come up with meaningful, important, and interesting attributes of them. Data extraction is also used to perform exploratory analysis and to extract relevant information from the data. There are many ways and technologies used for data manipulation such as Pandas, R, SQL, GraphQL etc. So GraphQL, which is one of the topics that I have chosen is also a data extracting method, but quite different from the rest.

So let’s start with the first one, GraphQL.

What is GraphQL?

Most applications today have the need to fetch data from a server where that data is stored in a database. It’s the responsibility of the API to provide an interface to the stored data that fits an application’s needs. Or else there should be some kind of filtration to the data that is taken through the API in order to match the needs of the application.

Most people get confused GraphQL as a database technology. This is a not quite accurate as GraphQL is a query language for APIs and not for databases. In that sense it’s database agnostic and effectively can be used in any context where an API is used. It enables declarative data fetching and thus ties in perfectly with React/React Native as a declarative framework for building user interfaces. GraphQL can either complement or entirely replace the usage of REST APIs.

The main difference between REST and GraphQL is that RESTful APIs have multiple endpoints that return fixed data structures whereas a GraphQL server only exposes a single endpoint and returns flexible data structures.

With a RESTful API, you will end up with data that you don’t need. For example, when calling the environment/build/<id> endpoint, you will get all the data pertaining to that particular build. You could get back data like, date_created, date_updated, status etc. But all we needed was the author’s name. This is a clear example of over-fetching in a RESTful API.

In the case of environment/build/<id> was not sufficient to retrieve what we are looking for. To get the code bases that build belongs to, we had to make a call to another endpoint environment/build/<id>/codebases. This situation is called under-fetching.

GraphQL uses its own type system to define the schema of an API, with its syntax called Schema Definition Language (SDL). The schema serves as a bridge between the server and the client to define how a client can access the data.

Once the schema is defined, the front-end and back-end developers can work on their own, as the front-end can be easily tested with mock data. The front-end can also get useful information from the schema, such as its types, queries and mutations. The schema also provides type safety, which is a plus for the front-end and back-end development, as it catches type errors early.

Example Query :

query {

books {

id

title

author

isbn

price }

}

Here we get a single endpoint in which we can get all the necessary data without any over or under fetching through GraphQL server.

Number of popular programming languages support GraphQL.

In addition to advancement in data fetching, performance optimization, versioning and easy to start are few of the reasons that you should try out GraphQL.

What is Grafana?

Climate Station Dashboard using Grafana ( https://developers.bigclown.com/integrations/grafana-for-visualization )

Grafana is an open-source, dashboarding, analytics, and monitoring platform that is tinkered for connection with a variety of sources like Elasticsearch, Influxdb, Graphite, Prometheus, AWS Cloud Watch etc. Most Importantly Grafana is the ability to bring several data sources together in one dashboard with adding rows that will host individual panels (in the UI).

Grafana allows you to query, visualize, alert on and understand your metrics no matter where they are stored. You can create, explore, and share dashboards with Grafana and foster a data driven culture.

If we talk about authentication, Grafana can be integrated with LDAP, Google Auth, Grafana.com, Github. However in the case of your company handles authentication using a technique like SSO (Single Sign-On), Grafana makes it work easily with your existing workflow.

Grafana also has already created dashboards which are build by the community. We can select those by data source, panel type, category and collector. You can have a look at those dashboards here.

If you have the data sources that are compatible with Grafana, all you have to do is set those data sources and then create the panels that you need in order to visualize the content that you want to display. Here you will start with an empty dashboard with one row. In that row you can put any kind and any number of panels you want each panel is responsible for one visual.

You can select the panels you prefer and just add those to your dashboard. It’s simple as that. You can choose the type of panel and explore the ways to interact with it. You can easily create dynamic & reusable dashboards with variables that appear as dropdowns at the top of the dashboard in which you can change and interact. Further you can have drilled down views by making queries. In addition to that you get the facility of having split views to compare with one another.

You can create live dashboards with minimum effort and even explore the logs separately. Another useful feature is the space given to users to define alert rules for their important metrics. Grafana will continuously look for the latest data and send notifications to systems like Slack, PagerDuty, VictorOps etc.

Grafana is a tool that is exceptionally user friendly and has plenty of attractive designs for the use to choose from.

What is Dash?

Dash is an Open Source Python library for creating reactive, Web-based applications. Dash started as a public proof-of-concept on GitHub 2 years ago. We kept this prototype online, but subsequent work on Dash occurred behind closed doors. We used feedback from private trials at banks, labs, and data science teams to guide the product forward. Today, we’re excited to announce the first public release of Dash that is both enterprise-ready and a first-class member of Plotly’s open-source tools. Dash can be downloaded today from Python’s package manager with pip install dash — it’s entirely open-source and MIT licensed. You’ll find a getting started guide here and the Dash code on GitHub here.

Sample dashboard from the Dash App Gallery

Dash empowers data science teams to focus on the data and models, while still producing and deploying enterprise-ready apps. What would typically require a team of back-end developers, front-end developers can all be done with Dash.

Dash is a Python-based framework which is predominantly used to create web applications that include intense data visualizations.As the name suggests, Dash is specialized for creating web-based dashboard applications. But this is achieved by using purely Python or R with the help of some Python libraries.

This is particularly suited to anyone who works with data in Python or R. And Dash abstracts away all of the technologies and protocols that are required to build an interactive web-based application, so it makes it much more convenient for the developers. They currently have a open source version as well as an enterprise version.

We can make awesome data visualizations applications with attractive themes and any kind of graphs. The beauty of this is all this is done in pure Python or R. Further you have more flexibility when we use Dash since we can start creating a dashboard from scratch using Python. So that you can create your own custom dashboard.

Written on top of Flask, Plotly.js, and React.js, Dash is ideal for building data visualization apps with highly attractive Dashboards. It’s particularly suited for anyone who works with data in Python.

Dash Apps are rendered in the web browser so that they are cross platform and mobile ready. Dash is a high level framework which abstracts the lower level implementation of UI components of the graphs, tables etc.

Here are some more pros of using Dash for your data visualization :

Ability to create reports in the form of printable PDFs or interactive, analytic web apps

Automatable reporting and monitoring streaming data flows

Historical archiving, versioning, and status tracking, facilitate auditing and compliance and create institutional memory

Since Dash is relatively new framework compared to the two that we discussed earlier, you can get a overall descriptive knowledge about Dash and how to create a dashboard using Dash here.

Comparison

GraphQL is clearly in a different path than Dash and Grafana as it’s about querying the necessary data from wherever you have stored them. GraphQL helps us to get exactly what we need, nothing more- nothing less. When we consider an application like a dashboard this is really useful when it comes to efficiency. Another thing is despite what many of the pundits and startups would like to be able to convince everyone of, GraphQL will not replace REST. But it’s indeed an easy, efficient and effective way of querying and manipulating data.

Grafana and Dash can be considered as ways to visualize data that you collect from different sources. In Grafana you begin by creating a new and blank Grafana dashboard by clicking on the Dashboard link. Grafana is shipped with a variety of panels that can be added to your dashboard. Panels are not very useful unless some type of graph is associated with them. Graphs depend on data, so each panel that you add to the dashboard will be associated with a data source. So grafana will let you add predefined panels and graphs with a data source connected to it.

But in Dash we can build our own very much customized dashboards from the scratch. But it makes really easy when it comes to plotting graphs and visualizing data. We can use dataframes and then feed those to the graph elements in Dash. So Dash is more customizable and is based on Python. It’s based on React and Flask, while Grafana is written in Go programming language(created by Google) and Node.js. Even though you may think using Dash might take more time and effort, it not that difficult if you’re familiar with Python or R.

All in all what I suggest is to try both Grafana and Dash if you have a requirement to create a Dashboard. What’s best depends on your requirements. But they both gives you amazing stylish visualizations which are informative as well as attractive at the same time.