Is D3.js as hard as it looks? Creating a Data Exploration App from Scratch
By: Anil Gurbuz
Tags: Data Visualization, Web Development, Movies, Data Exploration, D3js

Is D3.js as hard as it looks? Creating a Data Exploration App from Scratch

Photo by Myke Simon on Unsplash

D3 library of JavaScript is known for its flexibility and power for creating interactive and/or animated visualizations and also its steep learning curve.

This library is developed by Mike Bostock who also provided enormous number of high-quality visualizations and their codes. Those examples can be found on the official website here. I found those example visualizations as great resources even for brainstorming to decide what type of visualization could be a good fit for your purpose though, the main power is coming from the codes provided which can be adapted pretty easily after understanding some basics of D3.

Is it really as hard as it looks?

It is a fact that the initial investment cost is higher compared to other data visualization tools though I would argue that the difficulty of this library is exaggerated. I didn’t have any experience even in JavaScript when started learning d3 — by starting a visualization project directly — and I would say after learning some concepts such as data join — enter, update, exit cycles — things got much clear and even very complex animationS/interactive visualizations appeared quite achievable.

What is this project then?

This is a data exploration and visualization project which was done as an assignment a bit of time ago. It is all about creating a data visualization tool to compare movies and movie databases.

We are all familiar with IMDb which is a for-profit organization, on the other hand, TMDb is another platform that is crowdsourced. I initially wanted to investigate if there is any significant difference between the movie taste of their users. Hence, I developed the app.

I am adding the link to the visualisation below, you can go and discover what type of movies are liked by society or making more profit for their lucky producers.

PS: I know the design of the web page is against all the heuristics but the only concern was the design of the visualization at the time.

In the next part, I will try to explain some aspects of D3 that I used in my project and hope that it can be helpful for some beginners surfing on the web looking for what the hell is Enter-Update-Exit cycle — We all have been there.

How to create interactive/animated visualizations using D3?

Obviously, this is a very broad question and there can be a lot of answers to that but all of those answers will require learning how D3 binds data to DOM elements as a first step. Hence, we will look into it.

If you are asking yourself what is a DOM element, this link might help to understand the concept of DOM. In short, a DOM element is a container storing objects within a webpage, and yes D3 assigns data to those containers so we can use any dimension of that data to make changes on the corresponding part of the webpage/container/DOM element.

How to control data binding in D3? — Enter, Update, Exit Cycle

We control data binding to DOM elements through a 3 step process;

Enter-Update-Exit cycle.

Enter is the step where we select the data that doesn’t have a matching DOM element and create new DOM elements to assign the selected data. Update is the step where we select the DOM elements and data that are bound and update properties of selected DOM elements due to any change in the existing bound data of the DOM element. This step requires both the DOM element and the matching data to exist. Exit is the step where we select the DOM elements that don’t have matching data and remove those DOM elements.

Enter-Update-Exit Cycle in Action

In this visualization we will be creating DOM elements for rendering the circles in the scatter plot and the bound data will be about the movies as each point in the plot will be representing one movie in either IMDb or TMDb.

Let’s see some user actions and corresponding reactions of the app to get a sense of how data binding controls all those action-reaction cycles then we will be looking at the code.

Adding a link one more time here to reach the app.

User filters data using slider or buttons

The reaction of the app when the user filters data using sliders as in the gif :

Reaction 1.1: Rescales the x-axis to get a more appropriate range of values on the x-axis for the new set of points.

Reaction 1.2: Removes (falling down of the points) and adds points (falling into visualization from top).

User hovers points

2. Reaction of the app when the user hovers on & off points as in the gif:

Reaction2.1: Makes an API request, structures the response and renders the poster and metadata of the movie represented by the hovered point.

Reaction2.2: Increases opacity of the point hovered on.

Reaction2.3: Rolls back to the initial state of the visualization when the user hovers off.

Code

The code below shows the “render” function which is called every time user makes a change by filtering the data. This code includes the whole enter-update-exit cycle for the circles of the scatter plot.

Some Variables from the code:

d: Data that will be rendered. It stores the Profit, Rating, Vote_Count information of the movies. Poster images are not included in this loaded data set as those images are got with an API request simultaneously with the user action.

xScale: D3 scale object. Technically a mapper for your input data value to axis value.

Rescaling the x-axis

We start by rescaling the x-axis — Reaction 1.1 — by changing the domain of the mapper so that our x-axis will only include the values that are kept after filtering of the user. For example, If the user wants to see movies with Min. Profit 600M$, then, the x-axis will start from 600M$ with the help of this modification at line 5–7 of the code.

But there is one more point, we don’t want our axis to just to disappear and reappear with a new value to look smooth. Hence, transition.duration(1000) part makes the change of the DOM element occur in a second instead of doing it instantly. — DOM element is “xAxis” in this case. The element that is selected by d3.select().

Enter Phase

This phase starts with selectAll("circle").data(d) with this code, we select all DOM elements of the existing circles then assign a row of data to each circle. With the commands after enter() , we create a new circle for the data that doesn’t have any bound DOM element. For example, if we have 100 circles in the plot and we changed the filter and make it 150 which means our data points are more than DOM elements. This is handled in entering step by creating 50 extra DOM elements (“circles”) to match with the data at hand.

As mentioned above, enter is the step that we create new DOM elements for the data that doesn’t have a bounded DOM element. append("circle") is the command creating those new elements then we specify attributes of those circles. For example, attr("cx",d=> xScale(d.Profit)) specify the “cx” attribute of the circle which is the “Profit” attribute of data “d”. xScale turns raw Profit value into a coordinate in the plot.

.on("mouseover"... can be considered as another property of those circles which describes the reaction of the DOM elements once the user hovers. We already mentioned that as Reaction 2.1 and Reaction 2.2. Hence, those reactions are implemented by defining a function for the “mouseover” property.

Using d3.json() function, we are making an API request and get the the poster image and information rendered under the movie poster. This way we don’t keep poster images in our server and get it instantly from the API when the user hovers on an circle.

showPoster is the function that I implemented to size the poster image and render it. This function is also used within .on("mouseout"... to render nothing when the user takes the mouse out from the circle. Also d3.select(this) within .on("mouseover"...) and .on("mouse out"... selects the DOM element which the user hovered.

Update Phase

This is the step that the attributes of existing DOM elements are updated. If a DOM element selected in d3.select, this step reflects the change in the data to the attributes of DOM elements. The update step doesn’t have a keyword and it can be initiated after the enter step by selecting existing DOM elements that are already bound with data. This step only makes a change if both data and DOM elements exist and there is a change in the data.

Exit Phase

We remove the DOM elements here if they don’t have attached data. This happens in the app when we apply filter by moving oscar statues to right. We filter the data so some of the DOM elements have their data filtered. Hence, those DOM elements are removed at this step.

Mycircle.exit() selects the circles that got their data filtered and using attr("cy",... , we change the location of those circles before remove() the command. We also use transition().duration(1000) so those points don’t get their y-axis value changed instantly though those points transit to their new position in 1 second. This way we can make the points fall down as described in Reaction 1.2 instead of just getting disappeared.

Conclusion

Enter-update-exit phases are just representing the steps that we select data that doesn’t have DOM, data, and DOM that both have each other, or DOM elements that don’t have data. This way we control data — DOM element binding and manipulate properties of DOM elements. This concept can be a bit confusing at the beginning but once you get it, it seems quite intuitive.