Don’t over-plan your Data Visualization
By: Oskar Kocol
Tags: D3js, Maps, Visualization, Editors Pick, Data Visualization

Implementing basic functionality and the first speed bump

I like to start my design by implementing basic functionality. The zooms, pans, sliders — the more general they are, the more likely I am to include them in the first step. In my map, the first two things that I wanted to include were the drag and zoom behaviors. I structured them

wind_map = { const svg = d3.create('svg')

.attr('viewBox', [0, 0, width, height])

.attr("fill", "none")



const map = svg.append("g")

.attr("id", "map")

.call(d3.drag()

.on("start", dragstarted)

.on("drag", dragged)

.on("end", dragend)) //...

}

With dragstarted() , dragged() and dragend() as helper functions responsible for behavior at the start, during, and at the end of the drag event. But it quickly came to my attention that the default behavior of d3.drag() allows dragging in all directions. In the case of the globe, it was possible to change the roll, pitch, and yaw of the globe and turn the globe to the position where the North is not facing top of the visualization. See what I mean on the gif below.

Diverging from the North facing the top of the visualization is hardly ever justified.

Such functionality would introduce unnecessary confusion and distraction. I stopped this by setting the rotation around y axis (or, in more detail, setting the Euler rotation angle γ to 0).

I implemented the zooming behavior with d3.zoom() , using the same structure as the dragging behavior. I added one more call that used zoomstarted() , zoomed() and zoomend() helper function to implement behavior at the start, during, and at the and of the zoom.

map.call(d3.zoom()

.scaleExtent([400, 1000])

.on("start", zoomstarted)

.on("zoom", zoomed)

.on("end", zoomend))

As soon as the zoom was present on the chart, it became clear that it will have to be tweaked too. The default d3.zoom() allows for dragging during the zoom. It looked as a redundant distraction, that was quickly fixed with the deletion of a couple of lines.

With a globe that could be zoomed and dragged around, I was ready to move to the next stage.

The basic functionality is ready.

Getting the data

To get started on the wind overlay that will introduce the color to the wind map, I needed the data. Getting the data file with the data was easy, but extracting it was not.

On the website of the National Oceanic and Atmospheric Administration (NOAA), you can find a Global Forecast System. It gives a forecast for a range of parameters (such as temperature, cloudiness, wind strength and direction) for any place on Earth.

The data is packed into a GRIB format which is quite tricky to work with. It encodes values for 2-dimensional data on an equally spaced grid. For example, a GRIB file for temperature with a 1-degree resolution would give the temperature for:

(0⁰E, 0⁰N), (1⁰E, 0⁰N),… (180⁰E, 0⁰N), (179⁰W, 0⁰N),… (2⁰W, 0⁰N), (1⁰W, 0⁰N)



(0⁰E, 1⁰N), (1⁰E, 1⁰N),… (180⁰E, 1⁰N), (179⁰W, 1⁰N),… (2⁰W, 1⁰N), (1⁰W, 1⁰N)



(0⁰E, 2⁰N), (1⁰E, 2⁰N),… (180⁰E, 2⁰N), (179⁰W, 2⁰N),… (2⁰W, 2⁰N), (1⁰W, 2⁰N)



...

But what makes GRIB files difficult to work with is that they are encoded in binary. Unless you use a separate program or a library to decode it, there is no way to make sense out of the raw GRIB file. This, in turn, introduces a need for a backend decoding and serving decoded GRIB files.

At first, I was planning on making my visualization automatically download the most recent GFS data from the NOAA website, but the complexity of dealing with GRIB files using only JavaScript cooled down my excitement. I decided to use a static JSON file with the wind data from 2014-01-31T00:00:00.000Z . Why this particular date? It looks like a busy day with a lot of little weather systems going around the planet.

Creating the color scale

At first glance, the original color scale from earth.nullschool.net is built from two parts:

The first part starting with blue and ending with pink is a mirrored Sinebow Scale that ending on magenta. The second part is magenta that has its alpha gradually increased until reaching white.

The color scale used on the original visualization

I had a peek at the Github repo of the Wind Map and I was able to find that a lot of logic behind the color scales is hardcoded. I jumped into creating my own color scales and I ended up with three different versions:

The basic, quick and dirty scale using the default d3.interpolateSinebow . It essentially ignores the magenta-to-white portion of the color scale.

The accurate color scale where I used my own Sinebow Interpolator. It is as close as it gets to the original implementation.

The efficient color scale that uses my own Sinebow Interpolator. It is essentially a list of hardcoded if statements. I was hoping to achieve better efficiency when generating a wind overlay with this kind of color scale.

With the color scales ready, there was nothing else left to do other than getting started with the color (generating wind overlay).

First attempt to add the wind overlay

I ended up going through 2 failed attempts to efficiently add colors to the wind map before finding the 3rd one that I ended up using. Each of the three approaches worked, but the first two weren’t efficient enough, and they made the visualization look choppy and unappealing.

The first attempt was a quick and dirty scan through all the pixels on the chart. I would check whether the pixel was in the visualization. If it was, I would look up its coordinates, estimate the wind strength at that point using Bilinear Interpolation, look up the color, and finally plot the pixel. The process could be described in the following pseudocode:

for x in range(0, width):

for y in range(0, height):

flag1 = check_if_x_and_y_is_in_projection(x, y) if not flag1:

color = black longitude, latitude = calculate_coordinates(x, y) wind_strength = calculate_wind_strength(longitude, latitude) color = color_scale(wind_strength)

This worked, but the rendering was taking ages. Generating the layer during the zoom or drag was simply impossible and it had to be turned off. Even using the super-efficient color scale would not help. You can see the delays in real-time on the gif below:

Generating the wind overlay is inefficient

I tried to figure out how Cameron Beccario was capable of implementing his wind overlay using a very similar approach and yet have it run efficiently. I wasn’t able to get to the root of it, but one thing that I noticed was that he would look up every other pixel (therefore reducing the time needed to render by half).

I did not want to be forced to turn the colors off when dragging or zooming. I knew that there must be a way to achieve it and I moved to the next step.

Second attempt to add the wind overlay using raster reprojection on Canvas

The logic behind this one was to generate a 2D wind overlay once and save it as an image. Then, on an update, I would call raster reprojection — that is transforming a map encoded in raster image (like .jpeg or .png ) to a different map projection (in our case, the orthographic globe).

The assumption here was that the reprojecting of an image would be less computationally heavy than regenerating. This turned out to be true. But still not quick enough to allow smooth dragging and zooming. You can see the choppiness yourself on the quick prototype below: