How to Create a Simple Yet Effective Scatterplot
By: Jonathan Dunne
Tags: Data Science, Charts, Excel, Programming, Data Visualization

How to Create a Simple Yet Effective Scatterplot

How to answer the question, “Do I have a correlation, linear or otherwise?”

Introduction

Back in January, I spoke about how to create a simple yet effective bar chart. In this second installment of my “How to” series, I want to focus on another common chart type: the scatterplot.

A scatterplot is a useful way to determine the relationship between two variables. Scatterplots display numerical values across an x and y-axis using the cartesian coordinates of observations with a dataset. Scatterplots can also be used to display observations across multiple categories or groups. It is this type of use case that makes scatterplots a popular chart type.

Figure 1: An example of multi-group scatterplot showing statin dose with the drop in high-density lipoprotein cholesterol.

Figure 1 above provides an example of a scatterplot. Data was measured for the level of statin administered to a patient along with the increase or decrease of high-density lipoproteins (HDL). The observations themselves are typically represented by a circular dot.

The purpose of the chart is to understand if there is an underlying relationship (correlation) between statin dose and adjustment in HDL. Furthermore, by colour coding the different groups, it helps provide a measure of diffusion within each group.

The origins of the scatterplot date back to the 1830s. According to Friendly and Denis [1], one of the first illustrations of a scatterplot was presented by Sir John Frederick W. Herschel for his paper “On the Investigation of the Orbits of Revolving Double Stars.” Herschel was a polymath and used his multidisciplinary skills to track the angle of interstellar bodies over time. An example of his chart is shown below in Figure 2.

Figure 2: Herschel’s first scatterplot showing the angle of observed satellite data over time. (Source: [1])

In this article, I will describe step by step how to create a simple yet effective scatterplot. Once again, I will use Microsoft Excel to explain the process. However, I will provide source code for both Python and R using Bokeh and Plotly, respectively. I should add that the techniques I describe are not tool specific, but can be applied across any dataviz framework.

All examples and code can are available at my GitHub repository here. All code can be used freely as part of the general license agreement.

Datasets Used

Where possible, I use datasets that are in the public domain; the benefit of such an approach is that these datasets are open and accessible. It allows dataviz practitioners to collaborate with like-minded individuals, teams and especially you the reader.

For the worked example in this article, I will use the Titanic dataset. This dataset contains nontechnical data. Therefore readers can focus on chart construction rather than developing domain knowledge of the dataset.

Before we begin, a small technical note on the dataset. There are 891 records with the dataset. However, for passenger age, there are 177 missing values. When dealing with missing values in a dataset, one can remove the rows within a CSV file, or fill in the missing values using a technique called imputation. That is to say, using the median of the available passenger ages (calculated as 28) and replace the blank passenger ages with a value of 28.

Let’s address some questions that concern this dataset using a scatterplot as a means of visual storytelling.

Scatterplots

In this section, we ask two questions. First, what was the relationship between the fare paid by passengers and their age? Second, can we group this relationship by whether the passenger survived or perished?

Step 1 — Create the chart.

To create the chart, I arranged the data into six columns. I took the original Survived column and replaced the ‘1’ and ‘0’ with lived and survived labels. Next, with some judicious copy and pasting, I arranged the age and fare paid by each passenger according to their survival status. Table 1 below shows the final data arrangement. Finally, don’t forget to impute the missing passenger ages with the median value.

Table 1: Arrangement of Titanic passenger data for the scatterplot

Select the six columns for plotting, and click:

Insert -> XY (Scatter) -> Scatter.

Next, we need to assign the correct values to each survival category. Right-click the newly created chart, click:

Select Data -> Select the ‘Fare’ series in the Legend Entries (Series): box and click the minus button.

This action will remove the series. Repeat for the ‘Age’ series.

Press the plus button under the Legend Entries (Series): box and add the following text for the Name, X values and Y values respectively;

=”Lived”, =’Scatter Plot’!$B$2:$B$343 and =’Scatter Plot’!$C$2:$C$343.

Click the plus button once more and add the following text for Name, X values and Y values respectively;

=” Died”, =’Scatter Plot’!$E$2:$E$343 and =’Scatter Plot’!$F$2:$F$343.

Finally, click the ok button to create an initial scatter plot.

Figure 3: An initial scatterplot showing the relationship between passenger age and ticket price

Figure 3 above depicts an initial scatterplot. The two colours used are default scatterplot colours (Orange Hex code ED7D31, Blue Hex code 4472C4)

It is reasonable to say that this scatterplot has a number of drawbacks. Over the next seven steps, we can employ a series of techniques to increase the impact of our scatterplot.

Step 2 — Adjust Chart Aspect Ratio.

In my previous article on bar charts, I discussed the prior research that has been conducted into what the optimal ratio of a charts hight relative to its length.

To recap; William Playfair, the inventor of the bar, line and pie chart, preferred an aspect ratio between 1:1.4 to 1:1.8 [2]. Leonard Zusne, in his book Visual perception of form, notes “experiments in graphical perception; viewer responses were found to be highly context-dependent.” [3]. Furthermore, Edward Tufte (an expert in Data Visualisation) mentions “If the nature of the data suggests the shape of the graphic, follow that suggestion.” and “Otherwise, move toward horizontal graphics about 50 per cent wider than tall.”

The function of a scatterplot is to display the relationship between two variables. However, if the width of a chart is longer than the height, any relationship between the two variables may be exaggerated. To keep this distortion to a minimum, I suggest an aspect ratio of between 1:1 and 1:1.2 for scatterplots.

To obtain the aspect ratio proportion mentioned above, I experimented with a series of different chart heights and widths. I found that, as the length of the chart becomes longer (and the char height stays the same) the distance between the plotted dots becomes wider.

To adjust, the chart aspect ratio, double click the chart -> In the Format Chart Area, select Size and Properties -> Adjust the Height value to 18cm and the Width value to 21.6cm. These values correspond to an aspect ratio of 1:1.2. Figure 4 below illustrates the resized chart. It is also possible to change the aspect ratio of a chart using the Scale Height and Width control from the Size and Properties control. For example, to adjust a scatter plot aspect ratio to 1:1.2, leave the Scale Height at 100% and adjust the Scale Width to 120%.

Figure 4: Our scatterplot resized with an improved aspect ratio.

Step 3 — Add a Title.

To add a chart title, select the title text a the top centre of the chart. Provide an explanatory title such as “Relationship between Passenger Age and Fare Price, by Survival Group”. The default font size for this chart is size 14. If you find this text too small, adjust the size to your personal preference. Take note, with font sizes above 16; the text size will wrap to a second line unless you adjust the width of your title text box. Figure 5 below shows our scatterplot with an updated title.

Figure 5: A basic scatterplot, with a title added.

Step 4 — Mute or Remove Gridlines.

The use of gridlines is an individual choice. For some charts, it makes sense to use them, for other chart types, the use of gridlines, merely adds to the noise to ink ratio. That is to say, the use of gridlines may distract the viewer from the key takeaway of a chart.

To delete the gridlines within a chart, select the horizontal gridline and press the delete key (Windows) or Fn+backspace (Mac). To mute gridlines, select the chart gridlines, and double click. The Format Major Gridlines control panel should appear on the right-hand side of the screen. Adjust the width from 0.75pt (default value) to 0.25pt. Next, change the dash type from the continuous line (default value) to round dot (or another dot, dash type of your preference). You should end up with a chart similar to Figure 6 below.

Figure 6: A basic Scatterplot, with a title added and gridlines muted. The title begins to make the chart content more explanatory.

The chart lines are visible but are in the background. However, they enable the viewer to discern the coordinates of a data point more readily.

Step 5 — Add Axis Names.

At the halfway stage, our chart is beginning to take shape. To make the chart more understandable, we shall add a horizontal and vertical axis name. To add axis labels, select the chart -> Select Chart Design -> Add Chart Element -> Axis Titles (Primary Horizontal & Primary Vertical). Next, provide an axis title that clarifies rather than invites more questions. A useful rule of thumb is to use a variable name then add the unit of measurement in brackets.

When dealing with a unit of currency (Dollar, Euro, Pound), denote the currency as a symbol or words. Words may be more effective for less well-known currencies. Additionally, ensure the axis name is readable. The default font for axis names in Excel is 10pt. I have increased the size to 14pt for legibility. Figure 7 below illustrates the updated chart.

Figure 7: A basic Scatterplot, with a title added and gridlines muted and axis labels added.

Step 6 — Position the Legend.

For folks following closely, you could see that Excel added the legend when the chart was created, which means we don’t need to add a legend. However, the default position was to the middle bottom right.

There are two challenges; first axis labels generally sit within the middle of the axis line (you’ll notice I had to move the legend to make it visual in our previous step); Second, if a user has to frequently move their eye gaze downward to the bottom of the chart to view the group labels. For two groups, there is a small degree of context switching. For more than two groups, the switch is more significant.

To fix this problem, we can position the chart within the chart space itself but leaving a distance as to not distract the viewer. To accomplish this task, simply drag the legend to a location that is viewable, with little to no ‘gaze switching’.

You can see in Figure 8 below, I have placed the legend within the coordinates of 400 to 500 (X-Axis) and 50 to 60 (Y-Axis). The default font size for a legend is 9pt. I have increased the point to 10pt for clarity.

Figure 8: Our Scatterplot with a legend positioned for clarity.

Step 7 — Adjust Dot Colours.

To tell an effective story through visual means, we know it is important to highlight the data points that convey a story effectively. In this exercise, we want to explore the relationship between the fare and age of titanic passengers, and we also want to understand this relationship between the groups of passengers that survived and those that perished.

Let us consider the power of visual perception briefly. Stephen Few, a leading author in the realm of Data Visualisation, writes “The degree to which your data presentation effectively and efficiently communicates depends on how well you tap into the power of visual perception” [5].

Using the technique of preattentive attributes, we can highlight specific areas or regions of a chart to provide an “Ah-ha” moment. A preattentive process is a process that happens within 200ms after viewing a visual medium (e.g. Our Titanic Scatterplot). Chart areas viewed within the 200ms have a greater likelihood of being stored in one’s memory.

To adjust the scatterplot dot colours, click an existing data point related to the “Lived” group (Note: not all dots will be selected, but the entire fare and age columns adjacent to the Lived column will be). Click the Marker button, then expand the Marker Options section. Select Built-In, leave the Type as circular and adjust the size from 5 to 8pt.

Next, expand the Fill section, ensure Solid Fill is selected and adjust the colour. I have chosen a soft grey (colour code BFBFBF). Finally, expand the Border section. Ensure Solid Fill is selected and change the colour. I’m going to use a darker grey to provide a subtle definition to the dot (colour code 7F7F7F).

We now repeat the process for the “Died” group. As the process is the same, I won’t repeat the steps. To provide a contrast between the two groups I picked (Hex colour code 3A6587) as my primary fill colour, and chose (Hex colour code 002060) as my border colour.

On the one hand, the choice of colours can be an individual / personal choice relative to the question we want our chart to answer. On the other hand, the colours chosen should provide a degree of contrast to allow chart viewers to distinguish between the two groups [6]. Figure 9 below illustrates the updated chart.

Figure 9: Our Scatterplot with added preattentive attributes

Step 8 — Adjust Legend Colours.

Our final chart enhancement is one of the most subtle yet effective changes one can make. Before we discuss the adjustment, let us consider the theory behind it briefly.

In the 1890s, the Gestalt School of Psychology research determined that the human mind tends to perceive objects as part of a greater whole and as elements of more complex systems [7]. To that end, six principles were developed: proximity, similarity, enclosure, closure, continuity, and connection. To provide more detail on each of these principles is beyond the scope of this article.

However, the principle of similarity states that things that share visual characteristics such as shape, size, colour, texture, value or orientation are perceived as being related. To increase the relationship between the chart and the legend, I will colour the legend text to the same colours as my columns. The benefit is that the chart viewer perceives the similarity between the chart colours and the legend text.

To adjust the First, Select the “Died” legend item -> Select Font Colour (From the Main Menu) to match column colour to text colour (Hint: Use Hex colour code 3A6587). Next, repeat the same step for the “Lived” legend item (Hint: Use Hex colour code 7F7F7F). Figure 10 below provides an example of the updated chart.

Figure 10: Our Scatterplot with the Gestalt Theory principle similarity added.

Chart Insights

Before we conclude, I shall discuss briefly some insights provided by both the data and the adjusted chart. For the most part, we can see that passengers that died paid under 20 (the mean fare was found to be 22) pounds per ticket. Additionally, that deceased passenger age varied from 14 to 45 years, generally speaking.

Conversely, passengers who survived have a broader dispersion of fare prices. While there are several low price fares, the chart suggests a significant proportion of ticket prices above fifty pounds (the mean fare was found to be 48). For survivors, we can see a small cluster of young people. Additionally, one can see a wider dispersion of age ranges between survivors aged eighteen through sixty years of age.

Code Corner

For Dataviz practitioners that use Plotly or Bokeh, I have provided a working chart in both R and python. The good news is that the chart parameters can be reused will little to no adjustments. However, there is a limitation with both Plotly (for R and Python) and Bokeh as of the time of writing; neither framework provides the ability to change the colour of the legend text as discussed in step eight.

However, it is possible to adjust the colour of the legend text using plotly.js and D3.js. I don’t currently have javascript flavours of my chart available in my Github repository at the moment. I do plan to add charts using D3, in due course.

Conclusion

Currently, we don’t have chart applications that provide the best of breed visual techniques. However, it is possible to create these charts yourself employing eight systematic steps.

As we can see from the step by step approach, there isn’t one distinct step that provides the viewer with that “Ah-ha” moment. It is the combination of subtle adjustments incorporating Gestalt theory and preattentive attributes that provide charts with key insights.

In the next article in this ongoing series, I shall discuss how to make a line chart more effective.

References

[1] Friendly, Michael, and Daniel Denis. “The early origins and development of the scatterplot.” Journal of the History of the Behavioral Sciences 41.2 (2005): 103–130.

[2] Playfair, William. Playfair’s commercial and political atlas and statistical breviary. Cambridge University Press, 2005.

[3] Zusne, Leonard. Visual perception of form. Academic Press, 1970.

[4] Tufte, Edward R. The visual display of quantitative information. Vol. 2. Cheshire, CT: Graphics Press, 2001.

[5] Few, Stephen. Tapping the Power of Visual Perception, 2004

[6] Ware, Colin. Visual thinking: For design. Elsevier, 2010.

[7] Behrens, Roy R. Design in the visual arts. Pearson College, 1983.