Six Chart Design Lessons from Visualizations of COVID-19
By: Amanda Makulec
Tags: Evaluation, Covid 19, Public Health, Design, Data Visualization

A month later, I still couldn’t get a clear definition for what qualified as a “recovered” case, even asking colleagues working at the CDC. The Morbidity and Mortality Weekly Reports (MMWRs) had started to contain early learning about the novel coronavirus and the growing case counts in the US. I felt alarmed when I saw that Tableau had launched a ‘ready to use’ workbook for anyone to jump start their analysis of the case data and wrote 10 Considerations Before You Create Another Chart about COVID-19, which still has relevant lessons months later.

Now, I’ve delivered nearly 50 talks, written articles, and given podcast interviews about the messiness and challenges of visualizing COVID-19 data, thanks to my intersecting public health and dataviz expertise. I collaborated with Tableau as a public health data advisor on the COVID-19 Resource Hub to share useful context and information for analysts from a public health perspective.

The US is still hungry for information, though I know many of us are just fatigued at this point. Maybe some are hoping for a glimmer of a number that will give some hope of a return closer to ‘normal’? But instead, we’re peaking in a third wave as I write this article.

What have we learned as the charts evolved?

Have we ever watched visualizations on a topic evolve in real time on a topic with such a personal impact to our lives? The only parallel I can think of is visualizing polls and elections, but seldom are those charts updated for six or more months with significant changes made to the design.

In the midst of a global pandemic, I’ve watched visualization designers grapple with incomplete datasets — particularly in the early months — even John Burn-Murdoch at the Financial Times, an outlet widely praised for their data journalism in the pandemic, was putting out tweets asking for insights about different countries’ data gaps.

Often, visualizers were learning about the complexity of epidemiology in real time — or, in the worse cases, spinning up dashboards and charts of this trending news topic without much consideration of those complexities. At the same time, public health professionals were managing a pandemic response without clear national guidelines and leadership here in the US.

Charts, maps, and graphs are powerful tools for communicating information about current events like the COVID-19 crisis, and the world has looked to visualizations for answers. Ben Schneiderman, a leader in the data visualization field, called this “data visualization’s breakthrough moment.” Simple is often effective, with many small multiple tables and line charts packed with data. But all of that simplicity can mask complexity.

As researchers and evaluators, the American Evaluation Association (AEA) community understands the challenges of communicating complex information in easy-to-understand visualizations (this blog on visualizing regression results from Stephanie Evergreen comes to mind). While your evaluations and studies may not be COVID-related, you have likely faced some of the same design challenges as those charting the pandemic.

Lessons that evaluators and researchers can apply to their own visualizations

The journey through the COVID charts of 2020 has important reminders for any of us creating visualizations in the social sciences. Here are six lessons you can apply when visualizing your research results.

1. Visually display uncertainty, sample sizes, and unknowns.

When we create visualizations, we imply a certain degree of factfulness and objectivity. That creates a challenge when visualizing incomplete, messy datasets. Matthew Kay conducted a tour on uncertainty visualization research at Tapestry in 2018, and paired with colleague Jessica Hullman to talk about how these principles connect to visualizing COVID in April 2020. I highly recommend watching both talks and following Multiple Views to learn more about the nuances of the different marks and visual encodings used to alert readers to uncertainty in a chart.

Visually displaying uncertainty in some encoding on your chart, rather than burying it in a footnote or accompanying text, is particularly important today.

Take the example below of a case trend chart from the Georgia Department of Public Health. The mark used to plot the data from the last 14 days, which is likely reporting incomplete information, changes from a bar to a dot, and there’s a lightly-shaded box over the window of uncertainty, reminding readers that data from that period are likely incomplete (which is further explained in the accompanying text). While there has been much to criticize about Georgia’s COVID dashboards, this chart provides more context than many others with similar information.

Source: Georgia Department of Public Health on October 26, 2020

Many early analyses and studies of COVID-19 drew from limited data available, with results summarized in MMWRs and pre-prints. Even today, as we grapple with gaps in detailed, disaggregated information for specific populations like schools, small denominators and convenience samples can limit the generalizability of findings.

We should be transparent about those limitations in the titles and sub-titles of charts — or risk they be taken out of context. For example, when making sweeping statements like, “Schools aren’t Superspreaders,” the authorburied that data was sourced from a small cohort of students in a convenience sample of schools. Further, we should carefully consider the language we use and avoid stigmatizing those infected. (Guidance on language choice is more detailed and nuanced then I’ll cover in this post.)

2. Enable understanding with reference lines and bands.

Many of the metrics you need to track and report on COVID-19 were unfamiliar to most Americans last year. What is test positivity, and why is it better if the number is low? What is R0? Adding clear reference lines to charts can aid in interpretation. In the example below from COVID Act Now, the physical reference lines also provide additional severity encoding and color, which can aid accessibility.

Chart from COVIDActNow.org sourced on August 5, 2020. For more details on test positivity and the complexity of this metric, see the COVID Tracking Project.

You can also add marks to add important dates like when key policy actions were taken or other visual representations that help readers answer why something happened and what the data means. Consider how you can aid understanding within the chart, and not just solely relying on sharing your summary findings in the accompanying text.

3. Add thoughtful annotation layers.

In addition to reference lines, annotation layers with short text statements further explaining individual data points can help readers quickly make sense of key pieces of information. The text may include direct labeling of individual data points, adding precision where it is needed, and allowing the shape of the marks on the chart to otherwise communicate the broad story.

Data journalists have excelled in their use of annotation layers and key callouts to communicate changes in the global and local progression of COVID-19. The trackers from the Financial Times’ John Burn-Murdoch explained daily updates on country progressions, accompanied by Twitter threads calling attention to more insights or data gaps. Instead of wondering what to look at or where to focus our attention, annotation layers give us context clues.

Source: Twitter from March 11, 2020

Other annotation layers and descriptive text enable us to explore a chart more effectively, as on Our World in Data. We can borrow from these approaches as researchers by adding these kinds of key callouts in our charts, particularly for dissemination to audiences who may not have all of our context.

4. Anticipate opportunities for misinterpretation and design with them in mind.

How will someone new interpret your chart or graph? In the rush to create infographics communicating risk about COVID-19, we saw a number of misguided comparisons made between early COVID deaths and the flu or other diseases that understate the severity of the illness and the necessity of taking proactive public health prevention measures. Pre-prints of new research on COVID-19 were scoured for new insights by journalists and others, with numbers from tables or charts sometimes taken out of context or without the necessary framing of limitations.

If you’re communicating research findings to a wide audience, seek feedback on your chart before you publish. How would someone without any subject matter expertise interpret your graph? What number would they focus on in the table? And what conclusion might they draw?

When communicating research about key public health issues or other topics that motivate individual action, add clear information on where the reader can find more details and learn more about a given topic whether to better understand the science behind a behavior change recommendation or to view the original data source. For COVID-19, this could include the clear guidance to wear a mask, physically distance, and wash hands frequently with a link to the CDC website on prevention.

5. Collaborate with subject matter experts.

Evaluators bring a deep toolkit of research methods to the table, but don’t always evaluate programs within their primary area of expertise. Widely engage subject matter experts in the research design process — they can provide important insights and context in the dissemination phase.

Today, we’re further challenged to decouple our own experiences from the impact of new research about COVID-19. It’s easy to have confirmation bias and quickly click retweet when a chart confirms our beliefs about a prevention measure like mask wearing. It’s easy to misinterpret a model.

Some of the most illuminating visualizations of COVID-19 have resulted from the collaboration of experts in public health and data visualization. Visualization designers leverage their knowledge of how to communicate data clearly and effectively, and health experts ensure the visualizations communicate accurately and with appropriate language. These collaborations have extended into some exceptional data simulations, like the People of the Pandemic game.

6. Remember the people behind the numbers.

Perhaps more important than any other lesson from visualizing the pandemic: remember that behind these numbers are people. Giorgia Luipi writes in her Data Humanism Manifesto that ‘data is people’ rather than just numbers, which carries far beyond visualizing public health data.

The more we aware we are of how individuals see themselves or their communities represented in our graphs, the more we can avoid missteps in design and language that distract from the information we’re looking to communicate and respect the personal ways people connect to our numbers. Luipi and her team reimagined Andrew Cuomo’s slides from his COVID briefings, shifting away from austere bar charts and depicting the lives lost as individual dots reminiscent of stars in the sky. While the visuals are certainly less precise, they give us pause in a different way.

Image by Giorgia Luipi & her team at Pentagram, shared in Fast Company

In her essay on The Ethics of Visualizing During a Pandemic, Bridget Cogley lists various values we carry with us in our work, and challenges us to think about how those values change in the middle of a pandemic. If you read only one linked article in this essay, please make it this one.

Image from Bridget Cogley’s The Ethics of Visualizing During a Pandemic

I’m particularly struck by the ‘do no harm’ principle: we face an infodemic rife with misinformation as part of the current pandemic, and charts can be used not only to inform, but also to mislead. Before we hit publish on a new visualization, we must consider how what we’ve created could do harm — through misinforming, confusing, countering public health prevention recommendations, or using poorly-chosen language.

The six considerations shared here aren’t new ideas, but in a time when we’re facing an infodemic in parallel with a pandemic, evaluating the unintended consequences of our visualization and ensuring we communicate what we know (and acknowledge what we don’t) clearly has never been more critical. As we communicate research and evaluations that impact policy, future program investments and structures, and even individual actions, we should take the same care.

Amanda Makulec is the Senior Data Visualization Lead at Excella and holds a Masters of Public Health from the Boston University School of Public Health. She worked with data in global health programs for eight years before joining Excella, where she leads teams and develops user-centered data visualization products for federal, non-profit, and private sector clients. Amanda volunteers as the Operations Director for the Data Visualization Society and is a co-organizer for Data Visualization DC. Find her on Twitter at @abmakulec