Covid-19 Interactive Visualization
By: Abhishek M
Tags: Covid 19, Python, Visualization, Data Science, Dashboard

Visualizing covid-19 data to understand the impact it had on the American population.

The novel corona virus has caused more than 200000 deaths worldwide.

In this post i’m going to walk you through how to build an interactive dashboard of your own using only python tools and api’s and publish it on a provider like binder or heroku.

First off, the raw dataset is available and provided by Johns Hopkins and can be found on their github page. This is focused only on US data but the same can be extended to other countries as well.

Get started by making sure you have python, Jupyter notebook installed and ready to go. I’ll be detailing only a few of the operations performed. Go ahead and try all of it by yourself :) or find the entire project on my GitHub.

You will need to install the following python files in order to perform analysis and visualize your data.

import pandas as pd

import matplotlib.pyplot as plt

from IPython.display import Markdown, display

from datetime import datetime as dt

from ipywidgets import interact

import chart_studio.plotly as py

import plotly.graph_objs as go

from plotly.offline import iplot, init_notebook_mode

import cufflinks

from plotly.subplots import make_subplots

cufflinks.go_offline(connected=True)

init_notebook_mode(connected=True)

import dateutil

import plotly.express as px

pandas — To read the data and create subsets of data using dataframes.

matplotlib — Python’s go to visualization library.

datetime — Intuitive library to deal with time series data and works well with dataframes.

ipywidgets — Creating widgets in Jupyter to make the dashboard more interactive.

plotly/cufflinks — Better and interactive visualizations.

Gather the data from the links provided in the git repo.

x1 = 'https://raw.githubusercontent.com/CSSEGISandData/COVID-19/master/csse_covid_19_data/csse_covid_19_time_series/time_series_covid19_confirmed_US.csv'

x2 = 'https://raw.githubusercontent.com/CSSEGISandData/COVID-19/master/csse_covid_19_data/csse_covid_19_time_series/time_series_covid19_deaths_US.csv'

x3 = 'https://raw.githubusercontent.com/abhim-12/Covid-19/master/population.csv'

x4 = 'https://raw.githubusercontent.com/abhim-12/Covid-19/master/state-codes.csv'

Use the pandas read.csv() function to read the data into dataframes:

conf = pd.read_csv(x1)

death = pd.read_csv(x2)

pop = pd.read_csv(x3) # US Population data

codes = pd.read_csv(x4) # US State Codes

Use the head() function to get an understanding of what the data contains, the columns, data types etc.

Perform some data cleaning (like renaming columns, dropping) to make it more coherent and presentable. Also remove any unnecessary columns.

del death['Population']

def filt(df):

df.drop(columns=['iso3','code3','Admin2','Country_Region'],axis=1,inplace=True)

df.rename(columns={'Province_State':'State'},inplace=True)

df=df.reset_index()

del df['index']

The first dataframe (z2) we create merges the confirmed cases and deaths frames State-wise. Next we create a few columns to infer based on the data the severity of the situation in each state.

cols = conf.columns.to_list()[-1]

ctemp = conf.groupby('State').sum()[conf.columns[-1]].sort_values(ascending=False).reset_index()

dtemp = death.groupby('State').sum()[death.columns[-1]].sort_values(ascending=False).reset_index()

ctemp = ctemp[['State',conf.columns[-1]]]

dtemp = dtemp[['State',death.columns[-1]]] # State level aggregated data

z2 = pd.merge(ctemp,dtemp,on=['State'],how='inner').merge(temp2,on='State',how='left')



z2.rename(columns={deaths:'Deaths',confirm:'Cases'},inplace=True) z2['CMR'] = (z2['Deaths']/z2['Population'])*100000 # Crude Mortality Rate per 100000

z2['Infected %'] = (z2['Cases']/z2['Population'])*100 # Percentage of population infected z2['Case Fatality Rate'] = (z2['Deaths']/z2['Cases'])*100 # Case Fatality Rate - Ratio of Deaths out of those infected

z2 = z2[['State','Cases','Deaths','Population','Infected %','CMR','Case Fatality Rate']]

Crude Mortality, Case Fatality Rate and %Infected are calculated and i chose to highlight the states where CFR was greater than the average. The result looks something like this (Find more about styling dataframes here).

Styling the data also helps convey information better, using any method you like.

Our first visualization is how the virus has affected the country as a whole, so we do 2 visualizations — 1)A timeline of impact and 2)A choropleth plot

def timeline(Condition):

tempd = cc.groupby('Date').sum().reset_index()

temph = tempd.loc[tempd['PerDay_Confirm']>0].reset_index(drop=True)

if Condition=='Deaths':

title = 'Timeline of Deaths'

color='red'

if Condition=='Cases':

Condition='Confirm'

title = 'Timeline of Cases'

color='orange'

temph.iplot(x='Date',y=Condition,xTitle='Dates',yTitle='Count',title=title,mode='lines+markers',color=color)

interact(timeline, Condition=['Deaths','Cases'])

So the function defined above generates a time series plot of the impact of the virus. The iplot package allows for plotting interactive plots combined with jupyter widgets. The function interact takes as parameter - the function name, and a variable Condition which is a list containing ‘Cases’ and ‘Deaths’. The plot generates a drop down list where the user can switch between Cases and Deaths across time. The plotly package makes our plots interactive showing us the numbers at each data point without any additional code.

Interactive Time series plot of Impact of Covid-19

Another interesting visualization you can do is to animate the plot showing the effect on each state day by day.

def timeline2(Condition):

fig = px.choropleth(tempz,

locations='Code',

color=Condition,

hover_name='State',

locationmode='USA-states',

animation_frame='Date',

scope='usa'

)

fig.update_layout(

title_text = 'Spread of Coronavirus',

title_x = 0.5,

geo=dict(

showframe = False,

showcoastlines = True

)) fig.show()

interact(timeline2,Condition=['Confirm','Deaths'])

Another intermediate data frame i created contains a merged version of the deaths and confirmed cases each day in each State (State-Day Level).

def transform(df,s): #s is a String with 'Confirm' or 'Deaths'

# To dcast the data and get it at a state-date level



temp = df.copy(deep=True)

temp.drop(columns=['UID','iso2','Lat','Long_','FIPS'],inplace=True)

temp = temp.melt(id_vars=['State','Combined_Key'],var_name='Date',value_name=s) # Source data is cumulative, this gets the numbers per day



temp2 = temp.groupby(['Combined_Key'])[s].diff().fillna(0).reset_index()

p = 'PerDay'+'_'+s

temp2.rename(columns={s:p},inplace=True)

temp2 = pd.concat([temp,temp2],axis=1)

del temp2['index']

temp2 = temp2.sort_values(['Combined_Key','Date',s,p]).reset_index(drop=True)

return temp2

The function is called on each individual dataset(conf & death) and the result is merged to create the output as below:

Now you can perform numerous visualization on the above data frame. One such visualization i performed was displaying the Cases or Deaths Per Day in a Province in a State.

zz['Province'] = [s.split(',')[0] for s in zz['Combined_Key']]

province = zz['Province'].unique()

# Plots Cases/Deaths for a Province per Day

def plots2(State,Province,Condition):

a = zz.loc[(zz['Province']==Province) & (zz['State']==State) ]

if Condition.startswith('C'):

Condition = 'PerDay_Confirm'

color = 'orange'

if Condition.startswith('D'):

Condition = 'PerDay_Deaths'

color = 'red'

a = a[a[Condition].gt(0)]

a = a.sort_values('Date',ascending=True)

a.iplot(x='Date',y=Condition,kind='bar',xTitle='Dates',yTitle=Condition.split('_')[1],title=(Condition.split('_')[1] + " in " + Province + ", " + State),color=color)

interact(plots2,State=z2['State'].sort_values(),Province = province,Condition=['Cases','Deaths'])

You can perform numerous similar plots using the examples i have shown. Moving on to how to create a dashboard out of your notebook.

Voila — voila is a python library that works on top of jupyter to transform your notebooks into dashboards. Installing and running voila is fairly straightforward and instructions are provided on the github page.

Running voila from cmd : voila \path\to

otebook

otebook.ipynb — theme=dark

Your interactive will now look something like this :

Voila dashboard

Now that your dashboard is complete time to deploy it. As i said earlier you can use heroku or binder. Both require you to push your data to git (make sure you have git installed).

Heroku provides a cli to push your projects. You can find more details here. The voila documentation provides clear steps to deploy projects.

And “voila!” you’re done. You have created a dashboard entirely in python without requiring external software and a dashboard that can be shared with your peers, colleagues or others. Add more components use the template for future projects as you see fit.