“Covid-19”, Data Analysis From the Inception to Predicting the Uncertain Future Through Machine Learning
By: Rohan Sai . N
Tags: Data Analysis, Covid19 Crisis, Machine Learning, Data Visualization

One might consider Nuclear Warfare and a Climatic catastrophe as the biggest failure that humanity could ever experience but, People failed to acknowledge that the emergence of any Novel infectious disease has the potential to wipe out a major portion of human existence within no time. As they did in the past. Despite the intense studies on the patterns of these epidemic outbreaks, when, where and how these outbreaks trigger is out of the comprehension. Severe respiratory disease was reported in Wuhan, Hubei province, China. As of 25 January 2020, at least 1,975 cases had been reported since the first patient was hospitalised on 12 December 2019. After the phylogenetic analysis of the complete viral genome, it was found to be closely related to SARS-like virus which is related to the family Coronaviridae and initially named it as SARS-CoV-2 then later bolted down to the name Covid19. This outbreak highlights the ongoing ability of viral spill-over from animals to cause severe disease in humans.

In this Advanced Internet era, one can easily collect the data from the open-sourced communities, to understand, manipulate and to derive patterns that could lead to potentially positive insights. This blog gives you, intricate Knowledge of every aspect of thought integrated in this crisis period. This Exploratory Data Analysis and Machine Learning Implementation are based on a Johns Hopkins University CSSE Time Series Datasets.

Every Plot Shown in the following sections is from the web application I Developed and Deployed, called “The COVID19 Reconnaissance & Forecasting Web Application”. If you are arriving here, from the Application, I ensure, you will have a good read. if you are directly reading this blog, I recommend you to try out the Application, it gives you an Intuitive explanation of everything only

through interactive plots, which I’ve created using the Python Magic.

Understanding the Data and Preprocessing

Unlike the other Problems like Stock market analysis, Predicting stuff like Financial statistics from a dataset etc, you are required to have a certain level of Domain Knowledge to even begin to interpret the datasets. Covid19 Dataset is pretty straight forward to anyone to dig into the typical approaches to implement different visualisations. I chose to work with The Time Series Datasets, where each column represents a Date with a Numerical value in each cell representing the number of recorded cases that day.

Snapshot of Confirmed Dataset

Initially, in any possible Data Analysis or Machine Learning problem, No Dataset is free from scrap values, place holders, unique characters or even missing values, unless it’s Artificially Created or Preprocessed. Preliminary checks for any of the above kind, should not be ignored in any case, any sophisticated algorithm doesn’t accept uncleaned data at any cost, it throws an error almost immediately when it encounters something unfamiliar rather than the conventional numerical as it should be. At many instances, the error message is not even comprehensible enough to even trace out the problem directly to its origin, in that case, you are required to Reproduce the entire scenario once again to recreate the problem and resolve it. which is clearly a waste of time. So, as seen from the above picture we do have missing values in the Province/State column, which I imputed with a string saying ‘Unregistered’ for now, the Pythonic way of doing it will be

confirmed_df.fillna(value='Unregistered', inplace=True)

recovered_df.fillna(value='Unregistered', inplace=True)

deaths_df.fillna(value='Unregistered', inplace=True)

Exploratory Data Analysis

Data, should speak to eyes because they were the best judge of proportion, being able to estimate it with more quickness and accuracy than any other of our organs. A really good data Visualization produces form and shape to a number of separate Ideas, which are otherwise Abstract and Unconnected. William Playfair

Synopsis of the World Scenario