Exploratory Data Analysis of Cryptocurrency Historical Data
By: Hamza Ahmad
Tags: Bitcoin, Eda, Time Series Analysis, Cryptocurrency, Data Science

The word cryptocurrency has taken the financial world by storm, and yet there is a lack of formal and open research being conducted on the data of the digital assets. This blog post intents to explore the cryptocurrency historical prices dataset available on Kaggle here.

Through the exploration and initial analysis of this data, we aim to gain a basic understanding of the trends and form the basis for the implementation of a predictive model. Our EDA will lead towards answering a number of questions regarding the data in future stages, such as:

What will the future price of the currencies be? Which currencies are more volatile and which ones are more stable? How do the price fluctuations of currencies correlate with each other? What do the seasonal trend in the price fluctuations look like? What factors affect the bitcoin and ether prices? Directional prediction of bitcoin and ether prices Actual bitcoin price prediction

Understanding the Data

The data was extracted from the historical data tab in the coin market cap

The data spans over the period 2010 - 2018

There are 2 kinds of data: Prices and a Complete Dataset

and a The Prices data contains the daily performance of 17 different cryptocurrencies

data contains the daily performance of 17 different cryptocurrencies The Complete Dataset contain mining, block and transaction details of Bitcoin

Features

The Price Data are for 15 cryptocurrencies, containing the following features:

Date: date of observation

date of observation Open: Opening price on the given day

Opening price on the given day High: Highest price on the given day

Highest price on the given day Low: Lowest price on the given day

Lowest price on the given day Close: Closing price on the given day

Closing price on the given day Volume: Volume of transactions on the given day

Volume of transactions on the given day Market Cap: Market capitalization in USD

The Complete Dataset is for the Bitcoin currency, containing 24 features related to the transactions, blocks and minings over the course of 8 years.

Prices Data

Cleaning and Pre-Processing

First we clean the data, by converting the date to the correct format and checking for null values. We observed that some data values were erroneous (identified by the character ‘-’ ). These missing values accounted for a small portion of the whole data set (763 missing values out of a total of 13902). Hence, we decided to drop the records containing missing values.

The new shape of the data set is now (12360,7).

It is also necessary to look at the outliers and decide what to do with them. We look at the outliers in each currency by running the following command:

for key in df.keys():

columns = df[key].columns.drop(["Date"]

Q1 = df[key][columns].quantile(0.25)

Q3 = df[key][columns].quantile(0.75)

IQR = Q3 - Q1 temp = df[key][columns]

len(temp[~((temp < (Q1 - 1.5 * IQR)) |(temp > (Q3 + 1.5 * IQR))).any(axis=1)])

There are 577 outliers in the dataset, but not removing them could actually be useful, allowing us to see the anomalies in the data .

Analysis

Since the data is for 17 currencies over a wide period of time, looking at the skew line plot of the data was not meaningful as there were large variations for the same features across different currencies.

We would like to see how the closing values of all seventeen currencies correlate with each other. First we decide to use pairplots to visualize this. The pairplots for Bitcoin show positive linear relationships for a number of features, such as Open and Close.

Pairplots for Bitcoin’s Features

Multi-collinearity is another big aspect we should take a look at. It can be visualized to some extent through a Correlation Matrix.

The regions colored off-white to orange indicate very high correlation values. An example of this is the nem currency and the dash currency. The closing values of these currencies have a correlation value near 0.9. To look at the relationship of such features more closely, we will look at the scatter plots between these features.

The scatter plots below support a positive linear relationship between the features that were off-white in the confusion matrix above. The values on both axes are increasing linearly for all these feature combinations.