Using Bitcoin Data in Python. Getting Started with the Bitcoin... | by Stephen Lewis | Towards Data Science

Bitcoin and cryptocurrencies, in general, have been on the up in recent months. From the sudden crash in March, Bitcoin has responded very well and has since almost doubled in price. Add to this the much-hyped halving event earlier this month, and there are now a lot of people curious and looking to start investing and trading.
Well, here I present the Python code you need to start analysing Bitcoin and get it into a usable form. I will hopefully demonstrate just how easy this is in Python using the pandas library. The full code can be found at the bottom of the article.
If you are not interested in cryptocurrencies then this might also be a good tutorial demonstrating the power of Python and pandas in data manipulation and analysis and might have some useful snippets of code to apply to other projects.
To get started, first, you will need to install the pandas library if you don’t have it already. Simply run the following in the command line.
pip install pandas
Or if you are using Anaconda like I am, you can open the Anaconda terminal in the environment you are using and then enter the following command.
conda install pandas
If no errors occur you should have the library installed correctly. We will do a final test in a minute.
For data analysis, I find Jupyter Notebook to be the best IDE as it allows you to run snippets of code in blocks and therefore can be a great time saver when importing and manipulating large datasets. However, any IDE will do.
Open a new script and save it. The first task is to test if the pandas library is installed correctly. Run the following code to import the library into your script. If no errors occur then you can proceed with the rest of the tutorial. If not, you will have to get the pandas library installed correctly — google will be your friend here.
import pandas as pd
The data I will be using is from Kaggle which is the worlds largest data science community and one of the best places to find a huge variety of datasets. This particular dataset is historical minute data of Bitcoin from 2012/01/01–2020/04/22.
Once you have downloaded and saved the csv file into the same directory as your script, you will now want to load that data. To do this we use the pd.read_csv() method in pandas, passing the data filename as a string. This creates a data frame with all the data from the csv file.
We will also run the .head() method on the data frame which shows us the first 5 rows of my data. As you can see, there is one row of data followed by four successive rows of NaN values. This is because some data is missing and therefore we have some cleaning up to do. This is due to a number of reasons but it seems that early in this dataset the data was not archived every minute so large chunks are missing.
raw_data = pd.read_csv("bitstampUSD_1-min_data_2012-01-01_to_2020-04-22.csv")raw_data.head()
Running the .tail() method on the data frame will shows the last 5 rows and you can see that these rows are in fact full. Also, note the number of rows in this dataset from the data frame index column (far left). Over four million! Way too many for Excel but Python and pandas handle it with ease.
raw_data.tail()
So now we want to create a data frame with all the rows with NaN values in them removed. In pandas, this is incredibly easy and we just use the .dropna() method on our data frame and assign that to a new data frame we call df. This is the data frame containing all the useful data that we will use in our analysis going forward.
We also call the .reset_index() method on the data frame so that the index column is reset. This assigns an index value starting at 0 for the first row of data, 1 for the second row and so on. We will also pass the argument inplace=True. This is commonly used in pandas and what it does is it tell Python that we want our original data frame to become the new edited version of that data frame. If inplace is not set to True then the data frame we are working on will be unchanged after the line is run. Explore this behaviour using the .head() method and looking at the index values.
We also pass a second argument drop=True. By default, the .reset_index() method will reset the index of our data but the original index column will be copied to a new column called “index”. We have no use for this additional column and so we will drop this. This could also be achieved using the .drop() method on the data frame and specifying the column name to drop.
df = raw_data.dropna()df.reset_index(inplace=True, drop=True)df.head()
You can see that the ‘Timestamp’ column is in a weird format, this is in fact the UNIX time format and so we would like to convert this to UTC time. We do this using the pd.to_datetime() method. We will pass the arguments for the data we want to perform this function on, which is the ‘Timestamp’ Column of our data and also we will pass the argument unit=’s’ which tells the function that the units of our inputted UNIX times are seconds. We save the output of this function to the ‘Timestamp’ column in our data frame using the line of code below. Note that in pandas to reference the data inside a column, we use the following notion df[‘Timestamp’]. Where the string passed inside the square brackets is the name of the column.
df['Timestamp'] = pd.to_datetime(df['Timestamp'], unit='s')df.head()
We can see here that part of the dataset is actually from 2011. For completeness, I will remove these four rows. The easiest way to do this when there is only a few rows to remove is to simply specify the indices of the rows in a list and pass this into the .drop() method. We also have to pass in the argument inplace=True again. And then finally fix the index like we did before.
df.drop([0,1,2,3], inplace=True)df.reset_index(inplace=True, drop=True)df.head()
So now we have our data in a nice structure and hopefully all the data in our data frame. A very quick check to verify that everything looks good at this point is to simply plot the price of bitcoin over time. We will do this by
running the .plot() method on the data frame and then specifying the columns we want to be on out x and y axes. The .plot() method in pandas actually uses the matplotlib library in the background to display the graph and will try to choose sensible default settings for the axes. The graph can of course be customised if you wish.
df.plot(x="Timestamp",y="Weighted_Price")
Great! So our data looks to be correct and therefore we can proceed to the more interesting part which is to analyse or manipulate the data. One thing that you might want to do is to create a new column that processes the data in some way. An example would be to find the difference between each successive point in time.
We can do this using a method built into pandas called .diff(). We simply have to specify the column we wish to create using the following notation df[“Title of new column”] and set this equal to the column we wish to find the difference of with the .diff() method.
df["closeDiff"] = df["Close"].diff()df.head()
Notice the new column we created called “closeDiff”.
Now lets imagine that we want to create data frames for each of the years in our data. For example, a data frame df19 for all the data from 2019. One way to do this would be to first create a column called ‘year’ that contains values of the year at each data point. And then we will create a new dataframe containing only the rows of data where the ‘year’ column equals 2019.
To do this we will first create a ‘year’ column. We feed our ‘Timestamp’ column in to the pd.DatetimeIndex class and ask for the .year attribute. This returns the value of the year for each of our time stamps and we assigned this data to a new column that we define as ‘year’.
df["year"] = pd.DatetimeIndex(df["Timestamp"]).yeardf.head()
So now we want a new data frame df19, which contains all the rows in our dataframe df, that have the value 2019 in the ‘year’ column. To do that we write the following line of code. This notation can be slightly confusing but here we simply are saying, get all the rows with the data frame df where the ‘year’ column equals 2019 and assign it to a new data frame called df19.
df19 =  df[df["year"] == 2019]df19.reset_index(inplace=True, drop=True)df19.head()
Now we can plot just like before the data but this time just for the year 2019.
df19.plot(x="Timestamp",y="Weighted_Price")
So now you have clean data and perhaps you have added to the data or made a subset. It is common that you want to save this data so then you don’t have to run this code again. This is very easy using the .to_csv() method on the data frame. Below is the code I use to save the df19 data frame so I can use it easily in the future. The first argument you pass to the method is the filename you wish to save it as and the index=False tells Python not to save the index of the data frame to a column in the csv file which is usually unwanted.
df19.to_csv('btc19.csv', index=False)
There you have it, a basic framework to get you started using this Bitcoin historical dataset along with some basic things that you can do in Python and pandas. The full streamlined code without the .head() and .tail() calls can be found below.