The Simplest Data Science Project Using Pandas & Matplotlib | by Shivangi Sareen | Towards Data Science

We try so hard to find the perfect dataset to test out and improve our skills. But the truth is that no dataset is perfect. You make it perfect or at least improve it by preprocessing the data — an essential part of any data related project.
So, just pick up any dataset and get working! I experimented and the journey was nothing short of fun and full of learning.
All of the code below was implemented in Jupyter Notebook.
I got a hold of a dataset (from Kaggle) of forest fires in Brazil, which houses the largest rainforest on Earth — Amazon. I didn’t want to be picky and so this dataset was a complete random choice.
About the data:
year is the year when the forest fire happened;state is the Brazilian state;month is the month when the forest fire happened;number is the number of forest fires reported;date is the date when the forest fire was reported
Going through the csv file (amazon.csv), you notice that some numbers are in decimal. 2.588 numbers of forest fires doesn't make sense. That's because the decimal is how thousands are formatted. So, 2.588 means 2588 forest fires. This can easily be accounted for when reading the csv file.
You’ll also notice that the month column is in Portuguese. There's an upcoming fix for that too.
When I imported the file for the first time after downloading it, I got an error: UnicodeDecodeError: 'utf-8' codec can't decode byte in position : invalid continuation byte. To fix it, I opened the csv in Sublime Text and: Save with Encoding -> UTF-8. However, this caused errors in the date column. So, I simply opened up the original csv and exported it as csv. Weird but it worked.
For this project, I set up a virtual environment using virtualenv. Check out this post for all the steps. We’re using three major libraries: pandas, matplotlib, googletrans.!pip3 install the these packages (if you haven’t already) before importing them.
import syssys.path.insert(1, './lib/python3.7/site-packages')import pandas as pdimport googletransimport matplotlib.pyplot as plt; plt.rcdefaults() 
The sys.path.insert step is done to make sure that the virtual environment’s site-packages directory is included in the system path. Read more about it in this python virtual environment post.
Make sure amazon.csv is in your working directory.
data = pd.read_csv("amazon.csv", thousands = '.')
The thousands = "." parameter makes up for the decimal formatting.
data.shape
data.head()
data.describe(include= "all")
Gives a nice summary of the data. Such as the count of all the columns, the highest occurring value in each column (if applicable) and its frequency.
data.isna().sum()
None? Great!
The first thought I had was to visualise the number of forest fires over the years, over the months. You need to be able to identify smaller pieces of the bigger picture.
Let’s drop rows from the dataset that aren’t contributing to the number of forest fires. So, any row with number column value as 0, must be dropped. We first convert the 0s to NaN and then drop rows with NaN in the specific column number.
data = data.replace(0, np.nan)data2 = data.dropna(subset=['number'])
Our dataset is now reduced from 6454 to 5837 rows.
data2.describe(include= "all")
forest_fire_per_month = data2.groupby('month')['number'].sum() : grouping the data by month and summing the numbers. The output is a pandas series.
print(forest_fire_per_month) : we notice the result is in alphabetical order. To get it back to the monthly order, we use the reindex property of dataframes :
forest_fire_per_month = forest_fire_per_month.reindex(months_unique, axis=0) 
where,
months_unique = list(data.month.unique())
Next we convert the series into a dataframe by: forest_fire_per_month = forest_fire_per_month.to_frame()
forest_fire_per_month.head()
This doesn’t look right. That’s because month is being considered as the index of the dataframe. To set a default index:forest_fire_per_month.reset_index(level=0, inplace=True)
forest_fire_per_month.head()
googletrans is a python library that implements Google Translate API. I tested it out on months_unique.
translator = Translator() #create an object of Translator for month in months_unique:     detected = translator.detect(month)         translated = translator.translate(month)         print(detected)         print(translated)         print("...")
Member variables of the returned Translator() object:
src — source language (default: auto)
dest — destination language (default: en)
origin — original text
text — translated text
pronunciation — pronunciation
These variables can be individually accessed using the . operator.
Example: translated.text will output the translated English text.
We use an instance of Translator to translate the month column in forest_fire_per_month to English.
translator2 = Translator() #create a new object of Translator. #cannot use the one defined before as that contains the latest #month’s calculated valuefor i, m in enumerate(forest_fire_per_month['month']):    translated = translator2.translate(m)      month1 = translated.text        forest_fire_per_month.at[i, 'month'] = month1
The enumerate() function assigns an index to each item in an iterable object that can be used to reference the item later.The enumerate() function in Python is commonly used instead of the for loop. That’s because enumerate() can iterate over the index of an item, as well as the item itself.
print(forest_fire_per_month)
It worked on all but one case!
Let’s bar graph it.
plt.figure(figsize=(25, 15)) #specify width and height #plt.bar(x-values, y-values) plt.bar(forest_fire_per_month['month'],forest_fire_per_month['number'], color = (0.5,0.1,0.5,0.6)) #use .suptitle for the actual title and .title for the subheadingplt.suptitle('Amazon Forest Fires Over the Months', fontsize=20) plt.title('Using Data from Years 1998 - 2017', fontsize=20)  plt.xlabel('Month', fontsize=20) plt.ylabel('Number of Forest Fires', fontsize=20)#plt.text(x-coordinate, y-coordinate, valueOfText, alignmnet)#this adds text at the top of each bar indicating its valuefor i, num in enumerate(forest_fire_per_month['number']):    plt.text(        i,        num + 10000,        num,        ha='center',        fontsize=15)   #plt.setp is to set a property on an artist object.#plt.gca() gets the current axes (gca) instance on the current figure #matching the given keyword args.#xticklabels and yticklabels are nothing but the values of the #lables on the x and y axis.#The code below lets us set the fontsize and alignment of the x and #y axis tick labelsplt.setp(plt.gca().get_xticklabels(),         rotation=45,         horizontalalignment='right',         fontsize=20)plt.setp(plt.gca().get_yticklabels(), fontsize=20)
The output:
This is a very small documentation of what can be done with this dataset. You only need to start. And the process is full of learning new things.
So, just download a random dataset and get going!
Super helpful resources:
For googletrans
For looping with enumerate()