Beginner’s Guide to Jupyter Notebook | by Shinichi Okada | Towards Data Science

Table of ContentsIntroduction1. Installing Anaconda2. Jupyter Notebook help and shortcuts3. Python libraries4. Finding data5. Importing data6. Data attributes7. Missing Value8. Dealing with NaN values9. How to deal categorical data10. Descriptive statistics11. Basic plotting12. Matplotlib13. Seaborn14. What’s next?
In this beginner’s guide to Jupyter Notebook, we go through how to get started with basic descriptive statistics. This article is intended for high-school students (or above) with little experience with coding.
If you want to read about Jupyter Notebook tips please read this article.
The Jupyter Notebook is an open-source web application that allows you to create and share documents that contain live code, equations, visualizations, and narrative text. (description from Project Jupyter)
The Jupyter system supports over 100 programming languages including Python. Currently, Python is the most popular language for Data Science.
You can see the original chart here.
I created a Jupyer slideshow and you can download it here. You need to install RISE to use this slideshow.
We are going to use Python libraries, Numpy, Pandas, Matplotlib and Seaborn. Python library is a collection of functions and methods that allows you to perform many actions without writing your code.
We are going to install Anaconda. Anaconda is the standard platform for Python data science. Anaconda will install all necessary packages for our purpose.
From https://www.anaconda.com/distribution/, download a 3.7 version and install it.
After installing the Anaconda, start Anaconda-Navigator.
Press Launch Jupyter Notebook. It will start a terminal and open a browser.
To create a folder, click the New button on the top right. Then click Folder.
You need to rename the folder to Statistics.
Create a folder called Data and we are going to store all our data in this folder.
In the Data folder, create a text file called mydata1.csv.
Open http://bit.ly/2M1yrW7 in a browser. Command(or ctrl in windows) + A to select all and Command + C to copy. Command + V to paste it to the mydata1.csv and Command + S to save the file.
In the Statistics folder, create a file called Test by New > Python 3.
That’s all for setup. Let’s start writing some codes.
towardsdatascience.com
Pressing h toggles the keyboards shortcuts screen. You can see all commands by pressing ⌘⇧f. I use the following shortcuts often and it will be useful if you can memorize them.
Numpy and Pandas are two of the most used libraries in scientific computing, Data science, and machine learning. Pandas is built on top of Numpy. Both offer similar functions. Numpy provides objects for multi-dimensional array and Pandas provides an in-memory 2D table object called DataFrame.
Python offers a variety of packages for plotting data. Seaborn and Matplotlib are two of Python’s most powerful visualization libraries.
Use import ... as ... to import a library. We use %matplotlib inline to show graphs inline. In our Test file please write/copy the following.
import numpy as npimport pandas as pdimport seaborn as snsimport matplotlib.pyplot as plt%matplotlib inline
After typing the above, press SHIFT + RETURN. This will run the code you wrote. If you have written correctly, nothing is going to happen. But if you have misspelled then it will return an error. Please read the last line of error and fix the problem.
There are many resources you can find online. This is a collection of data sources. CSV file is a comma-separated values file and is the easiest file to work on. Most of the data sources provided a CSV file. Find a couple of data you are interested in from the list. Download them to your hard drive.
We are going to read the CSV file and store the data in a variable called df. In the next cell please write the following.
df = pd.read_csv('data/mydata1.csv')
Please do not forget to press SHIFT + RETURN.
We are going to see the overview of our data using shape, info(), head(), tail(), describe().
We use shape to print the dimension of our data. info() prints a concise summary of our data. Press SHIFT + RETURN to run the program.
df.shape
It prints out (569, 33).
df.info()
head() prints the first 5 rows of data as the default. If you want to return the first 10 rows, use head(10).
df.head()
tail() prints the last 5 rows of data as the default. You can use any number as tail(10).
df.tail()
For the Statistical overview, describe() prints the mean, standard deviation, min, etc.
df.describe()
First, we import sample data which has ?, na, NA, N/A, and no data. We check if the data frame has any null values using `isnull()`. `any()` outputs by columns if there are any null values.
Please note that NaN will replace data items where there is no data.Comparing the first two tables, ? and na are not recognized as null. So we need to use `na_values` to recognize them as null values. This will change to NAN.
towardsdatascience.com
One way to deal with null values is to drop rows or columns. dropna() has the default value of axis=0. So dropna() and dropna(axis=0) outputs the same table. This will drop all rows where there is a null value.
`axis=1` will drop columns where there is NaN in the column.
We can fill up NaN value with the column mean. The following fill NaN in each column with its column mean.
You can fill up with the median.
Or any value you think it is appropriate.
Our data have a cat, dog, and fox under the col3. We use `get_dummies()` to convert categorical variable to dummy/indicator variables. This will create columns with all categories and add either 0 or 1.
We concatenate these dummies to our data.
We don’t need col3 anymore, so let’s drop it.
towardsdatascience.com
Unless you specify the column, mean, max, min, etc. returns the result of all columns. Here we are going to print only all_mean.
df=pd.read_csv('http://bit.ly/2vmLyvU')all_mean = df.mean()all_max = df.max()all_min = df.min()all_median = df.median()all_std = df.std()all_25_quantile = df.quantile(0.25)print('The means of all data.')print(all_mean)
As an exercise, can you print all_median, all_max?
Let’s focus on radius_mean in our data. We set a variable called target.
df=pd.read_csv('http://bit.ly/2vmLyvU')target = 'radius_mean'
We find the max, min, and range of radius_mean.
max = df[target].max()min = df[target].min()range =  max - minprint('The max is %.1f, the min is %.1f and the range is %.1f' % (max, min, range))
The print command prints out within the bracket. Three %.1 print with one decimal place and will be replaced with max, min, and range. You can find out more details in this article.
Let’s find out the mean, median, and mode.
mean = df[target].mean()median = df[target].median()mode = df[target].mode()print('The mean is %.1f, the median is %.1f and the mode is %.1f.' % (mean, median, mode))
The default value ofquartile is 0.5 which is the same as the median or 50%-tile. You can specify the percentile, quantile(0.25).
q1 = df[target].quantile(0.25)q3 = df[target].quantile(0.75)iqr = q3-q1print('The lower quartile is %.1f, the upper quartile is %.1f and the interquartile range is %.1f' % (q1, q3, iqr))
Panda’s plot() draws a line graph as the default. You can set the kind parameter with bar, hist, box, pie, etc.
df[target].plot()
The above diagram is the line graph of radius_mean.
By adding kind='hist' to plot(), you can plot a histogram.
df[target].plot(kind='hist')
By adding hist() after plot also prints a histogram.
df[target].plot.hist()
You can add bins=100 to use 100 histogram bins.
df[target].plot.hist(bins=100)
plot(kind='box') or plot.box() shows a box-whisker plot.
df[target].plot(kind='box')
Matplotlib has a lot of flexibility. Here you change the outlier's color and symbol and showing three box plots together.
data =[df['radius_mean'],df['texture_mean'],df['texture_worst']]green_diamond = dict(markerfacecolor='g', marker='D')fig7, ax7 = plt.subplots()ax7.set_title('Multiple Samples with Different sizes')ax7.boxplot(data,flierprops=green_diamond)plt.show()
Matplotlib’s histogram has a lot of flexibility. You can add different parameters.
plt.hist([df['radius_mean'],           df['texture_mean']],          histtype='barstacked',         orientation='horizontal',bins=20)plt.show()
Seaborn builds on top of Matplotlib and introduces additional plot types. It also makes your traditional Matplotlib plots look a bit prettier. (from Quora)
sns.boxplot(x='diagnosis', y='perimeter_mean', data=df)
Get full access to every story on Medium by becoming a member.