The AI Alpha Geek: It starts with EDA! - Part B
Tags: datascience, datavisualization, eventdriven, machinelearning
Published: 2020-09-30 20:07:43

---
title: The AI Alpha Geek: It starts with EDA! - Part B
published: true
description: 
tags: #datascience #datavisualization #eda #machinelearning
cover_image: https://dev-to-uploads.s3.amazonaws.com/i/ivgxhnina509pp8cy05d.jpg
---

| [The AI Alpha Geek: It starts with EDA! - Part A](https://dev.to/joyadauche/the-ai-alpha-geek-it-starts-with-eda-part-a-2l1i) |         
| ------------------------------------------------|


Before we start exploring each individual feature, let's take a look at some statistics for the dataset produced by `train_df.drop('PassengerId', axis=1).describe()` below:

![Summary Stats](https://dev-to-uploads.s3.amazonaws.com/i/fl0lh2knsvkz5h6ym6io.png)

In the summary statistics above, looking at the **Age** feature for example:
- the **count** is 714, which tells us there are 177 missing entries since the total entries are 891 - we would need to deal with this later on when handling missing values,
- the **mean** age is 29.699, which is the average age of passengers who were aboard i.e the value 29.699 was the typical or normal age of the passengers aboard,
- the **std (standard deviation)** of 14.526 tells us that most of the passengers are in the age range (29.699-14.526) to (29.699+14.526),
- the **min** age is 0.42, which tells us the least age is for a baby on board,
- the **25th percentile** is 20.125 years shows that 25% of passengers is less than 20.125 years, 
- the **50th percentile**, which is the median is 28 years, tells us that half of the passengers onboard are below 28 years old - seems most of the passengers were young,
- the **75th percentile**, which is 38, tells us that 75% of the passengers are less than 38 years, and
- the **max** age is 80 years, which is the age of the eldest passenger onboard - luckily, it seems there are no aliens onboard.

Now, it's time for some **univariate analysis** - this is just descriptive analysis of one variable at a time which it helps us understand the data distribution for that variable and even detect outliers. Let's start with the **categorical variables** - 

{% gist https://gist.github.com/joyadauche/97fe09d38a4ced41225b201b1a948a61 file=eda_part_b.py %}

In the code example above, taking a look at the output for the target variable, **Survived**, below -
![Output Example](https://dev-to-uploads.s3.amazonaws.com/i/4nf3mhcbo8nwgn5cbc0o.png)

- **value_counts()** is used to get the counts of unique values for this column - and it seems a lot more people did not survive. Note that it is not a perfectly balanced dataset but this is not a case where the number of those who didn't survive is far more significant than those who survived.
- to get the percentages of each class (i.e survived - 1 and deceased - 0), set the **normalize** parameter of value_counts() to True.
- to have a better view of the count for each class, we use **count plot** via Seaborn. The **label_chart()** is just a helper function to label the chart.

Let's see some insights gathered from the code output from **eda_part_b.py** above - 
- For the **Pclass** feature, it seems a lot more people that were on board are in class 3 and from [Part A]((https://dev.to/joyadauche/the-ai-alpha-geek-it-starts-with-eda-part-a-2l1i)) of this series, we saw that these are people in the lower socio-economic class, which seem to mean most onboard got the cheap ticket,
- Seems more males boarded when you look at the **Sex** feature, as 64.76% of passengers are males,
- Most passengers boarded from the Southampton port, and it seems most passengers came alone since most have 0 siblings and/or travelled with just a nanny.


So, all these give us more insights to explore further - Stay tuned for the next parts on this topic, on this same series, where we go-ahead to explore individual numerical variables for patterns. Wish you an awesome October!