Machine Learning Overview
Tags: jupiter, numpy, pandas, matplotlib
Published: 2023-10-05 14:25:37

Welcome to the Machine Learning. Machine Learning is a way of taking data and turning it into insights. We use computer power to analyze. Examples from the past to build a model that can predict the result of new example.

**Course Basics**

We will use several Python packages that are helpful in solving Machine Learning problems. We will be using pandas, numpy, matplotlib, and scikit-learn.

**1 Pandas** is used for reading data and data manipulation
**2 Numpy** is used for computation of numerical data

**3. Matplotlib** is used for graphing data  

**4. Scikit-learn** is used for machine learning models.

Each of these packages are quite expensive, but we will review the functions we will be using. We will also review some basic statistics as it is the foundation of Machine Learning.

*What this Course Entails*
We well be focusing on classification problems.These are problems where we're are predicting which class something belongs to:

***Examples  will Include***
1. Predicting who would survive the Titanic Crash
2. Determining a handwritten digit from an image 
3. Using Biospy data to clarify if a lump is cancerous 

We'll be using a number of popular techniques to tackle these problems. We'll get into each of them in details in the upcoming modules.

- Logistic Regression
- Decision Trees
- Random Forests
- Neural Networks

By the end of this course, you'll be able to take a classification dataset and use Python to build several different models to determine the best model for the given problem.

**For Example**
*Predicting if a credit card charge is fraudulent*
*Determining if an image is of car, bus or bike*

> Machine Learning can be used to solve a broad range of problems. This course will focus on ****Supervised Learning**** and ****Classification****


**Averages**
When dealing with data, we often need to calculate some simple statistics.
Let's say we have a list of ages of people in a class. We have them in ascending order since it will be easier to do the calculations.

```
15, 16, 18, 19, 22, 24, 29, 30, 34
```
 
The *Mean* is the most commonly known average.
Add up all the values and divide by the total number of values.

```
 (15 + 16 + 18 + 19 + 22 + 24 + 29 + 30 + 34) / 9 = 207 / 9 = *23*
```

The **Median** is the value in the middle. In this case, since there are **9** values, the middle value is the **5th**, which is *22*.

> In statistics, both the **Mean** and the **median** are called averages. The layman's **Average** is the **mean**

**Percentiles**
The median can also be thought of as the **50th Percentile**. This means that 50% of the data is less than the median and 50% of the data is greater than the median. This tells us where the middle of the data is, but we often want more of an understanding of the distribution of the data. We'll often look at the **25th** percentile and the **75th** percentile. 

The **25th** percentile is the value that is one quarter of the way through the data. This is the value where **25%** of the data is less than it (and **75%** of the data is greater than it).

Similarly, the **75%** percentile is three quarters of the way through the data. This is the value where **75%** of the data is less than it (and **25%** of the data is greater than it).

```
15, 16, 18, 19, 22, 24, 29, 30, 34
```

We have **9** values, so **25%** of the data would be approximately 2 datapoints. So, the **3rd** datapoint is greater than **25%** of the data. Thus the **25th** percentile is **18**(the 3rd Datapoint).

Similarly, **75%** of the data is approximately 6 datapoint. So, the 7th datapoint is greater than **75%** of the data. Thus, the **75%** percentile is **29** (the **7th** datapoint).

The full range of our data is between `15 and 34. ` The **25th** and **75th** percentile tells us that half our data is between `18 and 19`. This helps us again understanding of how the data is distributed.

> If there is an even number of datapoints, to find the median (or 50th percentile), you take the mean of two values in the middle.


## Standard Deviation & Variance

We can get a deeper understanding of the distribution of our data with the **Standard deviation** and **Variance**. are measures of how dispersed or spread out the data is.

We measure how far each datapoint i9s from the mean.

Let's Look at our group of Ages again:

```
15, 16, 18, 19, 22, 24, 29, 30, 34
```

Recall that the mean is `23`.

Let's calculate how far each value is from the mean. `15` is  `8` away from the mean (since `23-15 = 8`).

Here's a list of all these distances.

```

8, 7, 5, 4, 1, 1, 6, 7, 11
```

We square these values and add them together.

```
8 + 7 + 5 + 4 + 1 + 1 + 6 + 7 + 11
= 64 + 49 + 25 + 16 + 1 + 1 + 36 + 49 + 121 
= 362
```
We divide this value by the total number of values and that gives us the variance.

```
362 / 9 = 40.22
```

To get the standard deviation, we just take the square root of this number and get: `6.34`

If our is normally distributed like the graph below, `68% ` of the population is within one standard deviation of the mean. in the graph, we've highlighted the area within one standard deviation of the mean . You can see that the shaded area is about two thirds ( more precisely 68%) of the total area is under the curve. If we assume that our data is normally distributed, we can say that 68% of the data is within 1 standard deviation of the mean.


![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/wi4updq2dp6wph44hcjw.png)

In our age example, while the ages are likely not exactly normally distributed, we assume that we are and say that approximately 68% of the population has an age within one standard deviation of the mean. Since the mean is `23` and the standard deviation is  `6.34`, we can say that approximately `68%` of the ages in our population are between `16.66` (`23-6.34`) and `29.34` (`23 + 6.34`)


> Even though data is never perfect normal distribution, we can still use the standard deviation to again insight about how the data is distributed.