Getting Started with Pandas – Lesson 2
Tags: datascience
Published: 2021-12-01 21:26:51

---
title: Getting Started with Pandas – Lesson 2
published: true
date: 2021-09-09 05:00:59 UTC
tags: DataScience
canonical_url: https://apiumhub.com/tech-blog-barcelona/getting-started-with-pandas-lesson-2/
---

We will start today with the second deliverable of out data science training with Pandas. In this article we will summarize different pandas python functions to treat concepts such as indexing, selection and filtering.

Creating, Getting Info, Selecting and Util Functions

Before we start, we are going to visualize a little example dataset that we are going to follow to put the examples. It is a well-known dataset that contains wine information.

![L4cf9q sQuElZ4nStA8Y4OM82Cvsmwo5fSH95LYVhLCcikX7L5PVEOeTIeo7TOPJ2zt2ThhToK3Ha6k8AsX6SDyuJj63d](https://lh4.googleusercontent.com/L4cf9q-sQuElZ4nStA8Y4OM82Cvsmwo5fSH95LYVhLCcikX7L5PVEOeTIeo7TOPJ2zt2ThhToK3Ha6k8AsX6SDyuJj63d-9Kk8vCRICuexlOio3R71ptIO6N6ehTesS_Y62aZjs4=s0 "Getting Started with Pandas - Lesson 2 2")

## Indexing, Selecting and Filtering

In this chapter we will explain some key functions that can be useful to obtain a high level vision and an statistical overview of our dataset.

We will start with info() function, that offers us insigths about the number of columns, name of every column, number of non-null elements of each column and data type of everycolumn. Therefore, This function gives us a detailed overview of the data columns.

```
df.info()

Wines Dataset: 

<class 'pandas.core.frame.DataFrame'>
RangeIndex: 1599 entries, 0 to 1598
Data columns (total 12 columns):
 # Column Non-Null Count Dtype  
--- ------ -------------- -----  
 0 fixed acidity 1599 non-null float64
 1 volatile acidity 1599 non-null float64
 2 citric acid 1599 non-null float64
 3 residual sugar 1599 non-null float64
 4 chlorides 1599 non-null float64
 5 free sulfur dioxide 1599 non-null float64
 6 total sulfur dioxide 1599 non-null float64
 7 density 1599 non-null float64
 8 pH 1599 non-null float64
 9 sulphates 1599 non-null float64
 10 alcohol 1599 non-null float64
 11 quality 1599 non-null int64  
dtypes: float64(11), int64(1)
memory usage: 150.0 KB
```

### Dtypes

Dtypes attribute show us the data structure associated with each column. 

```
df.dtypes

Wines Dataset: 

fixed acidity float64
volatile acidity float64
citric acid float64
residual sugar float64
chlorides float64
free sulfur dioxide float64
total sulfur dioxide float64
density float64
pH float64
sulphates float64
alcohol float64
quality int64
dtype: object
```

Describe() function gives us a statistical overview distribution of each column, giving us main metrics that offer the first useful insights : the **mode** represents the most prevalent value , **mean and std** gives us an overview of the average value and the deviations from that value, **median** offers a hint of how the 50 % of this column is **above or below** a given value. 

However, it is important to note that we should analyze the % of NaNs (missing data) that these columns have in order to have a general overview of what these metrics means but this will be analysed in the next articles..

```
df.describe()

Wines Dataset: 

fixed acidity	volatile acidity	citric acid	residual sugar	chlorides	free sulfur dioxide	total sulfur dioxide	density	pH	sulphates	alcohol	quality
count	1599.000000	1599.000000	1599.000000	1599.000000	1599.000000	1599.000000	1599.000000	1599.000000	1599.000000	1599.000000	1599.000000	1599.000000
mean	8.319637	0.527821	0.270976	2.538806	0.087467	15.874922	46.467792	0.996747	3.311113	0.658149	10.422983	5.636023
std	1.741096	0.179060	0.194801	1.409928	0.047065	10.460157	32.895324	0.001887	0.154386	0.169507	1.065668	0.807569
min	4.600000	0.120000	0.000000	0.900000	0.012000	1.000000	6.000000	0.990070	2.740000	0.330000	8.400000	3.000000
25%	7.100000	0.390000	0.090000	1.900000	0.070000	7.000000	22.000000	0.995600	3.210000	0.550000	9.500000	5.000000
50%	7.900000	0.520000	0.260000	2.200000	0.079000	14.000000	38.000000	0.996750	3.310000	0.620000	10.200000	6.000000
75%	9.200000	0.640000	0.420000	2.600000	0.090000	21.000000	62.000000	0.997835	3.400000	0.730000	11.100000	6.000000
max	15.900000	1.580000	1.000000	15.500000	0.611000	72.000000	289.000000	1.003690	4.010000	2.000000	14.900000	8.000000
```

## Indexing and Selection

Here we are going to take a deep dive into explaining the two main indexing and selection pandas functions : ‘iloc’ and ‘loc’

+ .loc is primarily label based, but may also be used with a boolean array. .loc will raise KeyError when the items are not found. Allowed inputs are:

    – A single label, e.g. 5 or ‘a’ (Note that 5 is interpreted as a label of the index. This use is not an integer position along the index.).

    – A list or array of labels [‘a’, ‘b’, ‘c’].

    – A slice object with labels ‘a’:’f’ (Note that contrary to usual Python slices, both the start and the stop are included, when present in the index! See Slicing with labels and Endpoints are inclusive.)

    – A boolean array (any NA values will be treated as False).

    – A callable function with one argument (the calling Series or DataFrame) and that returns valid output for indexing (one of the above).

+ .iloc is primarily integer position based (from 0 to length-1 of the axis), but may also be used with a boolean array. .iloc will raise IndexError if a requested indexer is out-of-bounds, except slice indexers which allow out-of-bounds indexing. (this conforms with Python/NumPy slice semantics). Allowed inputs are:

  – An integer e.g. 5.

  – A list or array of integers [4, 3, 0].

  – A slice object with ints 1:7.

  – A boolean array (any NA values will be treated as False).

  – A callable function with one argument (the calling Series or DataFrame) and that returns valid output for indexing (one of the above).

Let´s dive into how these functions work through an example:

### iloc` examples

+ Get First Row

```
df.iloc[0]
```

+ Get first column

```
df.iloc[:, 0]
```

+ Get first colum of the first row

```
df.iloc[0:1, 0]
```

+ Get rows from 3 to 5

```
df.iloc[3:5]
```

+ Get rows 3, 7, 10

```
df.iloc[[3, 7, 10]]
```

+ Get last five rows

```
df.iloc[-5:]
```

### `loc` examples

+ Get First Row of colum ‘quality’

```
df.loc[0, 'quality']
```

+ Get all rows from columns ‘quality’, ‘sulphates’, ‘alcohol’

```
df.loc[:, ['quality', 'sulphates', 'alcohol']]
```

+ Get from row called ‘litres’ forward from columns ‘quality’ to ‘alcohol’

```
df1.loc['litres':, 'quality':'alcohol']
```

+ Get rows from 3 to 5 (Different from iloc)

```
df.loc[3:5]
```

### Conditional Selection

One of the most useful analytical vector thoughts that we can use for data science is to filter data with given conditions. With that aim, loc function allow us to filter by given conditions.

+ Get all wines which quality is greater than 6

```
wines.loc[wines.quality > 6]
```

+ Get all wines which quality is greater than 5 and less than 8

```
wines.loc[(wines.quality > 5) & (wines.quality < 8)]
```

+ Get all wines which quality is equal to 5 or equal to 7

```
wines.loc[(wines.quality == 5) | (wines.quality == 7)]
```

### Training your abilities

If you want to bring your skills further in Data Science, we have created a course that you can download for free [here](https://hub.docker.com/repository/docker/davsuacar/datascience-train)).

Over the next chapter, we will get a deep dive into the functions we use to extract info from our data and start to transform it to prepare the input of our prediction models.