Pandas - Brief
Tags: panda, beginners, datascience, tutorial
Published: 2022-06-14 16:42:02

What is pandas?
Pandas is python package built on two python packages **Matplotlib** and **Numpy**.
14 million users

**DataFrame**: 2 dimensional, Mutable, heterogeneous(Can be),Tabular Data structure
![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/v9dlzekzufdlc5oj34sc.png)

- `.info()` Method:
Generates Summary of the dataFrame with column names, Non-null counts, Dtype, memory Usage.
![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/jkmeczmmzlhxzjvhuyeq.png)
- `.head()` Method:
returns the first few rows (the “head” of the DataFrame).
![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/ev1s4nfyx7cn45jyy0ic.png)
- `.describe()` Method:
use for calculating statistical properties like mean, max, std Deviation, percentiles
![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/o15x0zzu79x2kgwjoc48.png)
- `.values`
Returns Numpy representation of the dataFrame.
But new method that is `to_numpy()` should be used rather than `.values`.
- `.columns`
List all column heading for database and its data types.
- `.index`
List all index in the dataFrame. These index means numbers of rows
![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/h6j8jd7fzkn07qslnu2m.png)
 
- `.shape` Function:
Returns the tuple of shape such as rows and columns
- `.size` Function:
Returns overall number of elements in that data frame
- `.ndim` Function:
Returns dimensions of Database
 
- DataFrame column selecting
You can select also multiple columns in database by double square bracket syntax. First square bracket is for syntax of dataFrame selection and second is for List of columns.

```
column1 = dataFrame['columnName']
column1 = dataFrame.columnName
column1 = dataFrame[['columnName', 'col2']]
```
- DataFrame row selecting with logical testing
- And or Operators in row selection
- Specific Value row selection:
This selects particular row from given column where value is value.
We can use different logical operator here also
```
row1 = dataFrame.[dataFrame.column == 'Value']
row1 = dataFrame.[dataFrame[column]== 'Value']

```
- Sorting Dataframe:
```
sortedDataFrame = dataFrame.sort_values('column_to_sort')
sortedDataFrame = dataFrame.sort_values(by = ['column_to_sort1', 'column_to_sort2'])
```
Sorting can be perform on numbers, Dates.
Extra Attributes - 
ascending = True / False, 
na_position = first/ last - where to put Nan Values.
Example: 
```
homelessness_reg_fam = homelessness.sort_values(['region','family_members'],ascending=[True,False])
```
- `isin()` Method:
`isin()` is used in filtering DataFrame. With Particular Value and particular column.

```
# The Mojave Desert states
canu = ["California", "Arizona", "Nevada", "Utah"]

# Filter for rows in the Mojave Desert states
mojave_homelessness = homelessness[homelessness.state.isin(canu)]
```
- Adding New Column to Database:
Terms for adding new columns: Mutating/transforming DataFrame or feature engineering

```
dataframe['new_column'] = old_column.some_transformation
```
- Summary Statistics
Summary statistics is the way you can summarise and know more about your data.
`mean()`, `median()`,`mode()`,`min()`,`max()`,`var()`, `std()`, ` sum()`, `quantile()`, `agg()`, 
`agg()` method is use to calculate custom summary statistic. 
`agg()` function takes more than one parameter function in the form of list.
Example of custom percentile is as follows. 
```
def percentile30(column):
   return column.quantile(0.4)

dataFrame[columnName].agg(percentile30)
```
Functions like `min`,`max` works on Date columns also.

**Calculating Cumulative Statistics**
`cumsum()`, `cummax()`,`cummin()`,`cumprod()`





To be Continued...