【Python】EDAメモ


# この記事の目的
データ分析の最初に行う, EDA（探索的データ解析）においてよく使うコードをメモしておく. 
今回は特に, 分類問題（タイタニック号の乗客生存予測など）の場合を想定している. 




# コード解説

## ライブラリ読み込み
```python
import numpy as np
import pandas as pd
pd.set_option('display.max_columns', 100)

import seaborn as sns
import matplotlib.pyplot as plt
import japanize_matplotlib

import os
from tqdm import tqdm
import warnings
warnings.filterwarnings('ignore')
```

## データの読み込み
```python
df = sns.load_dataset("titanic")
df = df.replace(float("nan"),np.nan) #後のunique()の計算のため
df
```
![image.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/540956/f50e8ecf-257e-9b6c-f424-52f35cac6381.png)

## ぞれぞれの変数の中身を確認

```python
for colname in df.columns:
    uni = len(df[colname].unique())
    print("{0:<20} : {1}".format(colname, uni))
```
![image.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/540956/c8456726-a094-e4a4-e328-11e980cc2682.png)

## 目的変数の設定, 変数の型定義

```python
target="survived"
cate_list = ["pclass", "sex", "sibsp", "parch", "embarked", "class", 
                 "who", "adult_male", "deck", "embark_town", "alone"] #aliveは無視
num_list = ["age", "fare"]
all_list = cate_list + num_list


```

## NaNなどの確認
[こちら](https://qiita.com/yuto16/items/3f16919e2528a4970ee7) の記事を参照

![image.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/540956/071652c2-cb8d-ce5d-69e7-24c5844b9916.png)


## カテゴリカルデータ
まず簡単に確認するだけなら, 以下の２種類のようにかける. 

```python
sns.countplot(x="pclass", hue=target, data=df)
```
![image.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/540956/a2c1ff77-49bd-200c-c6cb-8e8163718939.png)

```python
sns.catplot(x="pclass", hue=target, data=df,kind="count")
```
![image.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/540956/37896022-a932-2815-f8c7-87995b333d9e.png)

さらに, NaNや平均についても知りたい場合は以下の関数を定義して使う. 

```python
def category_plot(x, hue, data, order=[]):
    #NaNを文字列に
    flag_nan = False
    data[x] = data[x].astype("str").replace("nan","NaN")
    if "NaN" in data[x].values:
        flag_nan = True

    x_unique_list = sorted(data[x].unique())
    x_unique_len = len(x_unique_list)
    x_unique_len_dropna = x_unique_len-1 if flag_nan else x_unique_len

    hue_unique_list = sorted(data[hue].unique())
    hue_unique_len = len(hue_unique_list)

    if order==[]:
        if flag_nan:
            order = x_unique_list
            order.remove("NaN")
            order = order + ["NaN"]
        else:
            order = x_unique_list
    else:
        pass
    
    colors = plt.get_cmap("tab10").colors

    sns.countplot(x=x, hue=hue, data=data, order=order,hue_order=hue_unique_list)

    for i,ui in enumerate(hue_unique_list):
        h = data.loc[data[hue]==ui,:].shape[0] / x_unique_len_dropna
        plt.plot([0-0.5,x_unique_len_dropna-1+0.5],[h,h],color=colors[i], linestyle="dashed", label="{0} (average)".format(ui))
    plt.legend()
    plt.show()

```

```python
category_plot(x="pclass", hue=target, data=df)
```
![image.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/540956/073f1d59-23e5-705d-3835-26b059e1c5c0.png)


```python
category_plot(x="embarked", hue=target, data=df)
```
![image.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/540956/157f7801-0fa6-7afa-9bd8-4226a5ea0081.png)

```python
category_plot(x="deck", hue=target, data=df)
```
![image.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/540956/0da7fbd9-e14e-9d17-6355-5d3c49554628.png)



## 数値データ

以下の２種類のプロットを見れば大体OK


```python
sns.catplot(x=target, y="age", data=df,kind="swarm")

```
![image.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/540956/70227291-bbd9-ec61-d27a-3872e1dbba2d.png)

```python
sns.catplot(x=target, y="age", data=df,kind="violin")
```
![image.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/540956/330efdff-f533-4d4d-e304-a0a28e948876.png)





# 参考
seaborn：[seaborn.catplot](https://seaborn.pydata.org/generated/seaborn.catplot.html)
seaborn：[seaborn.countplot](https://seaborn.pydata.org/generated/seaborn.countplot.html#seaborn.countplot)