Learn Python In Seminar〜自動車の走行距離を予測するモデル作成〜

# 概要

自動車の属性データからガソリン１ガロンあたりの走行距離を予測するモデル作成を行います。
データは[こちら](https://signate.jp/competitions/121)からダウンロードしてください。

他にも「[Learn Python In Seminar](https://qiita.com/k8m/items/4952894a80c1833e5501)」のシリーズとして書いているので、よければご覧ください！

---

# 利用するデータの説明

| カラム | ヘッダ名称 | データ型 | 説明 |
|:-:|:-:|:-:|:-:|
| 1 | mpg | float | ガソリン１ガロンあたりの走行距離  |
| 2 | cylinders | varchar |  シリンダー |
| 3 | displacement | float  | 排気量  |
| 4 | horsepower | float | 馬力  |
| 5 | weight | float |  重量 |
| 6 | acceleration | float | 加速度  |
| 7 | model year | varchar | 年式  |
| 8 | origin | varchar | 起源  |
| 9 | car name | varchar | 車名  |

---

# ライブラリのインストール

```
import numpy as np
import pandas as pd
import seaborn as sns
from matplotlib import pyplot as plt
from sklearn.metrics import r2_score
from sklearn.model_selection import train_test_split
from sklearn.metrics import mean_squared_error as MSE
from sklearn.linear_model import LinearRegression as LR
```

---

# データの読み込み

```
df_train = pd.read_csv('data/train.tsv', sep='\t')

print(df_train.head())
```

↓

```
   id   mpg  cylinders  displacement horsepower  weight  acceleration  model year  origin          car name
0   0  29.0          4         135.0      84.00  2525.0          16.0          82       1    dodge aries se
1   3  31.9          4          89.0      71.00  1925.0          14.0          79       2  vw rabbit custom
2   9  19.0          6         156.0      108.0  2930.0          15.5          76       3    toyota mark ii
3  11  28.0          4          90.0      75.00  2125.0          14.5          74       1        dodge colt
4  13  37.7          4          89.0      62.00  2050.0          17.3          81       3     toyota tercel
```

---

# 欠損値の確認

```
print(df_train.isnull().sum())
```

↓

```
id              0
mpg             0
cylinders       0
displacement    0
horsepower      0
weight          0
acceleration    0
model year      0
origin          0
car name        0
dtype: int64
```

---

# 基本統計量

```
print(df_train.describe(include='all'))
```

↓

```
                id         mpg   cylinders  displacement horsepower       weight  acceleration  model year      origin    car name
count   199.000000  199.000000  199.000000    199.000000        199   199.000000    199.000000  199.000000  199.000000         199
unique         NaN         NaN         NaN           NaN         71          NaN           NaN         NaN         NaN         167
top            NaN         NaN         NaN           NaN      90.00          NaN           NaN         NaN         NaN  ford pinto
freq           NaN         NaN         NaN           NaN         12          NaN           NaN         NaN         NaN           4
mean    200.170854   24.307035    5.296482    183.311558        NaN  2883.839196     15.647739   76.165829    1.582915         NaN
std     113.432759    7.797665    1.644562     98.400457        NaN   819.766870      2.701885    3.802928    0.798932         NaN
min       0.000000    9.000000    3.000000     71.000000        NaN  1613.000000      8.500000   70.000000    1.000000         NaN
25%      98.500000   18.000000    4.000000     98.000000        NaN  2217.500000     14.000000   73.000000    1.000000         NaN
50%     202.000000   24.000000    4.000000    140.000000        NaN  2702.000000     15.500000   76.000000    1.000000         NaN
75%     294.500000   30.500000    6.000000    250.000000        NaN  3426.500000     17.150000   80.000000    2.000000         NaN
max     396.000000   44.600000    8.000000    454.000000        NaN  5140.000000     23.700000   82.000000    3.000000         NaN
```

---

# 相関係数

```
print(df_train.corr())
```

↓

```
                    id       mpg  cylinders  displacement    weight  acceleration  model year    origin
id            1.000000 -0.052688   0.103419      0.098416  0.070563     -0.087649   -0.093272 -0.031421
mpg          -0.052688  1.000000  -0.770160     -0.804635 -0.820660      0.379721    0.568471  0.533671
cylinders     0.103419 -0.770160   1.000000      0.950600  0.893256     -0.479561   -0.303462 -0.497373
displacement  0.098416 -0.804635   0.950600      1.000000  0.933038     -0.523955   -0.329817 -0.579805
weight        0.070563 -0.820660   0.893256      0.933038  1.000000     -0.401757   -0.265562 -0.535893
acceleration -0.087649  0.379721  -0.479561     -0.523955 -0.401757      1.000000    0.194854  0.189193
model year   -0.093272  0.568471  -0.303462     -0.329817 -0.265562      0.194854    1.000000  0.124279
origin       -0.031421  0.533671  -0.497373     -0.579805 -0.535893      0.189193    0.124279  1.000000
```

---

# 相関係数の可視化

![heatmap.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/542018/677a0f99-b2b0-2906-c6f9-5a800efcfb36.png)

- mpg(ガロンあたりの走行距離)とdisplacement、weightに強い負の相関関係


---

# 相関関係の可視化

```
sns.pairplot(df_train[["mpg", "weight", "displacement"]])
```

↓

![pairplot2.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/542018/ade0f40d-1dc5-4e9b-fdb2-a6b3a3146455.png)

---

# 重回帰モデルの作成

```
# データの準備
y_train = df_train[["mpg"]] # 被説明変数
X_train = df_train[["weight", "displacement"]] # 説明変数

# モデルの生成
lr = LR()

# モデルの学習
lr.fit(X_train, y_train)
```

---

# 学習結果

```
print(lr.coef_) # 傾き
print(lr.intercept_) # 切片
print(lr.score(X_train, y_train)) # 決定係数
```

↓

```
[[-0.00513703 - 0.02383218]] # 傾き
[43.4901176] # 切片
0.6851901992035567 # 決定係数
```

- y = b1 × -0.00513703 + b2 × -0.02383218 + 43.4901176

---

# 学習結果と実データの比較

```
df_train['pred'] = y_pred_train

# 実データと予測結果の差をresとして代入
df_train['res'] = df_train['mpg'] - df_train['pred']

# ソート
print(df_train.sort_values(by='res'))
```

↓

```
mpg   displacement  weight  model year                            car name       pred        res
19.0         122.0  2310.0          73                          ford pinto  28.716052 - 9.716052
20.0         140.0  2408.0          72                      chevrolet vega  27.783644 - 7.783644
18.0         121.0  2933.0          72                     volvo 145e(sw)  25.539515 - 7.539515
19.0         121.0  2868.0          73                         volvo 144ea  25.873422 - 6.873422
21.0         140.0  2401.0          73                      chevrolet vega  27.819603 - 6.819603
...           ...     ...         ...                                 ...        ...        ...
41.5          98.0  2144.0          80                           vw rabbit  30.140772  11.359228
43.1          90.0  1985.0          78     volkswagen rabbit custom diesel  31.148217  11.951783
44.6          91.0  1850.0          80                 honda civic 1500 gl  31.817884  12.782116
43.4          90.0  2335.0          80                  vw dasher(diesel)  29.350257  14.049743
38.0         262.0  3015.0          82   oldsmobile cutlass ciera(diesel)  21.757941  16.242059
```

- model yearが古い時にマイナス、新しい時にプラスにずれている
- すでにmodel yearとmpgに相関関係が確認

---

# 特徴量の追加

説明変数にmodel yearを追加

```
y_train = df_train[["mpg"]]
X_train = df_train[["weight", "displacement", "model year"]]


lr = LR()
lr.fit(X_train, y_train)
y_pred_train = lr.predict(X_train)

print(lr.score(X_train, y_train))
```

↓

```
0.806202914350324
```

- model yearを説明変数に設定したところ決定変数の値が約0.2上昇

---

# 再検証

```
df_train['pred'] = y_pred_train
df_train['res'] = df_train['mpg'] - df_train['pred']
print(df_train.sort_values(by='res'))
```

↓

```
mpg  displacement  weight  model year                           car name       pred        res
19.0         122.0  2310.0          73                         ford pinto  25.822268 - 6.822268
22.0         232.0  2835.0          82                     ford granada l  28.804316 - 6.804316
19.8         200.0  2990.0          79                   mercury zephyr 6  25.711137 - 5.911137
21.5         121.0  2600.0          77                           bmw 320i  27.049786 - 5.549786
23.5         173.0  2725.0          81                 chevrolet citation  29.040204 - 5.540204
 ...           ...     ...         ...                                ...        ...        ...
36.4         121.0  2950.0          80                audi 5000s(diesel)  27.133402   9.266598
44.6          91.0  1850.0          80                honda civic 1500 gl  34.205448  10.394552
38.0         262.0  3015.0          82  oldsmobile cutlass ciera(diesel)  27.517075  10.482925
43.1          90.0  1985.0          78    volkswagen rabbit custom diesel  31.838867  11.261133
43.4          90.0  2335.0          80                 vw dasher(diesel)  31.161030  12.238970
```
- 通常年式だけではなく、車の種類とセットのため、car nameもセットで関係していると予測

---

# ダミー変数化

ダミー変数化できるカラムを自動で認識

```
X_train = pd.get_dummies(
    df_train[["weight", "displacement", "model year", "car name"]]) 
print(X_train)
```

↓

```
weight  displacement  model year  ...  car name_vw dasher(diesel)  car name_vw rabbit  car name_vw rabbit custom
2525.0         135.0          82  ...                            0                   0                    0
1925.0          89.0          79  ...                            0                   0                    1
2930.0         156.0          76  ...                            0                   0                    0
2125.0          90.0          74  ...                            0                   0                    0
2050.0          89.0          81  ...                            0                   0                    0
...           ...         ...  ...                          ...                 ...                  ...
2110.0          85.0          80  ...                            0                   0                    0
3570.0         302.0          78  ...                            0                   0                    0
3433.0         304.0          70  ...                            0                   0                    0
2335.0          90.0          80  ...                            1                   0                    0
2265.0          98.0          73  ...                            0                   0                    0
```

---

# 再度モデルの作成と検証

```
y_train = df_train[["mpg"]]
X_train = pd.get_dummies(
    df_train[["weight", "displacement", "model year", "car name"]]) 

lr = LR()
lr.fit(X_train, y_train)
y_pred_train = lr.predict(X_train)

df_train['pred'] = y_pred_train
df_train['res'] = df_train['mpg'] - df_train['pred']
print(df_train.sort_values(by='res'))

print(lr.score(X_train, y_train))
```

↓

```
mpg  displacement  weight  model year            car name       pred       res
29.0          90.0  1937.0          76           vw rabbit  34.656354 - 5.656354
31.8          85.0  2020.0          79          datsun 210  36.276274 - 4.476274
19.0         122.0  2310.0          73          ford pinto  23.364118 - 4.364118
25.0         116.0  2220.0          76           opel 1900  27.625883 - 2.625883
23.5         173.0  2725.0          81  chevrolet citation  26.067394 - 2.567394
 ...           ...     ...         ...                 ...        ...       ...
28.0         116.0  2123.0          71           opel 1900  25.374117  2.625883
26.5         140.0  2565.0          76          ford pinto  23.871847  2.628153
30.5          97.0  2190.0          77   volkswagen dasher  27.862259  2.637741
40.8          85.0  2110.0          80          datsun 210  36.323726  4.476274
41.5          98.0  2144.0          80           vw rabbit  35.843646  5.656354

0.9834170393004322
```

- 予測データと実データの差が減少
- car nameのダミー変数化前に比べて決定係数が約0.3上昇

---

# 結果

- horsepower、weight、model yearに自動車の走行距離(mpg)に影響
- car nameをダミー変数化することで決定係数が大幅に改善

---

# 課題

- car nameを説明変数に入れると未知の車種を予測できなかった(過学習)
  →メーカーと車種を分けて分析する必要がある
- originやcylindersが未検証
- null以外の欠損値があとで発見