Data Every Day: インド肝臓患者レコード

# tldr

Kggleの[Indian Liver Patient Records](https://www.kaggle.com/uciml/indian-liver-patient-records)を[Liver Disease Prediction - Data Every Day #04](https://www.youtube.com/watch?v=moqaKwvXyV0&list=PLs_CQl4mUvI7Z9HnwoeOS5EUu9gyvekjK&index=47)に沿ってやっていきます。

実行環境はGoogle Colaboratorです。

# インポート


```python
import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
import seaborn as sns

import sklearn.preprocessing as sp
from sklearn.model_selection import train_test_split
import sklearn.linear_model as slm

import tensorflow as tf
```

# データのダウンロード

Google Driveをマウントします。


```python
from google.colab import drive
drive.mount('/content/drive')
```

    Mounted at /content/drive


KaggleのAPIクライアントを初期化し、認証します。
認証情報はGoogle Drive内（`/content/drive/My Drive/Colab Notebooks/Kaggle`）に`kaggle.json`として置いてあります。


```python
import os
kaggle_path = "/content/drive/My Drive/Colab Notebooks/Kaggle"
os.environ['KAGGLE_CONFIG_DIR'] = kaggle_path

from kaggle.api.kaggle_api_extended import KaggleApi
api = KaggleApi()
api.authenticate() 
```

Kaggle APIを使ってデータをダウンロードします。


```python
dataset_id = 'uciml/indian-liver-patient-records'
dataset = api.dataset_list_files(dataset_id)
file_name = dataset.files[0].name
file_path = os.path.join(api.get_default_download_dir(), file_name)
file_path
```




    '/content/indian_liver_patient.csv'




```python
api.dataset_download_file(dataset_id, file_name, force=True, quiet=False)
```

    100%|██████████| 23.4k/23.4k [00:00<00:00, 4.49MB/s]

    Downloading indian_liver_patient.csv to /content
    


    





    True



# データの読み込み


```python
data = pd.read_csv(file_path)
```


```python
data
```




<div>
<style scoped>
    .dataframe tbody tr th:only-of-type {
        vertical-align: middle;
    }

    .dataframe tbody tr th {
        vertical-align: top;
    }

    .dataframe thead th {
        text-align: right;
    }
</style>
<table border="1" class="dataframe">
  <thead>
    <tr style="text-align: right;">
      <th></th>
      <th>Age</th>
      <th>Gender</th>
      <th>Total_Bilirubin</th>
      <th>Direct_Bilirubin</th>
      <th>Alkaline_Phosphotase</th>
      <th>Alamine_Aminotransferase</th>
      <th>Aspartate_Aminotransferase</th>
      <th>Total_Protiens</th>
      <th>Albumin</th>
      <th>Albumin_and_Globulin_Ratio</th>
      <th>Dataset</th>
    </tr>
  </thead>
  <tbody>
    <tr>
      <th>0</th>
      <td>65</td>
      <td>Female</td>
      <td>0.7</td>
      <td>0.1</td>
      <td>187</td>
      <td>16</td>
      <td>18</td>
      <td>6.8</td>
      <td>3.3</td>
      <td>0.90</td>
      <td>1</td>
    </tr>
    <tr>
      <th>1</th>
      <td>62</td>
      <td>Male</td>
      <td>10.9</td>
      <td>5.5</td>
      <td>699</td>
      <td>64</td>
      <td>100</td>
      <td>7.5</td>
      <td>3.2</td>
      <td>0.74</td>
      <td>1</td>
    </tr>
    <tr>
      <th>2</th>
      <td>62</td>
      <td>Male</td>
      <td>7.3</td>
      <td>4.1</td>
      <td>490</td>
      <td>60</td>
      <td>68</td>
      <td>7.0</td>
      <td>3.3</td>
      <td>0.89</td>
      <td>1</td>
    </tr>
    <tr>
      <th>3</th>
      <td>58</td>
      <td>Male</td>
      <td>1.0</td>
      <td>0.4</td>
      <td>182</td>
      <td>14</td>
      <td>20</td>
      <td>6.8</td>
      <td>3.4</td>
      <td>1.00</td>
      <td>1</td>
    </tr>
    <tr>
      <th>4</th>
      <td>72</td>
      <td>Male</td>
      <td>3.9</td>
      <td>2.0</td>
      <td>195</td>
      <td>27</td>
      <td>59</td>
      <td>7.3</td>
      <td>2.4</td>
      <td>0.40</td>
      <td>1</td>
    </tr>
    <tr>
      <th>...</th>
      <td>...</td>
      <td>...</td>
      <td>...</td>
      <td>...</td>
      <td>...</td>
      <td>...</td>
      <td>...</td>
      <td>...</td>
      <td>...</td>
      <td>...</td>
      <td>...</td>
    </tr>
    <tr>
      <th>578</th>
      <td>60</td>
      <td>Male</td>
      <td>0.5</td>
      <td>0.1</td>
      <td>500</td>
      <td>20</td>
      <td>34</td>
      <td>5.9</td>
      <td>1.6</td>
      <td>0.37</td>
      <td>2</td>
    </tr>
    <tr>
      <th>579</th>
      <td>40</td>
      <td>Male</td>
      <td>0.6</td>
      <td>0.1</td>
      <td>98</td>
      <td>35</td>
      <td>31</td>
      <td>6.0</td>
      <td>3.2</td>
      <td>1.10</td>
      <td>1</td>
    </tr>
    <tr>
      <th>580</th>
      <td>52</td>
      <td>Male</td>
      <td>0.8</td>
      <td>0.2</td>
      <td>245</td>
      <td>48</td>
      <td>49</td>
      <td>6.4</td>
      <td>3.2</td>
      <td>1.00</td>
      <td>1</td>
    </tr>
    <tr>
      <th>581</th>
      <td>31</td>
      <td>Male</td>
      <td>1.3</td>
      <td>0.5</td>
      <td>184</td>
      <td>29</td>
      <td>32</td>
      <td>6.8</td>
      <td>3.4</td>
      <td>1.00</td>
      <td>1</td>
    </tr>
    <tr>
      <th>582</th>
      <td>38</td>
      <td>Male</td>
      <td>1.0</td>
      <td>0.3</td>
      <td>216</td>
      <td>21</td>
      <td>24</td>
      <td>7.3</td>
      <td>4.4</td>
      <td>1.50</td>
      <td>2</td>
    </tr>
  </tbody>
</table>
<p>583 rows × 11 columns</p>
</div>



# 下準備


```python
data.info()
```

    <class 'pandas.core.frame.DataFrame'>
    RangeIndex: 583 entries, 0 to 582
    Data columns (total 11 columns):
     #   Column                      Non-Null Count  Dtype  
    ---  ------                      --------------  -----  
     0   Age                         583 non-null    int64  
     1   Gender                      583 non-null    object 
     2   Total_Bilirubin             583 non-null    float64
     3   Direct_Bilirubin            583 non-null    float64
     4   Alkaline_Phosphotase        583 non-null    int64  
     5   Alamine_Aminotransferase    583 non-null    int64  
     6   Aspartate_Aminotransferase  583 non-null    int64  
     7   Total_Protiens              583 non-null    float64
     8   Albumin                     583 non-null    float64
     9   Albumin_and_Globulin_Ratio  579 non-null    float64
     10  Dataset                     583 non-null    int64  
    dtypes: float64(5), int64(5), object(1)
    memory usage: 50.2+ KB



```python
data.isna().sum()
```




    Age                           0
    Gender                        0
    Total_Bilirubin               0
    Direct_Bilirubin              0
    Alkaline_Phosphotase          0
    Alamine_Aminotransferase      0
    Aspartate_Aminotransferase    0
    Total_Protiens                0
    Albumin                       0
    Albumin_and_Globulin_Ratio    4
    Dataset                       0
    dtype: int64




```python
data['Albumin_and_Globulin_Ratio'] = data['Albumin_and_Globulin_Ratio'].fillna(data['Albumin_and_Globulin_Ratio'].mean())
```

# エンコード


```python
def binary_encode(df, column, positive_value):
    df = df.copy()
    df[column] = df[column].apply(lambda x: 1 if x == positive_value else 0)
    return df
```


```python
data = binary_encode(data, 'Gender', 'Male')
```


```python
data = binary_encode(data, 'Dataset', 1)
```

# 分割とスケーリング


```python
y = data['Dataset']
X = data.drop('Dataset', axis=1)
```


```python
scaler = sp.StandardScaler()
X = scaler.fit_transform(X)
```


```python
X_train, X_test, y_train, y_test = train_test_split(X, y, train_size=0.8)
```

# トレーニング


```python
X.shape
```




    (583, 10)




```python
y.sum() / len(y)
```




    0.7135506003430532




```python
model = tf.keras.Sequential([
    tf.keras.layers.Dense(64, activation='relu', input_shape=(10,)),
    tf.keras.layers.Dense(64, activation='relu'),
    tf.keras.layers.Dense(1, activation='sigmoid')
])

model.summary()

model.compile(
    optimizer='adam',
    loss='binary_crossentropy',
    metrics=[
        'accuracy',
        tf.keras.metrics.AUC(name='auc'),
    ],
)

batch_size = 64
epochs = 100

history = model.fit(
    X_train,
    y_train,
    validation_split=0.2,
    batch_size=batch_size,
    epochs=epochs,
    verbose=0,
)
```

    Model: "sequential_3"
    _________________________________________________________________
    Layer (type)                 Output Shape              Param #   
    =================================================================
    dense_9 (Dense)              (None, 64)                704       
    _________________________________________________________________
    dense_10 (Dense)             (None, 64)                4160      
    _________________________________________________________________
    dense_11 (Dense)             (None, 1)                 65        
    =================================================================
    Total params: 4,929
    Trainable params: 4,929
    Non-trainable params: 0
    _________________________________________________________________


# 結果


```python
plt.figure(figsize=(14, 10))

epochs_range = range(1, epochs+1)
train_loss = history.history['loss']
val_loss = history.history['val_loss']

plt.plot(epochs_range, train_loss, label='Training Loss')
plt.plot(epochs_range, val_loss, label='Validation Loss')

plt.title('Training and Validation Loss')
plt.xlabel('Epoch')
plt.ylabel('Loss')
plt.legend()

plt.show()
```


![Indian_Liver_Patient_Records_30_0.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/141539/bb50a319-da7e-7cec-2374-0adbebab6f66.png)



```python
np.argmin(val_loss)
```




    56




```python
model.evaluate(X_test, y_test)
```

    4/4 [==============================] - 0s 4ms/step - loss: 0.6818 - accuracy: 0.6325 - auc: 0.7016





    [0.681831419467926, 0.632478654384613, 0.7015877962112427]