Data Every Day: Top Women Chess Players

# tldr

Kggleの[Top Women Chess Players
](https://www.kaggle.com/vikasojha98/top-women-chess-players)を[Chess Grandmaster Prediction - Data Every Day #023](https://www.youtube.com/watch?v=GXx_q6cKErQ&list=PLs_CQl4mUvI7Z9HnwoeOS5EUu9gyvekjK&index=23)に沿ってやっていきます。

実行環境はGoogle Colaboratorです。

# インポート


```
import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
import seaborn as sns

import sklearn.preprocessing as sp
from sklearn.model_selection import train_test_split

from sklearn.linear_model import LogisticRegression

from sklearn.metrics import f1_score

```

# データのダウンロード

Google Driveをマウントします。


```
from google.colab import drive
drive.mount('/content/drive')
```

    Drive already mounted at /content/drive; to attempt to forcibly remount, call drive.mount("/content/drive", force_remount=True).


KaggleのAPIクライアントを初期化し、認証します。
認証情報はGoogle Drive内（`/content/drive/My Drive/Colab Notebooks/Kaggle`）に`kaggle.json`として置いてあります。


```
import os
kaggle_path = "/content/drive/My Drive/Colab Notebooks/Kaggle"
os.environ['KAGGLE_CONFIG_DIR'] = kaggle_path

from kaggle.api.kaggle_api_extended import KaggleApi
api = KaggleApi()
api.authenticate() 
```

Kaggle APIを使ってデータをダウンロードします。


```
dataset_id = 'vikasojha98/top-women-chess-players'
dataset = api.dataset_list_files(dataset_id)
file_name = dataset.files[0].name
file_path = os.path.join(api.get_default_download_dir(), file_name)
file_path
```

    Warning: Looks like you're using an outdated API Version, please consider updating (server 1.5.10 / client 1.5.9)





    '/content/top_women_chess_players_aug_2020.csv'




```
api.dataset_download_file(dataset_id, file_name, force=True, quiet=False)
```

    100%|██████████| 459k/459k [00:00<00:00, 53.4MB/s]

    Downloading top_women_chess_players_aug_2020.csv to /content
    


    





    True



# データの読み込み

Padasを使ってダウンロードしてきたCSVファイルを読み込みます。


```
data = pd.read_csv(file_path)
```


```
data
```




<div>
<style scoped>
    .dataframe tbody tr th:only-of-type {
        vertical-align: middle;
    }

    .dataframe tbody tr th {
        vertical-align: top;
    }

    .dataframe thead th {
        text-align: right;
    }
</style>
<table border="1" class="dataframe">
  <thead>
    <tr style="text-align: right;">
      <th></th>
      <th>Fide id</th>
      <th>Name</th>
      <th>Federation</th>
      <th>Gender</th>
      <th>Year_of_birth</th>
      <th>Title</th>
      <th>Standard_Rating</th>
      <th>Rapid_rating</th>
      <th>Blitz_rating</th>
      <th>Inactive_flag</th>
    </tr>
  </thead>
  <tbody>
    <tr>
      <th>0</th>
      <td>700070</td>
      <td>Polgar, Judit</td>
      <td>HUN</td>
      <td>F</td>
      <td>1976.0</td>
      <td>GM</td>
      <td>2675</td>
      <td>2646.0</td>
      <td>2736.0</td>
      <td>wi</td>
    </tr>
    <tr>
      <th>1</th>
      <td>8602980</td>
      <td>Hou, Yifan</td>
      <td>CHN</td>
      <td>F</td>
      <td>1994.0</td>
      <td>GM</td>
      <td>2658</td>
      <td>2621.0</td>
      <td>2601.0</td>
      <td>NaN</td>
    </tr>
    <tr>
      <th>2</th>
      <td>5008123</td>
      <td>Koneru, Humpy</td>
      <td>IND</td>
      <td>F</td>
      <td>1987.0</td>
      <td>GM</td>
      <td>2586</td>
      <td>2483.0</td>
      <td>2483.0</td>
      <td>NaN</td>
    </tr>
    <tr>
      <th>3</th>
      <td>4147103</td>
      <td>Goryachkina, Aleksandra</td>
      <td>RUS</td>
      <td>F</td>
      <td>1998.0</td>
      <td>GM</td>
      <td>2582</td>
      <td>2502.0</td>
      <td>2441.0</td>
      <td>NaN</td>
    </tr>
    <tr>
      <th>4</th>
      <td>700088</td>
      <td>Polgar, Susan</td>
      <td>HUN</td>
      <td>F</td>
      <td>1969.0</td>
      <td>GM</td>
      <td>2577</td>
      <td>NaN</td>
      <td>NaN</td>
      <td>wi</td>
    </tr>
    <tr>
      <th>...</th>
      <td>...</td>
      <td>...</td>
      <td>...</td>
      <td>...</td>
      <td>...</td>
      <td>...</td>
      <td>...</td>
      <td>...</td>
      <td>...</td>
      <td>...</td>
    </tr>
    <tr>
      <th>8548</th>
      <td>3302288</td>
      <td>Reinkens, Natalia</td>
      <td>BOL</td>
      <td>F</td>
      <td>NaN</td>
      <td>NaN</td>
      <td>1801</td>
      <td>NaN</td>
      <td>NaN</td>
      <td>wi</td>
    </tr>
    <tr>
      <th>8549</th>
      <td>343960</td>
      <td>Saffova, Michaela</td>
      <td>CZE</td>
      <td>F</td>
      <td>1994.0</td>
      <td>NaN</td>
      <td>1801</td>
      <td>1791.0</td>
      <td>1765.0</td>
      <td>NaN</td>
    </tr>
    <tr>
      <th>8550</th>
      <td>5038294</td>
      <td>Shetye, Siddhali</td>
      <td>IND</td>
      <td>F</td>
      <td>1992.0</td>
      <td>NaN</td>
      <td>1801</td>
      <td>1884.0</td>
      <td>1824.0</td>
      <td>wi</td>
    </tr>
    <tr>
      <th>8551</th>
      <td>2072491</td>
      <td>Trakru, Priya</td>
      <td>USA</td>
      <td>F</td>
      <td>2001.0</td>
      <td>WFM</td>
      <td>1801</td>
      <td>NaN</td>
      <td>NaN</td>
      <td>wi</td>
    </tr>
    <tr>
      <th>8552</th>
      <td>4666399</td>
      <td>Vorpahl, Sina Fleur</td>
      <td>GER</td>
      <td>F</td>
      <td>1982.0</td>
      <td>NaN</td>
      <td>1801</td>
      <td>NaN</td>
      <td>NaN</td>
      <td>wi</td>
    </tr>
  </tbody>
</table>
<p>8553 rows × 10 columns</p>
</div>



# 下準備

## 不要な列の削除


```
data = data.drop(['Fide id', 'Name', 'Gender'], axis=1)
```

# 欠損値の処理


```
data.isnull().sum()
```




    Federation            0
    Year_of_birth       292
    Title              5435
    Standard_Rating       0
    Rapid_rating       4945
    Blitz_rating       5081
    Inactive_flag      2701
    dtype: int64




```
data.dtypes
```




    Federation          object
    Year_of_birth      float64
    Title               object
    Standard_Rating      int64
    Rapid_rating       float64
    Blitz_rating       float64
    Inactive_flag       object
    dtype: object



NULL値が入っている数値のセルに平均値を入れていく


```
numerical_features = ['Year_of_birth', 'Standard_Rating', 'Rapid_rating', 'Blitz_rating']
```


```
for column in numerical_features:
    data[column] = data[column].fillna(data[column].mean())
```


```
data.isnull().sum()
```




    Federation            0
    Year_of_birth         0
    Title              5435
    Standard_Rating       0
    Rapid_rating          0
    Blitz_rating          0
    Inactive_flag      2701
    dtype: int64




```
data['Title'].unique()
```




    array(['GM', 'IM', 'WGM', 'FM', 'WFM', 'WIM', nan, 'CM', 'WCM', 'WH'],
          dtype=object)




```
data['Inactive_flag'].unique()
```




    array(['wi', nan], dtype=object)



「Inactive_flag」は活動していない場合のみ「wi」が入っているので、NA値には「wa」をいれる


```
data['Inactive_flag'] = data['Inactive_flag'].fillna('wa')
```


```
title_dummies = pd.get_dummies(data['Title'])
title_dummies
```




<div>
<style scoped>
    .dataframe tbody tr th:only-of-type {
        vertical-align: middle;
    }

    .dataframe tbody tr th {
        vertical-align: top;
    }

    .dataframe thead th {
        text-align: right;
    }
</style>
<table border="1" class="dataframe">
  <thead>
    <tr style="text-align: right;">
      <th></th>
      <th>CM</th>
      <th>FM</th>
      <th>GM</th>
      <th>IM</th>
      <th>WCM</th>
      <th>WFM</th>
      <th>WGM</th>
      <th>WH</th>
      <th>WIM</th>
    </tr>
  </thead>
  <tbody>
    <tr>
      <th>0</th>
      <td>0</td>
      <td>0</td>
      <td>1</td>
      <td>0</td>
      <td>0</td>
      <td>0</td>
      <td>0</td>
      <td>0</td>
      <td>0</td>
    </tr>
    <tr>
      <th>1</th>
      <td>0</td>
      <td>0</td>
      <td>1</td>
      <td>0</td>
      <td>0</td>
      <td>0</td>
      <td>0</td>
      <td>0</td>
      <td>0</td>
    </tr>
    <tr>
      <th>2</th>
      <td>0</td>
      <td>0</td>
      <td>1</td>
      <td>0</td>
      <td>0</td>
      <td>0</td>
      <td>0</td>
      <td>0</td>
      <td>0</td>
    </tr>
    <tr>
      <th>3</th>
      <td>0</td>
      <td>0</td>
      <td>1</td>
      <td>0</td>
      <td>0</td>
      <td>0</td>
      <td>0</td>
      <td>0</td>
      <td>0</td>
    </tr>
    <tr>
      <th>4</th>
      <td>0</td>
      <td>0</td>
      <td>1</td>
      <td>0</td>
      <td>0</td>
      <td>0</td>
      <td>0</td>
      <td>0</td>
      <td>0</td>
    </tr>
    <tr>
      <th>...</th>
      <td>...</td>
      <td>...</td>
      <td>...</td>
      <td>...</td>
      <td>...</td>
      <td>...</td>
      <td>...</td>
      <td>...</td>
      <td>...</td>
    </tr>
    <tr>
      <th>8548</th>
      <td>0</td>
      <td>0</td>
      <td>0</td>
      <td>0</td>
      <td>0</td>
      <td>0</td>
      <td>0</td>
      <td>0</td>
      <td>0</td>
    </tr>
    <tr>
      <th>8549</th>
      <td>0</td>
      <td>0</td>
      <td>0</td>
      <td>0</td>
      <td>0</td>
      <td>0</td>
      <td>0</td>
      <td>0</td>
      <td>0</td>
    </tr>
    <tr>
      <th>8550</th>
      <td>0</td>
      <td>0</td>
      <td>0</td>
      <td>0</td>
      <td>0</td>
      <td>0</td>
      <td>0</td>
      <td>0</td>
      <td>0</td>
    </tr>
    <tr>
      <th>8551</th>
      <td>0</td>
      <td>0</td>
      <td>0</td>
      <td>0</td>
      <td>0</td>
      <td>1</td>
      <td>0</td>
      <td>0</td>
      <td>0</td>
    </tr>
    <tr>
      <th>8552</th>
      <td>0</td>
      <td>0</td>
      <td>0</td>
      <td>0</td>
      <td>0</td>
      <td>0</td>
      <td>0</td>
      <td>0</td>
      <td>0</td>
    </tr>
  </tbody>
</table>
<p>8553 rows × 9 columns</p>
</div>




```
title_dummies.sum()
```




    CM        8
    FM       36
    GM       37
    IM      119
    WCM     247
    WFM    1545
    WGM     316
    WH        1
    WIM     809
    dtype: int64



今回はグランド・マスターかどうか判断するのでGMのみ残します。


```
data = pd.concat([data, title_dummies['GM']], axis=1)
data = data.drop('Title', axis=1)
```


```
data.isnull().sum()
```




    Federation         0
    Year_of_birth      0
    Standard_Rating    0
    Rapid_rating       0
    Blitz_rating       0
    Inactive_flag      0
    GM                 0
    dtype: int64



# エンコード


```
data['Inactive_flag'].unique()
```




    array(['wi', 'wa'], dtype=object)




```
バイナリ値は `LabelEncoder`を使います。
```


```
encoder = LabelEncoder()
data['Inactive_flag'] = encoder.fit_transform(data['Inactive_flag'])
```


```
data
```




<div>
<style scoped>
    .dataframe tbody tr th:only-of-type {
        vertical-align: middle;
    }

    .dataframe tbody tr th {
        vertical-align: top;
    }

    .dataframe thead th {
        text-align: right;
    }
</style>
<table border="1" class="dataframe">
  <thead>
    <tr style="text-align: right;">
      <th></th>
      <th>Federation</th>
      <th>Year_of_birth</th>
      <th>Standard_Rating</th>
      <th>Rapid_rating</th>
      <th>Blitz_rating</th>
      <th>Inactive_flag</th>
      <th>GM</th>
    </tr>
  </thead>
  <tbody>
    <tr>
      <th>0</th>
      <td>HUN</td>
      <td>1976.000000</td>
      <td>2675</td>
      <td>2646.000000</td>
      <td>2736.000000</td>
      <td>1</td>
      <td>1</td>
    </tr>
    <tr>
      <th>1</th>
      <td>CHN</td>
      <td>1994.000000</td>
      <td>2658</td>
      <td>2621.000000</td>
      <td>2601.000000</td>
      <td>0</td>
      <td>1</td>
    </tr>
    <tr>
      <th>2</th>
      <td>IND</td>
      <td>1987.000000</td>
      <td>2586</td>
      <td>2483.000000</td>
      <td>2483.000000</td>
      <td>0</td>
      <td>1</td>
    </tr>
    <tr>
      <th>3</th>
      <td>RUS</td>
      <td>1998.000000</td>
      <td>2582</td>
      <td>2502.000000</td>
      <td>2441.000000</td>
      <td>0</td>
      <td>1</td>
    </tr>
    <tr>
      <th>4</th>
      <td>HUN</td>
      <td>1969.000000</td>
      <td>2577</td>
      <td>1931.680155</td>
      <td>1925.155242</td>
      <td>1</td>
      <td>1</td>
    </tr>
    <tr>
      <th>...</th>
      <td>...</td>
      <td>...</td>
      <td>...</td>
      <td>...</td>
      <td>...</td>
      <td>...</td>
      <td>...</td>
    </tr>
    <tr>
      <th>8548</th>
      <td>BOL</td>
      <td>1985.291732</td>
      <td>1801</td>
      <td>1931.680155</td>
      <td>1925.155242</td>
      <td>1</td>
      <td>0</td>
    </tr>
    <tr>
      <th>8549</th>
      <td>CZE</td>
      <td>1994.000000</td>
      <td>1801</td>
      <td>1791.000000</td>
      <td>1765.000000</td>
      <td>0</td>
      <td>0</td>
    </tr>
    <tr>
      <th>8550</th>
      <td>IND</td>
      <td>1992.000000</td>
      <td>1801</td>
      <td>1884.000000</td>
      <td>1824.000000</td>
      <td>1</td>
      <td>0</td>
    </tr>
    <tr>
      <th>8551</th>
      <td>USA</td>
      <td>2001.000000</td>
      <td>1801</td>
      <td>1931.680155</td>
      <td>1925.155242</td>
      <td>1</td>
      <td>0</td>
    </tr>
    <tr>
      <th>8552</th>
      <td>GER</td>
      <td>1982.000000</td>
      <td>1801</td>
      <td>1931.680155</td>
      <td>1925.155242</td>
      <td>1</td>
      <td>0</td>
    </tr>
  </tbody>
</table>
<p>8553 rows × 7 columns</p>
</div>




```
data['Federation'].unique()
```




    array(['HUN', 'CHN', 'IND', 'RUS', 'UKR', 'LTU', 'GEO', 'KAZ', 'IRI',
           'GER', 'SWE', 'BUL', 'TUR', 'GRE', 'AZE', 'FRA', 'ROU', 'USA',
           'MGL', 'POL', 'BLR', 'QAT', 'ESP', 'ENG', 'INA', 'ARM', 'CZE',
           'PER', 'SRB', 'NED', 'SCO', 'UZB', 'ITA', 'CUB', 'VIE', 'ECU',
           'AUS', 'ARG', 'CRO', 'SVK', 'SGP', 'ISR', 'LUX', 'SLO', 'EST',
           'CAN', 'LAT', 'AUT', 'SUI', 'MNC', 'MDA', 'BRA', 'BEL', 'COL',
           'PHI', 'PAR', 'BRU', 'MEX', 'BIH', 'MAS', 'NOR', 'MNE', 'TKM',
           'IRL', 'VEN', 'EGY', 'IRQ', 'FIN', 'BOL', 'DEN', 'MKD', 'KGZ',
           'ESA', 'CHI', 'RSA', 'FID', 'UAE', 'LBN', 'MYA', 'ISL', 'BAN',
           'POR', 'KSA', 'NAM', 'URU', 'ALG', 'WLS', 'PUR', 'ALB', 'KOR',
           'TJK', 'SRI', 'JAM', 'ANG', 'NGR', 'BAR', 'BER', 'ZIM', 'BOT',
           'JPN', 'DOM', 'CRC', 'SYR', 'GUA', 'SEY', 'JOR', 'NZL', 'MAR',
           'MAC', 'TTO', 'NCA', 'ZAM', 'PAN', 'THA', 'GCI', 'AHO', 'HKG',
           'MLT', 'HON', 'LBA', 'SUR', 'UGA', 'CPV', 'MAD'], dtype=object)




```
data = data.drop('Federation', axis=1)
```

# データの分割

XとYにデータを分割します。


```
y = data['GM']
X = data.drop('GM', axis=1)
```

# スケーリング

`MinMaxScaler`を使ってデータを0-1の間にスケーリングします。


```
scaler = sp.MinMaxScaler()
X = pd.DataFrame(scaler.fit_transform(X), columns=X.columns)
```


```
X
```




<div>
<style scoped>
    .dataframe tbody tr th:only-of-type {
        vertical-align: middle;
    }

    .dataframe tbody tr th {
        vertical-align: top;
    }

    .dataframe thead th {
        text-align: right;
    }
</style>
<table border="1" class="dataframe">
  <thead>
    <tr style="text-align: right;">
      <th></th>
      <th>Year_of_birth</th>
      <th>Standard_Rating</th>
      <th>Rapid_rating</th>
      <th>Blitz_rating</th>
      <th>Inactive_flag</th>
    </tr>
  </thead>
  <tbody>
    <tr>
      <th>0</th>
      <td>0.622222</td>
      <td>1.000000</td>
      <td>1.000000</td>
      <td>1.000000</td>
      <td>1.0</td>
    </tr>
    <tr>
      <th>1</th>
      <td>0.822222</td>
      <td>0.980549</td>
      <td>0.982419</td>
      <td>0.914394</td>
      <td>0.0</td>
    </tr>
    <tr>
      <th>2</th>
      <td>0.744444</td>
      <td>0.898169</td>
      <td>0.885373</td>
      <td>0.839569</td>
      <td>0.0</td>
    </tr>
    <tr>
      <th>3</th>
      <td>0.866667</td>
      <td>0.893593</td>
      <td>0.898734</td>
      <td>0.812936</td>
      <td>0.0</td>
    </tr>
    <tr>
      <th>4</th>
      <td>0.544444</td>
      <td>0.887872</td>
      <td>0.497665</td>
      <td>0.485831</td>
      <td>1.0</td>
    </tr>
    <tr>
      <th>...</th>
      <td>...</td>
      <td>...</td>
      <td>...</td>
      <td>...</td>
      <td>...</td>
    </tr>
    <tr>
      <th>8548</th>
      <td>0.725464</td>
      <td>0.000000</td>
      <td>0.497665</td>
      <td>0.485831</td>
      <td>1.0</td>
    </tr>
    <tr>
      <th>8549</th>
      <td>0.822222</td>
      <td>0.000000</td>
      <td>0.398734</td>
      <td>0.384274</td>
      <td>0.0</td>
    </tr>
    <tr>
      <th>8550</th>
      <td>0.800000</td>
      <td>0.000000</td>
      <td>0.464135</td>
      <td>0.421687</td>
      <td>1.0</td>
    </tr>
    <tr>
      <th>8551</th>
      <td>0.900000</td>
      <td>0.000000</td>
      <td>0.497665</td>
      <td>0.485831</td>
      <td>1.0</td>
    </tr>
    <tr>
      <th>8552</th>
      <td>0.688889</td>
      <td>0.000000</td>
      <td>0.497665</td>
      <td>0.485831</td>
      <td>1.0</td>
    </tr>
  </tbody>
</table>
<p>8553 rows × 5 columns</p>
</div>



# トレーニング

データをトレーニングデータとテストデータに分割します


```
X_train, X_test, y_train, y_test = train_test_split(X, y, train_size=0.8)
```


```
今回はグランドマスターか否かという判定なので ロジスティック回帰を使用します。
```


```
model = LogisticRegression()
model.fit(X_train, y_train)
```




    LogisticRegression(C=1.0, class_weight=None, dual=False, fit_intercept=True,
                       intercept_scaling=1, l1_ratio=None, max_iter=100,
                       multi_class='auto', n_jobs=None, penalty='l2',
                       random_state=None, solver='lbfgs', tol=0.0001, verbose=0,
                       warm_start=False)



# 結果

モデルの精度


```
print(f'Model Accuracy: {model.score(X_test, y_test)}')
```

    Model Accuracy: 0.9959088252483927


とても良い精度がでましたが、F1スコアを見てみます


```
y_pred = model.predict(X_test)
```


```
print(f'Model F1 Score: {f1_score(y_test, y_pred)}')
```

    Model F1 Score: 0.2222222222222222


低い結果になりました。
そもそもテストデータの0,1はそれぞれどのくらいの割合なのでしょうか。



```
print(f'Percent Grandmaster: {y_test.sum() / len(y)}')
```

    Percent Grandmaster: 0.0009353443236291359


これからわかる通り、0,1の割合にかなり差があります。こういった場合、単純にすべてをグランド・マスターではないと判断するモデルでも相当な精度がでてしまいます。もっと大量のサンプルが必要かなと思います。