kaggle/titanic 欠損値の補完と特徴量エンジニアリング


#テーマ : kaggle/titanic における、特徴量エンジニアリングと欠損値の補完について

今回こちらの課題を行うにあたり、特徴量と欠損値に目を付けた予測モデルの開発を行った。

特徴量エンジニアリングは、機械学習モデルのパフォーマンスを向上させるために、特徴量とも呼ばれる追加の予測因子を構築してデータセットに追加する ... つまり、モデルの予測精度を高めるための次の最前線は、データセット自体を改善することである。

-

##今回の特徴量

kaggle/titanic

- PassengerId: 乗客のID
- Survived: 生存(1)、死亡(0)
- Pclass: 乗客の階級
- Name: 乗客の名前
- Sex: 性別
- Age: 年齢
- SibSp: タイタニック号に乗っていた兄弟、姉妹、義兄弟、義姉妹、夫、妻の数（自分を除く）
- Parch: タイタニック号に乗っていた母親、父親、息子、娘の数
- Ticket: チケットナンバー
- Fare: 乗船料金
- Cabin: キャビン番号
- Embarked: 乗船場
 


##予測方法

titanicの特徴量を考察して新しく特徴量を作りだし、trainデータを決定木で分類したものをリーフごとにロジスティック回帰で分類し、最後に勾配ブースティングにかけたモデルにより最終的な予測を行った。


-


##はじめに

まずは、ライブラリのインポートとtrain/testデータの読み込みとデータの中身を確認。


```python
import pandas as pd
import numpy as np
from sklearn.linear_model import LogisticRegression  # as LR
import matplotlib.pyplot as plt
from sklearn.grid_search import GridSearchCV

from sklearn.model_selection import cross_val_score
from sklearn.tree import DecisionTreeClassifier
from sklearn import tree
from sklearn.ensemble import RandomForestClassifier, GradientBoostingClassifier

#データの取得
train = pd.read_csv("~/train.csv")
test = pd.read_csv("~/test.csv")

train.head(10)
```

<div>
<style scoped>
</style>
<table border="1" class="dataframe">
  <thead>
    <tr style="text-align: right;">
      <th></th>
      <th>PassengerId</th>
      <th>Survived</th>
      <th>Pclass</th>
      <th>Name</th>
      <th>Sex</th>
      <th>Age</th>
      <th>SibSp</th>
      <th>Parch</th>
      <th>Ticket</th>
      <th>Fare</th>
      <th>Cabin</th>
      <th>Embarked</th>
    </tr>
  </thead>
  <tbody>
    <tr>
      <th>0</th>
      <td>1</td>
      <td>0</td>
      <td>3</td>
      <td>Braund, Mr. Owen Harris</td>
      <td>male</td>
      <td>22.0</td>
      <td>1</td>
      <td>0</td>
      <td>A/5 21171</td>
      <td>7.2500</td>
      <td>NaN</td>
      <td>S</td>
    </tr>
    <tr>
      <th>1</th>
      <td>2</td>
      <td>1</td>
      <td>1</td>
      <td>Cumings, Mrs. John Bradley (Florence Briggs Th...</td>
      <td>female</td>
      <td>38.0</td>
      <td>1</td>
      <td>0</td>
      <td>PC 17599</td>
      <td>71.2833</td>
      <td>C85</td>
      <td>C</td>
    </tr>
    <tr>
      <th>2</th>
      <td>3</td>
      <td>1</td>
      <td>3</td>
      <td>Heikkinen, Miss. Laina</td>
      <td>female</td>
      <td>26.0</td>
      <td>0</td>
      <td>0</td>
      <td>STON/O2. 3101282</td>
      <td>7.9250</td>
      <td>NaN</td>
      <td>S</td>
    </tr>
    <tr>
      <th>3</th>
      <td>4</td>
      <td>1</td>
      <td>1</td>
      <td>Futrelle, Mrs. Jacques Heath (Lily May Peel)</td>
      <td>female</td>
      <td>35.0</td>
      <td>1</td>
      <td>0</td>
      <td>113803</td>
      <td>53.1000</td>
      <td>C123</td>
      <td>S</td>
    </tr>
    <tr>
      <th>4</th>
      <td>5</td>
      <td>0</td>
      <td>3</td>
      <td>Allen, Mr. William Henry</td>
      <td>male</td>
      <td>35.0</td>
      <td>0</td>
      <td>0</td>
      <td>373450</td>
      <td>8.0500</td>
      <td>NaN</td>
      <td>S</td>
    </tr>
    <tr>
      <th>5</th>
      <td>6</td>
      <td>0</td>
      <td>3</td>
      <td>Moran, Mr. James</td>
      <td>male</td>
      <td>NaN</td>
      <td>0</td>
      <td>0</td>
      <td>330877</td>
      <td>8.4583</td>
      <td>NaN</td>
      <td>Q</td>
    </tr>
    <tr>
      <th>6</th>
      <td>7</td>
      <td>0</td>
      <td>1</td>
      <td>McCarthy, Mr. Timothy J</td>
      <td>male</td>
      <td>54.0</td>
      <td>0</td>
      <td>0</td>
      <td>17463</td>
      <td>51.8625</td>
      <td>E46</td>
      <td>S</td>
    </tr>
    <tr>
      <th>7</th>
      <td>8</td>
      <td>0</td>
      <td>3</td>
      <td>Palsson, Master. Gosta Leonard</td>
      <td>male</td>
      <td>2.0</td>
      <td>3</td>
      <td>1</td>
      <td>349909</td>
      <td>21.0750</td>
      <td>NaN</td>
      <td>S</td>
    </tr>
    <tr>
      <th>8</th>
      <td>9</td>
      <td>1</td>
      <td>3</td>
      <td>Johnson, Mrs. Oscar W (Elisabeth Vilhelmina Berg)</td>
      <td>female</td>
      <td>27.0</td>
      <td>0</td>
      <td>2</td>
      <td>347742</td>
      <td>11.1333</td>
      <td>NaN</td>
      <td>S</td>
    </tr>
    <tr>
      <th>9</th>
      <td>10</td>
      <td>1</td>
      <td>2</td>
      <td>Nasser, Mrs. Nicholas (Adele Achem)</td>
      <td>female</td>
      <td>14.0</td>
      <td>1</td>
      <td>0</td>
      <td>237736</td>
      <td>30.0708</td>
      <td>NaN</td>
      <td>C</td>
    </tr>
  </tbody>
</table>
</div>





##特徴量の作成

head()でデータの中身を確認後、新しく作成できる特徴量を考察し、その結果家族の人数と旅行者が団体旅行者か単身旅行者かの特徴量を作り出せることを思い付いたので早速作成することにした

考察元のカラム名:
SibSp: タイタニック号に乗っていた兄弟、姉妹、義兄弟、義姉妹、夫、妻の数（自分を除く）
Parch: タイタニック号に乗っていた母親、父親、息子、娘の数

SibSp + Pearch + 1　を家族数として捉え(今回名前は考慮せず)
それを元に単体旅行者(1人)と団体旅行者(2人以上)を区別


```python
#単身旅行者と団体旅行者の特徴量を新しく作成
train['Family_group'] = train.SibSp + train.Parch
test['Family_group'] = test.SibSp + test.Parch

#家族人数の特徴量を新しく作成
train['Family'] = train.SibSp + train.Parch + 1
test['Family'] = test.SibSp + test.Parch + 1

#Family_groupの特徴量をダミー変数に変換
train['Family_group'] = np.where(train['Family_group'] >= 1, 1, 0)
test['Family_group'] = np.where(test['Family_group'] >= 1, 1, 0)

train.head(10)
```




<div>
<style scoped>
   
</style>
<table border="1" class="dataframe">
  <thead>
    <tr style="text-align: right;">
      <th></th>
      <th>PassengerId</th>
      <th>Survived</th>
      <th>Pclass</th>
      <th>Name</th>
      <th>Sex</th>
      <th>Age</th>
      <th>SibSp</th>
      <th>Parch</th>
      <th>Ticket</th>
      <th>Fare</th>
      <th>Cabin</th>
      <th>Embarked</th>
      <th>Family_group</th>
      <th>Family</th>
    </tr>
  </thead>
  <tbody>
    <tr>
      <th>0</th>
      <td>1</td>
      <td>0</td>
      <td>3</td>
      <td>Braund, Mr. Owen Harris</td>
      <td>male</td>
      <td>22.0</td>
      <td>1</td>
      <td>0</td>
      <td>A/5 21171</td>
      <td>7.2500</td>
      <td>NaN</td>
      <td>S</td>
      <td>1</td>
      <td>2</td>
    </tr>
    <tr>
      <th>1</th>
      <td>2</td>
      <td>1</td>
      <td>1</td>
      <td>Cumings, Mrs. John Bradley (Florence Briggs Th...</td>
      <td>female</td>
      <td>38.0</td>
      <td>1</td>
      <td>0</td>
      <td>PC 17599</td>
      <td>71.2833</td>
      <td>C85</td>
      <td>C</td>
      <td>1</td>
      <td>2</td>
    </tr>
    <tr>
      <th>2</th>
      <td>3</td>
      <td>1</td>
      <td>3</td>
      <td>Heikkinen, Miss. Laina</td>
      <td>female</td>
      <td>26.0</td>
      <td>0</td>
      <td>0</td>
      <td>STON/O2. 3101282</td>
      <td>7.9250</td>
      <td>NaN</td>
      <td>S</td>
      <td>0</td>
      <td>1</td>
    </tr>
    <tr>
      <th>3</th>
      <td>4</td>
      <td>1</td>
      <td>1</td>
      <td>Futrelle, Mrs. Jacques Heath (Lily May Peel)</td>
      <td>female</td>
      <td>35.0</td>
      <td>1</td>
      <td>0</td>
      <td>113803</td>
      <td>53.1000</td>
      <td>C123</td>
      <td>S</td>
      <td>1</td>
      <td>2</td>
    </tr>
    <tr>
      <th>4</th>
      <td>5</td>
      <td>0</td>
      <td>3</td>
      <td>Allen, Mr. William Henry</td>
      <td>male</td>
      <td>35.0</td>
      <td>0</td>
      <td>0</td>
      <td>373450</td>
      <td>8.0500</td>
      <td>NaN</td>
      <td>S</td>
      <td>0</td>
      <td>1</td>
    </tr>
    <tr>
      <th>5</th>
      <td>6</td>
      <td>0</td>
      <td>3</td>
      <td>Moran, Mr. James</td>
      <td>male</td>
      <td>NaN</td>
      <td>0</td>
      <td>0</td>
      <td>330877</td>
      <td>8.4583</td>
      <td>NaN</td>
      <td>Q</td>
      <td>0</td>
      <td>1</td>
    </tr>
    <tr>
      <th>6</th>
      <td>7</td>
      <td>0</td>
      <td>1</td>
      <td>McCarthy, Mr. Timothy J</td>
      <td>male</td>
      <td>54.0</td>
      <td>0</td>
      <td>0</td>
      <td>17463</td>
      <td>51.8625</td>
      <td>E46</td>
      <td>S</td>
      <td>0</td>
      <td>1</td>
    </tr>
    <tr>
      <th>7</th>
      <td>8</td>
      <td>0</td>
      <td>3</td>
      <td>Palsson, Master. Gosta Leonard</td>
      <td>male</td>
      <td>2.0</td>
      <td>3</td>
      <td>1</td>
      <td>349909</td>
      <td>21.0750</td>
      <td>NaN</td>
      <td>S</td>
      <td>1</td>
      <td>5</td>
    </tr>
    <tr>
      <th>8</th>
      <td>9</td>
      <td>1</td>
      <td>3</td>
      <td>Johnson, Mrs. Oscar W (Elisabeth Vilhelmina Berg)</td>
      <td>female</td>
      <td>27.0</td>
      <td>0</td>
      <td>2</td>
      <td>347742</td>
      <td>11.1333</td>
      <td>NaN</td>
      <td>S</td>
      <td>1</td>
      <td>3</td>
    </tr>
    <tr>
      <th>9</th>
      <td>10</td>
      <td>1</td>
      <td>2</td>
      <td>Nasser, Mrs. Nicholas (Adele Achem)</td>
      <td>female</td>
      <td>14.0</td>
      <td>1</td>
      <td>0</td>
      <td>237736</td>
      <td>30.0708</td>
      <td>NaN</td>
      <td>C</td>
      <td>1</td>
      <td>2</td>
    </tr>
  </tbody>
</table>
</div>

###脱出船に乗れる確率

旅行者のグループ分けを行なった後にそれらの特徴量を使用して脱出船のある場所まで着いた際に特定の人物が脱出船に乗れるかどうかを確率で求めた特徴量を作成する。

```python
#単身旅行者の場合は確実に乗れるので確率は１とする。

#団体旅行者の中で敬称が無い人の脱出船に乗れる確率
escape_boarding_probability_average_train = 2 / ((sum(train['Family']) - sum(train['Family_group'] == 0))/sum(train['Family_group'] == 1))
escape_boarding_probability_average_test = 2 / ((sum(test['Family']) - sum(test['Family_group'] == 0))/sum(test['Family_group'] == 1))

print("団体旅行者の中で敬称が無い人の脱出船に乗れる確率(1／団体旅行者の中での平均家族数) : ")
print(str(escape_boarding_probability_average_train))
```

    団体旅行者の中で敬称が無い人の脱出船に乗れる確率(1／団体旅行者の中での平均家族数) : 
    0.6103448275862069



単身旅行者が脱出船に乗る順番が来た際の脱出船に乗れる確率は１００％なので’１’を代入しそれ以外は一先ず’NaN’をセットしておく。

```python
train['escape_boarding_probability_train'] = 1
test['escape_boarding_probability_test'] = 1

train['escape_boarding_probability_train'] = train['escape_boarding_probability_train'].replace(1, np.nan)
test['escape_boarding_probability_test'] = test['escape_boarding_probability_test'].replace(1, np.nan)

for i in range(1, 891):
    if (train['Family_group'][i] == 0):
        train['escape_boarding_probability_train'][i] = 1

for i in range(1, 418):
    if (test['Family_group'][i] == 0):
        test['escape_boarding_probability_test'][i] = 1        
train.head(100)        
```

<div>
<style scoped>
   
</style>
<table border="1" class="dataframe">
  <thead>
    <tr style="text-align: right;">
      <th></th>
      <th>PassengerId</th>
      <th>Survived</th>
      <th>Pclass</th>
      <th>Name</th>
      <th>Sex</th>
      <th>Age</th>
      <th>SibSp</th>
      <th>Parch</th>
      <th>Ticket</th>
      <th>Fare</th>
      <th>Cabin</th>
      <th>Embarked</th>
      <th>Family_group</th>
      <th>Family</th>
      <th>escape_boarding_probability_train</th>
    </tr>
  </thead>
  <tbody>
    <tr>
      <th>0</th>
      <td>1</td>
      <td>0</td>
      <td>3</td>
      <td>Braund, Mr. Owen Harris</td>
      <td>male</td>
      <td>22.00</td>
      <td>1</td>
      <td>0</td>
      <td>A/5 21171</td>
      <td>7.2500</td>
      <td>NaN</td>
      <td>S</td>
      <td>1</td>
      <td>2</td>
      <td>NaN</td>
    </tr>
    <tr>
      <th>1</th>
      <td>2</td>
      <td>1</td>
      <td>1</td>
      <td>Cumings, Mrs. John Bradley (Florence Briggs Th...</td>
      <td>female</td>
      <td>38.00</td>
      <td>1</td>
      <td>0</td>
      <td>PC 17599</td>
      <td>71.2833</td>
      <td>C85</td>
      <td>C</td>
      <td>1</td>
      <td>2</td>
      <td>NaN</td>
    </tr>
    <tr>
      <th>2</th>
      <td>3</td>
      <td>1</td>
      <td>3</td>
      <td>Heikkinen, Miss. Laina</td>
      <td>female</td>
      <td>26.00</td>
      <td>0</td>
      <td>0</td>
      <td>STON/O2. 3101282</td>
      <td>7.9250</td>
      <td>NaN</td>
      <td>S</td>
      <td>0</td>
      <td>1</td>
      <td>1.0</td>
    </tr>
    <tr>
      <th>3</th>
      <td>4</td>
      <td>1</td>
      <td>1</td>
      <td>Futrelle, Mrs. Jacques Heath (Lily May Peel)</td>
      <td>female</td>
      <td>35.00</td>
      <td>1</td>
      <td>0</td>
      <td>113803</td>
      <td>53.1000</td>
      <td>C123</td>
      <td>S</td>
      <td>1</td>
      <td>2</td>
      <td>NaN</td>
    </tr>
    <tr>
      <th>4</th>
      <td>5</td>
      <td>0</td>
      <td>3</td>
      <td>Allen, Mr. William Henry</td>
      <td>male</td>
      <td>35.00</td>
      <td>0</td>
      <td>0</td>
      <td>373450</td>
      <td>8.0500</td>
      <td>NaN</td>
      <td>S</td>
      <td>0</td>
      <td>1</td>
      <td>1.0</td>
    </tr>
    <tr>
      <th>5</th>
      <td>6</td>
      <td>0</td>
      <td>3</td>
      <td>Moran, Mr. James</td>
      <td>male</td>
      <td>NaN</td>
      <td>0</td>
      <td>0</td>
      <td>330877</td>
      <td>8.4583</td>
      <td>NaN</td>
      <td>Q</td>
      <td>0</td>
      <td>1</td>
      <td>1.0</td>
    </tr>
    <tr>
      <th>6</th>
      <td>7</td>
      <td>0</td>
      <td>1</td>
      <td>McCarthy, Mr. Timothy J</td>
      <td>male</td>
      <td>54.00</td>
      <td>0</td>
      <td>0</td>
      <td>17463</td>
      <td>51.8625</td>
      <td>E46</td>
      <td>S</td>
      <td>0</td>
      <td>1</td>
      <td>1.0</td>
    </tr>
    <tr>
      <th>7</th>
      <td>8</td>
      <td>0</td>
      <td>3</td>
      <td>Palsson, Master. Gosta Leonard</td>
      <td>male</td>
      <td>2.00</td>
      <td>3</td>
      <td>1</td>
      <td>349909</td>
      <td>21.0750</td>
      <td>NaN</td>
      <td>S</td>
      <td>1</td>
      <td>5</td>
      <td>NaN</td>
    </tr>
    <tr>
      <th>8</th>
      <td>9</td>
      <td>1</td>
      <td>3</td>
      <td>Johnson, Mrs. Oscar W (Elisabeth Vilhelmina Berg)</td>
      <td>female</td>
      <td>27.00</td>
      <td>0</td>
      <td>2</td>
      <td>347742</td>
      <td>11.1333</td>
      <td>NaN</td>
      <td>S</td>
      <td>1</td>
      <td>3</td>
      <td>NaN</td>
    </tr>
    <tr>
      <th>9</th>
      <td>10</td>
      <td>1</td>
      <td>2</td>
      <td>Nasser, Mrs. Nicholas (Adele Achem)</td>
      <td>female</td>
      <td>14.00</td>
      <td>1</td>
      <td>0</td>
      <td>237736</td>
      <td>30.0708</td>
      <td>NaN</td>
      <td>C</td>
      <td>1</td>
      <td>2</td>
      <td>NaN</td>
    </tr>
    <tr>
      <th>10</th>
      <td>11</td>
      <td>1</td>
      <td>3</td>
      <td>Sandstrom, Miss. Marguerite Rut</td>
      <td>female</td>
      <td>4.00</td>
      <td>1</td>
      <td>1</td>
      <td>PP 9549</td>
      <td>16.7000</td>
      <td>G6</td>
      <td>S</td>
      <td>1</td>
      <td>3</td>
      <td>NaN</td>
    </tr>
    <tr>
      <th>11</th>
      <td>12</td>
      <td>1</td>
      <td>1</td>
      <td>Bonnell, Miss. Elizabeth</td>
      <td>female</td>
      <td>58.00</td>
      <td>0</td>
      <td>0</td>
      <td>113783</td>
      <td>26.5500</td>
      <td>C103</td>
      <td>S</td>
      <td>0</td>
      <td>1</td>
      <td>1.0</td>
    </tr>
    <tr>
      <th>12</th>
      <td>13</td>
      <td>0</td>
      <td>3</td>
      <td>Saundercock, Mr. William Henry</td>
      <td>male</td>
      <td>20.00</td>
      <td>0</td>
      <td>0</td>
      <td>A/5. 2151</td>
      <td>8.0500</td>
      <td>NaN</td>
      <td>S</td>
      <td>0</td>
      <td>1</td>
      <td>1.0</td>
    </tr>
    <tr>
      <th>13</th>
      <td>14</td>
      <td>0</td>
      <td>3</td>
      <td>Andersson, Mr. Anders Johan</td>
      <td>male</td>
      <td>39.00</td>
      <td>1</td>
      <td>5</td>
      <td>347082</td>
      <td>31.2750</td>
      <td>NaN</td>
      <td>S</td>
      <td>1</td>
      <td>7</td>
      <td>NaN</td>
    </tr>
    <tr>
      <th>14</th>
      <td>15</td>
      <td>0</td>
      <td>3</td>
      <td>Vestrom, Miss. Hulda Amanda Adolfina</td>
      <td>female</td>
      <td>14.00</td>
      <td>0</td>
      <td>0</td>
      <td>350406</td>
      <td>7.8542</td>
      <td>NaN</td>
      <td>S</td>
      <td>0</td>
      <td>1</td>
      <td>1.0</td>
    </tr>
    <tr>
      <th>15</th>
      <td>16</td>
      <td>1</td>
      <td>2</td>
      <td>Hewlett, Mrs. (Mary D Kingcome)</td>
      <td>female</td>
      <td>55.00</td>
      <td>0</td>
      <td>0</td>
      <td>248706</td>
      <td>16.0000</td>
      <td>NaN</td>
      <td>S</td>
      <td>0</td>
      <td>1</td>
      <td>1.0</td>
    </tr>
    <tr>
      <th>16</th>
      <td>17</td>
      <td>0</td>
      <td>3</td>
      <td>Rice, Master. Eugene</td>
      <td>male</td>
      <td>2.00</td>
      <td>4</td>
      <td>1</td>
      <td>382652</td>
      <td>29.1250</td>
      <td>NaN</td>
      <td>Q</td>
      <td>1</td>
      <td>6</td>
      <td>NaN</td>
    </tr>
    <tr>
      <th>17</th>
      <td>18</td>
      <td>1</td>
      <td>2</td>
      <td>Williams, Mr. Charles Eugene</td>
      <td>male</td>
      <td>NaN</td>
      <td>0</td>
      <td>0</td>
      <td>244373</td>
      <td>13.0000</td>
      <td>NaN</td>
      <td>S</td>
      <td>0</td>
      <td>1</td>
      <td>1.0</td>
    </tr>
    <tr>
      <th>18</th>
      <td>19</td>
      <td>0</td>
      <td>3</td>
      <td>Vander Planke, Mrs. Julius (Emelia Maria Vande...</td>
      <td>female</td>
      <td>31.00</td>
      <td>1</td>
      <td>0</td>
      <td>345763</td>
      <td>18.0000</td>
      <td>NaN</td>
      <td>S</td>
      <td>1</td>
      <td>2</td>
      <td>NaN</td>
    </tr>
    <tr>
      <th>19</th>
      <td>20</td>
      <td>1</td>
      <td>3</td>
      <td>Masselmani, Mrs. Fatima</td>
      <td>female</td>
      <td>NaN</td>
      <td>0</td>
      <td>0</td>
      <td>2649</td>
      <td>7.2250</td>
      <td>NaN</td>
      <td>C</td>
      <td>0</td>
      <td>1</td>
      <td>1.0</td>
    </tr>
    <tr>
      <th>20</th>
      <td>21</td>
      <td>0</td>
      <td>2</td>
      <td>Fynney, Mr. Joseph J</td>
      <td>male</td>
      <td>35.00</td>
      <td>0</td>
      <td>0</td>
      <td>239865</td>
      <td>26.0000</td>
      <td>NaN</td>
      <td>S</td>
      <td>0</td>
      <td>1</td>
      <td>1.0</td>
    </tr>
    <tr>
      <th>21</th>
      <td>22</td>
      <td>1</td>
      <td>2</td>
      <td>Beesley, Mr. Lawrence</td>
      <td>male</td>
      <td>34.00</td>
      <td>0</td>
      <td>0</td>
      <td>248698</td>
      <td>13.0000</td>
      <td>D56</td>
      <td>S</td>
      <td>0</td>
      <td>1</td>
      <td>1.0</td>
    </tr>
    <tr>
      <th>22</th>
      <td>23</td>
      <td>1</td>
      <td>3</td>
      <td>McGowan, Miss. Anna "Annie"</td>
      <td>female</td>
      <td>15.00</td>
      <td>0</td>
      <td>0</td>
      <td>330923</td>
      <td>8.0292</td>
      <td>NaN</td>
      <td>Q</td>
      <td>0</td>
      <td>1</td>
      <td>1.0</td>
    </tr>
    <tr>
      <th>23</th>
      <td>24</td>
      <td>1</td>
      <td>1</td>
      <td>Sloper, Mr. William Thompson</td>
      <td>male</td>
      <td>28.00</td>
      <td>0</td>
      <td>0</td>
      <td>113788</td>
      <td>35.5000</td>
      <td>A6</td>
      <td>S</td>
      <td>0</td>
      <td>1</td>
      <td>1.0</td>
    </tr>
    <tr>
      <th>24</th>
      <td>25</td>
      <td>0</td>
      <td>3</td>
      <td>Palsson, Miss. Torborg Danira</td>
      <td>female</td>
      <td>8.00</td>
      <td>3</td>
      <td>1</td>
      <td>349909</td>
      <td>21.0750</td>
      <td>NaN</td>
      <td>S</td>
      <td>1</td>
      <td>5</td>
      <td>NaN</td>
    </tr>
    <tr>
      <th>25</th>
      <td>26</td>
      <td>1</td>
      <td>3</td>
      <td>Asplund, Mrs. Carl Oscar (Selma Augusta Emilia...</td>
      <td>female</td>
      <td>38.00</td>
      <td>1</td>
      <td>5</td>
      <td>347077</td>
      <td>31.3875</td>
      <td>NaN</td>
      <td>S</td>
      <td>1</td>
      <td>7</td>
      <td>NaN</td>
    </tr>
    <tr>
      <th>26</th>
      <td>27</td>
      <td>0</td>
      <td>3</td>
      <td>Emir, Mr. Farred Chehab</td>
      <td>male</td>
      <td>NaN</td>
      <td>0</td>
      <td>0</td>
      <td>2631</td>
      <td>7.2250</td>
      <td>NaN</td>
      <td>C</td>
      <td>0</td>
      <td>1</td>
      <td>1.0</td>
    </tr>
    <tr>
      <th>27</th>
      <td>28</td>
      <td>0</td>
      <td>1</td>
      <td>Fortune, Mr. Charles Alexander</td>
      <td>male</td>
      <td>19.00</td>
      <td>3</td>
      <td>2</td>
      <td>19950</td>
      <td>263.0000</td>
      <td>C23 C25 C27</td>
      <td>S</td>
      <td>1</td>
      <td>6</td>
      <td>NaN</td>
    </tr>
    <tr>
      <th>28</th>
      <td>29</td>
      <td>1</td>
      <td>3</td>
      <td>O'Dwyer, Miss. Ellen "Nellie"</td>
      <td>female</td>
      <td>NaN</td>
      <td>0</td>
      <td>0</td>
      <td>330959</td>
      <td>7.8792</td>
      <td>NaN</td>
      <td>Q</td>
      <td>0</td>
      <td>1</td>
      <td>1.0</td>
    </tr>
    <tr>
      <th>29</th>
      <td>30</td>
      <td>0</td>
      <td>3</td>
      <td>Todoroff, Mr. Lalio</td>
      <td>male</td>
      <td>NaN</td>
      <td>0</td>
      <td>0</td>
      <td>349216</td>
      <td>7.8958</td>
      <td>NaN</td>
      <td>S</td>
      <td>0</td>
      <td>1</td>
      <td>1.0</td>
    </tr>
    <tr>
      <th>...</th>
      <td>...</td>
      <td>...</td>
      <td>...</td>
      <td>...</td>
      <td>...</td>
      <td>...</td>
      <td>...</td>
      <td>...</td>
      <td>...</td>
      <td>...</td>
      <td>...</td>
      <td>...</td>
      <td>...</td>
      <td>...</td>
      <td>...</td>
    </tr>
    <tr>
      <th>70</th>
      <td>71</td>
      <td>0</td>
      <td>2</td>
      <td>Jenkin, Mr. Stephen Curnow</td>
      <td>male</td>
      <td>32.00</td>
      <td>0</td>
      <td>0</td>
      <td>C.A. 33111</td>
      <td>10.5000</td>
      <td>NaN</td>
      <td>S</td>
      <td>0</td>
      <td>1</td>
      <td>1.0</td>
    </tr>
    <tr>
      <th>71</th>
      <td>72</td>
      <td>0</td>
      <td>3</td>
      <td>Goodwin, Miss. Lillian Amy</td>
      <td>female</td>
      <td>16.00</td>
      <td>5</td>
      <td>2</td>
      <td>CA 2144</td>
      <td>46.9000</td>
      <td>NaN</td>
      <td>S</td>
      <td>1</td>
      <td>8</td>
      <td>NaN</td>
    </tr>
    <tr>
      <th>72</th>
      <td>73</td>
      <td>0</td>
      <td>2</td>
      <td>Hood, Mr. Ambrose Jr</td>
      <td>male</td>
      <td>21.00</td>
      <td>0</td>
      <td>0</td>
      <td>S.O.C. 14879</td>
      <td>73.5000</td>
      <td>NaN</td>
      <td>S</td>
      <td>0</td>
      <td>1</td>
      <td>1.0</td>
    </tr>
    <tr>
      <th>73</th>
      <td>74</td>
      <td>0</td>
      <td>3</td>
      <td>Chronopoulos, Mr. Apostolos</td>
      <td>male</td>
      <td>26.00</td>
      <td>1</td>
      <td>0</td>
      <td>2680</td>
      <td>14.4542</td>
      <td>NaN</td>
      <td>C</td>
      <td>1</td>
      <td>2</td>
      <td>NaN</td>
    </tr>
    <tr>
      <th>74</th>
      <td>75</td>
      <td>1</td>
      <td>3</td>
      <td>Bing, Mr. Lee</td>
      <td>male</td>
      <td>32.00</td>
      <td>0</td>
      <td>0</td>
      <td>1601</td>
      <td>56.4958</td>
      <td>NaN</td>
      <td>S</td>
      <td>0</td>
      <td>1</td>
      <td>1.0</td>
    </tr>
    <tr>
      <th>75</th>
      <td>76</td>
      <td>0</td>
      <td>3</td>
      <td>Moen, Mr. Sigurd Hansen</td>
      <td>male</td>
      <td>25.00</td>
      <td>0</td>
      <td>0</td>
      <td>348123</td>
      <td>7.6500</td>
      <td>F G73</td>
      <td>S</td>
      <td>0</td>
      <td>1</td>
      <td>1.0</td>
    </tr>
    <tr>
      <th>76</th>
      <td>77</td>
      <td>0</td>
      <td>3</td>
      <td>Staneff, Mr. Ivan</td>
      <td>male</td>
      <td>NaN</td>
      <td>0</td>
      <td>0</td>
      <td>349208</td>
      <td>7.8958</td>
      <td>NaN</td>
      <td>S</td>
      <td>0</td>
      <td>1</td>
      <td>1.0</td>
    </tr>
    <tr>
      <th>77</th>
      <td>78</td>
      <td>0</td>
      <td>3</td>
      <td>Moutal, Mr. Rahamin Haim</td>
      <td>male</td>
      <td>NaN</td>
      <td>0</td>
      <td>0</td>
      <td>374746</td>
      <td>8.0500</td>
      <td>NaN</td>
      <td>S</td>
      <td>0</td>
      <td>1</td>
      <td>1.0</td>
    </tr>
    <tr>
      <th>78</th>
      <td>79</td>
      <td>1</td>
      <td>2</td>
      <td>Caldwell, Master. Alden Gates</td>
      <td>male</td>
      <td>0.83</td>
      <td>0</td>
      <td>2</td>
      <td>248738</td>
      <td>29.0000</td>
      <td>NaN</td>
      <td>S</td>
      <td>1</td>
      <td>3</td>
      <td>NaN</td>
    </tr>
    <tr>
      <th>79</th>
      <td>80</td>
      <td>1</td>
      <td>3</td>
      <td>Dowdell, Miss. Elizabeth</td>
      <td>female</td>
      <td>30.00</td>
      <td>0</td>
      <td>0</td>
      <td>364516</td>
      <td>12.4750</td>
      <td>NaN</td>
      <td>S</td>
      <td>0</td>
      <td>1</td>
      <td>1.0</td>
    </tr>
    <tr>
      <th>80</th>
      <td>81</td>
      <td>0</td>
      <td>3</td>
      <td>Waelens, Mr. Achille</td>
      <td>male</td>
      <td>22.00</td>
      <td>0</td>
      <td>0</td>
      <td>345767</td>
      <td>9.0000</td>
      <td>NaN</td>
      <td>S</td>
      <td>0</td>
      <td>1</td>
      <td>1.0</td>
    </tr>
    <tr>
      <th>81</th>
      <td>82</td>
      <td>1</td>
      <td>3</td>
      <td>Sheerlinck, Mr. Jan Baptist</td>
      <td>male</td>
      <td>29.00</td>
      <td>0</td>
      <td>0</td>
      <td>345779</td>
      <td>9.5000</td>
      <td>NaN</td>
      <td>S</td>
      <td>0</td>
      <td>1</td>
      <td>1.0</td>
    </tr>
    <tr>
      <th>82</th>
      <td>83</td>
      <td>1</td>
      <td>3</td>
      <td>McDermott, Miss. Brigdet Delia</td>
      <td>female</td>
      <td>NaN</td>
      <td>0</td>
      <td>0</td>
      <td>330932</td>
      <td>7.7875</td>
      <td>NaN</td>
      <td>Q</td>
      <td>0</td>
      <td>1</td>
      <td>1.0</td>
    </tr>
    <tr>
      <th>83</th>
      <td>84</td>
      <td>0</td>
      <td>1</td>
      <td>Carrau, Mr. Francisco M</td>
      <td>male</td>
      <td>28.00</td>
      <td>0</td>
      <td>0</td>
      <td>113059</td>
      <td>47.1000</td>
      <td>NaN</td>
      <td>S</td>
      <td>0</td>
      <td>1</td>
      <td>1.0</td>
    </tr>
    <tr>
      <th>84</th>
      <td>85</td>
      <td>1</td>
      <td>2</td>
      <td>Ilett, Miss. Bertha</td>
      <td>female</td>
      <td>17.00</td>
      <td>0</td>
      <td>0</td>
      <td>SO/C 14885</td>
      <td>10.5000</td>
      <td>NaN</td>
      <td>S</td>
      <td>0</td>
      <td>1</td>
      <td>1.0</td>
    </tr>
    <tr>
      <th>85</th>
      <td>86</td>
      <td>1</td>
      <td>3</td>
      <td>Backstrom, Mrs. Karl Alfred (Maria Mathilda Gu...</td>
      <td>female</td>
      <td>33.00</td>
      <td>3</td>
      <td>0</td>
      <td>3101278</td>
      <td>15.8500</td>
      <td>NaN</td>
      <td>S</td>
      <td>1</td>
      <td>4</td>
      <td>NaN</td>
    </tr>
    <tr>
      <th>86</th>
      <td>87</td>
      <td>0</td>
      <td>3</td>
      <td>Ford, Mr. William Neal</td>
      <td>male</td>
      <td>16.00</td>
      <td>1</td>
      <td>3</td>
      <td>W./C. 6608</td>
      <td>34.3750</td>
      <td>NaN</td>
      <td>S</td>
      <td>1</td>
      <td>5</td>
      <td>NaN</td>
    </tr>
    <tr>
      <th>87</th>
      <td>88</td>
      <td>0</td>
      <td>3</td>
      <td>Slocovski, Mr. Selman Francis</td>
      <td>male</td>
      <td>NaN</td>
      <td>0</td>
      <td>0</td>
      <td>SOTON/OQ 392086</td>
      <td>8.0500</td>
      <td>NaN</td>
      <td>S</td>
      <td>0</td>
      <td>1</td>
      <td>1.0</td>
    </tr>
    <tr>
      <th>88</th>
      <td>89</td>
      <td>1</td>
      <td>1</td>
      <td>Fortune, Miss. Mabel Helen</td>
      <td>female</td>
      <td>23.00</td>
      <td>3</td>
      <td>2</td>
      <td>19950</td>
      <td>263.0000</td>
      <td>C23 C25 C27</td>
      <td>S</td>
      <td>1</td>
      <td>6</td>
      <td>NaN</td>
    </tr>
    <tr>
      <th>89</th>
      <td>90</td>
      <td>0</td>
      <td>3</td>
      <td>Celotti, Mr. Francesco</td>
      <td>male</td>
      <td>24.00</td>
      <td>0</td>
      <td>0</td>
      <td>343275</td>
      <td>8.0500</td>
      <td>NaN</td>
      <td>S</td>
      <td>0</td>
      <td>1</td>
      <td>1.0</td>
    </tr>
    <tr>
      <th>90</th>
      <td>91</td>
      <td>0</td>
      <td>3</td>
      <td>Christmann, Mr. Emil</td>
      <td>male</td>
      <td>29.00</td>
      <td>0</td>
      <td>0</td>
      <td>343276</td>
      <td>8.0500</td>
      <td>NaN</td>
      <td>S</td>
      <td>0</td>
      <td>1</td>
      <td>1.0</td>
    </tr>
    <tr>
      <th>91</th>
      <td>92</td>
      <td>0</td>
      <td>3</td>
      <td>Andreasson, Mr. Paul Edvin</td>
      <td>male</td>
      <td>20.00</td>
      <td>0</td>
      <td>0</td>
      <td>347466</td>
      <td>7.8542</td>
      <td>NaN</td>
      <td>S</td>
      <td>0</td>
      <td>1</td>
      <td>1.0</td>
    </tr>
    <tr>
      <th>92</th>
      <td>93</td>
      <td>0</td>
      <td>1</td>
      <td>Chaffee, Mr. Herbert Fuller</td>
      <td>male</td>
      <td>46.00</td>
      <td>1</td>
      <td>0</td>
      <td>W.E.P. 5734</td>
      <td>61.1750</td>
      <td>E31</td>
      <td>S</td>
      <td>1</td>
      <td>2</td>
      <td>NaN</td>
    </tr>
    <tr>
      <th>93</th>
      <td>94</td>
      <td>0</td>
      <td>3</td>
      <td>Dean, Mr. Bertram Frank</td>
      <td>male</td>
      <td>26.00</td>
      <td>1</td>
      <td>2</td>
      <td>C.A. 2315</td>
      <td>20.5750</td>
      <td>NaN</td>
      <td>S</td>
      <td>1</td>
      <td>4</td>
      <td>NaN</td>
    </tr>
    <tr>
      <th>94</th>
      <td>95</td>
      <td>0</td>
      <td>3</td>
      <td>Coxon, Mr. Daniel</td>
      <td>male</td>
      <td>59.00</td>
      <td>0</td>
      <td>0</td>
      <td>364500</td>
      <td>7.2500</td>
      <td>NaN</td>
      <td>S</td>
      <td>0</td>
      <td>1</td>
      <td>1.0</td>
    </tr>
    <tr>
      <th>95</th>
      <td>96</td>
      <td>0</td>
      <td>3</td>
      <td>Shorney, Mr. Charles Joseph</td>
      <td>male</td>
      <td>NaN</td>
      <td>0</td>
      <td>0</td>
      <td>374910</td>
      <td>8.0500</td>
      <td>NaN</td>
      <td>S</td>
      <td>0</td>
      <td>1</td>
      <td>1.0</td>
    </tr>
    <tr>
      <th>96</th>
      <td>97</td>
      <td>0</td>
      <td>1</td>
      <td>Goldschmidt, Mr. George B</td>
      <td>male</td>
      <td>71.00</td>
      <td>0</td>
      <td>0</td>
      <td>PC 17754</td>
      <td>34.6542</td>
      <td>A5</td>
      <td>C</td>
      <td>0</td>
      <td>1</td>
      <td>1.0</td>
    </tr>
    <tr>
      <th>97</th>
      <td>98</td>
      <td>1</td>
      <td>1</td>
      <td>Greenfield, Mr. William Bertram</td>
      <td>male</td>
      <td>23.00</td>
      <td>0</td>
      <td>1</td>
      <td>PC 17759</td>
      <td>63.3583</td>
      <td>D10 D12</td>
      <td>C</td>
      <td>1</td>
      <td>2</td>
      <td>NaN</td>
    </tr>
    <tr>
      <th>98</th>
      <td>99</td>
      <td>1</td>
      <td>2</td>
      <td>Doling, Mrs. John T (Ada Julia Bone)</td>
      <td>female</td>
      <td>34.00</td>
      <td>0</td>
      <td>1</td>
      <td>231919</td>
      <td>23.0000</td>
      <td>NaN</td>
      <td>S</td>
      <td>1</td>
      <td>2</td>
      <td>NaN</td>
    </tr>
    <tr>
      <th>99</th>
      <td>100</td>
      <td>0</td>
      <td>2</td>
      <td>Kantor, Mr. Sinai</td>
      <td>male</td>
      <td>34.00</td>
      <td>1</td>
      <td>0</td>
      <td>244367</td>
      <td>26.0000</td>
      <td>NaN</td>
      <td>S</td>
      <td>1</td>
      <td>2</td>
      <td>NaN</td>
    </tr>
  </tbody>
</table>

</div>


家族(団体旅行者)内での脱出船に乗る順番の確率を先ほどの'NaN'の部分に代入する。

```python
#敬称ごとにindexの格納
train_mr_index = train['Name'].str.contains(' Mr. ')
train_miss_index = train['Name'].str.contains(' Miss. ')
train_mrs_index = train['Name'].str.contains(' Mrs. ')
train_master_index = train['Name'].str.contains(' Master. ')
test_mr_index = test['Name'].str.contains(' Mr. ')
test_miss_index = test['Name'].str.contains(' Miss. ')
test_mrs_index = test['Name'].str.contains(' Mrs. ')
test_master_index = test['Name'].str.contains(' Master. ')

train['escape_boarding_probability_train'][train_mr_index] = 1 / train['Family'][train_mr_index] 
train['escape_boarding_probability_train'][train_miss_index] = train['Family'][train_miss_index] - 1 / train['Family'][train_miss_index]
train['escape_boarding_probability_train'][train_mrs_index] = 1 / train['Family'][train_mrs_index]
train['escape_boarding_probability_train'][train_master_index] = train['Family'][train_master_index] - 1 / train['Family'][train_master_index]   
train['escape_boarding_probability_train']=train['escape_boarding_probability_train'].fillna(escape_boarding_probability_average_train)

test['escape_boarding_probability_test'][test_mr_index] = 1 / test['Family'][test_mr_index] 
test['escape_boarding_probability_test'][test_miss_index] = test['Family'][test_miss_index] - 1 / test['Family'][test_miss_index]
test['escape_boarding_probability_test'][test_mrs_index] = 1 / test['Family'][test_mrs_index]
test['escape_boarding_probability_test'][test_master_index] = test['Family'][test_master_index] - 1 / test['Family'][test_master_index]    
test['escape_boarding_probability_test']=test['escape_boarding_probability_test'].fillna(escape_boarding_probability_average_test)
 
```

~特徴量作成終了~

#欠損値の補完

欠損値の補完について、今回は**"Age"**についての欠損値を敬称ごとに細かく補完したのでその部分を紹介。

敬称ごとの平均年齢を算出し、その値を各敬称ごとに補完(※敬称のない方については全体の中央値で補完を行う)



```python

#敬称ごとの平均値の算出
train_mr = train[train['Name'].str.contains(' Mr. ')]
train_miss = train[train['Name'].str.contains(' Miss. ')]
train_mrs = train[train['Name'].str.contains(' Mrs. ')]
train_master = train[train['Name'].str.contains(' Master. ')]
test_mr = test[test['Name'].str.contains(' Mr. ')]
test_miss = test[test['Name'].str.contains(' Miss. ')]
test_mrs = test[test['Name'].str.contains(' Mrs. ')]
test_master = test[test['Name'].str.contains(' Master. ')]

train_mr_num = train_mr['Age'].dropna().mean()
train_miss_num = train_miss['Age'].dropna().mean()
train_mrs_num = train_mrs['Age'].dropna().mean()
train_master_num = train_master['Age'].dropna().mean()
train_all_num = train['Age'].dropna().median()

test_mr_num = test_mr['Age'].dropna().mean()
test_miss_num = test_miss['Age'].dropna().mean()
test_mrs_num = test_mrs['Age'].dropna().mean()
test_master_num = test_master['Age'].dropna().mean()
test_all_num = test['Age'].dropna().median()

print("trainデータの敬称'Mr'の平均値 = " + str(train_mr_num))
print("trainデータの敬称'Miss'の平均値 = " + str(train_miss_num))
print("trainデータの敬称'Mrs'の平均値 = " + str(train_mrs_num))
print("trainデータの敬称'Master'の平均値 = " + str(train_master_num))
print("trainデータの中央値 = " + str(train_all_num), '\n')

print("testデータの敬称'Mr'の平均値 = " + str(test_mr_num))
print("testデータの敬称'Miss'の平均値 = " + str(test_miss_num))
print("testデータの敬称'Mrs'の平均値 = " + str(test_mrs_num))
print("testデータの敬称'Master'の平均値 = " + str(test_master_num))
print("testデータの中央値 = " + str(test_all_num))
```

    trainデータの敬称'Mr'の平均値 = 32.368090452261306
    trainデータの敬称'Miss'の平均値 = 21.773972602739725
    trainデータの敬称'Mrs'の平均値 = 35.898148148148145
    trainデータの敬称'Master'の平均値 = 4.574166666666667
    trainデータの中央値 = 28.0 
    
    testデータの敬称'Mr'の平均値 = 32.0
    testデータの敬称'Miss'の平均値 = 21.774843750000002
    testデータの敬称'Mrs'の平均値 = 38.903225806451616
    testデータの敬称'Master'の平均値 = 7.406470588235294
    testデータの中央値 = 27.0



```python
#欠損値”Age”に対しての敬称ごとの平均値の補完
train['Age'][train_mr_index] = train_mr['Age'].fillna(32)
train['Age'][train_miss_index] = train_master['Age'].fillna(22)
train['Age'][train_mrs_index] = train_mrs['Age'].fillna(36)
train['Age'][train_master_index] = train_master['Age'].fillna(5)
train['Age'] = train['Age'].fillna(28)

test['Age'][test_mr_index] = test_mr['Age'].fillna(32)
test['Age'][test_miss_index] = test_miss['Age'].fillna(22)
test['Age'][test_mrs_index] = test_mrs['Age'].fillna(39)
test['Age'][test_master_index] = test_master['Age'].fillna(7)
test['Age'] = test['Age'].fillna(27)

train.isnull().sum()
```

    PassengerId       0
    Survived          0
    Pclass            0
    Name              0
    Sex               0
    Age               0
    SibSp             0
    Parch             0
    Ticket            0
    Fare              0
    Cabin           687
    Embarked          2
    Family_group      0
    Family            0
    dtype: int64



その他の欠損値を補完後、名称をダミー変数に変換し分類が可能な型に変換して再度特徴量として格納する。



```python


#その他欠損値の補完
train['Embarked'] = train['Embarked'].fillna('S')
test['Fare'] = test['Fare'].fillna(test['Fare'].mean())

#性別と乗船港をダミー変数への変換
dummy_train = pd.get_dummies(train[['Sex', 'Embarked']])
dummy_test = pd.get_dummies(test[['Sex', 'Embarked']])

train_two = pd.concat([train.drop(["Sex", "Embarked"], axis = 1),dummy_train], axis = 1)
test_two = pd.concat([test.drop(["Sex", "Embarked"], axis = 1),dummy_test], axis = 1)

train_two.isnull().sum()
```




    PassengerId       0
    Survived          0
    Pclass            0
    Name              0
    Age               0
    SibSp             0
    Parch             0
    Ticket            0
    Fare              0
    Cabin           687
    Family_group      0
    Family            0
    Sex_female        0
    Sex_male          0
    Embarked_C        0
    Embarked_Q        0
    Embarked_S        0
    dtype: int64




```python
train_two.head(10)
```




<div>
<style scoped>
    
</style>
<table border="1" class="dataframe">
  <thead>
    <tr style="text-align: right;">
      <th></th>
      <th>PassengerId</th>
      <th>Survived</th>
      <th>Pclass</th>
      <th>Name</th>
      <th>Age</th>
      <th>SibSp</th>
      <th>Parch</th>
      <th>Ticket</th>
      <th>Fare</th>
      <th>Cabin</th>
      <th>Family_group</th>
      <th>Family</th>
      <th>Sex_female</th>
      <th>Sex_male</th>
      <th>Embarked_C</th>
      <th>Embarked_Q</th>
      <th>Embarked_S</th>
    </tr>
  </thead>
  <tbody>
    <tr>
      <th>0</th>
      <td>1</td>
      <td>0</td>
      <td>3</td>
      <td>Braund, Mr. Owen Harris</td>
      <td>22.0</td>
      <td>1</td>
      <td>0</td>
      <td>A/5 21171</td>
      <td>7.2500</td>
      <td>NaN</td>
      <td>1</td>
      <td>2</td>
      <td>0</td>
      <td>1</td>
      <td>0</td>
      <td>0</td>
      <td>1</td>
    </tr>
    <tr>
      <th>1</th>
      <td>2</td>
      <td>1</td>
      <td>1</td>
      <td>Cumings, Mrs. John Bradley (Florence Briggs Th...</td>
      <td>38.0</td>
      <td>1</td>
      <td>0</td>
      <td>PC 17599</td>
      <td>71.2833</td>
      <td>C85</td>
      <td>1</td>
      <td>2</td>
      <td>1</td>
      <td>0</td>
      <td>1</td>
      <td>0</td>
      <td>0</td>
    </tr>
    <tr>
      <th>2</th>
      <td>3</td>
      <td>1</td>
      <td>3</td>
      <td>Heikkinen, Miss. Laina</td>
      <td>28.0</td>
      <td>0</td>
      <td>0</td>
      <td>STON/O2. 3101282</td>
      <td>7.9250</td>
      <td>NaN</td>
      <td>0</td>
      <td>1</td>
      <td>1</td>
      <td>0</td>
      <td>0</td>
      <td>0</td>
      <td>1</td>
    </tr>
    <tr>
      <th>3</th>
      <td>4</td>
      <td>1</td>
      <td>1</td>
      <td>Futrelle, Mrs. Jacques Heath (Lily May Peel)</td>
      <td>35.0</td>
      <td>1</td>
      <td>0</td>
      <td>113803</td>
      <td>53.1000</td>
      <td>C123</td>
      <td>1</td>
      <td>2</td>
      <td>1</td>
      <td>0</td>
      <td>0</td>
      <td>0</td>
      <td>1</td>
    </tr>
    <tr>
      <th>4</th>
      <td>5</td>
      <td>0</td>
      <td>3</td>
      <td>Allen, Mr. William Henry</td>
      <td>35.0</td>
      <td>0</td>
      <td>0</td>
      <td>373450</td>
      <td>8.0500</td>
      <td>NaN</td>
      <td>0</td>
      <td>1</td>
      <td>0</td>
      <td>1</td>
      <td>0</td>
      <td>0</td>
      <td>1</td>
    </tr>
    <tr>
      <th>5</th>
      <td>6</td>
      <td>0</td>
      <td>3</td>
      <td>Moran, Mr. James</td>
      <td>32.0</td>
      <td>0</td>
      <td>0</td>
      <td>330877</td>
      <td>8.4583</td>
      <td>NaN</td>
      <td>0</td>
      <td>1</td>
      <td>0</td>
      <td>1</td>
      <td>0</td>
      <td>1</td>
      <td>0</td>
    </tr>
    <tr>
      <th>6</th>
      <td>7</td>
      <td>0</td>
      <td>1</td>
      <td>McCarthy, Mr. Timothy J</td>
      <td>54.0</td>
      <td>0</td>
      <td>0</td>
      <td>17463</td>
      <td>51.8625</td>
      <td>E46</td>
      <td>0</td>
      <td>1</td>
      <td>0</td>
      <td>1</td>
      <td>0</td>
      <td>0</td>
      <td>1</td>
    </tr>
    <tr>
      <th>7</th>
      <td>8</td>
      <td>0</td>
      <td>3</td>
      <td>Palsson, Master. Gosta Leonard</td>
      <td>2.0</td>
      <td>3</td>
      <td>1</td>
      <td>349909</td>
      <td>21.0750</td>
      <td>NaN</td>
      <td>1</td>
      <td>5</td>
      <td>0</td>
      <td>1</td>
      <td>0</td>
      <td>0</td>
      <td>1</td>
    </tr>
    <tr>
      <th>8</th>
      <td>9</td>
      <td>1</td>
      <td>3</td>
      <td>Johnson, Mrs. Oscar W (Elisabeth Vilhelmina Berg)</td>
      <td>27.0</td>
      <td>0</td>
      <td>2</td>
      <td>347742</td>
      <td>11.1333</td>
      <td>NaN</td>
      <td>1</td>
      <td>3</td>
      <td>1</td>
      <td>0</td>
      <td>0</td>
      <td>0</td>
      <td>1</td>
    </tr>
    <tr>
      <th>9</th>
      <td>10</td>
      <td>1</td>
      <td>2</td>
      <td>Nasser, Mrs. Nicholas (Adele Achem)</td>
      <td>14.0</td>
      <td>1</td>
      <td>0</td>
      <td>237736</td>
      <td>30.0708</td>
      <td>NaN</td>
      <td>1</td>
      <td>2</td>
      <td>1</td>
      <td>0</td>
      <td>1</td>
      <td>0</td>
      <td>0</td>
    </tr>
  </tbody>
</table>
</div>


最後に不要なデータをdropして、データクレンジングを完了する。

```python
#不要な特徴量の削除
train_three = train_two.drop(['PassengerId', 'Name', 'Ticket', 'Cabin','Parch', 'SibSp'], axis = 1)
x_test = test_two.drop(['PassengerId', 'Name', 'Ticket', 'Cabin','Parch', 'SibSp'], axis = 1)

train_three.isnull().sum()
```




    Survived        0
    Pclass          0
    Age             0
    Fare            0
    Family_group    0
    Family          0
    Sex_female      0
    Sex_male        0
    Embarked_C      0
    Embarked_Q      0
    Embarked_S      0
    dtype: int64




ここからは特徴量ごとにライブラリを使って分類し、モデルの作成を行う。

```python
#データフレーム型への変換
x_train_df = train_three.drop(['Survived'], axis = 1)
x_train = x_train_df

#目的変数の格納
y_train = train_three.Survived

#決定木の学習を行う
depth = 4
clf = tree.DecisionTreeClassifier(max_depth = depth)
clf.fit(x_train_df, y_train)



#applyクラスでリーフごとのリーフ番号を返す
x_train_leaf_no = clf.apply(x_train_df)
x_test_leaf_no = clf.apply(x_test)


#リーフごとにロジスティック回帰分析を行う

#全てのindexを０にした配列を用意しておく
x_train_proba = np.zeros(x_train.shape[0])
x_test_proba = np.zeros(x_test.shape[0])

#重複しないリーフ番号をリストに格納する
unique_leaf_no = list(set(x_train_leaf_no))

#ロジスティック回帰のハイパーパラメータのチューニング
param_grid = {'C': [0.001, 0.01, 0.1, 1, 10, 100]}

#格納したリーフ番号を取り出す
for i in unique_leaf_no :
    #取り出すリーフ番号の確認
    print('leaf no:', i)
    
    #trainデータのリーフ番号を指定して取り出したデータフレームを変数に格納
    leaf_data_train_x = x_train[x_train_leaf_no == i]
    leaf_data_train_y = y_train[x_train_leaf_no == i]
    #testデータのリーフ番号を指定して取り出したデータフレームを変数に格納
    leaf_data_test_x = x_test[x_test_leaf_no == i]
    

    #一度、ダミー変数のデータを除外する
    leaf_data_train_x_drop = leaf_data_train_x.drop(['Family_group', 'Pclass', 'Sex_female', 'Sex_male', 'Embarked_C', 'Embarked_S', 'Embarked_Q', 'escape_boarding_probability_train'], axis = 1)
    leaf_data_test_x = leaf_data_test_x.drop(['Family_group', 'Pclass', 'Sex_female', 'Sex_male', 'Embarked_C', 'Embarked_S', 'Embarked_Q', 'escape_boarding_probability_test'], axis = 1)

    
    
    #survivedの値に生存者と死亡者の両方がいる場合
    if len(set(leaf_data_train_y)) > 1:
        
        #GridSearchを行う
        try:
            grid_search = GridSearchCV(LogisticRegression(), param_grid, cv = 5, scoring = 'roc_auc')   
            grid_search.fit(leaf_data_train_x_drop, leaf_data_train_y)
            clf = LogisticRegression(C=grid_search.best_params_['C'],class_weight="balanced")
        except (ValueError, TypeError, NameError, SyntaxError):
            clf = LogisticRegression()
        
        #ロジスティック回帰分析を行う
        clf.fit(leaf_data_train_x_drop, leaf_data_train_y)
        
        #所属しているクラスの確率を戻す
        a = clf.predict_proba(leaf_data_train_x_drop)
        
        #生存の場合の確率のみを格納
        x_train_proba[x_train_leaf_no == i] = a[:,1]
       
        if len(leaf_data_test_x) > 0:
            b = clf.predict_proba(leaf_data_test_x)    
            x_test_proba[x_test_leaf_no == i] = b[:,1]
        
        
    #survivedの値に生存者と死亡者のどちらかしかいない場合    
    else:
        x_train_proba[x_train_leaf_no == i] = leaf_data_train_y.head(1)
        if len(leaf_data_test_x) > 0:
            x_test_proba[x_test_leaf_no == i] =leaf_data_train_y.head(1)
        
        

#ループ終了の確認
print("for loop end")

#生存と死亡の確率のデータフレームを結合
train_data = pd.concat([x_train, pd.DataFrame(x_train_proba)], axis =1)
test_data = pd.concat([x_test, pd.DataFrame(x_test_proba)], axis =1)

#ロジスティック回帰のハイパーパラメータのチューニング
param_grid = {'max_depth': [3,5,8,13,21,34]}

#GridSearchを行う
grid_search = GridSearchCV(GradientBoostingClassifier(n_estimators=100), param_grid, cv = 5, scoring = 'roc_auc')   
grid_search.fit(train_data, y_train)

#勾配ブースティングによる学習と予測
model = GradientBoostingClassifier(max_depth=grid_search.best_params_['max_depth'], n_estimators=100)
model.fit(train_data, y_train)
output = model.predict(test_data).astype(int)


#結果をCSVに変換
leaf_data_test = pd.DataFrame({
    "PassengerId": test["PassengerId"],
    "Survived": output
})
leaf_data_test.to_csv('training_camp06.csv', index = False)       
```

↑最後は予測したデータをcsvの形式で作成し、そのデータをkaggleに提出すると完了！


```python

```
上記カーネルへのリンク↓

https://github.com/happycatcat23/pyworks/blob/master/kaggle%E8%B3%87%E6%96%99/%E3%82%BF%E3%82%A4%E3%82%BF%E3%83%8B%E3%83%83%E3%82%AF/%E3%82%AB%E3%83%BC%E3%83%8D%E3%83%AB/%20.ipynb_checkpoints/trainingcamp_feature_quantity_addition-checkpoint.ipynb

##予測結果


今回、こちらのモデルで予測した場合の結果について、
スコア : 0.82296 上位5% (459位/10499チーム中) という結果が返ってきた。

![スクリーンショット 2019-01-13 11.18.11.png](https://qiita-image-store.s3.amazonaws.com/0/323243/7ac82696-efca-154c-ef7c-9eb0a5ac86d9.png)



##まとめ

分析ツールについての理解が乏しい状態で望んだ今回の課題であったが、逆にそれが既存の特徴量を活かして分析を行うという方針に至り、titanicという問題に素直に取り組むことが出来た。
ライブラリの活用方法に関して中身の理解があまり出来てない状態だったことが原因で、最善ではない部分もあったと思うので、これからライブラリの中身の理解について深めていきたい。
今回のtitanicで学んだことを活かしつつ他の課題に挑戦していこうと思う。