【機械学習初心者向け】Kaggleのタイタニック生存予測を実践してみた

# 1.はじめに

機械学習について少しずつアウトプットしてきましたが、一連の流れを実践したものをまだアウトプットしてなかったので、今回は有名なKaggleのタイタニック号生存予測モデルを通して全体像をみていきます。

よければ以前書いた記事も参考にしてください。

過去の記事
・機械学習初心者の私に告ぐ「4つの忠告」
https://qiita.com/takuya_tsurumi/items/38f2858221599d5f93bd

・機械学習のデータ前処理備忘録
https://qiita.com/takuya_tsurumi/items/53b9e3f7427b631b17cf

・【機械学習】決定木モデルの違いをまとめてみた
https://qiita.com/takuya_tsurumi/items/23fdc43ee0e54ec7c87e


# 2.データの内容

Kaggleのタイタニック号生存予測のデータは以下サイトからダウンロードできます。
https://www.kaggle.com/c/titanic/overview

こちらをダウンロードしてデータの中身を確認してみます。
中身は以下のような要素が入っています。


|No|カラム名|説明|備考|
|---|---|---|---|
|1|PassengerID|乗客者のID||
|2|Survived|生存の有無|今回の目的変数となる|
|3|Pclass|チケットの階級|
|4|Name|名前||
|5|Sex|性別||
|6|Age|年齢|端数は切り捨て|
|7|SibSp|タイタニック号に乗っている兄弟/配偶者数||
|8|Parch|タイタニック号に乗っている親/子供の数||
|9|Ticket|チケット番号||
|10|Fare|旅客運賃||
|11|Cabin|部屋番号||
|12|Embarked|乗船港||



こちらのデータを使って実践してみます。


# 3.実践

全体の流れとして以下のような構成です。

1.データの読み込み
2.データの確認
3.データの加工(数値への変換+欠損値処理)
4.モデルの生成
5.モデルの評価

まずは、ライブラリをインストールし、データの読み込みをします。

~~~Python
# ライブラリをインポート
%matplotlib inline
import numpy as np
import pandas as pd
import seaborn as sns
from sklearn.model_selection import train_test_split
from sklearn.ensemble import RandomForestClassifier
from sklearn.metrics import confusion_matrix, classification_report, accuracy_score

# データの読み込み
train = pd.read_csv("./data/train.csv", index_col='PassengerId')
test = pd.read_csv('./data/test.csv', index_col='PassengerId')
print(train.head())
print(test.head())
~~~

以下表示結果です。

~~~
            Survived  Pclass  \
PassengerId                     
1                   0       3   
2                   1       1   
3                   1       3   
4                   1       1   
5                   0       3   

                                                          Name     Sex   Age  \
PassengerId                                                                    
1                                      Braund, Mr. Owen Harris    male  22.0   
2            Cumings, Mrs. John Bradley (Florence Briggs Th...  female  38.0   
3                                       Heikkinen, Miss. Laina  female  26.0   
4                 Futrelle, Mrs. Jacques Heath (Lily May Peel)  female  35.0   
5                                     Allen, Mr. William Henry    male  35.0   

             SibSp  Parch            Ticket     Fare Cabin Embarked  
PassengerId                                                          
1                1      0         A/5 21171   7.2500   NaN        S  
2                1      0          PC 17599  71.2833   C85        C  
3                0      0  STON/O2. 3101282   7.9250   NaN        S  
4                1      0            113803  53.1000  C123        S  
5                0      0            373450   8.0500   NaN        S  
             Pclass                                          Name     Sex  \
PassengerId                                                                 
892               3                              Kelly, Mr. James    male   
893               3              Wilkes, Mrs. James (Ellen Needs)  female   
894               2                     Myles, Mr. Thomas Francis    male   
895               3                              Wirz, Mr. Albert    male   
896               3  Hirvonen, Mrs. Alexander (Helga E Lindqvist)  female   

              Age  SibSp  Parch   Ticket     Fare Cabin Embarked  
PassengerId                                                       
892          34.5      0      0   330911   7.8292   NaN        Q  
893          47.0      1      0   363272   7.0000   NaN        S  
894          62.0      0      0   240276   9.6875   NaN        Q  
895          27.0      0      0   315154   8.6625   NaN        S  
896          22.0      1      1  3101298  12.2875   NaN        S  

~~~

続いて全データをモデルが学習できるように加工のための準備をします。
やっていることとしては、訓練データとテストデータを一旦まとめています。

~~~Python
df = pd.concat([train, test], sort=False, axis='rows')
df
~~~

表示結果

~~~

	Survived	Pclass	Name	Sex	Age	SibSp	Parch	Ticket	Fare	Cabin	Embarked
PassengerId											
1	0.0	3	Braund, Mr. Owen Harris	male	22.0	1	0	A/5 21171	7.2500	NaN	S
2	1.0	1	Cumings, Mrs. John Bradley (Florence Briggs Th...	female	38.0	1	0	PC 17599	71.2833	C85	C
3	1.0	3	Heikkinen, Miss. Laina	female	26.0	0	0	STON/O2. 3101282	7.9250	NaN	S
4	1.0	1	Futrelle, Mrs. Jacques Heath (Lily May Peel)	female	35.0	1	0	113803	53.1000	C123	S
5	0.0	3	Allen, Mr. William Henry	male	35.0	0	0	373450	8.0500	NaN	S
6	0.0	3	Moran, Mr. James	male	NaN	0	0	330877	8.4583	NaN	Q
7	0.0	1	McCarthy, Mr. Timothy J	male	54.0	0	0	17463	51.8625	E46	S
8	0.0	3	Palsson, Master. Gosta Leonard	male	2.0	3	1	349909	21.0750	NaN	S
9	1.0	3	Johnson, Mrs. Oscar W (Elisabeth Vilhelmina Berg)	female	27.0	0	2	347742	11.1333	NaN	S
10	1.0	2	Nasser, Mrs. Nicholas (Adele Achem)	female	14.0	1	0	237736	30.0708	NaN	C
11	1.0	3	Sandstrom, Miss. Marguerite Rut	female	4.0	1	1	PP 9549	16.7000	G6	S
12	1.0	1	Bonnell, Miss. Elizabeth	female	58.0	0	0	113783	26.5500	C103	S
13	0.0	3	Saundercock, Mr. William Henry	male	20.0	0	0	A/5. 2151	8.0500	NaN	S
14	0.0	3	Andersson, Mr. Anders Johan	male	39.0	1	5	347082	31.2750	NaN	S
15	0.0	3	Vestrom, Miss. Hulda Amanda Adolfina	female	14.0	0	0	350406	7.8542	NaN	S
16	1.0	2	Hewlett, Mrs. (Mary D Kingcome)	female	55.0	0	0	248706	16.0000	NaN	S
17	0.0	3	Rice, Master. Eugene	male	2.0	4	1	382652	29.1250	NaN	Q
18	1.0	2	Williams, Mr. Charles Eugene	male	NaN	0	0	244373	13.0000	NaN	S
19	0.0	3	Vander Planke, Mrs. Julius (Emelia Maria Vande...	female	31.0	1	0	345763	18.0000	NaN	S
20	1.0	3	Masselmani, Mrs. Fatima	female	NaN	0	0	2649	7.2250	NaN	C
21	0.0	2	Fynney, Mr. Joseph J	male	35.0	0	0	239865	26.0000	NaN	S
22	1.0	2	Beesley, Mr. Lawrence	male	34.0	0	0	248698	13.0000	D56	S
23	1.0	3	McGowan, Miss. Anna "Annie"	female	15.0	0	0	330923	8.0292	NaN	Q
24	1.0	1	Sloper, Mr. William Thompson	male	28.0	0	0	113788	35.5000	A6	S
25	0.0	3	Palsson, Miss. Torborg Danira	female	8.0	3	1	349909	21.0750	NaN	S
26	1.0	3	Asplund, Mrs. Carl Oscar (Selma Augusta Emilia...	female	38.0	1	5	347077	31.3875	NaN	S
27	0.0	3	Emir, Mr. Farred Chehab	male	NaN	0	0	2631	7.2250	NaN	C
28	0.0	1	Fortune, Mr. Charles Alexander	male	19.0	3	2	19950	263.0000	C23 C25 C27	S
29	1.0	3	O'Dwyer, Miss. Ellen "Nellie"	female	NaN	0	0	330959	7.8792	NaN	Q
30	0.0	3	Todoroff, Mr. Lalio	male	NaN	0	0	349216	7.8958	NaN	S
...	...	...	...	...	...	...	...	...	...	...	...
1280	NaN	3	Canavan, Mr. Patrick	male	21.0	0	0	364858	7.7500	NaN	Q
1281	NaN	3	Palsson, Master. Paul Folke	male	6.0	3	1	349909	21.0750	NaN	S
1282	NaN	1	Payne, Mr. Vivian Ponsonby	male	23.0	0	0	12749	93.5000	B24	S
1283	NaN	1	Lines, Mrs. Ernest H (Elizabeth Lindsey James)	female	51.0	0	1	PC 17592	39.4000	D28	S
1284	NaN	3	Abbott, Master. Eugene Joseph	male	13.0	0	2	C.A. 2673	20.2500	NaN	S
1285	NaN	2	Gilbert, Mr. William	male	47.0	0	0	C.A. 30769	10.5000	NaN	S
1286	NaN	3	Kink-Heilmann, Mr. Anton	male	29.0	3	1	315153	22.0250	NaN	S
1287	NaN	1	Smith, Mrs. Lucien Philip (Mary Eloise Hughes)	female	18.0	1	0	13695	60.0000	C31	S
1288	NaN	3	Colbert, Mr. Patrick	male	24.0	0	0	371109	7.2500	NaN	Q
1289	NaN	1	Frolicher-Stehli, Mrs. Maxmillian (Margaretha ...	female	48.0	1	1	13567	79.2000	B41	C
1290	NaN	3	Larsson-Rondberg, Mr. Edvard A	male	22.0	0	0	347065	7.7750	NaN	S
1291	NaN	3	Conlon, Mr. Thomas Henry	male	31.0	0	0	21332	7.7333	NaN	Q
1292	NaN	1	Bonnell, Miss. Caroline	female	30.0	0	0	36928	164.8667	C7	S
1293	NaN	2	Gale, Mr. Harry	male	38.0	1	0	28664	21.0000	NaN	S
1294	NaN	1	Gibson, Miss. Dorothy Winifred	female	22.0	0	1	112378	59.4000	NaN	C
1295	NaN	1	Carrau, Mr. Jose Pedro	male	17.0	0	0	113059	47.1000	NaN	S
1296	NaN	1	Frauenthal, Mr. Isaac Gerald	male	43.0	1	0	17765	27.7208	D40	C
1297	NaN	2	Nourney, Mr. Alfred (Baron von Drachstedt")"	male	20.0	0	0	SC/PARIS 2166	13.8625	D38	C
1298	NaN	2	Ware, Mr. William Jeffery	male	23.0	1	0	28666	10.5000	NaN	S
1299	NaN	1	Widener, Mr. George Dunton	male	50.0	1	1	113503	211.5000	C80	C
1300	NaN	3	Riordan, Miss. Johanna Hannah""	female	NaN	0	0	334915	7.7208	NaN	Q
1301	NaN	3	Peacock, Miss. Treasteall	female	3.0	1	1	SOTON/O.Q. 3101315	13.7750	NaN	S
1302	NaN	3	Naughton, Miss. Hannah	female	NaN	0	0	365237	7.7500	NaN	Q
1303	NaN	1	Minahan, Mrs. William Edward (Lillian E Thorpe)	female	37.0	1	0	19928	90.0000	C78	Q
1304	NaN	3	Henriksson, Miss. Jenny Lovisa	female	28.0	0	0	347086	7.7750	NaN	S
1305	NaN	3	Spector, Mr. Woolf	male	NaN	0	0	A.5. 3236	8.0500	NaN	S
1306	NaN	1	Oliva y Ocana, Dona. Fermina	female	39.0	0	0	PC 17758	108.9000	C105	C
1307	NaN	3	Saether, Mr. Simon Sivertsen	male	38.5	0	0	SOTON/O.Q. 3101262	7.2500	NaN	S
1308	NaN	3	Ware, Mr. Frederick	male	NaN	0	0	359309	8.0500	NaN	S
1309	NaN	3	Peter, Master. Michael J	male	NaN	1	1	2668	22.3583	NaN	C
1309 rows × 11 columns

~~~

データの特性を describe() 関数で確認してみる

~~~Python
# 外れ値の確認
df.describe()
~~~

表示結果

~~~

	Survived	Pclass	Age	SibSp	Parch	Fare
count	891.000000	1309.000000	1046.000000	1309.000000	1309.000000	1308.000000
mean	0.383838	2.294882	29.881138	0.498854	0.385027	33.295479
std	0.486592	0.837836	14.413493	1.041658	0.865560	51.758668
min	0.000000	1.000000	0.170000	0.000000	0.000000	0.000000
25%	0.000000	2.000000	21.000000	0.000000	0.000000	7.895800
50%	0.000000	3.000000	28.000000	0.000000	0.000000	14.454200
75%	1.000000	3.000000	39.000000	1.000000	0.000000	31.275000
max	1.000000	3.000000	80.000000	8.000000	9.000000	512.329200

~~~

欠損値の有無を確認

~~~Python

# 　欠損値の確認
df.isnull().sum()

~~~

表示結果

~~~

Survived     418
Pclass         0
Name           0
Sex            0
Age          263
SibSp          0
Parch          0
Ticket         0
Fare           1
Cabin       1014
Embarked       2
dtype: int64

~~~

{Age, Cabin, Embarked} の3種類に欠損値が存在していることがわかります。
この欠損値をどう処理するのか検討する必要があります。

モデルに読み込ませる型(数値)に全体がなっているかを確認します。

~~~Python

# 型の確認
df.dtypes

~~~

表示結果

~~~

Survived    float64
Pclass        int64
Name         object
Sex          object
Age         float64
SibSp         int64
Parch         int64
Ticket       object
Fare        float64
Cabin        object
Embarked     object
dtype: object

~~~

object型である、{Name, Sex, Ticket, Cabin, Embarked}の5種類は数値型への変換を検討します。
まずは Name の変換を実施します。

~~~Python

# Nameの変換
## 要素の確認
df['Name'].value_counts()

~~~

表示結果

~~~

Connolly, Miss. Kate                                                                  2
Kelly, Mr. James                                                                      2
Saad, Mr. Amin                                                                        1
Lievens, Mr. Rene Aime                                                                1
Turja, Miss. Anna Sofia                                                               1
Collett, Mr. Sidney C Stuart                                                          1
Hassan, Mr. Houssein G N                                                              1
Penasco y Castellana, Mrs. Victor de Satode (Maria Josefa Perez de Soto y Vallejo)    1
Pavlovic, Mr. Stefo                                                                   1
Swift, Mrs. Frederick Joel (Margaret Welles Barron)                                   1
Lindahl, Miss. Agda Thorilda Viktoria                                                 1
Carter, Mr. William Ernest                                                            1
Ponesell, Mr. Martin                                                                  1
Mellinger, Miss. Madeleine Violet                                                     1
Ryerson, Mr. Arthur Larned                                                            1
Kenyon, Mr. Frederick R                                                               1
O'Connor, Mr. Patrick                                                                 1
Daly, Mr. Peter Denis                                                                 1
Gracie, Col. Archibald IV                                                             1
Samaan, Mr. Hanna                                                                     1
Johnson, Miss. Eleanor Ileen                                                          1
Walker, Mr. William Anderson                                                          1
Torber, Mr. Ernst William                                                             1
Vander Planke, Mr. Leo Edmondus                                                       1
Sage, Mr. John George                                                                 1
Mullens, Miss. Katherine "Katie"                                                      1
Mayne, Mlle. Berthe Antonine ("Mrs de Villiers")                                      1
Olsen, Mr. Henry Margido                                                              1
Behr, Mr. Karl Howell                                                                 1
Braund, Mr. Owen Harris                                                               1
                                                                                     ..
Mack, Mrs. (Mary)                                                                     1
Slayter, Miss. Hilda Mary                                                             1
McCrae, Mr. Arthur Gordon                                                             1
Goldsmith, Mrs. Frank John (Emily Alice Brown)                                        1
Blank, Mr. Henry                                                                      1
Dean, Miss. Elizabeth Gladys Millvina""                                               1
Potter, Mrs. Thomas Jr (Lily Alexenia Wilson)                                         1
Chaffee, Mrs. Herbert Fuller (Carrie Constance Toogood)                               1
O'Dwyer, Miss. Ellen "Nellie"                                                         1
Sage, Miss. Stella Anna                                                               1
Kallio, Mr. Nikolai Erland                                                            1
Demetri, Mr. Marinko                                                                  1
Boulos, Mrs. Joseph (Sultana)                                                         1
Landergren, Miss. Aurora Adelia                                                       1
Braund, Mr. Lewis Richard                                                             1
Wilkes, Mrs. James (Ellen Needs)                                                      1
Clark, Mrs. Walter Miller (Virginia McDowell)                                         1
Lundstrom, Mr. Thure Edvin                                                            1
Dorking, Mr. Edward Arthur                                                            1
Oliva y Ocana, Dona. Fermina                                                          1
Mernagh, Mr. Robert                                                                   1
Shutes, Miss. Elizabeth W                                                             1
Anderson, Mr. Harry                                                                   1
McGowan, Miss. Katherine                                                              1
Eitemiller, Mr. George Floyd                                                          1
Van Impe, Mr. Jean Baptiste                                                           1
Palsson, Miss. Stina Viola                                                            1
Smith, Mrs. Lucien Philip (Mary Eloise Hughes)                                        1
Bourke, Miss. Mary                                                                    1
Wick, Miss. Mary Natalie                                                              1
Name: Name, Length: 1307, dtype: int64

~~~

全てユニークなデータとなります。
{Mr., Ms., Miss., Mrs., Master.}で分類してみます。

~~~Python

print('Mr.:{}'.format(df['Name'].str.contains('Mr. ').sum()))
print('Miss.:{}'.format(df['Name'].str.contains('Miss. ').sum()))
print('Mrs.:{}'.format(df['Name'].str.contains('Mrs. ').sum()))
print('Master.:{}'.format(df['Name'].str.contains('Master. ').sum()))

~~~

表示結果

~~~

Mr.:761
Miss.:260
Mrs.:197
Master.:61

~~~

{Mr., Ms., Miss., Mrs., Master.}の合計値でも891にならないため、
一旦置き換えを実施し、その他の名前を確認します。

~~~Python

df.loc[df['Name'].str.contains('Mr. ') == True, 'Name'] = 0
df.loc[df['Name'].str.contains('Miss. ') == True, 'Name'] = 1
df.loc[df['Name'].str.contains('Mrs. ') == True, 'Name'] = 2
df.loc[df['Name'].str.contains('Master. ') == True, 'Name'] = 3
​
df['Name'].value_counts()

~~~


表示結果

~~~

0                                                           761
1                                                           258
2                                                           197
3                                                            61
Lahtinen, Rev. William                                        1
Bateman, Rev. Robert James                                    1
Uruchurtu, Don. Manuel E                                      1
Weir, Col. John                                               1
Moraweck, Dr. Ernest                                          1
Leader, Dr. Alice (Farnham)                                   1
Dodge, Dr. Washington                                         1
Stahelin-Maeglin, Dr. Max                                     1
Minahan, Dr. William Edward                                   1
Reynaldo, Ms. Encarnacion                                     1
Harper, Rev. John                                             1
Kirkland, Rev. Charles Leonard                                1
Montvila, Rev. Juozas                                         1
Sagesser, Mlle. Emma                                          1
Rothes, the Countess. of (Lucy Noel Martha Dyer-Edwards)      1
Oliva y Ocana, Dona. Fermina                                  1
Carter, Rev. Ernest Courtenay                                 1
Peruschitz, Rev. Joseph Maria                                 1
Brewe, Dr. Arthur Jackson                                     1
Reuchlin, Jonkheer. John George                               1
Aubart, Mme. Leontine Pauline                                 1
Pain, Dr. Alfred                                              1
Duff Gordon, Sir. Cosmo Edmund ("Mr Morgan")                  1
Astor, Col. John Jacob                                        1
Peuchen, Major. Arthur Godfrey                                1
Crosby, Capt. Edward Gifford                                  1
Gracie, Col. Archibald IV                                     1
Butt, Major. Archibald Willingham                             1
O'Donoghue, Ms. Bridget                                       1
Frauenthal, Dr. Henry William                                 1
Byles, Rev. Thomas Roussel Davids                             1
Simonius-Blumer, Col. Oberst Alfons                           1
Name: Name, dtype: int64
~~~

残りのものの数を確認し変換を実施します。

~~~Python
print('Dr.:{}'.format(df['Name'].str.contains('Dr. ').sum()))
print('Rev.:{}'.format(df['Name'].str.contains('Rev. ').sum()))
print('Col.:{}'.format(df['Name'].str.contains('Col. ').sum()))
print('Major.:{}'.format(df['Name'].str.contains('Major. ').sum()))
print('Jonkheer.:{}'.format(df['Name'].str.contains('Jonkheer. ').sum()))
print('Mme.:{}'.format(df['Name'].str.contains('Mme. ').sum()))
print('Capt.:{}'.format(df['Name'].str.contains('Capt. ').sum()))
print('Ms.:{}'.format(df['Name'].str.contains('Ms. ').sum()))
print('Mlle.:{}'.format(df['Name'].str.contains('Mlle. ').sum()))
print('Don.:{}'.format(df['Name'].str.contains('Don. ').sum()))
print('Countess.:{}'.format(df['Name'].str.contains('Countess. ').sum()))
print('Sir.:{}'.format(df['Name'].str.contains('Sir. ').sum()))

~~~

表示結果

~~~

Dr.:8
Rev.:8
Col.:4
Major.:2
Jonkheer.:1
Mme.:1
Capt.:1
Ms.:2
Mlle.:1
Don.:1
Countess.:1
Sir.:1

~~~

それぞれの値に対して数値に変換を行います。

~~~Python
In [11]:
df.loc[df['Name'].str.contains('Dr. ') == True, 'Name'] = 3
df.loc[df['Name'].str.contains('Rev. ') == True, 'Name'] = 4
df.loc[df['Name'].str.contains('Col. ') == True, 'Name'] = 5
df.loc[df['Name'].str.contains('Major. ') == True, 'Name'] = 6
df.loc[df['Name'].str.contains('Jonkheer. ') == True, 'Name'] = 7
df.loc[df['Name'].str.contains('Mme. ') == True, 'Name'] = 8
df.loc[df['Name'].str.contains('Capt. ') == True, 'Name'] = 9
df.loc[df['Name'].str.contains('Ms. ') == True, 'Name'] = 10
df.loc[df['Name'].str.contains('Mlle. ') == True, 'Name'] = 11
df.loc[df['Name'].str.contains('Don. ') == True, 'Name'] = 12
df.loc[df['Name'].str.contains('Countess. ') == True, 'Name'] = 13
df.loc[df['Name'].str.contains('Sir. ') == True, 'Name'] = 14
df.loc[df['Name'].str.contains('Dona. ') == True, 'Name'] = 15
​
df['Name'].value_counts()
~~~

表示結果

~~~

0     761
1     258
2     197
3      69
4       8
5       4
10      2
6       2
15      1
14      1
13      1
12      1
11      1
9       1
8       1
7       1
Name: Name, dtype: int64

~~~

Nameの変換が完了しました。
次に Sex を数値へ変換します。

~~~Python
# Sex の変換
## 要素の確認
df['Sex'].value_counts()
~~~

表示結果

~~~
male      843
female    466
Name: Sex, dtype: int64
~~~

maleとfemaleの2種類であるため、{male:0, female:1} へ変換します。

~~~Python
df['Sex'] = df['Sex'].replace({'male':0, 'female':1})
​
df['Sex'].value_counts()
~~~

表示結果

~~~
0    843
1    466
Name: Sex, dtype: int64
~~~

Sex の変換が完了しました。
次に Ticket を数値へ変換します。

~~~Python
# Ticket の変換
## 要素の確認
df['Ticket'].value_counts()
~~~

表示結果

~~~
CA. 2343              11
1601                   8
CA 2144                8
347077                 7
3101295                7
347082                 7
PC 17608               7
S.O.C. 14879           7
113781                 6
382652                 6
19950                  6
347088                 6
W./C. 6608             5
16966                  5
349909                 5
113503                 5
220845                 5
4133                   5
PC 17757               5
230136                 4
113760                 4
24160                  4
C.A. 33112             4
W./C. 6607             4
36928                  4
PC 17760               4
LINE                   4
C.A. 34651             4
2666                   4
C.A. 2315              4
                      ..
248744                 1
365235                 1
349228                 1
345765                 1
C 7077                 1
315090                 1
364511                 1
PC 17609               1
C.A. 33111             1
345775                 1
350409                 1
SOTON/O.Q. 3101308     1
367229                 1
PC 17596               1
35851                  1
W/C 14208              1
347464                 1
SOTON/O.Q. 3101306     1
SC/PARIS 2148          1
363272                 1
347064                 1
14311                  1
312992                 1
349215                 1
250650                 1
343120                 1
SC/A.3 2861            1
237393                 1
349236                 1
C.A. 15185             1
Name: Ticket, Length: 929, dtype: int64
~~~

複雑な形であるため、一旦カラムを削除する方針とします。
次に Cabin の欠損値処理と数値への変換を実施します。

~~~Python
# Cabinの処理
## 要素の確認
df['Cabin'].value_counts()
~~~

表示結果

~~~
C23 C25 C27        6
B57 B59 B63 B66    5
G6                 5
C22 C26            4
F33                4
C78                4
F2                 4
F4                 4
D                  4
B96 B98            4
A34                3
B58 B60            3
B51 B53 B55        3
C101               3
E34                3
E101               3
E67                2
D17                2
C65                2
C80                2
C46                2
D37                2
C92                2
E25                2
D10 D12            2
D26                2
B77                2
C106               2
E46                2
C68                2
                  ..
C111               1
F E69              1
E52                1
B52 B54 B56        1
E58                1
B79                1
B37                1
C95                1
B102               1
C87                1
E38                1
C50                1
C148               1
B10                1
C49                1
B61                1
A36                1
D34                1
B39                1
A32                1
E60                1
B101               1
C70                1
C82                1
E39 E41            1
E10                1
A21                1
E63                1
A18                1
D56                1
Name: Cabin, Length: 186, dtype: int64
~~~

それぞれTIcketの頭文字で分類をしてみます。

~~~Python
print('A:{}'.format(df['Cabin'].str.contains('A').sum()))
print('B:{}'.format(df['Cabin'].str.contains('B').sum()))
print('C:{}'.format(df['Cabin'].str.contains('C').sum()))
print('D:{}'.format(df['Cabin'].str.contains('D').sum()))
print('E:{}'.format(df['Cabin'].str.contains('E').sum()))
print('F:{}'.format(df['Cabin'].str.contains('F').sum()))
print('G:{}'.format(df['Cabin'].str.contains('G').sum()))
print('T:{}'.format(df['Cabin'].str.contains('T').sum()))
~~~

表示結果

~~~
A:22
B:65
C:94
D:46
E:44
F:21
G:9
T:1
~~~

一時的に欠損値には0を、その他のデータには数が多い順番に1から数値を振っていきます。

~~~Python
df['Cabin'] = df['Cabin'].fillna(0)
df.loc[df['Cabin'].str.contains('C') == True, 'Cabin'] = 1
df.loc[df['Cabin'].str.contains('B') == True, 'Cabin'] = 2
df.loc[df['Cabin'].str.contains('D') == True, 'Cabin'] = 3
df.loc[df['Cabin'].str.contains('E') == True, 'Cabin'] = 4
df.loc[df['Cabin'].str.contains('A') == True, 'Cabin'] = 5
df.loc[df['Cabin'].str.contains('F') == True, 'Cabin'] = 6
df.loc[df['Cabin'].str.contains('G') == True, 'Cabin'] = 7
df.loc[df['Cabin'].str.contains('T') == True, 'Cabin'] = 8
​
df['Cabin'].value_counts()
~~~

表示結果

~~~
0    1014
1      94
2      65
3      46
4      44
5      22
6      18
7       5
8       1
Name: Cabin, dtype: int64
~~~


Cabin の変換が完了しました。
次に Embarked の欠損値処理と数値への変換を実施します。

~~~Python
# Embarkedの処理
## 要素の確認
df['Embarked'].value_counts()
~~~

表示結果

~~~
S    914
C    270
Q    123
Name: Embarked, dtype: int64
~~~

欠損値が 2 だけなので、一番多い S に合わせます。

~~~Python
df['Embarked'] = df['Embarked'].fillna('S')
​
df['Embarked'].value_counts()
~~~

表示結果

~~~

S    916
C    270
Q    123
Name: Embarked, dtype: int64
~~~

{S:0, C:1, Q:2}へ変換します。

~~~Python
df.loc[df['Embarked'] == 'S', 'Embarked'] = 0
df.loc[df['Embarked'] == 'C', 'Embarked'] = 1
df.loc[df['Embarked'] == 'Q', 'Embarked'] = 2
​
df['Embarked'].value_counts()
~~~

表示結果

~~~
0    916
1    270
2    123
Name: Embarked, dtype: int64
~~~

Embarked の処理が完了しました。
Age の欠損値処理をします。
一時的に中央値を代入します。

~~~Python
df['Age'] = df['Age'].fillna(df['Age'].mean())
​
df.isnull().sum()
~~~

表示結果

~~~
Survived    418
Pclass        0
Name          0
Sex           0
Age           0
SibSp         0
Parch         0
Ticket        0
Fare          1
Cabin         0
Embarked      0
dtype: int64
~~~

Fareの欠損値は2つのみなので、一旦0で穴埋めし、訓練データとテストデータを分割します。

~~~Python
df['Fare'] = df['Fare'].fillna(0)
train = df.loc[:891]
train
~~~

表示結果

~~~

	Survived	Pclass	Name	Sex	Age	SibSp	Parch	Ticket	Fare	Cabin	Embarked
PassengerId											
1	0.0	3	0	0	22.000000	1	0	A/5 21171	7.2500	0	0
2	1.0	1	2	1	38.000000	1	0	PC 17599	71.2833	1	1
3	1.0	3	1	1	26.000000	0	0	STON/O2. 3101282	7.9250	0	0
4	1.0	1	2	1	35.000000	1	0	113803	53.1000	1	0
5	0.0	3	0	0	35.000000	0	0	373450	8.0500	0	0
6	0.0	3	0	0	29.881138	0	0	330877	8.4583	0	2
7	0.0	1	0	0	54.000000	0	0	17463	51.8625	4	0
8	0.0	3	3	0	2.000000	3	1	349909	21.0750	0	0
9	1.0	3	2	1	27.000000	0	2	347742	11.1333	0	0
10	1.0	2	2	1	14.000000	1	0	237736	30.0708	0	1
11	1.0	3	1	1	4.000000	1	1	PP 9549	16.7000	7	0
12	1.0	1	1	1	58.000000	0	0	113783	26.5500	1	0
13	0.0	3	0	0	20.000000	0	0	A/5. 2151	8.0500	0	0
14	0.0	3	0	0	39.000000	1	5	347082	31.2750	0	0
15	0.0	3	1	1	14.000000	0	0	350406	7.8542	0	0
16	1.0	2	2	1	55.000000	0	0	248706	16.0000	0	0
17	0.0	3	3	0	2.000000	4	1	382652	29.1250	0	2
18	1.0	2	0	0	29.881138	0	0	244373	13.0000	0	0
19	0.0	3	2	1	31.000000	1	0	345763	18.0000	0	0
20	1.0	3	2	1	29.881138	0	0	2649	7.2250	0	1
21	0.0	2	0	0	35.000000	0	0	239865	26.0000	0	0
22	1.0	2	0	0	34.000000	0	0	248698	13.0000	3	0
23	1.0	3	1	1	15.000000	0	0	330923	8.0292	0	2
24	1.0	1	0	0	28.000000	0	0	113788	35.5000	5	0
25	0.0	3	1	1	8.000000	3	1	349909	21.0750	0	0
26	1.0	3	2	1	38.000000	1	5	347077	31.3875	0	0
27	0.0	3	0	0	29.881138	0	0	2631	7.2250	0	1
28	0.0	1	0	0	19.000000	3	2	19950	263.0000	1	0
29	1.0	3	1	1	29.881138	0	0	330959	7.8792	0	2
30	0.0	3	0	0	29.881138	0	0	349216	7.8958	0	0
...	...	...	...	...	...	...	...	...	...	...	...
862	0.0	2	0	0	21.000000	1	0	28134	11.5000	0	0
863	1.0	1	2	1	48.000000	0	0	17466	25.9292	3	0
864	0.0	3	1	1	29.881138	8	2	CA. 2343	69.5500	0	0
865	0.0	2	0	0	24.000000	0	0	233866	13.0000	0	0
866	1.0	2	2	1	42.000000	0	0	236852	13.0000	0	0
867	1.0	2	1	1	27.000000	1	0	SC/PARIS 2149	13.8583	0	1
868	0.0	1	0	0	31.000000	0	0	PC 17590	50.4958	5	0
869	0.0	3	0	0	29.881138	0	0	345777	9.5000	0	0
870	1.0	3	3	0	4.000000	1	1	347742	11.1333	0	0
871	0.0	3	0	0	26.000000	0	0	349248	7.8958	0	0
872	1.0	1	2	1	47.000000	1	1	11751	52.5542	3	0
873	0.0	1	0	0	33.000000	0	0	695	5.0000	2	0
874	0.0	3	0	0	47.000000	0	0	345765	9.0000	0	0
875	1.0	2	2	1	28.000000	1	0	P/PP 3381	24.0000	0	1
876	1.0	3	1	1	15.000000	0	0	2667	7.2250	0	1
877	0.0	3	0	0	20.000000	0	0	7534	9.8458	0	0
878	0.0	3	0	0	19.000000	0	0	349212	7.8958	0	0
879	0.0	3	0	0	29.881138	0	0	349217	7.8958	0	0
880	1.0	1	2	1	56.000000	0	1	11767	83.1583	1	1
881	1.0	2	2	1	25.000000	0	1	230433	26.0000	0	0
882	0.0	3	0	0	33.000000	0	0	349257	7.8958	0	0
883	0.0	3	1	1	22.000000	0	0	7552	10.5167	0	0
884	0.0	2	0	0	28.000000	0	0	C.A./SOTON 34068	10.5000	0	0
885	0.0	3	0	0	25.000000	0	0	SOTON/OQ 392076	7.0500	0	0
886	0.0	3	2	1	39.000000	0	5	382652	29.1250	0	2
887	0.0	2	4	0	27.000000	0	0	211536	13.0000	0	0
888	1.0	1	1	1	19.000000	0	0	112053	30.0000	2	0
889	0.0	3	1	1	29.881138	1	2	W./C. 6607	23.4500	0	0
890	1.0	1	0	0	26.000000	0	0	111369	30.0000	1	1
891	0.0	3	0	0	32.000000	0	0	370376	7.7500	0	2

~~~

~~~Python
X_test = df.loc[892:]
X_test = X_test.drop(['Survived', 'Ticket'], axis='columns')
X_test
~~~

表示結果

~~~

	Pclass	Name	Sex	Age	SibSp	Parch	Fare	Cabin	Embarked
PassengerId									
892	3	0	0	34.500000	0	0	7.8292	0	2
893	3	2	1	47.000000	1	0	7.0000	0	0
894	2	0	0	62.000000	0	0	9.6875	0	2
895	3	0	0	27.000000	0	0	8.6625	0	0
896	3	2	1	22.000000	1	1	12.2875	0	0
897	3	0	0	14.000000	0	0	9.2250	0	0
898	3	1	1	30.000000	0	0	7.6292	0	2
899	2	0	0	26.000000	1	1	29.0000	0	0
900	3	2	1	18.000000	0	0	7.2292	0	1
901	3	0	0	21.000000	2	0	24.1500	0	0
902	3	0	0	29.881138	0	0	7.8958	0	0
903	1	0	0	46.000000	0	0	26.0000	0	0
904	1	2	1	23.000000	1	0	82.2667	2	0
905	2	0	0	63.000000	1	0	26.0000	0	0
906	1	2	1	47.000000	1	0	61.1750	4	0
907	2	2	1	24.000000	1	0	27.7208	0	1
908	2	0	0	35.000000	0	0	12.3500	0	2
909	3	0	0	21.000000	0	0	7.2250	0	1
910	3	1	1	27.000000	1	0	7.9250	0	0
911	3	2	1	45.000000	0	0	7.2250	0	1
912	1	0	0	55.000000	1	0	59.4000	0	1
913	3	3	0	9.000000	0	1	3.1708	0	0
914	1	2	1	29.881138	0	0	31.6833	0	0
915	1	0	0	21.000000	0	1	61.3792	0	1
916	1	2	1	48.000000	1	3	262.3750	2	1
917	3	0	0	50.000000	1	0	14.5000	0	0
918	1	1	1	22.000000	0	1	61.9792	2	1
919	3	0	0	22.500000	0	0	7.2250	0	1
920	1	0	0	41.000000	0	0	30.5000	5	0
921	3	0	0	29.881138	2	0	21.6792	0	1
...	...	...	...	...	...	...	...	...	...
1280	3	0	0	21.000000	0	0	7.7500	0	2
1281	3	3	0	6.000000	3	1	21.0750	0	0
1282	1	0	0	23.000000	0	0	93.5000	2	0
1283	1	2	1	51.000000	0	1	39.4000	3	0
1284	3	3	0	13.000000	0	2	20.2500	0	0
1285	2	0	0	47.000000	0	0	10.5000	0	0
1286	3	0	0	29.000000	3	1	22.0250	0	0
1287	1	2	1	18.000000	1	0	60.0000	1	0
1288	3	0	0	24.000000	0	0	7.2500	0	2
1289	1	2	1	48.000000	1	1	79.2000	2	1
1290	3	0	0	22.000000	0	0	7.7750	0	0
1291	3	0	0	31.000000	0	0	7.7333	0	2
1292	1	1	1	30.000000	0	0	164.8667	1	0
1293	2	0	0	38.000000	1	0	21.0000	0	0
1294	1	1	1	22.000000	0	1	59.4000	0	1
1295	1	0	0	17.000000	0	0	47.1000	0	0
1296	1	0	0	43.000000	1	0	27.7208	3	1
1297	2	0	0	20.000000	0	0	13.8625	3	1
1298	2	0	0	23.000000	1	0	10.5000	0	0
1299	1	0	0	50.000000	1	1	211.5000	1	1
1300	3	1	1	29.881138	0	0	7.7208	0	2
1301	3	1	1	3.000000	1	1	13.7750	0	0
1302	3	1	1	29.881138	0	0	7.7500	0	2
1303	1	2	1	37.000000	1	0	90.0000	1	2
1304	3	1	1	28.000000	0	0	7.7750	0	0
1305	3	0	0	29.881138	0	0	8.0500	0	0
1306	1	15	1	39.000000	0	0	108.9000	1	1
1307	3	0	0	38.500000	0	0	7.2500	0	0
1308	3	0	0	29.881138	0	0	8.0500	0	0
1309	3	3	0	29.881138	1	1	22.3583	0	1
418 rows × 9 columns

~~~

訓練データを説明変数と目的変数に分割します。

~~~Python
X_train = train.drop(['Survived', 'Ticket'], axis='columns')
y_train = train['Survived']
​
print(X_train.head())
print(y_train.head())
~~~

表示結果

~~~
            Pclass  Name  Sex   Age  SibSp  Parch     Fare  Cabin  Embarked
PassengerId                                                                 
1                 3     0    0  22.0      1      0   7.2500      0         0
2                 1     2    1  38.0      1      0  71.2833      1         1
3                 3     1    1  26.0      0      0   7.9250      0         0
4                 1     2    1  35.0      1      0  53.1000      1         0
5                 3     0    0  35.0      0      0   8.0500      0         0
PassengerId
1    0.0
2    1.0
3    1.0
4    1.0
5    0.0
Name: Survived, dtype: float64

~~~

使用するモデルを定義し、訓練データを学習させます。
今回はRandomForestClassifierを使用します。

~~~Python
model = RandomForestClassifier(n_estimators=200, random_state=71)
model.fit(X_train, y_train)
~~~

表示結果

~~~
RandomForestClassifier(bootstrap=True, class_weight=None, criterion='gini',
            max_depth=None, max_features='auto', max_leaf_nodes=None,
            min_impurity_decrease=0.0, min_impurity_split=None,
            min_samples_leaf=1, min_samples_split=2,
            min_weight_fraction_leaf=0.0, n_estimators=200, n_jobs=None,
            oob_score=False, random_state=71, verbose=0, warm_start=False)
~~~

定義した予測させます。

~~~Python
y_pred = model.predict(X_test)
~~~

正解ラベルを読み込みます。

~~~Python
y_true = pd.read_csv('./data/gender_submission.csv', index_col='PassengerId')
y_true
~~~

表示結果

~~~

	Survived
PassengerId	
892	0
893	1
894	0
895	0
896	1
897	0
898	1
899	0
900	1
901	0
902	0
903	0
904	1
905	0
906	1
907	1
908	0
909	0
910	1
911	1
912	0
913	0
914	1
915	0
916	1
917	0
918	1
919	0
920	0
921	0
...	...
1280	0
1281	0
1282	0
1283	1
1284	0
1285	0
1286	0
1287	1
1288	0
1289	1
1290	0
1291	0
1292	1
1293	0
1294	1
1295	0
1296	0
1297	0
1298	0
1299	0
1300	1
1301	1
1302	1
1303	1
1304	1
1305	0
1306	1
1307	0
1308	0
1309	0
418 rows × 1 columns
~~~

confusiojn_matrix と classification_report を使用し、評価を確認します。

~~~Python
print('正答率：{}'.format(accuracy_score(y_true=y_true , y_pred=y_pred))+ '\n')

print('混合行列:\n{}'.format(confusion_matrix(y_true=y_true , y_pred=y_pred))+ '\n')

print('精度の評価:\n{}'.format(classification_report(y_true=y_true , y_pred=y_pred)))
~~~

表示結果

~~~

正答率：0.8229665071770335

混合行列:
[[225  41]
 [ 33 119]]

精度の評価:
              precision    recall  f1-score   support

           0       0.87      0.85      0.86       266
           1       0.74      0.78      0.76       152

   micro avg       0.82      0.82      0.82       418
   macro avg       0.81      0.81      0.81       418
weighted avg       0.83      0.82      0.82       418

~~~

正答率として8割を超える結果が出ました。
こちらをkaggleで評価する場合、PassengerIDを付与したcsvファイルを作成して提出しましょう。





# 4.まとめ

機械学習の一連の流れを実装してみました。
まだまだ改善の余地がある結果でした。
コードをみていただくとわかる通り、ほとんどが前処理です。
Kaggleのデータはある程度まとまっているデータが多いので、前処理にかける時間もそこまで多くないかもしれませんが、実務で行う際は、どの変数が効いてくるのか等をトライアンドエラーを繰り返しながら探していく作業をしていきます。(想像以上に泥臭かったです。)

またKaggleのKernelも参考になるという話をよく聞くので、kernelを参考にしつつ今後も精度を改善していきたいと思います。