Pythonで機械学習 - Data Understanding

Pythonで

データの操作にはPandasを使います。


```python
import pandas as pd
```

Jupyterでの描画のためmatplotlibも使用します。


```python
import matplotlib.pyplot as plt
%matplotlib inline
```

タイタニックのデータを読み込みます。Pandasのread_csvという関数でCSVファイルを取り込み、DataFrame型として保持します。


```python
train = pd.read_csv("train.csv")
```

まずはデータの中身をざっと見てみます。


```python
train.head()
```




<div>
<table border="1" class="dataframe">
  <thead>
    <tr style="text-align: right;">
      <th></th>
      <th>PassengerId</th>
      <th>Survived</th>
      <th>Pclass</th>
      <th>Name</th>
      <th>Sex</th>
      <th>Age</th>
      <th>SibSp</th>
      <th>Parch</th>
      <th>Ticket</th>
      <th>Fare</th>
      <th>Cabin</th>
      <th>Embarked</th>
    </tr>
  </thead>
  <tbody>
    <tr>
      <th>0</th>
      <td>1</td>
      <td>0</td>
      <td>3</td>
      <td>Braund, Mr. Owen Harris</td>
      <td>male</td>
      <td>22.0</td>
      <td>1</td>
      <td>0</td>
      <td>A/5 21171</td>
      <td>7.2500</td>
      <td>NaN</td>
      <td>S</td>
    </tr>
    <tr>
      <th>1</th>
      <td>2</td>
      <td>1</td>
      <td>1</td>
      <td>Cumings, Mrs. John Bradley (Florence Briggs Th...</td>
      <td>female</td>
      <td>38.0</td>
      <td>1</td>
      <td>0</td>
      <td>PC 17599</td>
      <td>71.2833</td>
      <td>C85</td>
      <td>C</td>
    </tr>
    <tr>
      <th>2</th>
      <td>3</td>
      <td>1</td>
      <td>3</td>
      <td>Heikkinen, Miss. Laina</td>
      <td>female</td>
      <td>26.0</td>
      <td>0</td>
      <td>0</td>
      <td>STON/O2. 3101282</td>
      <td>7.9250</td>
      <td>NaN</td>
      <td>S</td>
    </tr>
    <tr>
      <th>3</th>
      <td>4</td>
      <td>1</td>
      <td>1</td>
      <td>Futrelle, Mrs. Jacques Heath (Lily May Peel)</td>
      <td>female</td>
      <td>35.0</td>
      <td>1</td>
      <td>0</td>
      <td>113803</td>
      <td>53.1000</td>
      <td>C123</td>
      <td>S</td>
    </tr>
    <tr>
      <th>4</th>
      <td>5</td>
      <td>0</td>
      <td>3</td>
      <td>Allen, Mr. William Henry</td>
      <td>male</td>
      <td>35.0</td>
      <td>0</td>
      <td>0</td>
      <td>373450</td>
      <td>8.0500</td>
      <td>NaN</td>
      <td>S</td>
    </tr>
  </tbody>
</table>
</div>



[Kaggle](https://www.kaggle.com/c/titanic/data)によると各項目の定義は以下のとおりです。

|Variable|Definition|Key/Remark|
|:--|:--|:--|:--|
|survival|Survival|0 = No, 1 = Yes|
|pclass|Ticket class|1 = 1st(Upper), 2 = 2nd(Middle), 3 = 3rd(Lower)|
|sex|Sex||
|Age|Age in years|Age is fractional if less than 1. If the age is estimated, is it in the form of xx.5|
|sibsp|# of siblings / spouses aboard the Titanic|Sibling = brother, sister, stepbrother, stepsister / Spouse = husband, wife (mistresses and fiancés were ignored)|
|parch|# of parents / children aboard the Titanic|Parent = mother, father / Child = daughter, son, stepdaughter, stepson / Some children travelled only with a nanny, therefore parch=0 for them.|	
|ticket|Ticket number||
|fare|Passenger fare||
|cabin|Cabin number||
|embarked|Port of Embarkation|C = Cherbourg, Q = Queenstown, S = Southampton|

データの件数をカウントします。shape()で行数、列数を取得できます。全部で891行あります。


```python
train.shape
```




    (891, 12)



各列に欠損値がないかを、count()関数で確認します。Age, Cabinには欠損値があるようです。とくにCabinは欠損値が大半なので、変数としては使えなそうです。


```python
train.count()
```




    PassengerId    891
    Survived       891
    Pclass         891
    Name           891
    Sex            891
    Age            714
    SibSp          891
    Parch          891
    Ticket         891
    Fare           891
    Cabin          204
    Embarked       889
    dtype: int64



データの分布を可視化してみます。


```python
train.Survived.value_counts().plot.bar()
```









![output_16_1.png](https://qiita-image-store.s3.amazonaws.com/0/186483/c94b5dcc-b90f-aeb9-33eb-073877abb474.png)



```python
train.Pclass.value_counts().sort_index().plot.bar()
```









![image.png](https://qiita-image-store.s3.amazonaws.com/0/186483/d0f0b006-a7ab-923e-05f9-ea1befdb632d.png)


```python
train.Sex.value_counts().plot.bar()
```









![image.png](https://qiita-image-store.s3.amazonaws.com/0/186483/853fa4e3-5f43-ba8c-30ac-11aa62b8b3ee.png)




```python
train.Age.plot.hist()
```









![image.png](https://qiita-image-store.s3.amazonaws.com/0/186483/823d41fb-0c70-c6f8-4332-39038524b3e4.png)



```python
train.SibSp.value_counts().sort_index().plot.bar()
```









![image.png](https://qiita-image-store.s3.amazonaws.com/0/186483/df810045-057f-187e-880b-aa583a4f3de3.png)




```python
train.Parch.value_counts().sort_index().plot.bar()
```









![image.png](https://qiita-image-store.s3.amazonaws.com/0/186483/01265a92-18ea-bfde-e828-0a022772b8a1.png)




```python
train.Fare.plot.hist()
```









![image.png](https://qiita-image-store.s3.amazonaws.com/0/186483/ae67e56c-6eac-1746-91f6-4bf896e9ecd3.png)




```python
train.Embarked.value_counts().plot.bar()
```









![image.png](https://qiita-image-store.s3.amazonaws.com/0/186483/d556eb48-9f26-b85e-9820-dd14c45e6e0e.png)

等級の高い（数字の小さい）船室の方が生存率が高い


```python
train.groupby(["Pclass"]).agg(["count","sum"])["Survived"].plot.bar()
```




    <matplotlib.axes._subplots.AxesSubplot at 0x10864b110>




![output_23_1.png](https://qiita-image-store.s3.amazonaws.com/0/186483/ee2b89a9-d59b-d446-1b38-e24040d2d560.png)



女性の方が生存率が高い


```python
train.groupby(["Sex"]).agg(["count","sum"])["Survived"].plot.bar()
```




    <matplotlib.axes._subplots.AxesSubplot at 0x10851b590>




![output_25_1.png](https://qiita-image-store.s3.amazonaws.com/0/186483/6788dd0b-fa65-da91-da59-3ea7b6590248.png)




```python
train.groupby(["SibSp"]).agg(["count","sum"])["Survived"].plot.bar()
```




    <matplotlib.axes._subplots.AxesSubplot at 0x10921c690>




![output_26_1.png](https://qiita-image-store.s3.amazonaws.com/0/186483/7680e021-7d7c-5983-a291-80ca9817ba90.png)




```python
train.groupby(["Parch"]).agg(["count","sum"])["Survived"].plot.bar()
```




    <matplotlib.axes._subplots.AxesSubplot at 0x10921c3d0>




![output_27_1.png](https://qiita-image-store.s3.amazonaws.com/0/186483/df2be4fa-3de8-fb53-2aad-40126fd72e12.png)




```python
train["NumFamily"] = train["SibSp"] + train["Parch"]
train.groupby(["NumFamily"]).agg(["count","sum"])["Survived"].plot.bar()
```




    <matplotlib.axes._subplots.AxesSubplot at 0x10a26e450>




![output_28_1.png](https://qiita-image-store.s3.amazonaws.com/0/186483/b9e55a29-1f27-b008-a583-fc66f2d98926.png)



8歳未満は特に生存率が高い(AgeBand=6は欠損値)


```python
import numpy as np
bins = np.histogram(
    train["Age"].fillna(train["Age"].median())
    ,bins=10)[1]
print bins
train["AgeBand"] = np.digitize(train["Age"],bins)
train.groupby(["AgeBand"]).agg(["count","sum"])["Survived"].plot.bar()
```

    [  0.42    8.378  16.336  24.294  32.252  40.21   48.168  56.126  64.084
      72.042  80.   ]





    <matplotlib.axes._subplots.AxesSubplot at 0x1090af0d0>




![output_30_2.png](https://qiita-image-store.s3.amazonaws.com/0/186483/0a898642-aa0f-0ded-d587-332e1f226fce.png)




```python
bins = np.histogram(
    train["Fare"].fillna(train["Fare"].median())
    ,bins=10)[1]
print bins
train["FareBand"] = np.digitize(train["Fare"],bins)
train.groupby(["FareBand"]).agg(["count","sum"])["Survived"].plot.bar()


```

    [   0.        51.23292  102.46584  153.69876  204.93168  256.1646
      307.39752  358.63044  409.86336  461.09628  512.3292 ]





    <matplotlib.axes._subplots.AxesSubplot at 0x1093adc10>




![output_31_2.png](https://qiita-image-store.s3.amazonaws.com/0/186483/01905212-ef3e-6c8d-2dc2-c0101b0ac455.png)




[戻る](https://qiita.com/nittaryo1977/private/b16386ef78152a20bc2b)