Pandas: データフレームについて--02:欠損値の取り扱い

# 欠損値の取り扱い

## 1.　NaN を含むデータフレームの例

NaN は np.nan として入力する。


```python
import numpy as np
import pandas as pd

df = pd.DataFrame({
        'a': [1, 2, 3, 4, 5],
        'b': [1.2, np.nan, 5.6, 7.8, 9.0],
        'c': ['a', 'b', 'c', 'd', 'e']
    })
df
```




<div>
<style scoped>
    .dataframe tbody tr th:only-of-type {
        vertical-align: middle;
    }

    .dataframe tbody tr th {
        vertical-align: top;
    }

    .dataframe thead th {
        text-align: right;
    }
</style>
<table border="1" class="dataframe">
  <thead>
    <tr style="text-align: right;">
      <th></th>
      <th>a</th>
      <th>b</th>
      <th>c</th>
    </tr>
  </thead>
  <tbody>
    <tr>
      <th>0</th>
      <td>1</td>
      <td>1.2</td>
      <td>a</td>
    </tr>
    <tr>
      <th>1</th>
      <td>2</td>
      <td>NaN</td>
      <td>b</td>
    </tr>
    <tr>
      <th>2</th>
      <td>3</td>
      <td>5.6</td>
      <td>c</td>
    </tr>
    <tr>
      <th>3</th>
      <td>4</td>
      <td>7.8</td>
      <td>d</td>
    </tr>
    <tr>
      <th>4</th>
      <td>5</td>
      <td>9.0</td>
      <td>e</td>
    </tr>
  </tbody>
</table>
</div>



## 2.　欠損値を取り除く　`.dropna`

1 つでも欠損値のある行は，1 行全部を削除する。

    dropna(axis: 'Axis' = 0, how: 'str' = 'any', thresh=None,
           subset=None, inplace: 'bool' = False)


```python
df.dropna()
```




<div>
<style scoped>
    .dataframe tbody tr th:only-of-type {
        vertical-align: middle;
    }

    .dataframe tbody tr th {
        vertical-align: top;
    }

    .dataframe thead th {
        text-align: right;
    }
</style>
<table border="1" class="dataframe">
  <thead>
    <tr style="text-align: right;">
      <th></th>
      <th>a</th>
      <th>b</th>
      <th>c</th>
    </tr>
  </thead>
  <tbody>
    <tr>
      <th>0</th>
      <td>1</td>
      <td>1.2</td>
      <td>a</td>
    </tr>
    <tr>
      <th>2</th>
      <td>3</td>
      <td>5.6</td>
      <td>c</td>
    </tr>
    <tr>
      <th>3</th>
      <td>4</td>
      <td>7.8</td>
      <td>d</td>
    </tr>
    <tr>
      <th>4</th>
      <td>5</td>
      <td>9.0</td>
      <td>e</td>
    </tr>
  </tbody>
</table>
</div>



## 3.　欠損値を特定の値に置き換える

### 3.1.　`fillna`

欠損値を　`value` で指定する値に置き換える。

    fillna(value: 'object | ArrayLike | None' = None,
        method: 'FillnaOptions | None' = None,
        axis: 'Axis | None' = None, inplace: 'bool' = False,
        limit=None, downcast=None)`


```python
df.fillna(99999)
```




<div>
<style scoped>
    .dataframe tbody tr th:only-of-type {
        vertical-align: middle;
    }

    .dataframe tbody tr th {
        vertical-align: top;
    }

    .dataframe thead th {
        text-align: right;
    }
</style>
<table border="1" class="dataframe">
  <thead>
    <tr style="text-align: right;">
      <th></th>
      <th>a</th>
      <th>b</th>
      <th>c</th>
    </tr>
  </thead>
  <tbody>
    <tr>
      <th>0</th>
      <td>1</td>
      <td>1.2</td>
      <td>a</td>
    </tr>
    <tr>
      <th>1</th>
      <td>2</td>
      <td>99999.0</td>
      <td>b</td>
    </tr>
    <tr>
      <th>2</th>
      <td>3</td>
      <td>5.6</td>
      <td>c</td>
    </tr>
    <tr>
      <th>3</th>
      <td>4</td>
      <td>7.8</td>
      <td>d</td>
    </tr>
    <tr>
      <th>4</th>
      <td>5</td>
      <td>9.0</td>
      <td>e</td>
    </tr>
  </tbody>
</table>
</div>



### 3.2.　`bfill`

欠損値を直後の非欠損値で置き換える。

`fillna` で `method='bfill'` を指定するのと同じ。

    bfill(axis: 'None | Axis' = None, inplace: 'bool' = False,
          limit: 'None | int' = None, downcast=None)


```python
df.bfill()
```




<div>
<style scoped>
    .dataframe tbody tr th:only-of-type {
        vertical-align: middle;
    }

    .dataframe tbody tr th {
        vertical-align: top;
    }

    .dataframe thead th {
        text-align: right;
    }
</style>
<table border="1" class="dataframe">
  <thead>
    <tr style="text-align: right;">
      <th></th>
      <th>a</th>
      <th>b</th>
      <th>c</th>
    </tr>
  </thead>
  <tbody>
    <tr>
      <th>0</th>
      <td>1</td>
      <td>1.2</td>
      <td>a</td>
    </tr>
    <tr>
      <th>1</th>
      <td>2</td>
      <td>5.6</td>
      <td>b</td>
    </tr>
    <tr>
      <th>2</th>
      <td>3</td>
      <td>5.6</td>
      <td>c</td>
    </tr>
    <tr>
      <th>3</th>
      <td>4</td>
      <td>7.8</td>
      <td>d</td>
    </tr>
    <tr>
      <th>4</th>
      <td>5</td>
      <td>9.0</td>
      <td>e</td>
    </tr>
  </tbody>
</table>
</div>




```python
df.fillna(method='bfill') # df.bifill() と同じ
```




<div>
<style scoped>
    .dataframe tbody tr th:only-of-type {
        vertical-align: middle;
    }

    .dataframe tbody tr th {
        vertical-align: top;
    }

    .dataframe thead th {
        text-align: right;
    }
</style>
<table border="1" class="dataframe">
  <thead>
    <tr style="text-align: right;">
      <th></th>
      <th>a</th>
      <th>b</th>
      <th>c</th>
    </tr>
  </thead>
  <tbody>
    <tr>
      <th>0</th>
      <td>1</td>
      <td>1.2</td>
      <td>a</td>
    </tr>
    <tr>
      <th>1</th>
      <td>2</td>
      <td>5.6</td>
      <td>b</td>
    </tr>
    <tr>
      <th>2</th>
      <td>3</td>
      <td>5.6</td>
      <td>c</td>
    </tr>
    <tr>
      <th>3</th>
      <td>4</td>
      <td>7.8</td>
      <td>d</td>
    </tr>
    <tr>
      <th>4</th>
      <td>5</td>
      <td>9.0</td>
      <td>e</td>
    </tr>
  </tbody>
</table>
</div>



### 3.3.　`ffill`

欠損値を直前の非欠損値で置き換える。

`fillna` で `method='ffill'` を指定するのと同じ。

    ffill(axis: 'None | Axis' = None, inplace: 'bool' = False,
          limit: 'None | int' = None, downcast=None)


```python
df.ffill()
```




<div>
<style scoped>
    .dataframe tbody tr th:only-of-type {
        vertical-align: middle;
    }

    .dataframe tbody tr th {
        vertical-align: top;
    }

    .dataframe thead th {
        text-align: right;
    }
</style>
<table border="1" class="dataframe">
  <thead>
    <tr style="text-align: right;">
      <th></th>
      <th>a</th>
      <th>b</th>
      <th>c</th>
    </tr>
  </thead>
  <tbody>
    <tr>
      <th>0</th>
      <td>1</td>
      <td>1.2</td>
      <td>a</td>
    </tr>
    <tr>
      <th>1</th>
      <td>2</td>
      <td>1.2</td>
      <td>b</td>
    </tr>
    <tr>
      <th>2</th>
      <td>3</td>
      <td>5.6</td>
      <td>c</td>
    </tr>
    <tr>
      <th>3</th>
      <td>4</td>
      <td>7.8</td>
      <td>d</td>
    </tr>
    <tr>
      <th>4</th>
      <td>5</td>
      <td>9.0</td>
      <td>e</td>
    </tr>
  </tbody>
</table>
</div>




```python
df.fillna(method='ffill') # df.ffill() と同じ
```




<div>
<style scoped>
    .dataframe tbody tr th:only-of-type {
        vertical-align: middle;
    }

    .dataframe tbody tr th {
        vertical-align: top;
    }

    .dataframe thead th {
        text-align: right;
    }
</style>
<table border="1" class="dataframe">
  <thead>
    <tr style="text-align: right;">
      <th></th>
      <th>a</th>
      <th>b</th>
      <th>c</th>
    </tr>
  </thead>
  <tbody>
    <tr>
      <th>0</th>
      <td>1</td>
      <td>1.2</td>
      <td>a</td>
    </tr>
    <tr>
      <th>1</th>
      <td>2</td>
      <td>1.2</td>
      <td>b</td>
    </tr>
    <tr>
      <th>2</th>
      <td>3</td>
      <td>5.6</td>
      <td>c</td>
    </tr>
    <tr>
      <th>3</th>
      <td>4</td>
      <td>7.8</td>
      <td>d</td>
    </tr>
    <tr>
      <th>4</th>
      <td>5</td>
      <td>9.0</td>
      <td>e</td>
    </tr>
  </tbody>
</table>
</div>



## 4.　もっと知りたい人

    help(df.dropna) # Help on method dropna in module pandas.core.frame
    help(df.fillna) # Help on method fillna in module pandas.core.frame
    help(df.bfill)  # Help on method bfill in module pandas.core.frame
    help(df.ffill)  # Help on method ffill in module pandas.core.frame