訓練用データとテスト用データをダミー変数化したときに列数が合わない問題を解決する

このようなデータがあるとします。

```python
import pandas as pd

train = pd.DataFrame({'Categorical':['A','B','C','E','G','I','J','K'],
                     'Numerical':[1, 2, 3, 4, 5, 6, 7, 8]})
test = pd.DataFrame({'Categorical':['A','B','D','F','H','J','K'],
                    'Numerical':[1, 2, 3, 4, 5, 6, 7]})
print(train)
print(test)
```

```
  Categorical  Numerical
0           A          1
1           B          2
2           C          3
3           E          4
4           G          5
5           I          6
6           J          7
7           K          8
  Categorical  Numerical
0           A          1
1           B          2
2           D          3
3           F          4
4           H          5
5           J          6
6           K          7
```
train, testをそれぞれダミー変数化します。

```python
train = pd.get_dummies(train)
test = pd.get_dummies(test)
print(train)
print(test)
```



```
   Numerical  Categorical_A  Categorical_B  Categorical_C  Categorical_E  \
0          1              1              0              0              0   
1          2              0              1              0              0   
2          3              0              0              1              0   
3          4              0              0              0              1   
4          5              0              0              0              0   
5          6              0              0              0              0   
6          7              0              0              0              0   
7          8              0              0              0              0   

   Categorical_G  Categorical_I  Categorical_J  Categorical_K  
0              0              0              0              0  
1              0              0              0              0  
2              0              0              0              0  
3              0              0              0              0  
4              1              0              0              0  
5              0              1              0              0  
6              0              0              1              0  
7              0              0              0              1  
   Numerical  Categorical_A  Categorical_B  Categorical_D  Categorical_F  \
0          1              1              0              0              0   
1          2              0              1              0              0   
2          3              0              0              1              0   
3          4              0              0              0              1   
4          5              0              0              0              0   
5          6              0              0              0              0   
6          7              0              0              0              0   

   Categorical_H  Categorical_J  Categorical_K  
0              0              0              0  
1              0              0              0  
2              0              0              0  
3              0              0              0  
4              1              0              0  
5              0              1              0  
6              0              0              1  
```

列数並びに列名が合いません。

そこで不足している列の値を全て0で補う関数を定義します。

```python
def fill_missing_columns(df_a, df_b):
    columns_for_b = set(df_a.columns) - set(df_b.columns)
    for column in columns_for_b:
        df_b[column] = 0
    columns_for_a = set(df_b.columns) - set(df_a.columns)
    for column in columns_for_a:
        df_a[column] = 0
```

関数を実行します。

```python
fill_missing_columns(train, test)

train.sort_index(axis=1, inplace=True)
test.sort_index(axis=1, inplace=True)

print('train')
print(train.shape)
print(train)
print('test')
print(test.shape)
print(test)
```

```
train
(8, 12)
   Categorical_A  Categorical_B  Categorical_C  Categorical_D  Categorical_E  \
0              1              0              0              0              0   
1              0              1              0              0              0   
2              0              0              1              0              0   
3              0              0              0              0              1   
4              0              0              0              0              0   
5              0              0              0              0              0   
6              0              0              0              0              0   
7              0              0              0              0              0   

   Categorical_F  Categorical_G  Categorical_H  Categorical_I  Categorical_J  \
0              0              0              0              0              0   
1              0              0              0              0              0   
2              0              0              0              0              0   
3              0              0              0              0              0   
4              0              1              0              0              0   
5              0              0              0              1              0   
6              0              0              0              0              1   
7              0              0              0              0              0   

   Categorical_K  Numerical  
0              0          1  
1              0          2  
2              0          3  
3              0          4  
4              0          5  
5              0          6  
6              0          7  
7              1          8  
test
(7, 12)
   Categorical_A  Categorical_B  Categorical_C  Categorical_D  Categorical_E  \
0              1              0              0              0              0   
1              0              1              0              0              0   
2              0              0              0              1              0   
3              0              0              0              0              0   
4              0              0              0              0              0   
5              0              0              0              0              0   
6              0              0              0              0              0   

   Categorical_F  Categorical_G  Categorical_H  Categorical_I  Categorical_J  \
0              0              0              0              0              0   
1              0              0              0              0              0   
2              0              0              0              0              0   
3              1              0              0              0              0   
4              0              0              1              0              0   
5              0              0              0              0              1   
6              0              0              0              0              0   

   Categorical_K  Numerical  
0              0          1  
1              0          2  
2              0          3  
3              0          4  
4              0          5  
5              0          6  
6              1          7   
```

列名並びに列数を合わせることができました。

### 追記

@nkay さんにコメントでスマートな下記の書き方を教えていただきました。ありがとうございます。


```python

import pandas as pd

train = pd.DataFrame({'Categorical':['A','B','C','E','G','I','J','K'],
                     'Numerical':[1, 2, 3, 4, 5, 6, 7, 8]})
test = pd.DataFrame({'Categorical':['A','B','D','F','H','J','K'],
                    'Numerical':[1, 2, 3, 4, 5, 6, 7]})


cat_list = {*train.Categorical, *test.Categorical}
# 方法1
train.Categorical = train.Categorical.astype(pd.CategoricalDtype(cat_list))
# 方法2
test.Categorical = pd.Categorical(test.Categorical, cat_list)


train = pd.get_dummies(train)
test = pd.get_dummies(test)
print(train)
print(test)
```

参考： 
https://pandas.pydata.org/pandas-docs/version/0.23/generated/pandas.api.types.CategoricalDtype.html
https://pandas.pydata.org/pandas-docs/version/0.23.4/generated/pandas.Categorical.html