【機械学習】kerasでの実装【レビュー文分析】

# 参考動画
<iframe width="560" height="315" src="https://www.youtube.com/embed/Vo9Atzozv0w" title="YouTube video player" frameborder="0" allow="accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture" allowfullscreen></iframe>

・上の動画を参考にして、**Python**で深層学習の実装をしてみます。
・ライブラリは**Keras**を使用します。

## データの取得
```python
import numpy as np
import matplotlib.pyplot as plt
from keras.datasets import imdb
from keras.preprocessing.text import Tokenizer
from keras import models
from keras import layers
from keras.callbacks import EarlyStopping
from sklearn.model_selection import train_test_split

#データの呼び出し
#imdbは映画のレビュー文を集めてそれが肯定的か否定的か評価するデータ
(train_data, train_labels), (test_data, test_labels) = imdb.load_data(num_words=10000)

Downloading data from https://storage.googleapis.com/tensorflow/tf-keras-datasets/imdb.npz
   16384/17464789 [..............................] - ET  
499712/17464789 [..............................] - ET 
1064960/17464789 [>.............................] - ET 
1490944/17464789 [=>............................] - ET 
2228224/17464789 [==>...........................] - ET 
2834432/17464789 [===>..........................] - ET 
3407872/17464789 [====>.........................] - ET 
4046848/17464789 [=====>........................] - ET 
4571136/17464789 [======>.......................] - ET 
5210112/17464789 [=======>......................] - ET 
5701632/17464789 [========>.....................] - ET 
6373376/17464789 [=========>....................] - ET 
7028736/17464789 [===========>..................] - ET 
7520256/17464789 [===========>..................] - ET 
8126464/17464789 [============>.................] - ET 
8781824/17464789 [==============>...............] - ET 
9338880/17464789 [===============>..............] - ET 
9879552/17464789 [===============>..............] - ET
10567680/17464789 [=================>............] - ET
11141120/17464789 [==================>...........] - ET
11829248/17464789 [===================>..........] - ET
12369920/17464789 [====================>.........] - ET
12943360/17464789 [=====================>........] - ET
13533184/17464789 [======================>.......] - ET
14155776/17464789 [=======================>......] - ET
14811136/17464789 [========================>.....] - ET
15351808/17464789 [=========================>....] - ET
15892480/17464789 [==========================>...] - ET
16547840/17464789 [===========================>..] - ET
17154048/17464789 [============================>.] - ET
17465344/17464789 [==============================] - 2s 0us/step
17473536/17464789 [==============================] - 2s 0us/step

print(train_data.shape)
(25000,)
```


## データの確認
```python
print(train_data[0],train_labels[0])

#各番号それぞれ一つの単語を意味する
[1, 14, 22, 16, 43, 530, 973, 1622, 1385, 65, 458, 4468, 66, 3941, 4, 173, 36, 256, 5, 25, 100, 43, 838, 112, 
50, 670, 2, 9, 35, 480, 284, 5, 150, 4, 172, 112, 167, 
2, 336, 385, 39, 4, 172, 4536, 1111, 17, 546, 38, 13, 447, 4, 192, 50, 16, 6, 147, 2025, 19, 14, 22, 4, 1920, 
4613, 469, 4, 22, 71, 87, 12, 16, 43, 530, 38, 76, 15, 
13, 1247, 4, 22, 17, 515, 17, 12, 16, 626, 18, 2, 5, 62, 386, 12, 8, 316, 8, 106, 5, 4, 2223, 5244, 16, 480, 66, 3785, 33, 4, 130, 12, 16, 38, 619, 5, 25, 124, 51, 36, 135, 48, 25, 1415, 33, 6, 22, 12, 215, 28, 77, 52, 5, 14, 407, 16, 82, 2, 8, 4, 107, 117, 5952, 15, 256, 4, 2, 7, 3766, 5, 723, 36, 71, 43, 530, 476, 26, 400, 317, 46, 7, 4, 2, 1029, 13, 104, 88, 4, 381, 15, 297, 98, 
32, 2071, 56, 26, 141, 6, 194, 7486, 18, 4, 226, 22, 21, 134, 476, 26, 480, 5, 144, 30, 5535, 18, 51, 36, 28, 
224, 92, 25, 104, 4, 226, 65, 16, 38, 1334, 88, 12, 16, 283, 5, 16, 4472, 113, 103, 32, 15, 16, 5345, 19, 178, 32] 

#1:肯定的、0:否定的
1
```

## データをワンホットベクトル（文章を0か1で表したベクトル）の行列に直す
```python
tokenizer = Tokenizer(num_words=10000)
X_train = tokenizer.sequences_to_matrix(train_data, mode='binary')
X_test = tokenizer.sequences_to_matrix(train_data, mode='binary')

print(X_train.shape)
(25000, 10000)
```


## ラベルもnumpy配列に変換
```python
y_train = np.asarray(train_labels).astype('float32')
y_test = np.asarray(train_labels).astype('float32')

```

## トレーニングデータを更にトレーニングと検証に分割
```python
X_train, X_val, y_train, y_val = train_test_split(X_train, y_train, test_size=0.1, random_state=0)

```

## モデル作成
```python
model = models.Sequential()
model.add(layers.Dense(32, activation='relu', input_shape=(10000,)))
model.add(layers.Dense(16, activation='relu'))
model.add(layers.Dense(1, activation='sigmoid'))

#モデルの内容を表示
print(model.summary())
Model: "sequential"
_________________________________________________________________
 Layer (type)                Output Shape
Param #
=================================================================
 dense (Dense)               (None, 32)
320032


 dense_1 (Dense)             (None, 16)
528


 dense_2 (Dense)             (None, 1)
17


=================================================================
Total params: 320,577
Trainable params: 320,577
Non-trainable params: 0
_________________________________________________________________
```

## 学習処理の内容
```python
model.compile(optimizer='Adam', loss = 'binary_crossentropy', metrics=['accuracy'])
```

## 学習の実行
```python
history = model.fit(x=X_train, y=y_train, epochs=20, batch_size=512, verbose=1, validation_data=(X_val,y_val))
```

## historyの中身を確認
```python
history_dict = history.history

print(history_dict.keys())

dict_keys(['loss', 'accuracy', 'val_loss', 'val_accuracy'])
```

## 正確度と損失の可視化
```python
#正確度の可視化
acc = history_dict['accuracy']
val_acc = history_dict['val_accuracy']
epochs = range(1, len(acc)+1)
plt.figure(figsize=(12, 8))
plt.plot(epochs, acc, label='acc')
plt.plot(epochs, val_acc, label='val_acc')
plt.ylim((0,1))
plt.legend(loc='best')
plt.show()

#損失の可視化
acc = history_dict['loss']
val_acc = history_dict['val_loss']
epochs = range(1, len(acc)+1)
plt.figure(figsize=(12, 8))
plt.plot(epochs, acc, label='loss')
plt.plot(epochs, val_acc, label='val_loss')
plt.ylim((0,1))
plt.legend(loc='best')
plt.show()
```
![model acc valacc1.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/2658998/17977cde-49c7-c13a-ab3b-46f3ce87a652.png)
![model loss valloss1.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/2658998/91625ea2-fa68-c648-c55b-0d8197a31f66.png)
・普通のaccとlossはいい感じだけどval（検証用）にはしっかり反映されていない。
→過学習
→モデルが複雑すぎ。

・accもlossも早い段階で結果が頭打ちになる。
→アーリーストップを加える

## アーリーストップを加える
```python
#アーリーストップを加える
callbacks = [EarlyStopping(monitor='val_accuracy', patience=3)]

#学習の実行
history1 = model.fit(x=X_train, y=y_train, epochs=20, batch_size=512, 
                    verbose=1, callbacks=callbacks, validation_data=(X_val,y_val))
```

## テストセットを使ってスコアを見る
```python
score = model.evaluate(X_test, y_test)
print("Test set loss: {}, Test set acc: {}".format(score[0], score[1]))

Test set loss: 0.08818113058805466, Test set acc: 0.977400004863739
```

## アーリーストップを加えた後の学習状況
![model1 acc valacc1.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/2658998/66f5b18e-b957-263c-da45-e273f187aa77.png)
![mode1 loss valloss1.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/2658998/76bc91d8-6ea5-87ff-307d-c1b499e15b68.png)
epochs=5くらいですでに評価は頭打ち。


## モデルを改善してみる
```python
model2 = models.Sequential()
model2.add(layers.Dense(1, activation='relu', input_shape=(10000,)))
#model.add(layers.Dense(16, activation='relu'))
#model.add(layers.Dense(1, activation='sigmoid'))

model2.compile(optimizer='Adam', loss = 'binary_crossentropy', metrics=['accuracy'])

#学習の実行
history2 = model2.fit(x=X_train, y=y_train, epochs=20, batch_size=512, 
                    verbose=1, callbacks=callbacks, validation_data=(X_val,y_val))

#テストセットを使ってスコアを見る
score = model2.evaluate(X_test, y_test)
print("Test set loss: {}, Test set acc: {}".format(score[0], score[1]))

Test set loss: 0.20287659764289856, Test set acc: 0.9389600157737732
```

![model2 acc valacc1.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/2658998/e1f30201-8cdd-a9bf-702f-bf6c85a21d81.png)
![model2 loss valloss1.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/2658998/f2bf38da-5ef2-4b55-163d-818f480346d9.png)

過学習は発生していない。
しかし、損失は前のモデルの方が少ない？