DonkeyCar_v2.5.7 学習モデルの入力に対する中間層の出力確認


・DonkeyCarで作成した学習モデルの入力に対する各中間層の出力を確認。
・学習モデルは画像のどこに注目しているのか？
  　-スクリーンが映り込んでいるところは白線が・・・・
　 　→ 自動走行させたとき安定しなかったポイントなので関係あるかもしれない。
　　   (スクリーンの画面が学習時より暗かったなど・・・)

#1.直線:右カーブ手前
![2_img_in.png](https://qiita-image-store.s3.amazonaws.com/0/336848/cc1c8409-950f-7b66-0c91-afb4964f7575.png)
![2_Conv2D_0.png](https://qiita-image-store.s3.amazonaws.com/0/336848/d6428f70-c7c9-a031-916a-7a09d55d0168.png)
![2_Conv2D_1.png](https://qiita-image-store.s3.amazonaws.com/0/336848/10e294a2-11dc-f31c-e23f-0c387733382e.png)
![2_Conv2D_2.png](https://qiita-image-store.s3.amazonaws.com/0/336848/e458b255-9f6a-7226-242c-380fb7282083.png)
![2_Conv2D_3.png](https://qiita-image-store.s3.amazonaws.com/0/336848/3d506d1f-1f7e-92a3-0d16-ac16ddb98da1.png)
![2_Conv2D_4.png](https://qiita-image-store.s3.amazonaws.com/0/336848/01c10d39-f10a-9f3e-dd49-29a735503177.png)

#2.右カーブ：出口
![3_img_in.png](https://qiita-image-store.s3.amazonaws.com/0/336848/876ee85a-1cae-8c61-d7d4-9d706ee835b9.png)
![3_Conv2D_0.png](https://qiita-image-store.s3.amazonaws.com/0/336848/cbfc39d1-5263-67c2-9a86-57282d2e2c92.png)
![3_Conv2D_1.png](https://qiita-image-store.s3.amazonaws.com/0/336848/9c49ef92-8233-6483-46d5-75b90f212a93.png)
![3_Conv2D_2.png](https://qiita-image-store.s3.amazonaws.com/0/336848/a2090f14-b816-cc72-b855-f3212248f918.png)
![3_Conv2D_3.png](https://qiita-image-store.s3.amazonaws.com/0/336848/f52f1bbd-4ab3-9ed5-406f-c87461abcb6e.png)
![3_Conv2D_4.png](https://qiita-image-store.s3.amazonaws.com/0/336848/23bd1192-1051-3e04-a8f9-4ab258969d40.png)

#3.直線
![6_img_in.png](https://qiita-image-store.s3.amazonaws.com/0/336848/8470e5b0-8eaa-fefc-3d75-6a1f72b7ef65.png)
![6_Conv2D_0.png](https://qiita-image-store.s3.amazonaws.com/0/336848/6b3ec5e8-f3a3-0e67-e26d-bf614ffaf556.png)
![6_Conv2D_1.png](https://qiita-image-store.s3.amazonaws.com/0/336848/8caa406e-3a9b-91ea-d05e-df2275633105.png)
![6_Conv2D_2.png](https://qiita-image-store.s3.amazonaws.com/0/336848/7d93efe1-ea24-87a4-0f0d-96fd703781b4.png)
![6_Conv2D_3.png](https://qiita-image-store.s3.amazonaws.com/0/336848/6fd57b17-2951-5189-7404-1d90d078a694.png)
![6_Conv2D_4.png](https://qiita-image-store.s3.amazonaws.com/0/336848/6c79afe8-189b-288e-dc15-c66d906bd974.png)

#4.左カーブ：スクリーンに影響されてる?
![5_img_in.png](https://qiita-image-store.s3.amazonaws.com/0/336848/d718ebac-c5c9-6f58-4285-06c951411553.png)
![5_Conv2D_0.png](https://qiita-image-store.s3.amazonaws.com/0/336848/28258d2e-fbf3-8046-ad6a-ebae35ae4eaf.png)
![5_Conv2D_1.png](https://qiita-image-store.s3.amazonaws.com/0/336848/89cf35bd-efd5-e350-5ec3-677292e18675.png)
![5_Conv2D_2.png](https://qiita-image-store.s3.amazonaws.com/0/336848/5da81840-80c5-459f-07b7-4f4bf46065cf.png)
![5_Conv2D_3.png](https://qiita-image-store.s3.amazonaws.com/0/336848/5062625b-7d05-2967-f6a1-a36f040a86d5.png)
![5_Conv2D_4.png](https://qiita-image-store.s3.amazonaws.com/0/336848/5549cc33-9de2-8a64-2be9-a41053744ef5.png)

#自動走行時の動画：スマホで撮影
・2周目、画面左奥のカーブを曲がれていない。
　スクリーンを正面に捉えてしまい、そのままスクリーン方向に走っていた？
![ezgif.com-video-to-gif (3).gif](https://qiita-image-store.s3.amazonaws.com/0/336848/227e7033-0b63-fc17-48da-f3a9f1951152.gif)

#プログラム
```python

#インポート
import matplotlib.pyplot as plt
import numpy as np
import os

from keras import backend as K
from keras.preprocessing import image
from tensorflow.python.keras.models import Model, load_model
from tensorflow.python.keras.layers import Convolution2D

#モデル読み込み
model_path = 'mypilot'
model = load_model(model_path)

#画像読み込み
img = image.load_img('423_cam-image_array_.jpg', target_size=(120,160))
x = image.img_to_array(img)
x = np.expand_dims(x, axis=0)
x = x/255
plt.imshow(x[0])
str_pred = "angle_out:"+str(model.predict(x)[0])+",   throttle_out:"+str(model.predict(x)[1])
plt.title('img_in:\n' + str_pred)
plt.show()

#中間層の特徴量抽出
layers = model.layers[1:6]
layer_outputs = [layer.output for layer in layers]
activation_model = Model(inputs=model.input, outputs=layer_outputs)
activations = activation_model.predict(x)

#中間層の特徴量可視化
fig1 = plt.figure()
plt.tick_params(labeltop=False, top=False, labelleft=False, left=False, labelbottom=False, bottom=False)
plt.gca().spines['left'].set_visible(False)
plt.gca().spines['right'].set_visible(False)
plt.gca().spines['top'].set_visible(False)
plt.gca().spines['bottom'].set_visible(False)
first_layer_activation = activations[0]
str_shape = str(first_layer_activation.shape)
plt.title('conv2d:'+ str_shape)
print(first_layer_activation.shape)
for i in range(24):
   ax = fig1.add_subplot(8,3,i+1)
   ax.matshow(first_layer_activation[0, :, :, i], cmap='viridis')
   ax.tick_params(labeltop=False, top=False, labelleft=False, left=False, labelbottom=False, bottom=False)
plt.show()

fig1 = plt.figure()
plt.tick_params(labeltop=False, top=False, labelleft=False, left=False, labelbottom=False, bottom=False)
plt.gca().spines['left'].set_visible(False)
plt.gca().spines['right'].set_visible(False)
plt.gca().spines['top'].set_visible(False)
plt.gca().spines['bottom'].set_visible(False)
first_layer_activation = activations[1]
str_shape = str(first_layer_activation.shape)
plt.title('conv2d_1:'+ str_shape)
print(first_layer_activation.shape)
for i in range(32):
   ax = fig1.add_subplot(8,4,i+1)
   ax.matshow(first_layer_activation[0, :, :, i], cmap='viridis')
   ax.tick_params(labeltop=False, top=False, labelleft=False, left=False, labelbottom=False, bottom=False)
plt.show()

fig1 = plt.figure()
plt.tick_params(labeltop=False, top=False, labelleft=False, left=False, labelbottom=False, bottom=False)
plt.gca().spines['left'].set_visible(False)
plt.gca().spines['right'].set_visible(False)
plt.gca().spines['top'].set_visible(False)
plt.gca().spines['bottom'].set_visible(False)
first_layer_activation = activations[2]
str_shape = str(first_layer_activation.shape)
plt.title('conv2d_2:'+ str_shape)
print(first_layer_activation.shape)
for i in range(64):
   ax = fig1.add_subplot(8,8,i+1)
   ax.matshow(first_layer_activation[0, :, :, i], cmap='viridis')
   ax.tick_params(labeltop=False, top=False, labelleft=False, left=False, labelbottom=False, bottom=False)
plt.show()

fig1 = plt.figure()
plt.tick_params(labeltop=False, top=False, labelleft=False, left=False, labelbottom=False, bottom=False)
plt.gca().spines['left'].set_visible(False)
plt.gca().spines['right'].set_visible(False)
plt.gca().spines['top'].set_visible(False)
plt.gca().spines['bottom'].set_visible(False)
first_layer_activation = activations[3]
str_shape = str(first_layer_activation.shape)
plt.title('conv2d_3:'+ str_shape)
print(first_layer_activation.shape)
for i in range(64):
   ax = fig1.add_subplot(8,8,i+1)
   ax.matshow(first_layer_activation[0, :, :, i], cmap='viridis')
   ax.tick_params(labeltop=False, top=False, labelleft=False, left=False, labelbottom=False, bottom=False)
plt.show()

fig1 = plt.figure()
plt.tick_params(labeltop=False, top=False, labelleft=False, left=False, labelbottom=False, bottom=False)
plt.gca().spines['left'].set_visible(False)
plt.gca().spines['right'].set_visible(False)
plt.gca().spines['top'].set_visible(False)
plt.gca().spines['bottom'].set_visible(False)
first_layer_activation = activations[4]
str_shape = str(first_layer_activation.shape)
plt.title('conv2d_4:'+ str_shape)
print(first_layer_activation.shape)
for i in range(64):
   ax = fig1.add_subplot(8,8,i+1)
   ax.matshow(first_layer_activation[0, :, :, i], cmap='viridis')
   ax.tick_params(labeltop=False, top=False, labelleft=False, left=False, labelbottom=False, bottom=False)
plt.show()

```

#参考図書
・RとKerasによるディープラーニング
  https://www.oreilly.co.jp/books/9784873118574/
　テンソルの構造やDnkeycarのソースコードを読み解いたり、CNNの可視化方法を考えるのに参考になった。