FSSDを動かして遊んでみた♬

SSDの拡張版として、DSSDなどがあるが、今回は最近 @tanakataiki　さんにより 
2018年05月02日に実装公開された[FeatureFused-SSD～ssdkerasのその先3～](https://qiita.com/tanakataiki/items/36e71e7d2f5705bd98bb)を動かしてみた。
さらに、VGG16のネットワーク構造への組み込み等は参考になるので、ちょっと覗いてみた。
【文献】
①[Feature-Fused SSD: Fast Detection for Small Objects](https://arxiv.org/ftp/arxiv/papers/1709/1709.05054.pdf) 
Guimei Cao, Xuemei Xie, Wenzhe Yang, Quan Liao, Guangming Shi, Jinjian Wu School of Electronic Engineering, Xidian University, China 
②[FSSD:FeatureFusionSingleShotMultiboxDetector](https://arxiv.org/pdf/1712.00960.pdf)
Zuo-Xin Li Fu-Qiang Zhou Key Laboratory of Precision Opto-mechatronics Technology, Ministry of Education, Beihang University, Beijing 100191, China
③[DSSD:DeconvolutionalSingleShotDetector](https://arxiv.org/pdf/1701.06659.pdf)
Cheng-Yang Fu1∗, Wei Liu1∗, Ananth Ranga2, Ambrish Tyagi2, Alexander C. Berg1 1UNC Chapel Hill, 2Amazon Inc. 

###文献よりFSSDモデルとは
最初にSSDありき。。この発展形がDSSD、そしてさらなる構造変態がFSSDってことで、元祖から見ていくこととする。

ということで、文献③よりSSDとDSSDの構造を見ると以下のとおり
![DSSD_model.jpg](https://qiita-image-store.s3.amazonaws.com/0/233744/3db79231-1120-b80a-76b9-e928a1b261a0.jpeg)
肝心のDetection部分のDSSDの構造は以下のとおり
![DSSD_model2.jpg](https://qiita-image-store.s3.amazonaws.com/0/233744/e808c0ef-5e68-4b10-bae4-98c1b78f3afc.jpeg)
そして、文献①より、FSSDの構造は以下のとおり、上記のSSDやDSSDと比較すると変更部分がわかりやすい
![ffssd_model.jpg](https://qiita-image-store.s3.amazonaws.com/0/233744/566ffd13-22bf-466e-0672-57d8b045d996.jpeg)
Detection部分で使われているSSDとFSSDの構造の変更部分がわかりやすい
![SSDvsFSSD.jpg](https://qiita-image-store.s3.amazonaws.com/0/233744/0f02d3c5-93ab-92ad-2528-645452784cc4.jpeg)
そして、文献①はFSSDとして、以下二通りのDetection部分の変態構造を提案している。
![FSSD_2model.jpg](https://qiita-image-store.s3.amazonaws.com/0/233744/1e3e7149-751d-3e68-46b1-45ddc9755b66.jpeg)


###文献より性能比較
以下を見ると、性能的には、文献①より以下の表が示されている。
この表から、オリジナルなSSD300と比較するとDSSDはFPSが著しく悪化しているが、FSSDはどちらのモデルもあまりFPSが下がらずｍAPがアップするのがわかる。
![ffssd_performance.jpg](https://qiita-image-store.s3.amazonaws.com/0/233744/fac7c260-ed41-10f6-5426-a140a45d8d21.jpeg)
一方、文献②ではｍAP　vs　FPSのグラフが示されている。
この図は上記の表と若干数字が異なるように見えるが、それでもなんとなくSSD300(77.2,50)とFSSD(78.8,40)の位置関係は同じようだ。
![ffssd_mAPvsfps.jpg](https://qiita-image-store.s3.amazonaws.com/0/233744/3b21aad4-8f81-663d-9b06-e8b71a372112.jpeg)
※この図が正しいとするとYolov2（73.7,80↑）はいいのかも？？
※そして、SSD300とYolov2の間にもう一つモデルあってもいいが、Yolv3はどこに来るのか？
※Yoloについては再度挑戦してみよう
#環境
これは、model比較してみるとわかるが少なくともconv5-3まではSSDと同じである。
つまり、ssdkerasの環境で動くということだ。
必要だとすればその後のFusion層を学習する必要があるが、実はFusion層以外もその後の構造は同じ部分があり、SSDで学習したWeightsが使える。
そして、Prior_Boxなども同じ構成で動くことが分かる。
つまり、[前回までの記事](https://qiita.com/MuAuan/items/44d7babf1da92f9e822d)で利用していた、環境で本来うまく物体検出までできるはずである。

さらに、重要なことであるが、モデルサマリーを比較すると検出部分は以下のとおり、ほとんど変わらない。
以下だと、```conv4_3_norm_mbox_conf_flat[0][0]```が```fusion_conv4_5_mbox_conf_flat[0][0]```に置き換えたものになっている。だから、SSDのパラメータをそのままでも検出はある程度できる可能性がある。

```
SSD300の検出部一部
_____________________________________________________________________________________________
mbox_conf (Concatenate)         (None, 153468)       0           conv4_3_norm_mbox_conf_flat[0][0]
                                                                 fc7_mbox_conf_flat[0][0]
                                                                 conv6_2_mbox_conf_flat[0][0]
                                                                 conv7_2_mbox_conf_flat[0][0]
                                                                 conv8_2_mbox_conf_flat[0][0]
                                                                 pool6_mbox_conf_flat[0][0]
_____________________________________________________________________________________________
```

```
FSSDの検出部一部
_____________________________________________________________________________________________
mbox_conf (Concatenate)         (None, 153468)       0           fusion_conv4_5_mbox_conf_flat[0][0]
                                                                 fc7_mbox_conf_flat[0][0]
                                                                 conv6_2_mbox_conf_flat[0][0]
                                                                 conv7_2_mbox_conf_flat[0][0]
                                                                 conv8_2_mbox_conf_flat[0][0]
                                                                 pool6_mbox_conf_flat[0][0]
_____________________________________________________________________________________________
```
###結果
結果に際してもっとも重要なことは、Weightsである。
ここでは、以下の三種類を利用した
①[FeatureFused-SSD～ssdkerasのその先3～](https://qiita.com/tanakataiki/items/36e71e7d2f5705bd98bb)で提供されている「FeatureFusedSSD300weights_voc_2007.hdf5」
②[FeatureFused-SSD～ssdkerasのその先3～](https://qiita.com/tanakataiki/items/36e71e7d2f5705bd98bb)で提供されている「VGG16SSD512weights_voc_2007_class20.hdf5」
③SSD300で提供されている「weights_SSD300.hdf5」
####「SSDモデル＋weights_SSD300.hdf5」の結果
以下が比較ベースラインとなる。
![SSD2_recognized_picture002__004.png](https://qiita-image-store.s3.amazonaws.com/0/233744/f0009b21-626e-6ba1-9e66-c4826e768e32.png)
![SSD2_recognized_picture001__005.png](https://qiita-image-store.s3.amazonaws.com/0/233744/7b00786b-78bb-7fcd-d0c6-97102454c94a.png)

####「FSSDモデル＋FeatureFusedSSD300weights_voc_2007.hdf5」の結果
![recognized_picture002__FeatureFusedSSD300weights_voc_2007.png](https://qiita-image-store.s3.amazonaws.com/0/233744/ffd55978-fbfd-9c5d-055c-085c50c0960d.png)
![recognized_picture001__FeatureFusedSSD300weights_voc_2007.png](https://qiita-image-store.s3.amazonaws.com/0/233744/c34553cf-1f86-e127-7ee7-5fcaa824c055.png)


####「FSSDモデル＋weights_SSD300.hdf5」の結果
![FSSD_FeatureFusedSSD300weights_voc_2007recognized_picture002__004.png](https://qiita-image-store.s3.amazonaws.com/0/233744/f2deebfe-f894-6f50-4d7b-72f3cf389c09.png)
![FSSD_FeatureFusedSSD300weights_voc_2007recognizedrecognized_picture001__005.png](https://qiita-image-store.s3.amazonaws.com/0/233744/137824b6-29d5-942e-ab68-1d0784adc97f.png)


####「ssd512VGG16＋VGG16SSD512weights_voc_2007_class20.hdf5」の結果
pretrained VGG16を入力としたssd512VGG16の結果である。
![recognized_picture002__003.png](https://qiita-image-store.s3.amazonaws.com/0/233744/e14fa269-a499-e2d0-6397-b10436d3cb47.png)
![recognized_picture001__004.png](https://qiita-image-store.s3.amazonaws.com/0/233744/adabbb1c-697d-5a69-b935-e97d49f2ca4f.png)
実はOriginalはVGG16の
vgg16_input_shape = (224, 224, 3)
としているが、もともとVGG16も入力サイズフリーなので、以下のように書き換えると
vgg16_input_shape = (input_shape[1], input_shape[0], 3)
![VGG16changeinput_recognized_picture002__005.png](https://qiita-image-store.s3.amazonaws.com/0/233744/a800cd23-50c3-a34f-69df-34071bbe7439.png)
![VGG16changeinput_recognized_picture001__006.png](https://qiita-image-store.s3.amazonaws.com/0/233744/1516eac6-4a39-bfef-c8af-8b25245c570c.png)

###VGG16FineTuning版SSD300を作ってみた
上記の結果がちょっと気に入らないので、オリジナルはSSD512だが、SSD300用に書き換えてweights_SSD300.hdf5を使ってやってみた。
コードは以下のとおり
[SSD / FeatureFused-SSD / ssd300VGG16_alt.py ](https://github.com/MuAuan/SSD/blob/master/FeatureFused-SSD/ssd300VGG16_alt.py)
ということで、結果は以下のとおりです。
![finetuningVGG16_SSDrecognized_picture002__004.png](https://qiita-image-store.s3.amazonaws.com/0/233744/35c19fa7-398b-5d01-fba3-c40781570158.png)
![finetuningVGG16_recognized_picture001__005.png](https://qiita-image-store.s3.amazonaws.com/0/233744/023f46a6-6f3a-cf6c-f14c-89468eca0d93.png)
当然？だけど、上記のSSD300＋weights_SSD300.hdf5と同じくなった。
※ちょっと違うというか、精度良くなっているかも。。。
また、一番下にモデルサマリーを置いてあるが、FineTuningだけあって、VGG16部分のmodel出力は無い。

###まとめ
・FSSDモデルを概観して、動かしてみた
・Weightsを各種変更して傾向を見ると、やはりWeights次第な結果となった
・PretrainedVGG16を利用したSSD300を作成した。
　性能はPretrainedなだけあって若干良さそう。

実は動画物体検出もやったが、次回にしたいと思います。

###コードは以下においた
[SSD / FeatureFused-SSD /](https://github.com/MuAuan/SSD/tree/master/FeatureFused-SSD)

###モデルサマリー

```
model.summary() SSD300
_____________________________________________________________________________________________
Layer (type)                    Output Shape         Param #     Connected to
=============================================================================================
...
_____________________________________________________________________________________________
conv6_1 (Conv2D)                (None, 19, 19, 256)  262400      fc7[0][0]
_____________________________________________________________________________________________
conv6_2 (Conv2D)                (None, 10, 10, 512)  1180160     conv6_1[0][0]
_____________________________________________________________________________________________
conv7_1 (Conv2D)                (None, 10, 10, 128)  65664       conv6_2[0][0]
_____________________________________________________________________________________________
conv7_1z (ZeroPadding2D)        (None, 12, 12, 128)  0           conv7_1[0][0]
_____________________________________________________________________________________________
conv7_2 (Conv2D)                (None, 5, 5, 256)    295168      conv7_1z[0][0]
_____________________________________________________________________________________________
conv8_1 (Conv2D)                (None, 5, 5, 128)    32896       conv7_2[0][0]
_____________________________________________________________________________________________
conv4_3_norm (Normalize)        (None, 38, 38, 512)  512         conv4_3[0][0]
_____________________________________________________________________________________________
conv8_2 (Conv2D)                (None, 3, 3, 256)    295168      conv8_1[0][0]
_____________________________________________________________________________________________
pool6 (GlobalAveragePooling2D)  (None, 256)          0           conv8_2[0][0]
_____________________________________________________________________________________________
conv4_3_norm_mbox_conf (Conv2D) (None, 38, 38, 63)   290367      conv4_3_norm[0][0]
_____________________________________________________________________________________________
fc7_mbox_conf (Conv2D)          (None, 19, 19, 126)  1161342     fc7[0][0]
_____________________________________________________________________________________________
conv6_2_mbox_conf (Conv2D)      (None, 10, 10, 126)  580734      conv6_2[0][0]
_____________________________________________________________________________________________
conv7_2_mbox_conf (Conv2D)      (None, 5, 5, 126)    290430      conv7_2[0][0]
_____________________________________________________________________________________________
conv8_2_mbox_conf (Conv2D)      (None, 3, 3, 126)    290430      conv8_2[0][0]
_____________________________________________________________________________________________
conv4_3_norm_mbox_loc (Conv2D)  (None, 38, 38, 12)   55308       conv4_3_norm[0][0]
_____________________________________________________________________________________________
fc7_mbox_loc (Conv2D)           (None, 19, 19, 24)   221208      fc7[0][0]
_____________________________________________________________________________________________
conv6_2_mbox_loc (Conv2D)       (None, 10, 10, 24)   110616      conv6_2[0][0]
_____________________________________________________________________________________________
conv7_2_mbox_loc (Conv2D)       (None, 5, 5, 24)     55320       conv7_2[0][0]
_____________________________________________________________________________________________
conv8_2_mbox_loc (Conv2D)       (None, 3, 3, 24)     55320       conv8_2[0][0]
_____________________________________________________________________________________________
conv4_3_norm_mbox_conf_flat (Fl (None, 90972)        0           conv4_3_norm_mbox_conf[0][0]
_____________________________________________________________________________________________
fc7_mbox_conf_flat (Flatten)    (None, 45486)        0           fc7_mbox_conf[0][0]
_____________________________________________________________________________________________
conv6_2_mbox_conf_flat (Flatten (None, 12600)        0           conv6_2_mbox_conf[0][0]
_____________________________________________________________________________________________
conv7_2_mbox_conf_flat (Flatten (None, 3150)         0           conv7_2_mbox_conf[0][0]
_____________________________________________________________________________________________
conv8_2_mbox_conf_flat (Flatten (None, 1134)         0           conv8_2_mbox_conf[0][0]
_____________________________________________________________________________________________
pool6_mbox_conf_flat (Dense)    (None, 126)          32382       pool6[0][0]
_____________________________________________________________________________________________
conv4_3_norm_mbox_loc_flat (Fla (None, 17328)        0           conv4_3_norm_mbox_loc[0][0]
_____________________________________________________________________________________________
fc7_mbox_loc_flat (Flatten)     (None, 8664)         0           fc7_mbox_loc[0][0]
_____________________________________________________________________________________________
conv6_2_mbox_loc_flat (Flatten) (None, 2400)         0           conv6_2_mbox_loc[0][0]
_____________________________________________________________________________________________
conv7_2_mbox_loc_flat (Flatten) (None, 600)          0           conv7_2_mbox_loc[0][0]
_____________________________________________________________________________________________
conv8_2_mbox_loc_flat (Flatten) (None, 216)          0           conv8_2_mbox_loc[0][0]
_____________________________________________________________________________________________
pool6_mbox_loc_flat (Dense)     (None, 24)           6168        pool6[0][0]
_____________________________________________________________________________________________
mbox_conf (Concatenate)         (None, 153468)       0           conv4_3_norm_mbox_conf_flat[0][0]
                                                                 fc7_mbox_conf_flat[0][0]
                                                                 conv6_2_mbox_conf_flat[0][0]
                                                                 conv7_2_mbox_conf_flat[0][0]
                                                                 conv8_2_mbox_conf_flat[0][0]
                                                                 pool6_mbox_conf_flat[0][0]
_____________________________________________________________________________________________
pool6_reshaped (Reshape)        (None, 1, 1, 256)    0           pool6[0][0]
_____________________________________________________________________________________________
mbox_loc (Concatenate)          (None, 29232)        0           conv4_3_norm_mbox_loc_flat[0][0]
                                                                 fc7_mbox_loc_flat[0][0]
                                                                 conv6_2_mbox_loc_flat[0][0]
                                                                 conv7_2_mbox_loc_flat[0][0]
                                                                 conv8_2_mbox_loc_flat[0][0]
                                                                 pool6_mbox_loc_flat[0][0]
_____________________________________________________________________________________________
mbox_conf_logits (Reshape)      (None, 7308, 21)     0           mbox_conf[0][0]
_____________________________________________________________________________________________
conv4_3_norm_mbox_priorbox (Pri (None, 4332, 8)      0           conv4_3_norm[0][0]
_____________________________________________________________________________________________
fc7_mbox_priorbox (PriorBox)    (None, 2166, 8)      0           fc7[0][0]
_____________________________________________________________________________________________
conv6_2_mbox_priorbox (PriorBox (None, 600, 8)       0           conv6_2[0][0]
_____________________________________________________________________________________________
conv7_2_mbox_priorbox (PriorBox (None, 150, 8)       0           conv7_2[0][0]
_____________________________________________________________________________________________
conv8_2_mbox_priorbox (PriorBox (None, 54, 8)        0           conv8_2[0][0]
_____________________________________________________________________________________________
pool6_mbox_priorbox (PriorBox)  (None, 6, 8)         0           pool6_reshaped[0][0]
_____________________________________________________________________________________________
mbox_loc_final (Reshape)        (None, 7308, 4)      0           mbox_loc[0][0]
_____________________________________________________________________________________________
mbox_conf_final (Activation)    (None, 7308, 21)     0           mbox_conf_logits[0][0]
_____________________________________________________________________________________________
mbox_priorbox (Concatenate)     (None, 7308, 8)      0           conv4_3_norm_mbox_priorbox[0][0]
                                                                 fc7_mbox_priorbox[0][0]
                                                                 conv6_2_mbox_priorbox[0][0]
                                                                 conv7_2_mbox_priorbox[0][0]
                                                                 conv8_2_mbox_priorbox[0][0]
                                                                 pool6_mbox_priorbox[0][0]
_____________________________________________________________________________________________
predictions (Concatenate)       (None, 7308, 33)     0           mbox_loc_final[0][0]
                                                                 mbox_conf_final[0][0]
                                                                 mbox_priorbox[0][0]

=============================================================================================
Total params: 25,765,497
Trainable params: 25,765,497
Non-trainable params: 0
```

```
model.summary() ssd300FeatureFused
_____________________________________________________________________________________________
Layer (type)                    Output Shape         Param #     Connected to
=============================================================================================
...
_____________________________________________________________________________________________
conv6_1 (Conv2D)                (None, 19, 19, 256)  262400      fc7[0][0]
_____________________________________________________________________________________________
conv6_2 (Conv2D)                (None, 10, 10, 512)  1180160     conv6_1[0][0]
_____________________________________________________________________________________________
*deconv5_3 (Conv2DTranspose)     (None, 38, 38, 512)  2359808     conv5_3[0][0]*
_____________________________________________________________________________________________
conv7_1 (Conv2D)                (None, 10, 10, 128)  65664       conv6_2[0][0]
_____________________________________________________________________________________________
*conv4 (Conv2D)                  (None, 38, 38, 512)  2359808     conv4_3[0][0]*
_____________________________________________________________________________________________
*conv_deconv5_3 (Conv2D)         (None, 38, 38, 512)  2359808     deconv5_3[0][0]*
_____________________________________________________________________________________________
zero_padding2d_1 (ZeroPadding2D (None, 12, 12, 128)  0           conv7_1[0][0]
_____________________________________________________________________________________________
conv4_3_norm (Normalize)        (None, 38, 38, 512)  512         conv4[0][0]
_____________________________________________________________________________________________
*conv_5_3_norm (Normalize)       (None, 38, 38, 512)  512         conv_deconv5_3[0][0]*
_____________________________________________________________________________________________
conv7_2 (Conv2D)                (None, 5, 5, 256)    295168      zero_padding2d_1[0][0]
_____________________________________________________________________________________________
*add_1 (Add)                     (None, 38, 38, 512)  0           conv4_3_norm[0][0]
                                                                 conv_5_3_norm[0][0]*
_____________________________________________________________________________________________
conv8_1 (Conv2D)                (None, 5, 5, 128)    32896       conv7_2[0][0]
_____________________________________________________________________________________________
*fusion_conv4_5 (Activation)     (None, 38, 38, 512)  0           add_1[0][0]*
_____________________________________________________________________________________________
conv8_2 (Conv2D)                (None, 3, 3, 256)    295168      conv8_1[0][0]
_____________________________________________________________________________________________
pool6 (GlobalAveragePooling2D)  (None, 256)          0           conv8_2[0][0]
_____________________________________________________________________________________________
*fusion_conv4_5_mbox_conf (Conv2 (None, 38, 38, 63)   290367      fusion_conv4_5[0][0]*
_____________________________________________________________________________________________
fc7_mbox_conf (Conv2D)          (None, 19, 19, 126)  1161342     fc7[0][0]
_____________________________________________________________________________________________
conv6_2_mbox_conf (Conv2D)      (None, 10, 10, 126)  580734      conv6_2[0][0]
_____________________________________________________________________________________________
conv7_2_mbox_conf (Conv2D)      (None, 5, 5, 126)    290430      conv7_2[0][0]
_____________________________________________________________________________________________
conv8_2_mbox_conf (Conv2D)      (None, 3, 3, 126)    290430      conv8_2[0][0]
_____________________________________________________________________________________________
*fusion_conv4_5_mbox_loc (Conv2D (None, 38, 38, 12)   55308       fusion_conv4_5[0][0]*
_____________________________________________________________________________________________
fc7_mbox_loc (Conv2D)           (None, 19, 19, 24)   221208      fc7[0][0]
_____________________________________________________________________________________________
conv6_2_mbox_loc (Conv2D)       (None, 10, 10, 24)   110616      conv6_2[0][0]
_____________________________________________________________________________________________
conv7_2_mbox_loc (Conv2D)       (None, 5, 5, 24)     55320       conv7_2[0][0]
_____________________________________________________________________________________________
conv8_2_mbox_loc (Conv2D)       (None, 3, 3, 24)     55320       conv8_2[0][0]
_____________________________________________________________________________________________
*fusion_conv4_5_mbox_conf_flat ( (None, 90972)        0           fusion_conv4_5_mbox_conf[0][0]*
_____________________________________________________________________________________________
fc7_mbox_conf_flat (Flatten)    (None, 45486)        0           fc7_mbox_conf[0][0]
_____________________________________________________________________________________________
conv6_2_mbox_conf_flat (Flatten (None, 12600)        0           conv6_2_mbox_conf[0][0]
_____________________________________________________________________________________________
conv7_2_mbox_conf_flat (Flatten (None, 3150)         0           conv7_2_mbox_conf[0][0]
_____________________________________________________________________________________________
conv8_2_mbox_conf_flat (Flatten (None, 1134)         0           conv8_2_mbox_conf[0][0]
_____________________________________________________________________________________________
pool6_mbox_conf_flat (Dense)    (None, 126)          32382       pool6[0][0]
_____________________________________________________________________________________________
*fusion_conv4_5_mbox_loc_flat (F (None, 17328)        0           fusion_conv4_5_mbox_loc[0][0]*
_____________________________________________________________________________________________
fc7_mbox_loc_flat (Flatten)     (None, 8664)         0           fc7_mbox_loc[0][0]
_____________________________________________________________________________________________
conv6_2_mbox_loc_flat (Flatten) (None, 2400)         0           conv6_2_mbox_loc[0][0]
_____________________________________________________________________________________________
conv7_2_mbox_loc_flat (Flatten) (None, 600)          0           conv7_2_mbox_loc[0][0]
_____________________________________________________________________________________________
conv8_2_mbox_loc_flat (Flatten) (None, 216)          0           conv8_2_mbox_loc[0][0]
_____________________________________________________________________________________________
pool6_mbox_loc_flat (Dense)     (None, 24)           6168        pool6[0][0]
_____________________________________________________________________________________________
mbox_conf (Concatenate)         (None, 153468)       0          *fusion_conv4_5_mbox_conf_flat[0][0]*
                                                                 fc7_mbox_conf_flat[0][0]
                                                                 conv6_2_mbox_conf_flat[0][0]
                                                                 conv7_2_mbox_conf_flat[0][0]
                                                                 conv8_2_mbox_conf_flat[0][0]
                                                                 pool6_mbox_conf_flat[0][0]
_____________________________________________________________________________________________
pool6_reshaped (Reshape)        (None, 1, 1, 256)    0           pool6[0][0]
_____________________________________________________________________________________________
mbox_loc (Concatenate)          (None, 29232)        0          *fusion_conv4_5_mbox_loc_flat[0][0]*
                                                                 fc7_mbox_loc_flat[0][0]
                                                                 conv6_2_mbox_loc_flat[0][0]
                                                                 conv7_2_mbox_loc_flat[0][0]
                                                                 conv8_2_mbox_loc_flat[0][0]
                                                                 pool6_mbox_loc_flat[0][0]
_____________________________________________________________________________________________
mbox_conf_logits (Reshape)      (None, 7308, 21)     0           mbox_conf[0][0]
_____________________________________________________________________________________________
*fusion_conv4_5_mbox_priorbox (P (None, 4332, 8)      0           fusion_conv4_5[0][0]*
_____________________________________________________________________________________________
fc7_mbox_priorbox (PriorBox)    (None, 2166, 8)      0           fc7[0][0]
_____________________________________________________________________________________________
conv6_2_mbox_priorbox (PriorBox (None, 600, 8)       0           conv6_2[0][0]
_____________________________________________________________________________________________
conv7_2_mbox_priorbox (PriorBox (None, 150, 8)       0           conv7_2[0][0]
_____________________________________________________________________________________________
conv8_2_mbox_priorbox (PriorBox (None, 54, 8)        0           conv8_2[0][0]
_____________________________________________________________________________________________
pool6_mbox_priorbox (PriorBox)  (None, 6, 8)         0           pool6_reshaped[0][0]
_____________________________________________________________________________________________
mbox_loc_final (Reshape)        (None, 7308, 4)      0           mbox_loc[0][0]
_____________________________________________________________________________________________
mbox_conf_final (Activation)    (None, 7308, 21)     0           mbox_conf_logits[0][0]
_____________________________________________________________________________________________
mbox_priorbox (Concatenate)     (None, 7308, 8)      0           *fusion_conv4_5_mbox_priorbox[0][0*
                                                                 fc7_mbox_priorbox[0][0]
                                                                 conv6_2_mbox_priorbox[0][0]
                                                                 conv7_2_mbox_priorbox[0][0]
                                                                 conv8_2_mbox_priorbox[0][0]
                                                                 pool6_mbox_priorbox[0][0]
_____________________________________________________________________________________________
predictions (Concatenate)       (None, 7308, 33)     0           mbox_loc_final[0][0]
                                                                 mbox_conf_final[0][0]
                                                                 mbox_priorbox[0][0]
=============================================================================================
Total params: 32,845,433
Trainable params: 32,845,433
Non-trainable params: 0
```

```
input:VGG16_model
Layer (type)                    Output Shape         Param #     Connected to
=============================================================================================
input_2 (InputLayer)            (None, 300, 300, 3)  0
_____________________________________________________________________________________________
model_1 (Model)                 multiple             1735488     input_2[0][0]
_____________________________________________________________________________________________
conv4_1 (Conv2D)                (None, 37, 37, 512)  1180160     model_1[1][0]
_____________________________________________________________________________________________
conv4_2 (Conv2D)                (None, 37, 37, 512)  2359808     conv4_1[0][0]
_____________________________________________________________________________________________
conv4_3 (Conv2D)                (None, 37, 37, 512)  2359808     conv4_2[0][0]
_____________________________________________________________________________________________
pool4 (MaxPooling2D)            (None, 19, 19, 512)  0           conv4_3[0][0]
_____________________________________________________________________________________________
conv5_1 (Conv2D)                (None, 19, 19, 512)  2359808     pool4[0][0]
_____________________________________________________________________________________________
conv5_2 (Conv2D)                (None, 19, 19, 512)  2359808     conv5_1[0][0]
_____________________________________________________________________________________________
conv5_3 (Conv2D)                (None, 19, 19, 512)  2359808     conv5_2[0][0]
_____________________________________________________________________________________________
pool5 (MaxPooling2D)            (None, 19, 19, 512)  0           conv5_3[0][0]
_____________________________________________________________________________________________
fc6 (Conv2D)                    (None, 19, 19, 1024) 4719616     pool5[0][0]
_____________________________________________________________________________________________
drop6 (Dropout)                 (None, 19, 19, 1024) 0           fc6[0][0]
_____________________________________________________________________________________________
fc7 (Conv2D)                    (None, 19, 19, 1024) 1049600     drop6[0][0]
_____________________________________________________________________________________________
conv6_1 (Conv2D)                (None, 19, 19, 256)  262400      fc7[0][0]
_____________________________________________________________________________________________
conv6_2 (Conv2D)                (None, 10, 10, 512)  1180160     conv6_1[0][0]
_____________________________________________________________________________________________
conv7_1 (Conv2D)                (None, 10, 10, 128)  65664       conv6_2[0][0]
_____________________________________________________________________________________________
zero_padding2d_1 (ZeroPadding2D (None, 12, 12, 128)  0           conv7_1[0][0]
_____________________________________________________________________________________________
conv7_2 (Conv2D)                (None, 5, 5, 256)    295168      zero_padding2d_1[0][0]
_____________________________________________________________________________________________
conv8_1 (Conv2D)                (None, 5, 5, 128)    32896       conv7_2[0][0]
_____________________________________________________________________________________________
conv8_2 (Conv2D)                (None, 3, 3, 256)    295168      conv8_1[0][0]
_____________________________________________________________________________________________
conv9_1 (Conv2D)                (None, 3, 3, 128)    32896       conv8_2[0][0]
_____________________________________________________________________________________________
conv9_2 (Conv2D)                (None, 2, 2, 256)    295168      conv9_1[0][0]
_____________________________________________________________________________________________
conv10_1 (Conv2D)               (None, 2, 2, 128)    32896       conv9_2[0][0]
_____________________________________________________________________________________________
conv4_3_norm (Normalize)        (None, 37, 37, 512)  512         conv4_3[0][0]
_____________________________________________________________________________________________
conv10_2 (Conv2D)               (None, 1, 1, 256)    295168      conv10_1[0][0]
_____________________________________________________________________________________________
pool6 (GlobalAveragePooling2D)  (None, 256)          0           conv10_2[0][0]
_____________________________________________________________________________________________
conv4_3_norm_mbox_conf (Conv2D) (None, 37, 37, 84)   387156      conv4_3_norm[0][0]
_____________________________________________________________________________________________
fc7_mbox_conf (Conv2D)          (None, 19, 19, 126)  1161342     fc7[0][0]
_____________________________________________________________________________________________
conv6_2_mbox_conf (Conv2D)      (None, 10, 10, 126)  580734      conv6_2[0][0]
_____________________________________________________________________________________________
conv7_2_mbox_conf (Conv2D)      (None, 5, 5, 126)    290430      conv7_2[0][0]
_____________________________________________________________________________________________
conv8_2_mbox_conf (Conv2D)      (None, 3, 3, 126)    290430      conv8_2[0][0]
_____________________________________________________________________________________________
conv9_2_mbox_conf (Conv2D)      (None, 2, 2, 84)     193620      conv9_2[0][0]
_____________________________________________________________________________________________
conv4_3_norm_mbox_loc (Conv2D)  (None, 37, 37, 16)   73744       conv4_3_norm[0][0]
_____________________________________________________________________________________________
fc7_mbox_loc (Conv2D)           (None, 19, 19, 24)   221208      fc7[0][0]
_____________________________________________________________________________________________
conv6_2_mbox_loc (Conv2D)       (None, 10, 10, 24)   110616      conv6_2[0][0]
_____________________________________________________________________________________________
conv7_2_mbox_loc (Conv2D)       (None, 5, 5, 24)     55320       conv7_2[0][0]
_____________________________________________________________________________________________
conv8_2_mbox_loc (Conv2D)       (None, 3, 3, 24)     55320       conv8_2[0][0]
_____________________________________________________________________________________________
conv9_2_mbox_loc (Conv2D)       (None, 2, 2, 16)     36880       conv9_2[0][0]
_____________________________________________________________________________________________
conv4_3_norm_mbox_conf_flat (Fl (None, 114996)       0           conv4_3_norm_mbox_conf[0][0]
_____________________________________________________________________________________________
fc7_mbox_conf_flat (Flatten)    (None, 45486)        0           fc7_mbox_conf[0][0]
_____________________________________________________________________________________________
conv6_2_mbox_conf_flat (Flatten (None, 12600)        0           conv6_2_mbox_conf[0][0]
_____________________________________________________________________________________________
conv7_2_mbox_conf_flat (Flatten (None, 3150)         0           conv7_2_mbox_conf[0][0]
_____________________________________________________________________________________________
conv8_2_mbox_conf_flat (Flatten (None, 1134)         0           conv8_2_mbox_conf[0][0]
_____________________________________________________________________________________________
conv9_2_mbox_conf_flat (Flatten (None, 336)          0           conv9_2_mbox_conf[0][0]
_____________________________________________________________________________________________
pool6_mbox_conf_flat (Dense)    (None, 84)           21588       pool6[0][0]
_____________________________________________________________________________________________
conv4_3_norm_mbox_loc_flat (Fla (None, 21904)        0           conv4_3_norm_mbox_loc[0][0]
_____________________________________________________________________________________________
fc7_mbox_loc_flat (Flatten)     (None, 8664)         0           fc7_mbox_loc[0][0]
_____________________________________________________________________________________________
conv6_2_mbox_loc_flat (Flatten) (None, 2400)         0           conv6_2_mbox_loc[0][0]
_____________________________________________________________________________________________
conv7_2_mbox_loc_flat (Flatten) (None, 600)          0           conv7_2_mbox_loc[0][0]
_____________________________________________________________________________________________
conv8_2_mbox_loc_flat (Flatten) (None, 216)          0           conv8_2_mbox_loc[0][0]
_____________________________________________________________________________________________
conv9_2_mbox_loc_flat (Flatten) (None, 64)           0           conv9_2_mbox_loc[0][0]
_____________________________________________________________________________________________
pool6_mbox_loc_flat (Dense)     (None, 16)           4112        pool6[0][0]
_____________________________________________________________________________________________
mbox_conf (Concatenate)         (None, 177786)       0           conv4_3_norm_mbox_conf_flat[0][0]
                                                                 fc7_mbox_conf_flat[0][0]
                                                                 conv6_2_mbox_conf_flat[0][0]
                                                                 conv7_2_mbox_conf_flat[0][0]
                                                                 conv8_2_mbox_conf_flat[0][0]
                                                                 conv9_2_mbox_conf_flat[0][0]
                                                                 pool6_mbox_conf_flat[0][0]
_____________________________________________________________________________________________
pool6_reshaped (Reshape)        (None, 1, 1, 256)    0           pool6[0][0]
_____________________________________________________________________________________________
mbox_loc (Concatenate)          (None, 33864)        0           conv4_3_norm_mbox_loc_flat[0][0]
                                                                 fc7_mbox_loc_flat[0][0]
                                                                 conv6_2_mbox_loc_flat[0][0]
                                                                 conv7_2_mbox_loc_flat[0][0]
                                                                 conv8_2_mbox_loc_flat[0][0]
                                                                 conv9_2_mbox_loc_flat[0][0]
                                                                 pool6_mbox_loc_flat[0][0]
_____________________________________________________________________________________________
mbox_conf_logits (Reshape)      (None, 8466, 21)     0           mbox_conf[0][0]
_____________________________________________________________________________________________
conv4_3_norm_mbox_priorbox (Pri (None, 5476, 8)      0           conv4_3_norm[0][0]
_____________________________________________________________________________________________
fc7_mbox_priorbox (PriorBox)    (None, 2166, 8)      0           fc7[0][0]
_____________________________________________________________________________________________
conv6_2_mbox_priorbox (PriorBox (None, 600, 8)       0           conv6_2[0][0]
_____________________________________________________________________________________________
conv7_2_mbox_priorbox (PriorBox (None, 150, 8)       0           conv7_2[0][0]
_____________________________________________________________________________________________
conv8_2_mbox_priorbox (PriorBox (None, 54, 8)        0           conv8_2[0][0]
_____________________________________________________________________________________________
conv9_2_mbox_priorbox (PriorBox (None, 16, 8)        0           conv9_2[0][0]
_____________________________________________________________________________________________
pool6_mbox_priorbox (PriorBox)  (None, 4, 8)         0           pool6_reshaped[0][0]
_____________________________________________________________________________________________
mbox_loc_final (Reshape)        (None, 8466, 4)      0           mbox_loc[0][0]
_____________________________________________________________________________________________
mbox_conf_final (Activation)    (None, 8466, 21)     0           mbox_conf_logits[0][0]
_____________________________________________________________________________________________
mbox_priorbox (Concatenate)     (None, 8466, 8)      0           conv4_3_norm_mbox_priorbox[0][0]
                                                                 fc7_mbox_priorbox[0][0]
                                                                 conv6_2_mbox_priorbox[0][0]
                                                                 conv7_2_mbox_priorbox[0][0]
                                                                 conv8_2_mbox_priorbox[0][0]
                                                                 conv9_2_mbox_priorbox[0][0]
                                                                 pool6_mbox_priorbox[0][0]
_____________________________________________________________________________________________
predictions (Concatenate)       (None, 8466, 33)     0           mbox_loc_final[0][0]
                                                                 mbox_conf_final[0][0]
                                                                 mbox_priorbox[0][0]
=============================================================================================
Total params: 26,754,500
Trainable params: 26,754,500
Non-trainable params: 0
_____________________________________________________________________________________________
```

```
pretrainedVGG16_used_SSD300
__________________________________________________________________________________________________
Layer (type)                    Output Shape         Param #     Connected to
==================================================================================================
input_2 (InputLayer)            (None, 300, 300, 3)  0
__________________________________________________________________________________________________
model_1 (Model)                 (None, 37, 37, 256)  1735488     input_2[0][0]
__________________________________________________________________________________________________
conv4_1 (Conv2D)                (None, 37, 37, 512)  1180160     model_1[1][0]
__________________________________________________________________________________________________
conv4_2 (Conv2D)                (None, 37, 37, 512)  2359808     conv4_1[0][0]
__________________________________________________________________________________________________
conv4_3 (Conv2D)                (None, 37, 37, 512)  2359808     conv4_2[0][0]
__________________________________________________________________________________________________
pool4 (MaxPooling2D)            (None, 19, 19, 512)  0           conv4_3[0][0]
__________________________________________________________________________________________________
conv5_1 (Conv2D)                (None, 19, 19, 512)  2359808     pool4[0][0]
__________________________________________________________________________________________________
conv5_2 (Conv2D)                (None, 19, 19, 512)  2359808     conv5_1[0][0]
__________________________________________________________________________________________________
conv5_3 (Conv2D)                (None, 19, 19, 512)  2359808     conv5_2[0][0]
__________________________________________________________________________________________________
pool5 (MaxPooling2D)            (None, 19, 19, 512)  0           conv5_3[0][0]
__________________________________________________________________________________________________
fc6 (Conv2D)                    (None, 19, 19, 1024) 4719616     pool5[0][0]
__________________________________________________________________________________________________
fc7 (Conv2D)                    (None, 19, 19, 1024) 1049600     fc6[0][0]
__________________________________________________________________________________________________
conv6_1 (Conv2D)                (None, 19, 19, 256)  262400      fc7[0][0]
__________________________________________________________________________________________________
conv6_2 (Conv2D)                (None, 10, 10, 512)  1180160     conv6_1[0][0]
__________________________________________________________________________________________________
conv7_1 (Conv2D)                (None, 10, 10, 128)  65664       conv6_2[0][0]
__________________________________________________________________________________________________
conv7_1z (ZeroPadding2D)        (None, 12, 12, 128)  0           conv7_1[0][0]
__________________________________________________________________________________________________
conv7_2 (Conv2D)                (None, 5, 5, 256)    295168      conv7_1z[0][0]
__________________________________________________________________________________________________
conv8_1 (Conv2D)                (None, 5, 5, 128)    32896       conv7_2[0][0]
__________________________________________________________________________________________________
conv4_3_norm (Normalize)        (None, 37, 37, 512)  512         conv4_3[0][0]
__________________________________________________________________________________________________
conv8_2 (Conv2D)                (None, 3, 3, 256)    295168      conv8_1[0][0]
__________________________________________________________________________________________________
pool6 (GlobalAveragePooling2D)  (None, 256)          0           conv8_2[0][0]
__________________________________________________________________________________________________
conv4_3_norm_mbox_conf (Conv2D) (None, 37, 37, 63)   290367      conv4_3_norm[0][0]
__________________________________________________________________________________________________
fc7_mbox_conf (Conv2D)          (None, 19, 19, 126)  1161342     fc7[0][0]
__________________________________________________________________________________________________
conv6_2_mbox_conf (Conv2D)      (None, 10, 10, 126)  580734      conv6_2[0][0]
__________________________________________________________________________________________________
conv7_2_mbox_conf (Conv2D)      (None, 5, 5, 126)    290430      conv7_2[0][0]
__________________________________________________________________________________________________
conv8_2_mbox_conf (Conv2D)      (None, 3, 3, 126)    290430      conv8_2[0][0]
__________________________________________________________________________________________________
conv4_3_norm_mbox_loc (Conv2D)  (None, 37, 37, 12)   55308       conv4_3_norm[0][0]
__________________________________________________________________________________________________
fc7_mbox_loc (Conv2D)           (None, 19, 19, 24)   221208      fc7[0][0]
__________________________________________________________________________________________________
conv6_2_mbox_loc (Conv2D)       (None, 10, 10, 24)   110616      conv6_2[0][0]
__________________________________________________________________________________________________
conv7_2_mbox_loc (Conv2D)       (None, 5, 5, 24)     55320       conv7_2[0][0]
__________________________________________________________________________________________________
conv8_2_mbox_loc (Conv2D)       (None, 3, 3, 24)     55320       conv8_2[0][0]
__________________________________________________________________________________________________
conv4_3_norm_mbox_conf_flat (Fl (None, 86247)        0           conv4_3_norm_mbox_conf[0][0]
__________________________________________________________________________________________________
fc7_mbox_conf_flat (Flatten)    (None, 45486)        0           fc7_mbox_conf[0][0]
__________________________________________________________________________________________________
conv6_2_mbox_conf_flat (Flatten (None, 12600)        0           conv6_2_mbox_conf[0][0]
__________________________________________________________________________________________________
conv7_2_mbox_conf_flat (Flatten (None, 3150)         0           conv7_2_mbox_conf[0][0]
__________________________________________________________________________________________________
conv8_2_mbox_conf_flat (Flatten (None, 1134)         0           conv8_2_mbox_conf[0][0]
__________________________________________________________________________________________________
pool6_mbox_conf_flat (Dense)    (None, 126)          32382       pool6[0][0]
__________________________________________________________________________________________________
conv4_3_norm_mbox_loc_flat (Fla (None, 16428)        0           conv4_3_norm_mbox_loc[0][0]
__________________________________________________________________________________________________
fc7_mbox_loc_flat (Flatten)     (None, 8664)         0           fc7_mbox_loc[0][0]
__________________________________________________________________________________________________
conv6_2_mbox_loc_flat (Flatten) (None, 2400)         0           conv6_2_mbox_loc[0][0]
__________________________________________________________________________________________________
conv7_2_mbox_loc_flat (Flatten) (None, 600)          0           conv7_2_mbox_loc[0][0]
__________________________________________________________________________________________________
conv8_2_mbox_loc_flat (Flatten) (None, 216)          0           conv8_2_mbox_loc[0][0]
__________________________________________________________________________________________________
pool6_mbox_loc_flat (Dense)     (None, 24)           6168        pool6[0][0]
__________________________________________________________________________________________________
mbox_conf (Concatenate)         (None, 148743)       0           conv4_3_norm_mbox_conf_flat[0][0]
                                                                 fc7_mbox_conf_flat[0][0]
                                                                 conv6_2_mbox_conf_flat[0][0]
                                                                 conv7_2_mbox_conf_flat[0][0]
                                                                 conv8_2_mbox_conf_flat[0][0]
                                                                 pool6_mbox_conf_flat[0][0]
__________________________________________________________________________________________________
pool6_reshaped (Reshape)        (None, 1, 1, 256)    0           pool6[0][0]
__________________________________________________________________________________________________
mbox_loc (Concatenate)          (None, 28332)        0           conv4_3_norm_mbox_loc_flat[0][0]
                                                                 fc7_mbox_loc_flat[0][0]
                                                                 conv6_2_mbox_loc_flat[0][0]
                                                                 conv7_2_mbox_loc_flat[0][0]
                                                                 conv8_2_mbox_loc_flat[0][0]
                                                                 pool6_mbox_loc_flat[0][0]
__________________________________________________________________________________________________
mbox_conf_logits (Reshape)      (None, 7083, 21)     0           mbox_conf[0][0]
__________________________________________________________________________________________________
conv4_3_norm_mbox_priorbox (Pri (None, 4107, 8)      0           conv4_3_norm[0][0]
__________________________________________________________________________________________________
fc7_mbox_priorbox (PriorBox)    (None, 2166, 8)      0           fc7[0][0]
__________________________________________________________________________________________________
conv6_2_mbox_priorbox (PriorBox (None, 600, 8)       0           conv6_2[0][0]
__________________________________________________________________________________________________
conv7_2_mbox_priorbox (PriorBox (None, 150, 8)       0           conv7_2[0][0]
__________________________________________________________________________________________________
conv8_2_mbox_priorbox (PriorBox (None, 54, 8)        0           conv8_2[0][0]
__________________________________________________________________________________________________
pool6_mbox_priorbox (PriorBox)  (None, 6, 8)         0           pool6_reshaped[0][0]
__________________________________________________________________________________________________
mbox_loc_final (Reshape)        (None, 7083, 4)      0           mbox_loc[0][0]
__________________________________________________________________________________________________
mbox_conf_final (Activation)    (None, 7083, 21)     0           mbox_conf_logits[0][0]
__________________________________________________________________________________________________
mbox_priorbox (Concatenate)     (None, 7083, 8)      0           conv4_3_norm_mbox_priorbox[0][0]
                                                                 fc7_mbox_priorbox[0][0]
                                                                 conv6_2_mbox_priorbox[0][0]
                                                                 conv7_2_mbox_priorbox[0][0]
                                                                 conv8_2_mbox_priorbox[0][0]
                                                                 pool6_mbox_priorbox[0][0]
__________________________________________________________________________________________________
predictions (Concatenate)       (None, 7083, 33)     0           mbox_loc_final[0][0]
                                                                 mbox_conf_final[0][0]
                                                                 mbox_priorbox[0][0]
==================================================================================================
Total params: 25,765,497
Trainable params: 25,765,497
Non-trainable params: 0
__________________________________________________________________________________________________
```