再提出レポート（深層学習ー１）

順伝播（単層・単ユニット）
入力（２，３）　重み　０　１　　　　計算結果（２,６）　これにバイアス0.5
　　　　　　　　　　　０　２
を掛けて、出力が（１，３）　中間層出力はReLU関数を使うので、そのまま（１、３）
 
![image.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/435808/2f699eee-f3dd-1af4-d241-b300070d3cd4.png)

順伝播（単層・複数ユニット）
![image.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/435808/10387d01-4bec-41c0-5e0c-d1c79d0eca7b.png)
 

順伝播（３層・複数ユニット）
 ![image.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/435808/936fa08e-980a-1e1a-9faf-04b8311b70d0.png)
![image.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/435808/e37ecf1c-f1ff-b0e1-cc9f-bb80a02c4f3b.png)

 


多クラス分類
重み１を３層、重み２を４層にして、出力を４つに分類した結果。
出力値の合計を１とし、最も大きい値の要素と訓練データとの比較が誤差である。
 ![image.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/435808/619de9b6-6964-5444-7126-5b08d140b697.png)
![image.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/435808/31042e41-29b3-a418-56b9-0e91a3e66b57.png)

 


回帰
出力値と訓練データの差があり、その結果、誤差も大きい値である。
 ![image.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/435808/5f72a71c-53cd-8884-fffa-973af5f352a0.png)
![image.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/435808/650c6251-b477-edcf-ca00-f644e12f0a3b.png)

 


２値分類
 ![image.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/435808/aaaeb1c2-4bba-6990-992c-225d07479c52.png)



誤差逆伝播法
 ![image.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/435808/06a7fac0-5a60-9058-c876-22d3fee9b98a.png)
![image.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/435808/b4ec9693-e110-e4fe-e8f5-712701c03fde.png)

 


確率的勾配降下法

 ![image.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/435808/36046b9a-8921-3ec5-2093-389cc10b05c7.png)


ＭＮＩＳＴを使った検証
訓練データ、検証データとも正答率が0.9以上と良い結果が得られた。
 
 ![image.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/435808/6f3dafca-ba89-4664-858e-4f1117fcdb57.png)

![image.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/435808/3230e5dd-e957-ed72-8ae5-37af4fda317b.png)


勾配消失問題

ReLU－gauss

 ![image.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/435808/f50d69a4-c44a-988e-73c8-fb9e089a1a7b.png)


＜初期値設定について＞
Sigmoid-Xavier
![image.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/435808/68e14c30-b3c8-26e3-55d2-0039faf3afa1.png)

 

ReLU-He
![image.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/435808/82312cdb-59a3-4112-5b95-34e718330f55.png)

 


バッチ正則化
 ![image.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/435808/eac3080b-77a3-b6c1-fd13-5b283d9c8792.png)



確率的勾配降下法
・モメンタム
 ![image.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/435808/45df2c57-e296-8228-e282-c3449beb750c.png)


・AdaGrad
 ![image.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/435808/4a3c92d9-39b8-8133-df7f-509b03124f7a.png)


・RSMprop
 ![image.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/435808/e29dba5a-90d6-fc14-19b2-01d0d33e466f.png)


・Adam
 ![image.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/435808/9d490e30-2680-adc9-c02f-823f69052c33.png)



過学習
Ｌ１正則化
 ![image.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/435808/902f41f9-273f-c9a5-f512-6ba8b8e281cb.png)

Ｌ２正則化
 ![image.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/435808/4ab8f0a6-3cd2-661b-2c28-0a614c5e0923.png)


DropOut
 ![image.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/435808/8a23641c-085f-79dc-8a72-4ffe613088cc.png)


DropOut-L1正則化
 ![image.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/435808/348290a9-5da3-40e1-7254-47d34c1bbe79.png)



Im2col
 ![image.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/435808/211386e2-2f23-9275-3714-ef0ddc94a396.png)



Simple Convolution Network
 ![image.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/435808/ebfad0f0-32ea-d1dc-be85-2290b4233dd7.png)


Double Convolution Network
 ![image.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/435808/99200db7-66b5-3e87-5b9a-cea271012d69.png)



Double Convolution Network
![image.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/435808/e567785a-c276-2c38-2580-55294e9c3724.png)