再提出レポート（深層学習ー２）

＜後編＞
・simple RNN(Recursive Neural Network)
バイナリ加算
 
![image.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/435808/2944f354-5570-7109-17dc-75808c0715d3.png)

＜コード追記＞
![image.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/435808/b19e42cc-18b1-eb88-a10d-39cf3351a13e.png)
 

重み初期化方法の変更
<Xavier>
 ![image.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/435808/f0682766-359d-4696-d66e-b6ae55531ba7.png)


<He>
 ![image.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/435808/72c6c5a6-f3ff-ba8d-7021-47a0d7bdecb7.png)


重みの初期化方法を変更すると、収束するまでの時間が長くなる傾向にある。

活性化関数の変更
<ReLU>
 ![image.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/435808/8e20d517-a69f-5823-794b-4f1f02e39bb4.png)


<tanh>
 ![image.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/435808/a9a64f63-bed8-cba7-c289-6e506e03ec00.png)


活性化関数を、ReLU、tanhに変更した結果、両方とも良い結果が得られなかった。

・Predict sin
＜maxlen=2,iters_num=500＞
 ![image.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/435808/73a642c2-9b9d-f122-da9d-f233466cadd7.png)


＜maxlen=2,iters_num=100＞
 
![image.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/435808/3a85a629-e813-0ed8-cb58-9769cc56321a.png)


＜maxlen=5,iters_num=500＞
 
![image.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/435808/99ec7a97-c34e-05b6-b42d-c7a87ca7c15a.png)


＜maxlen=5,iters_num=3000＞
![image.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/435808/217dd2e1-dcce-67f8-0c89-f5a8c8db8328.png)
 

上記の結果から、「ひとつの時系列データ」を長くして、繰り返し数を多くすることによって、sin波を正確に再現できることが示唆された。

・Predict word
総単語数：9826
 ![image.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/435808/c35b0ae1-b243-7c0b-f0c4-2c37befca5cc.png)



出現回数の少ない単語を未知語（？？？）として表示
 
![image.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/435808/37c9fe4a-e6c0-e597-446c-3947b479cdb4.png)


Tensorflow
・constant
![image.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/435808/e1892869-4762-89c0-8f81-7e60d907f118.png)
 

・placeholder
![image.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/435808/dc554584-4eba-3431-3ec5-da96543d8cdc.png)
 

・variables
![image.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/435808/951cdedb-b139-a935-5f5b-7ea83522c00a.png)

 

・線形回帰

 ![image.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/435808/b4e802df-30c6-f28c-01bf-e8cb7df584f7.png)


・非線形回帰

![image.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/435808/9934c716-e42b-c3ef-c7d5-1d000b0e30d3.png)


y=30x2+0.5x+0.2
![image.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/435808/13bdde56-8ba5-f22c-f7d4-2b27752cd981.png)
![image.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/435808/f1c4be44-250e-a1a2-3f69-bc039f8c3cb6.png)
 
 

・分類1層(mnist)
 ![image.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/435808/589d3f13-2bc8-19d9-607d-685de577c678.png)


・分類3層(mnist)
![image.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/435808/40bb8654-f11d-c091-7ccc-6a77b50aa45c.png)
 


・分類CNN(mnist)
![image.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/435808/35a98026-048d-73e7-d34e-0fd63076e8a0.png)

 

・分類CNN(mnist):Dropout率=0
![image.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/435808/44383550-571f-d9b5-7e83-81062b2ce33a.png)

 


Keras
・線形回帰
![image.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/435808/971a15c9-3605-1d89-45ed-21c7ba86691e.png)

 




・単純パーセプトロン
![image.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/435808/d5085d75-25f9-03d3-e2a8-d1dfb7f82ec2.png)
 

・np.random.seed(1)
 ![image.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/435808/b881f55b-ec2f-49a2-5038-4c8e669fb6b5.png)


・epochs=100
 ![image.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/435808/f467a663-dde6-9445-8d58-8b139e17f9ea.png)

・AND回路
![image.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/435808/9c7ce56a-94e3-e194-6c52-101bdfc2b2e2.png)
 




・XOR回路
 ![image.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/435808/5b351e18-f144-3faf-7feb-38114a599461.png)


・batch size=10
 ![image.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/435808/02e4dda0-2403-e9e1-dc24-2dcf605dba4d.png)


・epochs=300
 
![image.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/435808/3463bc04-3d77-b6a6-0a58-849e1fb7958d.png)

上記ケースが最も誤差が小さく、正確な結果を表している。

分類(iris)

 ![image.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/435808/2952b0de-080e-75a7-3d39-1325ccefc3d4.png)


・sigmoid関数
![image.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/435808/91c99452-d293-81aa-454f-b73d32fb803c.png)

 
・from keras.optimizers import SGD
SGD(lr=0.1, momentum=0.0, decay=0.0, nesterov=False)

![image.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/435808/fcf69f66-50e0-d99b-bce6-6f4961f3410c.png)
 

分類(mnist)

![image.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/435808/03f41823-fd8c-4034-f29c-cfa17a89d927.png)

・one_hot_label=False
![image.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/435808/c7e370a9-0329-546a-3ec4-6ee056e15da6.png)
 

・sparse_categorical_crossentropy
![image.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/435808/a3daa416-2c76-60ea-fca6-b6633eed4deb.png)

 

・Adam　(Ir=0.1)
![image.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/435808/0efb5bd0-28a5-eb7f-21e0-f541879be58f.png)



・Adam　(Ir =0.00001)

![image.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/435808/dca9834a-747d-03b8-5e73-e747da6edd94.png)