Auto Encoderで再生した画像をMSE,binary_crossentropy,PSNRで評価する

#Auto Encoderで再生した画像をMSE,binary_crossentropy,PSNRで評価

AutoEncoderで再生した画像をMSE,binary_crossentropy,PSNRで評価し、
上位10個、下位10個を表示して確認すした。

やはり、直感的にはMSEが一番わかりやすい。
binary_crossentropyは元のアタイの範囲を0~1になるように1/2555にする前処理が必要。

```python
import keras
from keras.datasets import mnist
import matplotlib.pyplot as plt
import numpy as np
import pandas as pd
import cv2
%matplotlib inline
```

    Using TensorFlow backend.



```python
from keras import losses
# MSE
from sklearn.metrics import mean_absolute_error

orgs = []
gens = []
binary_crossentropys = np.zeros((4 * 6000),np.float)
mses = np.zeros((4 * 6000),np.float)
psnrs = np.zeros((4 * 6000),np.float)
for i in range(4):
    for j in range(6000):
        index = i*6000 + j
        org = cv2.imread(f'images/org_{i}_{j}.png', cv2.IMREAD_GRAYSCALE)
        gen = cv2.imread(f'images/gen_{i}_{j}.png', cv2.IMREAD_GRAYSCALE)
        orgs.append(org)
        gens.append(gen)
        binary_crossentropys[index] = float(losses.binary_crossentropy(org.reshape(784)/255, gen.reshape(784)/255))
        mses[index] = mean_absolute_error(org, gen)
        psnrs[index] = cv2.PSNR(org, gen)
df = pd.DataFrame({'org': orgs, 'gen': gens, 'binary_crossentropy': binary_crossentropys, 'mse': mses, 'psnr': psnrs})
```


```python
# MSE
mse_sort = df.sort_values(by='mse', ascending=False)
for i in range(10):
    row = mse_sort.iloc[i]
    mse = row['mse']
    plt.figure(figsize=(20, 20))
    plt.subplot(2,10,i*2+1)
    plt.title(f'mse: {mse}')
    plt.imshow(row['org'])
    plt.subplot(2,10,i*2+2)
    plt.imshow(row['gen'])
    plt.show()
```


    
![test_loss_2_0.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/490869/5a9f2856-64c1-3b21-5907-7052f4561b0d.png)
![test_loss_2_1.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/490869/35f76b2a-bb05-6996-3409-1f10453db231.png)
![test_loss_2_2.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/490869/fc219220-c0a7-7c5b-f188-e939c6745efe.png)
![test_loss_2_3.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/490869/5e234a33-9392-fdc6-e43e-b0964bafa9a0.png)
![test_loss_2_4.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/490869/60e15b3d-47e3-2485-6976-dd0547e99c79.png)
![test_loss_2_5.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/490869/50ac6cf1-9131-627b-8f95-dac59275b160.png)
![test_loss_2_6.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/490869/ea95a588-1b05-e5fe-451f-bd3997d2aa51.png)
![test_loss_2_7.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/490869/883d4f45-a38e-442b-f884-ff7d7771a239.png)
![test_loss_2_8.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/490869/5a541a9b-f4d2-7aa7-0ad2-c57b6cda5d65.png)
![test_loss_2_9.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/490869/52ab6203-9d5a-ad7f-f354-73534e4c277d.png)



```python
j = 0
for i in range(6000*4-1, 6000*4-11, -1):
    row = mse_sort.iloc[i]
    mse = row['mse']
    plt.figure(figsize=(20, 20))
    plt.subplot(2,10,j*2+1)
    plt.title(f'mse: {mse}')
    plt.imshow(row['org'])
    plt.subplot(2,10,j*2+2)
    plt.imshow(row['gen'])
    plt.show()
    j += 1
```

![test_loss_3_0.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/490869/59efb54d-00c5-e58d-122d-fc34b47abe35.png)
![test_loss_3_1.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/490869/220a0a25-8b60-042d-4c30-a3ec04f7bec6.png)
![test_loss_3_2.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/490869/b279499d-8c44-9b99-f41c-c38ee2eb4761.png)
![test_loss_3_3.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/490869/16d01ec0-a47d-bbeb-7654-e20a11aa47de.png)
![test_loss_3_4.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/490869/ce25d306-6b40-b194-1b72-ba487377e24e.png)
![test_loss_3_5.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/490869/fd26c91b-83c8-75fb-5151-0d7e550e5b8b.png)
![test_loss_3_6.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/490869/7e1f33b3-bc76-465d-51b1-90e54b61e2b3.png)
![test_loss_3_7.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/490869/d8539022-9a35-f018-7a12-664a6539350a.png)
![test_loss_3_8.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/490869/9933c4c0-d128-7c1d-896a-dda275b75397.png)
![test_loss_3_9.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/490869/693a824c-5126-85a6-4599-70392a564c09.png)
    

```python
# binary_crossentropy
binary_crossentropy_sort = df.sort_values(by='binary_crossentropy', ascending=False)
for i in range(10):
    row = mse_sort.iloc[i]
    mse = row['binary_crossentropy']
    plt.figure(figsize=(20, 20))
    plt.subplot(2,10,i*2+1)
    plt.title(f'binary_crossentropy: {mse}')
    plt.imshow(row['org'])
    plt.subplot(2,10,i*2+2)
    plt.imshow(row['gen'])
    plt.show()
```

![test_loss_4_0.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/490869/c377dfce-7b76-f79b-d9b2-dab1b9a6e704.png)
![test_loss_4_1.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/490869/8001bdb6-d317-ebce-37fa-5e590e0421ce.png)
![test_loss_4_2.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/490869/58e339f1-6592-e8d8-200d-e28129953ee7.png)
![test_loss_4_3.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/490869/9bf165ab-bd7c-d111-980d-b76a22a0ae74.png)
![test_loss_4_4.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/490869/e5cf2861-8d07-5434-47db-c9a48bd8bba0.png)
![test_loss_4_5.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/490869/95dd8fee-6db7-16f3-0db9-e0b5eb4d6f97.png)
![test_loss_4_6.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/490869/bfe9818e-a770-a377-54ed-916bdfd0b65d.png)
![test_loss_4_7.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/490869/34d30b0c-5f72-eb68-a2d4-d7cfa9aa506b.png)
![test_loss_4_8.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/490869/8d79baff-493c-3540-a727-43e089c9bd94.png)
![test_loss_4_9.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/490869/df203ab8-82bd-7efd-6229-9afa16d1d8e9.png)


```python
j = 0
for i in range(6000*4-1, 6000*4-11, -1):
    row = binary_crossentropy_sort.iloc[i]
    binary_crossentropy = row['binary_crossentropy']
    plt.figure(figsize=(20, 20))
    plt.subplot(2,10,j*2+1)
    plt.title(f'binary_crossentropy: {binary_crossentropy}')
    plt.imshow(row['org'])
    plt.subplot(2,10,j*2+2)
    plt.imshow(row['gen'])
    plt.show()
    j += 1
```

![test_loss_5_0.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/490869/4e213eab-69e6-a945-e209-39898c0d764e.png)
![test_loss_5_1.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/490869/6038f36d-9389-07be-74f7-fbdff4a229f4.png)
![test_loss_5_2.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/490869/e5acedb4-9eb1-0287-3cbc-6c27faba5fe3.png)
![test_loss_5_3.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/490869/7ef87429-965d-e51f-d9aa-8b85cf4cf161.png)
![test_loss_5_4.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/490869/5dc774fc-0494-6473-e563-506bb4873555.png)
![test_loss_5_5.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/490869/9f025874-60ba-9273-b93f-d9bd5b62a8bb.png)
![test_loss_5_6.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/490869/198ad1b3-900f-101a-5d62-20aa69aff082.png)
![test_loss_5_7.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/490869/52d5ee2a-0706-ffa6-4a9a-d30b70f6a99c.png)
![test_loss_5_8.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/490869/71cbe6c2-c7e2-8b40-7cf4-70761bc6d065.png)
![test_loss_5_9.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/490869/16f07f09-38e5-b302-0393-86d02a5bda10.png)


```python
# PSNR
psnr_sort = df.sort_values(by='psnr', ascending=False)
for i in range(10):
    row = mse_sort.iloc[i]
    psnr = row['psnr']
    plt.figure(figsize=(20, 20))
    plt.subplot(2,10,i*2+1)
    plt.title(f'psnr: {psnr}')
    plt.imshow(row['org'])
    plt.subplot(2,10,i*2+2)
    plt.imshow(row['gen'])
    plt.show()
```

![test_loss_6_0.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/490869/9a318ccb-e48c-b02a-9e8d-34cc8fa1d9e2.png)
![test_loss_6_1.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/490869/856691a7-577c-92cc-0cb2-d102d5142a24.png)
![test_loss_6_2.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/490869/64bef4dd-2044-9b5c-2565-fe40b0c97bbf.png)
![test_loss_6_3.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/490869/c6f28ffe-64d5-de02-2951-9bd171706fbb.png)
![test_loss_6_4.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/490869/8fc29405-5e15-24e9-b4a3-9f0a7d3bd496.png)
![test_loss_6_5.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/490869/ce7146ab-b31d-585d-6db3-61122c372212.png)
![test_loss_6_6.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/490869/cf961571-65e5-c5b7-5ab5-ac27354053b4.png)
![test_loss_6_7.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/490869/b9807cbe-8426-e637-33a4-74847c59db3a.png)
![test_loss_6_8.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/490869/fb299213-5858-9d27-5884-1a90f47f20bc.png)
![test_loss_6_9.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/490869/5ca8d45c-283f-5629-da00-7414ce6b1299.png)


```python
j = 0
for i in range(6000*4-1, 6000*4-11, -1):
    row = psnr_sort.iloc[i]
    psnr = row['psnr']
    plt.figure(figsize=(20, 20))
    plt.subplot(2,10,j*2+1)
    plt.title(f'psnr: {psnr}')
    plt.imshow(row['org'])
    plt.subplot(2,10,j*2+2)
    plt.imshow(row['gen'])
    plt.show()
    j += 1
```

![test_loss_7_0.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/490869/7b714d27-c5a3-ad53-5ce0-147b8bf4e8fc.png)
![test_loss_7_1.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/490869/324968be-d55a-7186-0430-ae73a144ea78.png)
![test_loss_7_2.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/490869/24063b7c-4dbb-7472-94ed-27fd501d8479.png)
![test_loss_7_3.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/490869/961e3b66-d336-605c-7429-f8c4aee6932e.png)
![test_loss_7_4.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/490869/b2f81c38-87d1-14b0-40d0-42b9fa776ecf.png)
![test_loss_7_5.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/490869/7768a914-2516-65cf-9d06-fefcadcccaa7.png)
![test_loss_7_6.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/490869/bbc144b5-0e4d-1d4d-a672-4faf6ccb0a66.png)
![test_loss_7_7.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/490869/ab5bf873-07f2-2eca-f6f0-86360bc450b3.png)
![test_loss_7_8.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/490869/ec6126d7-0efa-a20f-bd38-b229f8f2f272.png)
![test_loss_7_9.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/490869/7a7c30fa-b503-8fac-a09b-2c1dfff01542.png)