TensorFlowで input:100, output:100のネットワークでの学習を検討 > 結果の図示

```txt:動作環境
GeForce GTX 1070 (8GB)
ASRock Z170M Pro4S [Intel Z170chipset]
Ubuntu 14.04 LTS desktop amd64
TensorFlow v0.11
cuDNN v5.1 for Linux
CUDA v8.0
Python 2.7.6
IPython 5.1.0 -- An enhanced Interactive Python.
gcc (Ubuntu 4.8.4-2ubuntu1~14.04.3) 4.8.4
GNU bash, version 4.3.8(1)-release (x86_64-pc-linux-gnu)
```

TensorFlowを使って、input:100 node, output:100 nodeのネットワークの学習を試行中。

関連 http://qiita.com/7of9/items/7394f4326a88500a67b9

http://qiita.com/7of9/items/31d93a6bc53e0c3ef02a
にて二乗誤差が0.08程度での学習で停滞している。

その誤差でどういう結果が得られるか出力してみた。

### 学習&出力コード

```learn_in100out100.py
#!/usr/bin/env python
# -*- coding: utf-8 -*-
import sys
import tensorflow as tf
import tensorflow.contrib.slim as slim
import numpy as np

'''
v0.10 Mar. 14, 2017
    - output training_data_output and it's prediction
v0.9 Mar. 13, 2017
    - use [tf.nn.softplus] for hidden layers
v0.8 Mar. 13, 2017
    - toying with output activation function
        + [None], [tf.nn.softplus], [tf.nn.sigmoid]
v0.7 Feb. 22, 2017
    - treat each hidden layer to handle dropout instead of slim.stack()
v0.6 Feb. 19, 2017
    - add sys.stdout.flush() to immediately print
v0.5 Feb. 18, 2017
    - add fc_drop()
v0.4 Feb. 15, 2017
    - fix bug > ZeroDivisionError: float division by zero @ shuffle_batch()
v0.3 Feb. 15, 2017
    - tweak [batch_size] for shuffle_batch()
v0.2 Feb. 15, 2017
    - fix bug > different dimensions for placeholder and network
v0.1 Feb. 06, 2017
    - read [test_in.csv],[test_out.csv]
'''

'''
codingrule:PEP8
'''


def fc_drop(inputs, *args, **kwargs):
    # Thanks to: http://qiita.com/shngt/items/f532601b4f059ce8584f
    net = slim.fully_connected(inputs, *args, **kwargs)
    return slim.dropout(net, 0.9)

filename_inp = tf.train.string_input_producer(["test_in.csv"])
filename_out = tf.train.string_input_producer(["test_out.csv"])
NUM_INP_NODE = 100
NUM_OUT_NODE = 100

# parse csv
# a. input node
reader = tf.TextLineReader()
key, value = reader.read(filename_inp)
deflist = [[0.] for idx in range(NUM_INP_NODE)]
input1 = tf.decode_csv(value, record_defaults=deflist)
# b. output node
key, value = reader.read(filename_out)
deflist = [[0.] for idx in range(NUM_OUT_NODE)]
output1 = tf.decode_csv(value, record_defaults=deflist)
# c. pack
# inputs = tf.pack([input1])
inputs = input1
# outputs = tf.pack([output1])
outputs = output1

batch_size = 2
inputs_batch, output_batch = tf.train.shuffle_batch(
    [inputs, outputs], batch_size, capacity=10, min_after_dequeue=batch_size)

input_ph = tf.placeholder("float", [None, 100])
output_ph = tf.placeholder("float", [None, 100])

# network
# a. with stack
# hiddens = slim.stack(input_ph, slim.fully_connected, [7, 7, 7],
#                     activation_fn=tf.nn.sigmoid, scope="hidden")
# a2. larget network
# a2a. with sigmoid
hiddens = slim.stack(input_ph, slim.fully_connected, [100, 100, 100],
                     activation_fn=tf.nn.sigmoid, scope="hidden")
# a2b. with softplus
#hiddens = slim.stack(input_ph, slim.fully_connected, [100, 100, 100],
#                     activation_fn=tf.nn.softplus, scope="hidden")
# b. without stack
# sgm = tf.nn.sigmoid
# hiddens = slim.fully_connected(input_ph, 7, activation_fn=sgm, scope='hidden1')
# hiddens = slim.fully_connected(hiddens, 7, activation_fn=sgm, scope='hidden2')
# hiddens = slim.fully_connected(hiddens, 7, activation_fn=sgm, scope='hidden3')
# c. without stack, with dropout
# sgm = tf.nn.sigmoid
# hiddens = slim.fully_connected(input_ph, 7, activation_fn=sgm, scope='hidden1')
# drpout = slim.stack(hiddens, fc_drop, [7, 7], scope='hd_fc1')
# hiddens = slim.fully_connected(drpout, 7, activation_fn=sgm, scope='hidden2')
# drpout = slim.stack(hiddens, fc_drop, [7, 7], scope='hd_fc2')
# hiddens = slim.fully_connected(drpout, 7, activation_fn=sgm, scope='hidden3')


# a. None(Linear)
#prediction = slim.fully_connected(
#    hiddens, 100, activation_fn=None, scope="output")
# b. None(Linear)
prediction = slim.fully_connected(
    hiddens, 100, activation_fn=tf.nn.softplus, scope="output")
# c. None(Linear)
# prediction = slim.fully_connected(
#    hiddens, 100, activation_fn=tf.nn.sigmoid, scope="output")


loss = tf.contrib.losses.mean_squared_error(prediction, output_ph)

train_op = slim.learning.create_train_op(loss, tf.train.AdamOptimizer(0.000001))

init_op = tf.initialize_all_variables()

with tf.Session() as sess:
    coord = tf.train.Coordinator()
    threads = tf.train.start_queue_runners(sess=sess, coord=coord)

    try:
        sess.run(init_op)
        for idx in range(20000):
            # for idx in range(90000):
            inpbt, outbt = sess.run([inputs_batch, output_batch])
            _, t_loss = sess.run(
                [train_op, loss],
                feed_dict={input_ph: inpbt, output_ph: outbt})

            if (idx+1) % 100 == 0:
                print("%d,%f" % (idx+1, t_loss))
                sys.stdout.flush()
#if 1 // debug
        inpbt, outbt = sess.run([inputs_batch, output_batch])
        prd = sess.run(
            [prediction],
                feed_dict={input_ph: inpbt, output_ph: outbt})
        print(outbt)
        print(prd)
#endif
    finally:
        coord.request_stop()

    coord.join(threads)
```

### 結果の可視化

上記の出力で得られる標準出力をtrやgrepを組み合わせて1行１項目のファイルに変換。

- sample_train_170314.txt : 学習用の出力例
- sample_learn_170314.txt : 学習結果(prediction)

それをJupyterにて可視化。

```py:read_prediction_170316.ipynb
import numpy as np
import matplotlib.pyplot as plt
import matplotlib.cm as cm

in_1d = np.loadtxt('sample_train_170314.txt', delimiter = ' ')
in_2d = np.reshape(in_1d, (10,10))

out_1d = np.loadtxt('sample_learn_170314.txt', delimiter = ' ')
out_2d = np.reshape(out_1d, (10,10))

plt.imshow(in_2d, extent=(0,10,0,10),cmap=cm.gist_rainbow)
plt.show()

plt.imshow(out_2d, extent=(0,10,0,10),cmap=cm.gist_rainbow)
plt.show()

```

![qiita.png](https://qiita-image-store.s3.amazonaws.com/0/32870/5c0c79b3-29ca-5193-28fb-82f6f06bdf57.png)

学習できてない。