TensorFlow > mapping > R^2のinputをR^2のoutputにmappingする実装 v0.4 > (x1, x2)範囲を[0, 1)から[-2, 2)に変更

```txt:動作環境
GeForce GTX 1070 (8GB)
ASRock Z170M Pro4S [Intel Z170chipset]
Ubuntu 14.04 LTS desktop amd64
TensorFlow v0.11
cuDNN v5.1 for Linux
CUDA v8.0
Python 2.7.6
IPython 5.1.0 -- An enhanced Interactive Python.
gcc (Ubuntu 4.8.4-2ubuntu1~14.04.3) 4.8.4
GNU bash, version 4.3.8(1)-release (x86_64-pc-linux-gnu)
```

### 概要

http://users.monash.edu/~app/Lrn/LearningMDS.pdf
の5.4 (Simple) example of function approximation。

R^2(2次元の実数)のinputをR^2のoutputにmappingしている。

TensorFlowでの学習を試行錯誤中。

### 関連 

v0.1: http://qiita.com/7of9/items/16bf19555dd97b09ecb0

### (x1, x2)範囲の変更

(x1, x2)の配列作成時、Halton Sequenceの結果をそのまま使っていたため`[0, 1)`の範囲であった。
これを論文の`[-2, 2)`に合わせる。

http://qiita.com/7of9/items/5431ca6ff8eb79d5eee7#_reference-490cf191989e52437f3b
のコードを元に、prep_data_170321.pyとcheck_resultmap_170329.ipynbを変更した。

### prep_data_170321.py > v0.2

```prep_data_170321.py
from __future__ import absolute_import
from __future__ import division
from __future__ import print_function
import numpy as np
import math
import sys
from UtilHaltonSequence import CHaltonSequence

'''
on Python 2.7.6

v0.2 Apr. 01, 2017
  - change range from [0, 1) to [-2, 2)

v0.1 Mar. 21, 2017
  - output to STD
  - add func2d()
  - add get_2d_QMC()
  - add get_2d_random_nums()
'''
# codingrule:PEP8


def get_2d_random_nums(xsize, ysize):
    MAXVAL_PLUS_ONE = 65536
    ints = np.random.randint(MAXVAL_PLUS_ONE, size=(xsize, ysize))
    # print(ints)

    flts = ints / float(MAXVAL_PLUS_ONE)
    # print(flts)
    return flts


def get_2d_QMC(xsize):
    alist = []
    for idx in range(0, xsize):
        xwrk, ywrk = CHaltonSequence.calc_Halton_sequence(index=idx)
        alist.append([xwrk, ywrk])
    # print(alist)
    return np.array(alist)


def func2d(x1, x2):
    rho = x1 ** 2 + x2 ** 2
    epsilon = 10.**(-10)
    if abs(rho) < epsilon:
        return (0.0, 0.0)

    y1 = x1 * math.exp(-rho ** 2)
    try:
        # print(rho) # debug
        y2 = math.sin(2.0 * rho ** 2) / 4.0 / rho ** 2
    except ZeroDivisionError:
        y2 = 0.0
    return (y1, y2)

NUM_SAMPLES = 100

# 1. get random numbers in 2D
#
# a. np.random.randint()
# arr_2d = get_2d_random_nums(NUM_SAMPLES,2) # [0,1)
#
# b. QMC using Halton Sequence
arr_2d = get_2d_QMC(NUM_SAMPLES)  # [0, 1)
arr_2d = arr_2d * 4.0 - 2.0  # [-2, 2)

for x1, x2 in arr_2d:
    y1, y2 = func2d(x1, x2)
    print('%.7f, %.7f, %.7f, %.7f' % (x1, x2, y1, y2))
    # sys.exit()  # debug
```

### check_resultmap_170329.ipynb > v0.2

```py:check_resultmap_170329.ipynb
'''
v0.2 Arp. 01, 2017
    - change (x1,x2) range from [0, 1) to [-2, 2)
v0.1 Mar. 29, 2017
    - move to matplotlib on Jupyter
    - modify calc_conv() for 2 input nodes
=== [reproduce_sine.py] branched to [reproduce_xxxfunc_170328.py] ===
v0.3 Dec. 11, 2016
    - add output_debugPrint()
    - fix bug > calc_sigmoid() was using positive for exp()
v0.2 Dec. 10, 2016
    - calc_conv() takes [applyActFnc] argument
v0.1 Dec. 10, 2016
    - add calc_sigmoid()
    - add fully_connected network
    - add input data for sine curve
=== [read_model_var.py] branched to [reproduce_sine.py] ===

v0.4 Dec. 10, 2016
    - add 2x2 network example
v0.3 Dec. 07, 2016
    - calc_conv() > add bias
v0.2 Dec. 07, 2016
    - fix calc_conv() treating src as a list
v0.1 Dec. 07, 2016
    - add calc_conv()
'''
from __future__ import absolute_import
from __future__ import division
from __future__ import print_function
import numpy as np
import math
import sys
import matplotlib.pyplot as plt
import matplotlib.cm as cm
from UtilHaltonSequence import CHaltonSequence


# to ON/OFF debug print at one place
def output_debugPrint(str):
    #print(str)
    pass  # no operation

def calc_sigmoid(x):
    return 1.0 / (1.0 + math.exp(-x))


def calc_conv(src, weight, bias, applyActFnc):
    wgt = weight.shape
#   print wgt # debug
    #conv = list(range(bias.size))
    conv = [0.0] * bias.size

    # weight
    for idx1 in range(wgt[0]):
        for idx2 in range(wgt[1]):
            conv[idx2] = conv[idx2] + src[idx1] * weight[idx1, idx2]
    # bias
    for idx2 in range(wgt[1]):
        conv[idx2] = conv[idx2] + bias[idx2]
    # activation function
    if applyActFnc:
        for idx2 in range(wgt[1]):
            conv[idx2] = calc_sigmoid(conv[idx2])
    return conv  # return list


def get_2d_QMC(xsize):
    alist = []
    for idx in range(0, xsize):
        xwrk, ywrk = CHaltonSequence.calc_Halton_sequence(index=idx)
        alist.append([xwrk, ywrk])
    # print(alist)
    return np.array(alist)


def func2d(x1, x2):
    rho = x1 ** 2 + x2 ** 2
    epsilon = 10.**(-10)
    if abs(rho) < epsilon:
        return (0.0, 0.0)

    y1 = x1 * math.exp(-rho ** 2)
    try:
        # print(rho) # debug
        y2 = math.sin(2.0 * rho ** 2) / 4.0 / rho ** 2
    except ZeroDivisionError:
        y2 = 0.0
    return (y1, y2)


model_var = np.load('model_variables_170321.npy')

#output_debugPrint(("all shape:", (model_var.shape)))

NUM_SAMPLES = 100
inpdata = get_2d_QMC(NUM_SAMPLES)  # [0, 1)

prd_y1_1d = np.array([])
prd_y2_1d = np.array([])
ans_y1_1d = np.array([])
ans_y2_1d = np.array([])

for x1idx in range(-20, 20, 2):
    for x2idx in range(-20, 20, 2):
        inlist = (x1idx/10.0, x2idx/10.0)
        # input layer (2 node)
        #
        # hidden layer 1 (7 node)
        outdata = calc_conv(inlist, model_var[0], model_var[1], applyActFnc=True)
        # hidden layer 2 (7 node)
        outdata = calc_conv(outdata, model_var[2], model_var[3], applyActFnc=True)
        # hidden layer 3 (7 node)
        outdata = calc_conv(outdata, model_var[4], model_var[5], applyActFnc=True)
        # output layer (2 node)
        outdata = calc_conv(outdata, model_var[6], model_var[7], applyActFnc=False)
        prd_y1_1d = np.append(prd_y1_1d, outdata[0])
        prd_y2_1d = np.append(prd_y2_1d, outdata[1])

        answer = func2d(inlist[0], inlist[1])
        ans_y1_1d = np.append(ans_y1_1d, answer[0])
        ans_y2_1d = np.append(ans_y2_1d, answer[1])
        #print('%s %s %s' % (inlist, outdata, answer))
        
    
siz = 20
plt.subplot(223)
dat_2d = np.reshape(prd_y2_1d, (siz,siz))
plt.imshow(dat_2d, extent=(0, siz, 0, siz), cmap=cm.gist_rainbow)

plt.subplot(224)
dat_2d = np.reshape(ans_y2_1d, (siz,siz))
plt.imshow(dat_2d, extent=(0, siz, 0, siz), cmap=cm.gist_rainbow)

plt.subplot(221)
dat_2d = np.reshape(prd_y1_1d, (siz,siz))
plt.imshow(dat_2d, extent=(0, siz, 0, siz), cmap=cm.gist_rainbow)

plt.subplot(222)
dat_2d = np.reshape(ans_y1_1d, (siz,siz))
plt.imshow(dat_2d, extent=(0, siz, 0, siz), cmap=cm.gist_rainbow)

plt.show()

```

### min_queue_examples

learn_xxyyfunc_170321.pyのmin_queue_examplesは以下で計算。

```
batch_size = 4  # [4]
# Ref: cifar10_input.py
min_fraction_of_examples_in_queue = 0.2 # 0.4
NUM_EXAMPLES_PER_EPOCH_FOR_TRAIN = 100
min_queue_examples = int(NUM_EXAMPLES_PER_EPOCH_FOR_TRAIN * 
	min_fraction_of_examples_in_queue)
```

### 結果 > `[7, 7, 7]`のネットワーク

中間層が`[7, 7, 7]`でのネットワークの結果。
(learn_xxyyfunc_170321.py の hiddnes = slim.stack()参照)

![qiita.png](https://qiita-image-store.s3.amazonaws.com/0/32870/1699e65f-5b3e-856a-0e4f-973f105cf190.png)

右側が回答であるy1, y2の分布。
左側が学習したネットワークからの再現。

y1の分布は再現度がそこそこだが、y2は差異が大きそう。

### 結果 > `[100, 100, 100]`のネットワーク

中間層が`[100, 100, 100]`でのネットワークの結果。
(learn_xxyyfunc_170321.py の hiddnes = slim.stack()参照)

![qiita.png](https://qiita-image-store.s3.amazonaws.com/0/32870/eea7e53f-7328-bc7f-43bb-a88e62bc17f9.png)

y1の再現性は十分だろう。
y2もplateauの特徴が再現できるようになってきた。
( y2の具体的な形は　http://qiita.com/7of9/items/e8abb2e6cbb9825c186d )
y2の細かな振動は再現できないが、これらの特徴は線形方程式を解き直した時に希望精度に高められるのでは。

実利用時にどれくらいの規模のネットワークを扱うかはメモリと計算時間を検討した上で決めることになるだろう。