Repository: tensorlayer/TensorLayer

Issue:
How to initialize DeConv2dLayer weights to bilinear upsampling?
Is there some simple way of initializing a Deconv2D layer with Bilinear weights?

I could write my own initalizer, but seems like a common thing to do.
Would it make sense to add it as a library feature?

PS
Great work with TensorLayer! :)




Discussion:
@JoelKronander sorry of the late reply.
I am not quite understand what bilinear weights stand for, do you mean given a matrix
`[[1, 4], [1, 4]]`, you can upsample to `[[1, 2, 3, 4], [1, 2, 3, 4]]`?
Hi, 

By bilinear weights I mean initializing the DeConvLayer weights so it replicates a standard bilinear image upsampling (resampling). This is used in some Semantic Segmentation papers, such as [FCN](https://arxiv.org/abs/1605.06211) e.g. 

Below is some code to do that by creating bilinear weights in numpy and then passing them to a constant_initializer, then passed as the W_init parameter when creating the DeConv2dLayer. 
Not sure if/where it would make sense to add/contribute something like this to tensorlayer, custom initializers somewhere?

```
import tensorflow as tf
import tensorlayer as tl
import numpy as np
import skimage
from scipy.misc import imread, imresize, imsave

sess = tf.InteractiveSession()

imsize = 100
scale = 2
filter_size = (2 * scale - scale % 2)
num_channels = 3

#Create bilinear weights in numpy array
bilinear_kernel = np.zeros([filter_size, filter_size], dtype=np.float32)
scale_factor = (filter_size + 1) // 2
if filter_size % 2 == 1:
    center = scale_factor - 1
else:
    center = scale_factor - 0.5
for x in range(filter_size):
    for y in range(filter_size):
        bilinear_kernel[x,y] = (1 - abs(x - center) / scale_factor) * \
                               (1 - abs(y - center) / scale_factor)
weights = np.zeros((filter_size, filter_size, num_channels, num_channels))
for i in range(num_channels):
    weights[:, :, i, i] = bilinear_kernel

#assign numpy array to constant_initalizer and pass to get_variable
bilinear_init = tf.constant_initializer(value=weights, dtype=tf.float32)

x = tf.placeholder(tf.float32, [1, imsize, imsize, num_channels])

network = tl.layers.InputLayer(x, name='input_layer')
network = tl.layers.DeConv2dLayer(network,
                            shape = [filter_size, filter_size, num_channels, num_channels],
                            output_shape = [1, imsize*scale, imsize*scale, num_channels],
                            strides=[1, scale, scale, 1],
                            W_init=bilinear_init,
                            padding='SAME',
                            act=tf.identity, name='g/h1/decon2d')
y=network.outputs
sess.run(tf.global_variables_initializer())

#Random test image
img = np.random.random_sample((imsize,imsize,num_channels))

deconv_result = sess.run(y, feed_dict={x: [img]})
#skimag
upsampled_skimage = skimage.transform.rescale(img,
                                     scale,
                                     mode='constant', #zero padding as in tf conv2d_transpose
                                     cval=0,
                                     order=1, #corresponds to bilinear upsampling
                                     preserve_range=False)

print(np.allclose(upsampled_skimage,deconv_result))

imsave('skim.png',upsampled_skimage)
imsave('deconv2d.png', np.squeeze(deconv_result))
```
@JoelKronander oh, great you did it.
for contribution, you can put it into layers.py ;  when we collect a lot of special initializer , we may create init.py 
Ok, I will submit a pull request adding a function to get such an initializer in layers.py.
Where would it be appropriate to submit a corresponding test case of the function?
At some point I could also look into submitting a FCN segmantic segmentation example.

Resolving patch:
diff --git a/tensorlayer/layers.py b/tensorlayer/layers.py
--- a/tensorlayer/layers.py
+++ b/tensorlayer/layers.py
@@ -1540,6 +1540,68 @@ def __init__(
         # self.all_drop = dict(layer.all_drop)
         # self.all_layers.extend( [self.outputs] )
 
+## Initializers for Convuolutional Layers
+def deconv2d_bilinear_upsampling_initializer(shape):
+    """Returns initializer that can be passed to DeConv2dLayer to initalize the
+    weights to correspond to channel wise bilinear upsampling.
+    Used in some segmantic segmentation approches such as [FCN](https://arxiv.org/abs/1605.06211)
+
+    Parameters
+    ----------
+        shape : list of shape
+            shape of the filters, [height, width, output_channels, in_channels], must match that passed to DeConv2dLayer
+
+    Returns
+    ----------
+        tf.constant_initializer
+            with weights set to correspond to per channel bilinear upsampling when passed as W_int in DeConv2dLayer
+
+    Examples
+    --------
+    >>> rescale_factor = 2 #upsampling by a factor of 2, ie e.g 100->200
+    >>> filter_size = (2 * rescale_factor - rescale_factor % 2) #Corresponding bilinear filter size
+    >>> num_in_channels = 3
+    >>> num_out_channels = 3
+    >>> deconv_filter_shape = [filter_size, filter_size, num_out_channels, num_in_channels]
+    >>> x = tf.placeholder(tf.float32, [1, imsize, imsize, num_channels])
+    >>> network = tl.layers.InputLayer(x, name='input_layer')
+    >>> bilinear_init = deconv2d_bilinear_upsampling_initializer(shape=filter_shape)
+    >>> network = tl.layers.DeConv2dLayer(network,
+                            shape = filter_shape,
+                            output_shape = [1, imsize*rescale_factor, imsize*rescale_factor, num_out_channels],
+                            strides=[1, rescale_factor, rescale_factor, 1],
+                            W_init=bilinear_init,
+                            padding='SAME',
+                            act=tf.identity, name='g/h1/decon2d')
+    """
+    if shape[0] != shape[1]:
+        raise Exception('deconv2d_bilinear_upsampling_initializer only supports symmetrical filter sizes')
+    if shape[3] < shape [2]:
+        raise Exception('deconv2d_bilinear_upsampling_initializer behaviour is not defined for num_in_channels < num_out_channels ')
+
+    filter_size = shape[0]
+    num_out_channels = shape[2]
+    num_in_channels = shape[3]
+
+    #Create bilinear filter kernel as numpy array
+    bilinear_kernel = np.zeros([filter_size, filter_size], dtype=np.float32)
+    scale_factor = (filter_size + 1) // 2
+    if filter_size % 2 == 1:
+        center = scale_factor - 1
+    else:
+        center = scale_factor - 0.5
+    for x in range(filter_size):
+        for y in range(filter_size):
+            bilinear_kernel[x,y] = (1 - abs(x - center) / scale_factor) * \
+                                   (1 - abs(y - center) / scale_factor)
+    weights = np.zeros((filter_size, filter_size, num_out_channels, num_in_channels))
+    for i in range(num_out_channels):
+        weights[:, :, i, i] = bilinear_kernel
+
+    #assign numpy array to constant_initalizer and pass to get_variable
+    bilinear_weights_init = tf.constant_initializer(value=weights, dtype=tf.float32)
+    return bilinear_weights_init
+
 ## Convolutional layer (Simplified)
 def Conv2d(net, n_filter=32, filter_size=(3, 3), strides=(1, 1), act = None,
         padding='SAME', W_init = tf.truncated_normal_initializer(stddev=0.02), b_init = tf.constant_initializer(value=0.0),


Test patch: