Repository: Theano/Theano

Issue:
Add `conv2d_transpose` method to `theano.tensor.nnet.abstract_conv`
... and make it available in `theano.tensor.nnet`.
The semantics and name of the parameters was discussed IRL with @vdumoulin and @fvisin .

This is a follow-up to #3681 and #3872.



Discussion:
 I used the code below to install theano：
`pip install git+git://github.com/Theano/Theano.git`
But there is no thenao.tensor.nnet.conv2d_transpose() method.

IIRC you should create a new `conv2d_transpose` op in `theano.tensor.nnet.abstract_conv`, which will be an alias of `theano.tensor.nnet.abstract_conv.conv2d_grad_wrt_inputs` with more user-friendly parameter names. Then, you should make this new op available in `theano.tensor.nnet`.

@meanmee: this is an open issue, meaning it is not done yet. Someone has been assigned to it, though, so it should become available soon.

Oh sorry, I thought you had been assigned to this issue.

In order to design the interface of **Theano**'s `conv2d_transpose` op, I had a look at **TensorFlow**'s `conv2d_transpose` interface:

#### `tf.nn.conv2d_transpose(value, filter, output_shape, strides, padding='SAME', name=None)`

* `value`: A 4-D `Tensor` of type `float` and shape `[batch, height, width, in_channels]`.
* `filter`: A 4-D `Tensor` with the same type as `value` and shape
  `[height, width, output_channels, in_channels]`. `filter`'s `in_channels` dimension must
  match that of `value`.
* `output_shape`: A 1-D `Tensor` representing the output shape of the deconvolution op.
* `strides`: A list of `int`. The stride of the sliding window for each dimension of the input tensor.
* `padding`: A string, either `'VALID'` or `'SAME'`. The padding algorithm.
* `name`: Optional name for the returned tensor.

I think it makes sense to follow the same interface design and to adapt some of the names to be consistent with **Theano's** `conv2d` interface. In particular, having users explicitly specify the output shape disambiguates the case where strides are greater than one.

I propose the following interface:

#### `theano.tensor.nnet.conv2d_transpose(input, filters, output_shape, input_shape=None, filter_shape=None, border_mode='valid', subsample=(1, 1), filter_flip=True, filter_dilation=(1, 1))`

* `input`: Symbolic 4D tensor, mini-batch of feature map stacks, of shape
  `(batch size, input channels, input rows, input columns)`.
* `filters`: Symbolic 4D tensor, set of filters used in CNN layer of shape
  `(input channels, output channels, filter rows, filter columns)`. **Note: the order for
  `output channels` and `input channels` is reversed with respect to `conv2d`.**
* `output_shape`: `tuple` representing the shape of the output of `conv2d_transpose`.
* `input_shape`: ...
* `filter_shape`: ...
* `border_mode`: ...
* `subsample`: ...
* `filter_flip`: ...
* `filter_dilation`: ...
It makes sense to me, this function should (as much as possible) be centered on the transposed convolution itself (in terms of what the input and output are, for instance).

I think the following parameters also need some clarifications:
- `subsample`: it means that the original (non-transposed) convolution was doing some subsampling. So `conv2d_transpose` actually does something like "input dilation". TensorFlow names it "strides". I would either keep `subsample` with a warning, or rename it `input_dilation` and say that it corresponds to `subsample` in the non-transposed convolution.
- `border_mode`: what was "padding" for the forward convolution means "cropping" the output of the transposed one. "valid" corresponds to no cropping, "full" to maximal cropping.

Resolving patch:
diff --git a/theano/tensor/nnet/__init__.py b/theano/tensor/nnet/__init__.py
--- a/theano/tensor/nnet/__init__.py
+++ b/theano/tensor/nnet/__init__.py
@@ -32,6 +32,7 @@
 
 import warnings
 from .abstract_conv import conv2d as abstract_conv2d
+from .abstract_conv import conv2d_grad_wrt_inputs
 from .abstract_conv import conv3d
 
 
@@ -151,3 +152,86 @@ def conv2d(input, filters, input_shape=None, filter_shape=None,
     return abstract_conv2d(input, filters, input_shape, filter_shape,
                            border_mode, subsample, filter_flip,
                            filter_dilation)
+
+
+def conv2d_transpose(input, filters, output_shape, filter_shape=None,
+                     border_mode='valid', input_dilation=(1, 1),
+                     filter_flip=True, filter_dilation=(1, 1)):
+    """
+    This function will build the symbolic graph for applying a transposed
+    convolution over a mini-batch of a stack of 2D inputs with a set of 2D
+    filters.
+
+
+    Parameters
+    ----------
+    input: symbolic 4D tensor
+        Mini-batch of feature map stacks, of shape
+        (batch size, input channels, input rows, input columns).
+        See the optional parameter ``input_shape``.
+
+    filters: symbolic 4D tensor
+        Set of filters used in CNN layer of shape
+        (input channels, output channels, filter rows, filter columns).
+        See the optional parameter ``filter_shape``. **Note: the order for
+        ``output_channels`` and ``input_channels`` is reversed with respect to
+        ``conv2d``.**
+
+    output_shape: tuple/list of len 4 of int or Constant variable
+        The shape of the output of ``conv2d_transpose``. The last two elements
+        are allowed to be ``tensor.scalar`` variables.
+
+    filter_shape: None, tuple/list of len 4 of int or Constant variable
+        The shape of the filters parameter.
+        Optional, possibly used to choose an optimal implementation.
+        You can give ``None`` for any element of the list to specify that this
+        element is not known at compile time.
+
+    border_mode: str, int or tuple of two int
+        Refers to the ``border_mode`` argument of the corresponding forward
+        (non-transposed) convolution. See the argument description in
+        ``conv2d``.  What was ``padding`` for the forward convolution means
+        ``cropping`` the output of the transposed one. ``valid`` corresponds to
+        no cropping, ``full`` to maximal cropping.
+
+    input_dilation: tuple of len 2
+        Corresponds to ``subsample`` (also called strides elsewhere) in the
+        non-transposed convolution.
+
+    filter_flip: bool
+        If ``True``, will flip the filter rows and columns
+        before sliding them over the input. This operation is normally referred
+        to as a convolution, and this is the default. If ``False``, the filters
+        are not flipped and the operation is referred to as a cross-correlation.
+
+    filter_dilation: tuple of len 2
+        Factor by which to subsample (stride) the input.
+        Also called dilation elsewhere.
+
+    Returns
+    -------
+    Symbolic 4D tensor
+        Set of feature maps generated by the transposed convolution. Tensor is
+        of shape (batch size, output channels, output rows, output columns)
+
+    Notes
+    -----
+        If cuDNN is available, it will be used on the
+        GPU. Otherwise, it is the *CorrMM* convolution that will be used
+        "caffe style convolution".
+
+        This operation is also sometimes called "deconvolution".
+
+        The parameter filter_dilation is an implementation of `dilated
+        convolution <https://arxiv.org/pdf/1511.07122v3.pdf>`_.
+
+    """
+
+    return conv2d_grad_wrt_inputs(output_grad=input,
+                                  filters=filters,
+                                  input_shape=output_shape,
+                                  filter_shape=filter_shape,
+                                  border_mode=border_mode,
+                                  subsample=input_dilation,
+                                  filter_flip=filter_flip,
+                                  filter_dilation=filter_dilation)


Test patch:
diff --git a/theano/gpuarray/tests/test_abstractconv.py b/theano/gpuarray/tests/test_abstractconv.py
--- a/theano/gpuarray/tests/test_abstractconv.py
+++ b/theano/gpuarray/tests/test_abstractconv.py
@@ -252,3 +252,7 @@ def setUp(self):
         self.constant_tensor = gpuarray.array(
             numpy.zeros((3, 5, 7, 11), dtype='float32'),
             context=get_context(test_ctx_name))
+
+
+class TestConv2dTranspose(test_abstract_conv.TestConv2dTranspose):
+    mode = mode_with_gpu
diff --git a/theano/tensor/nnet/tests/test_abstract_conv.py b/theano/tensor/nnet/tests/test_abstract_conv.py
--- a/theano/tensor/nnet/tests/test_abstract_conv.py
+++ b/theano/tensor/nnet/tests/test_abstract_conv.py
@@ -9,7 +9,8 @@
 from theano import tensor
 from theano.gof.opt import check_stack_trace
 from theano.tests import unittest_tools as utt
-from theano.tensor.nnet import corr, corr3d, abstract_conv as conv
+from theano.tensor.nnet import (corr, corr3d, conv2d_transpose,
+                                abstract_conv as conv)
 from theano.tensor.nnet.abstract_conv import (get_conv_output_shape,
                                               get_conv_gradweights_shape,
                                               get_conv_gradinputs_shape,
@@ -1548,3 +1549,40 @@ def test_compare_1D_and_2D_upsampling_values(self):
         f_1D = theano.function([], mat_1D, mode=self.compile_mode)
         f_2D = theano.function([], mat_2D, mode=self.compile_mode)
         utt.assert_allclose(f_1D(), f_2D(), rtol=1e-06)
+
+
+class TestConv2dTranspose(unittest.TestCase):
+    mode = None
+
+    def test_interface(self):
+        """Test conv2d_transpose wrapper.
+
+        This method tests that the order of the filter's
+        axes expected by the function produces the correct
+        output shape.
+
+        """
+        mode = self.mode
+        if theano.config.mode == "FAST_COMPILE":
+            mode = theano.compile.get_mode(
+                mode).excluding("conv_gemm").excluding("AbstractConvCheck")
+
+        output = theano.function(
+            inputs=[],
+            outputs=conv2d_transpose(input=tensor.ones((2, 2, 4, 4)),
+                                     filters=tensor.ones((2, 1, 4, 4)),
+                                     output_shape=(2, 1, 10, 10),
+                                     input_dilation=(2, 2)),
+            mode=mode)()
+        expected_output = numpy.array(
+            [[[[2, 2, 4, 4, 4, 4, 4, 4, 2, 2],
+               [2, 2, 4, 4, 4, 4, 4, 4, 2, 2],
+               [4, 4, 8, 8, 8, 8, 8, 8, 4, 4],
+               [4, 4, 8, 8, 8, 8, 8, 8, 4, 4],
+               [4, 4, 8, 8, 8, 8, 8, 8, 4, 4],
+               [4, 4, 8, 8, 8, 8, 8, 8, 4, 4],
+               [4, 4, 8, 8, 8, 8, 8, 8, 4, 4],
+               [4, 4, 8, 8, 8, 8, 8, 8, 4, 4],
+               [2, 2, 4, 4, 4, 4, 4, 4, 2, 2],
+               [2, 2, 4, 4, 4, 4, 4, 4, 2, 2]]]] * 2)
+        numpy.testing.assert_equal(output, expected_output)