Utility function for calculating the shape of a conv output

username: Carson_McNeil
cooked: <p>Hello!<br>
Is there some utility function hidden somewhere for calculating the shape of the output tensor that would result from passing a given input tensor to (for example), a <code>nn.Conv2d</code> module?</p>
<p>To me this seems basic though, so I may be misunderstanding something about how pytorch is supposed to be used.</p>
<p>Use case:<br>
You have a (non-convolutional) custom module that needs to know the shape of its input in order to define its <code>nn.Parameter</code>s. I realize fully-convolutional architectures do not have this need, but I have a module that does. Your code is built so that the <code>Variable</code> passed to this module is the result of applying a <code>Conv2d</code> module to a tensor of known (constant) size. Given all the arguments to the <code>Conv2d</code> module (stride, kernel size, etc), the size of the output can be computed, but to my knowledge this function isn’t in pytorch.</p>
<p>Is this function just not in pytorch? (In which case I can implement it, for personal use, or submission) Or is my question based on a false premise?</p>

username: richard
cooked: <p><a href="https://pytorch.org/docs/master/generated/torch.nn.Conv2d.html#torch.nn.Conv2d" class="inline-onebox">Conv2d — PyTorch master documentation</a> describes the output size.</p>
<p><div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/2X/f/fab7651734d4000309a49736e88fc8e8b0bc2221.png" data-download-href="https://discuss.pytorch.org/uploads/default/fab7651734d4000309a49736e88fc8e8b0bc2221" title="image"><img src="https://discuss.pytorch.org/uploads/default/optimized/2X/f/fab7651734d4000309a49736e88fc8e8b0bc2221_2_690x151.png" alt="image" data-base62-sha1="zLWhWeiuB5RCHtTG7oOJ8BaL30d" width="690" height="151" srcset="https://discuss.pytorch.org/uploads/default/optimized/2X/f/fab7651734d4000309a49736e88fc8e8b0bc2221_2_690x151.png, https://discuss.pytorch.org/uploads/default/original/2X/f/fab7651734d4000309a49736e88fc8e8b0bc2221.png 1.5x, https://discuss.pytorch.org/uploads/default/original/2X/f/fab7651734d4000309a49736e88fc8e8b0bc2221.png 2x" data-dominant-color="F4F4F4"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">image</span><span class="informations">797×175 25.7 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>

username: richard
cooked: <p>I imagine if you wanted a function you could generate a random input, pass it to conv2d, and check the size of the output (this is definitely not efficient but works as a sanity check)</p>

username: Carson_McNeil
cooked: <p>Sure, so it’s not implemented yet then. I almost feel like it would be nice if this was just a method of the conv module: pass an input shape, and it returns the formula above.</p>

username: DuaneNielsen
cooked: <p>in case anyone else end’s up here…</p>
<pre><code class="lang-auto">"""
Utility function for computing output of convolutions
takes a tuple of (h,w) and returns a tuple of (h,w)
"""
def conv_output_shape(h_w, kernel_size=1, stride=1, pad=0, dilation=1):
    from math import floor
    if type(kernel_size) is not tuple:
        kernel_size = (kernel_size, kernel_size)
    h = floor( ((h_w[0] + (2 * pad) - ( dilation * (kernel_size[0] - 1) ) - 1 )/ stride) + 1)
    w = floor( ((h_w[1] + (2 * pad) - ( dilation * (kernel_size[1] - 1) ) - 1 )/ stride) + 1)
    return h, w
</code></pre>

username: MicPie
cooked: <p>Thank you very much for the code snippet!</p>
<p>I adapted it a little bit including a version for transposed convolutions:</p>
<pre><code class="lang-auto">def conv_output_shape(h_w, kernel_size=1, stride=1, pad=0, dilation=1):
    """
    Utility function for computing output of convolutions
    takes a tuple of (h,w) and returns a tuple of (h,w)
    """
    
    if type(h_w) is not tuple:
        h_w = (h_w, h_w)
    
    if type(kernel_size) is not tuple:
        kernel_size = (kernel_size, kernel_size)
    
    if type(stride) is not tuple:
        stride = (stride, stride)
    
    if type(pad) is not tuple:
        pad = (pad, pad)
    
    h = (h_w[0] + (2 * pad[0]) - (dilation * (kernel_size[0] - 1)) - 1)// stride[0] + 1
    w = (h_w[1] + (2 * pad[1]) - (dilation * (kernel_size[1] - 1)) - 1)// stride[1] + 1
    
    return h, w
</code></pre>
<pre><code class="lang-auto">def convtransp_output_shape(h_w, kernel_size=1, stride=1, pad=0, dilation=1):
    """
    Utility function for computing output of transposed convolutions
    takes a tuple of (h,w) and returns a tuple of (h,w)
    """
    
    if type(h_w) is not tuple:
        h_w = (h_w, h_w)
    
    if type(kernel_size) is not tuple:
        kernel_size = (kernel_size, kernel_size)
    
    if type(stride) is not tuple:
        stride = (stride, stride)
    
    if type(pad) is not tuple:
        pad = (pad, pad)
        
    h = (h_w[0] - 1) * stride[0] - 2 * pad[0] + kernel_size[0] + pad[0]
    w = (h_w[1] - 1) * stride[1] - 2 * pad[1] + kernel_size[1] + pad[1]
    
    return h, w
</code></pre>

username: sytelus
cooked: <p>I went ahead and decided to improve on <a class="mention" href="/u/micpie">@MicPie</a>’s solution little bit. Specifically, it now accepts 4-tuple in form of <code>((pad_up, pad_bottom), (pad_left, pad_right))</code> for padding to allow for asymmetric padding. The transposed convolution function supports dilation as well as output padding. Also added additional functions to compute padding for desired output shape.</p>
<p>Below is full code. Note that currently PytTorch Conv2D layer accepts only 2-tuple so it only allows symmetric padding. To use asymmetric padding, currently you must additional padding layer either usin <code>F.pad</code> or ZeroPadding layer. See <a href="https://github.com/pytorch/pytorch/issues/3867" rel="nofollow noopener">here</a> for more discussions.</p>
<pre><code class="lang-auto">import math

def num2tuple(num):
    return num if isinstance(num, tuple) else (num, num)

def conv2d_output_shape(h_w, kernel_size=1, stride=1, pad=0, dilation=1):
    h_w, kernel_size, stride, pad, dilation = num2tuple(h_w), \
        num2tuple(kernel_size), num2tuple(stride), num2tuple(pad), num2tuple(dilation)
    pad = num2tuple(pad[0]), num2tuple(pad[1])
    
    h = math.floor((h_w[0] + sum(pad[0]) - dilation[0]*(kernel_size[0]-1) - 1) / stride[0] + 1)
    w = math.floor((h_w[1] + sum(pad[1]) - dilation[1]*(kernel_size[1]-1) - 1) / stride[1] + 1)
    
    return h, w

def convtransp2d_output_shape(h_w, kernel_size=1, stride=1, pad=0, dilation=1, out_pad=0):
    h_w, kernel_size, stride, pad, dilation, out_pad = num2tuple(h_w), \
        num2tuple(kernel_size), num2tuple(stride), num2tuple(pad), num2tuple(dilation), num2tuple(out_pad)
    pad = num2tuple(pad[0]), num2tuple(pad[1])
    
    h = (h_w[0] - 1)*stride[0] - sum(pad[0]) + dialation[0]*(kernel_size[0]-1) + out_pad[0] + 1
    w = (h_w[1] - 1)*stride[1] - sum(pad[1]) + dialation[1]*(kernel_size[1]-1) + out_pad[1] + 1
    
    return h, w

def conv2d_get_padding(h_w_in, h_w_out, kernel_size=1, stride=1, dilation=1):
    h_w_in, h_w_out, kernel_size, stride, dilation = num2tuple(h_w_in), num2tuple(h_w_out), \
        num2tuple(kernel_size), num2tuple(stride), num2tuple(dilation)
    
    p_h = ((h_w_out[0] - 1)*stride[0] - h_w_in[0] + dilation[0]*(kernel_size[0]-1) + 1)
    p_w = ((h_w_out[1] - 1)*stride[1] - h_w_in[1] + dilation[1]*(kernel_size[1]-1) + 1)
    
    return (math.floor(p_h/2), math.ceil(p_h/2)), (math.floor(p_w/2), math.ceil(p_w/2))

def convtransp2d_get_padding(h_w_in, h_w_out, kernel_size=1, stride=1, dilation=1, out_pad=0):
    h_w_in, h_w_out, kernel_size, stride, dilation, out_pad = num2tuple(h_w_in), num2tuple(h_w_out), \
        num2tuple(kernel_size), num2tuple(stride), num2tuple(dilation), num2tuple(out_pad)
        
    p_h = -(h_w_out[0] - 1 - out_pad[0] - dialation[0]*(kernel_size[0]-1) - (h_w[0] - 1)*stride[0]) / 2
    p_w = -(h_w_out[1] - 1 - out_pad[1] - dialation[1]*(kernel_size[1]-1) - (h_w[1] - 1)*stride[1]) / 2
    
    return (math.floor(p_h/2), math.ceil(p_h/2)), (math.floor(p_w/2), math.ceil(p_w/2))
</code></pre>

username: Fengnan_Zhao
cooked: <p>Hi Guys, I recently jump on to the pytorch and also noticed the problem. Automatically working out the output shape is not only the requirement for Conv1d(2d).   In tensorflow,   once the shape of the first input layer is known,  TF can figure out output shape of following layers automatically.   Now it is 2021, just wondering if we have something in Pytorch too ?   thanks</p>

username: Joao_Pedro
cooked: <p>I agree! Having to update all shapes when you change a single layer is a major annoyance. Just moments ago I was trying to write code for a conv autoencoder in a more general way. My solution was to compute the output shape for a random tensor, inside the model initialization method.</p>

username: bkoyuncu
cooked: <p><a class="mention" href="/u/joao_pedro">@Joao_Pedro</a> Would you mind sharing it?</p>

username: Joao_Pedro
cooked: <p>I ended up using the “Lazy” modules, so PyTorch infers the tensor shapes when you first execute the forward method, just like Keras.</p>

username: TriKri
cooked: <p><a class="mention" href="/u/joao_pedro">@Joao_Pedro</a> Are there implementation of lazy modules in PyTorch, or did you implement the laziness yourself? I have recently implemented a lazy module myself, so I was wondering whether PyTorch maybe had support for this already, and in that case I should probably switch to using that.</p>

username: TriKri
cooked: <p>As the title to this thread suggests, it would be nice if PyTorch could provide a utility function for calculating the output shape of a <code>_ConvNd</code> layer given an input shape. Since PyTorch already must do this internally when <code>_ConvNd.forward</code> is called, I don’t think this should be too much of an issue to add to the API, so is this something that could be implemented? <img src="https://discuss.pytorch.org/images/emoji/apple/upside_down_face.png?v=12" title=":upside_down_face:" class="emoji" alt=":upside_down_face:" loading="lazy" width="20" height="20"></p>

username: ptrblck
cooked: <p>Lazy modules are implemented as described in <a href="https://discuss.pytorch.org/t/put-lazily-created-parameters-on-the-correct-device/160612/2">your other post</a>.</p>