How to create convnet for variable size input dimension images

username: kirk86
cooked: <p>Hey guys, I was wondering if it’s possible to create a convnet with variable size input images as training data. If so can someone provide a simple example on how to achieve sth like that. For instance in <code>tensorflow</code> I would go and simply define the input shape on the first conv layer as <code>(None, None, 3)</code>. Can we do sth like that in pytorch?</p>
<p>Thanks!</p>

username: smth
cooked: <p>it depends on what you want to do at the end.</p>
<p>If the convnet is for image classification (you want one output for an image, regardless of size), then you can use nn.AdaptiveAvgPooling right before the fully connected layers.</p>
<p>If you want dense classification (larger image = larger output), you can replace your nn.Linear layers at the end with nn.Conv2d with kernel size 1x1</p>

username: kirk86
cooked: <p>Hi Soumith , thanks a lot for the answer. Ultimately the goal is to do classification on image ROIs and since they are of different sizes that’s why I was asking. So for instance if we have a training loop like this:</p>
<pre><code class="lang-auto">for epoch in epochs:
    for sample in nb_samples:
        outputs = convnet(sample)
</code></pre>
<p>where each sample represents a batch and it actually is a list of images <code>[img_1, img_2,...,img_n]</code>. And, <code>img_1, img_2,...,img_n</code> represent ROIs extracted from the original images.</p>
<p>Would that work or do I need to specify sth beforehand in the convnet architecture to make it work with this kind of data?</p>
<p>I’m not quite sure about the benefits of <code>nn.AdaptiveAvgPooling</code> in this case. Would you mind elaborating a little bit. How could it help in terms of making the convnet deal with data where each sample has as variable size.</p>
<p>For instance:<br>
<code>img1.shape = 238, 126, 3</code><br>
<code>img2.shape = 68, 234, 3</code><br>
<code>img3.shape = 225, 98, 3</code><br>
<code>. . .</code><br>
<code>img_n = ...</code></p>

username: smth
cooked: <p>AdaptiveAvgPooling will take as input a variable sized image, and then downsample it to a fixed size. The way it does it is by adaptively changing the pooling filter size based on the input image size.</p>

username: kirk86
cooked: <p>Awesome. Thanks, much appreciated! <img src="//discuss.pytorch.org/images/emoji/apple/slight_smile.png?v=5" title=":slight_smile:" class="emoji" alt=":slight_smile:"></p>

username: zchrissirhcz
cooked: <p><a class="mention" href="/u/kirk86">@kirk86</a> Can you share your key code that loading different size input images?</p>
<p>I read the source code of dataloader, finding that <code>torch.stack()</code>  is used. This function expects all elements in the batch sequence the exactly same size. How do you handle that?</p>
<p><a class="mention" href="/u/smth">@smth</a><br>
If I set batch size to greater than 1, how can I use a dataloader of different size inputs?</p>
<p>Thanks.</p>

username: Yan_Wang
cooked: <p>Same problem here.  It would be much more convenient if we can do something like torch.stack with tensor of different sizes.</p>

username: jdhao
cooked: <p>as a workaround, for batchsize=1, you can manually accumulate loss, do the average, then backward and update weight.</p>

username: rbrigden
cooked: <p>Thanks for the workaround, Jiedong. Is there a more optimized method for GPU utilization? Calling .cuda() on each data example tensor is not improving training performance over the CPU alone.</p>

username: jon
cooked: <p>I might be missing something, but how does a kernel size of 1x1 solve this problem? How do we go from the 1x1 convolution of the preceding feature maps to a classification?</p>

username: ptrblck
cooked: <p>The number of kernels in the last conv layer (<code>out_channels</code>) will specify the number of classes, while the <code>1x1</code> kernel does not change the spatial size.<br>
Using this approach you can easily output the class logits for each pixel location e.g. in a segmentation use case.</p>

username: jon
cooked: <p>Thanks! But if we were trying to do simple classification of the entire image, then we would need some more logic here. In particular, if were trying to do NLL against say 10 classes, then perhaps one approach would be to average the values in each of the 10 feature maps of the last layer. I suppose this would be  the “obvious” spatial-dimension invariant simple classifier.</p>

username: Mylinda
cooked: <p>Hi smth! How should I do if the spatial size of the input for convolutional layers is the same but the depth size of the input is variable? Does the AdaptiveAvgPooling still work? Thanks!</p>