Learning with different sizes

username: Mrig
cooked: <p>I wanted to create an ensemble learning model, where I retrain the same custom model over the different size of the image.  Is it possible?</p>

username: ptrblck
cooked: <p>If your model is flexible regarding the input shapes, it would be possible.<br>
E.g. for a CNN you could use an adaptive pooling layer before flattening the activation and passing it to the linear layer to accept more flexible spatial shapes.</p>
<p>I’m not sure, if I understood your use case correctly, so please feel free to add more information. <img src="https://discuss.pytorch.org/images/emoji/apple/slight_smile.png?v=9" title=":slight_smile:" class="emoji" alt=":slight_smile:"></p>

username: Mrig
cooked: <p>Thanks for the quick reply, actually i am a keras user trying to switch to pytorch.<br>
I was using a keras model using Xception pretrained model</p>
<pre><code class="lang-auto">base_model0 = Xception(weights='imagenet',include_top=False,input_shape=(None,None,3))
base_model1 = Xception(weights='imagenet',include_top=False,input_shape=(None,None,3))
base_model2 = Xception(weights='imagenet',include_top=False,input_shape=(None,None,3))
base_model3 = Xception(weights='imagenet',include_top=False,input_shape=(None,None,3))
x0 = base_model0.output
x0 = layers.GlobalAveragePooling2D()(x0)
x1 = base_model1.output
x1 = layers.GlobalAveragePooling2D()(x1)
x2 = base_model2.output
x2 = layers.GlobalAveragePooling2D()(x2)
x3 = base_model3.output
x3 = layers.GlobalAveragePooling2D()(x3)
x = layers.concatenate([x0,x1,x2,x3])
x = layers.Dense(4,activation='sigmoid')(x)
model = Model(inputs=(base_model0.input, base_model1.input, base_model2.input, 
    base_model3.input), outputs=x)
</code></pre>
<p>each Xception base model has (NONE, NONE,3) shape signifying any shape of images be inserted,<br>
where NONE, NONE signifies height and width and ‘3’ the no. of channels.<br>
then each result of the layers go through Global average pooling and are concatenated to give an output.<br>
Although the input shapes were different but when it enters the Xception the output shape is same before going to the  Global average pooling .<br>
I just wanted to know if I could implement with Pytorch.</p>

username: ptrblck
cooked: <p>I don’t know, how the <code>Xception</code> model is implemented in Keras or what <code>GlobalAveragePooling2D</code> does (is it returning a single output scalar for each sample?).<br>
However, if you are using adaptive pooling layers (which define the output size instead of the kernel size), your spatial input shapes would be flexible.<br>
<a href="https://github.com/pytorch/vision/blob/ab73b44887eb9d7566b2a80fd45cb5f68b11d643/torchvision/models/resnet.py#L213">This line</a> shows the usage for the <code>ResNet</code> implementation.</p>
<p>Since PyTorch does not use any placeholder variables, you don’t have to set <code>None</code> for specific shapes.</p>

username: Mrig
cooked: <p>Thanks for the clarification.<br>
Are average pooling(in the example of your given link) and adaptive pooling the same?<br>
Is there some example codes or links where adaptive layer was first used to make it to desirable shape and then pass it in the model.</p>

username: ptrblck
cooked: <p>Average and adaptive pooling can be combined and these names do not refer to the same layer.<br>
In the linked example, an <a href="https://pytorch.org/docs/stable/nn.html#torch.nn.AdaptiveAvgPool2d"><code>nn.AdaptiveAvgPool2d</code></a> layer is used. The docs give you an example of how to use it, but let me know, if you get stuck.</p>

username: Mrig
cooked: <p>Hi again, As you said I have to do adaptive pooling before sending it to the network.<br>
So for example as VGG16 require input images to be of size 224X224X3, but we want to train on image on different size ( example 1024X1024X3)<br>
Should the code look like this?</p>
<pre><code class="lang-auto">model = models.vgg16(pretrained=False)
first_layer = [nn.AdaptiveMaxPool2d((224,224))]
first_layer.extend(list(model.features))  
model.features= nn.Sequential(*first_layer )
</code></pre>
<p>Also now I have to freeze my model from 2nd layer, and I only need to train the 1st layer and the classification part, how to do that?</p>

username: ptrblck
cooked: <aside class="quote no-group" data-username="Mrig" data-post="7" data-topic="89411">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/mrig/48/14538_2.png" class="avatar"> Mrig:</div>
<blockquote>
<p>As you said I have to do adaptive pooling before sending it to the network.<br>
So for example as VGG16 require input images to be of size 224X224X3, but we want to train on image on different size ( example 1024X1024X3)<br>
Should the code look like this?</p>
</blockquote>
</aside>
<p>No, you don’t need to add the adaptive pooling layer at the beginning of the model.<br>
<code>vgg16</code> uses this layer internally before feeding the activation to the linear layer in <a href="https://github.com/pytorch/vision/blob/ac3ba94440af8eaafc3c21353006b5d0fc9aed2a/torchvision/models/vgg.py#L44">this line of code</a>.</p>
<p>You can just use different shapes:</p>
<pre data-code-wrap="python"><code class="lang-python">model = models.vgg16()

x = torch.randn(1, 3, 224, 224)
out = model(x)

x = torch.randn(1, 3, 1024, 1024)
out = model(x)
</code></pre>
<aside class="quote no-group" data-username="Mrig" data-post="7" data-topic="89411">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/mrig/48/14538_2.png" class="avatar"> Mrig:</div>
<blockquote>
<p>Also now I have to freeze my model from 2nd layer, and I only need to train the 1st layer and the classification part, how to do that?</p>
</blockquote>
</aside>
<p>That wouldn’t be necessary, as pooling layers don’t have trainable parameters.</p>