Given input size: (512x1x1). Calculated output size: (512x0x0). Output size is too small

FTravi: <p>Hello everyone,<br>
I’m new to torch/PyTorch, and I’m currently trying to translate a script in Lua + torch into Python + PyTorch. The script in question implements a visual search model from a paper, and it can be found <a href="https://github.com/kreimanlab/VisualSearchZeroShot/blob/master/IVSNtopdown_30_31_naturaldesign.lua" rel="noopener nofollow ugc">here</a>.<br>
The model that’s used is Caffe VGG16, but it’s loaded through torch.<br>
Since it’s visual search, there are two different nn used: one for the stimuli (the image to be explored) and one for the target (the object to be found). They’re both very similar: the target model only has one extra layer (MaxPool2d, with kernel size 2x2 and stride 2). Both wrap VGG16 into an <em>nn.Sequential</em> container.<br>
Here’s how they look:</p>
<pre><code class="lang-auto">Sequential(
  (0): Conv2d(3, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
  (1): ReLU(inplace=True)
  (2): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
  (3): ReLU(inplace=True)
  (4): MaxPool2d(kernel_size=2, stride=2, padding=0, dilation=1, ceil_mode=False)
  (5): Conv2d(64, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
  (6): ReLU(inplace=True)
  (7): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
  (8): ReLU(inplace=True)
  (9): MaxPool2d(kernel_size=2, stride=2, padding=0, dilation=1, ceil_mode=False)
  (10): Conv2d(128, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
  (11): ReLU(inplace=True)
  (12): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
  (13): ReLU(inplace=True)
  (14): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
  (15): ReLU(inplace=True)
  (16): MaxPool2d(kernel_size=2, stride=2, padding=0, dilation=1, ceil_mode=False)
  (17): Conv2d(256, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
  (18): ReLU(inplace=True)
  (19): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
  (20): ReLU(inplace=True)
  (21): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
  (22): ReLU(inplace=True)
  (23): MaxPool2d(kernel_size=2, stride=2, padding=0, dilation=1, ceil_mode=False)
  (24): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
  (25): ReLU(inplace=True)
  (26): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
  (27): ReLU(inplace=True)
  (28): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
  (29): ReLU(inplace=True)
  (30): MaxPool2d(kernel_size=2, stride=2, padding=0, dilation=1, ceil_mode=False)
)
</code></pre>
<p>In Python, I’m using torchvision to load the VGG16 model. Up to the point where the models are fed by the input (line 105 in the Lua script quoted above), I’ve succesfully recreated every step. Everything is precisely the same: the dimensions and data.<br>
However, I’ve encountered two different problems:<br>
-I had to cast the tensors to 32-bit floats before feeding it to the nn, a step that’s not necessary in the Lua script (where they use 64-bit float tensors).<br>
-The error described in the title.</p>
<p>Now, I know this error arises when you use an image that’s below the minimum size required by the model. And that’s precisely it: the stimuli model is fed a 224x224 image, which doesn’t raise any errors; the target model is fed a 28x28 image, which does raise an error.<br>
The catch is: it doesn’t raise any errors in the Lua script, even though it’s a 28x28 image.</p>
<p>I’ve seen <a href="https://discuss.pytorch.org/t/runtimeerror-given-input-size-256x1x1-calculated-output-size-256x0x0-output-size-is-too-small/79600">this thread</a>, where ptrblck mentions that wrapping a model into an <em>nn.Sequential</em> container may cause some problems.<br>
If anyone can give me a hint of why that’s so, I’d greatly appreciate it.</p>
<p>Here’s how I load the model and manipulate it (it’s not pretty):</p>
<pre><code class="lang-auto">import torchvision.models as models
import torch.nn as nn

# Load the model
model = models.vgg16(pretrained=True)
# Ignore first module since it's the net itself
layers = [module for module in model.modules()][1:]
stimuliLayers = list(layers[0])[:-1]
targetLayers = list(layers[0])
model_stimuli = nn.Sequential(*stimuliLayers)
model_target = nn.Sequential(*targetLayers)
</code></pre>

ptrblck: <p>As described in the linked post, the functional API calls would be dropped.<br>
In your case <a href="https://github.com/pytorch/vision/blob/ced96a0ca66e35ae61b64267e083fc3e3a62172a/torchvision/models/vgg.py#L51">this flatten operation</a> would be missing.</p>
<p>Your error is raised, since <code>model.modules()</code> will recursively return all modules. You can check it via <code>print(model.modules())</code>.<br>
<code>model.children()</code> would yield the submodules in their order of initialization.<br>
However, even using this approach would assign the first <code>nn.Sequential</code> module (called <code>.features</code> in the original VGG16 model) to <code>model_stimuli</code> and <code>model_target</code> and will thus still raise the shape mismatch error.<br>
Due to these shortcomings, I would thus only recommend to use <code>nn.Sequential</code> for very simple modules and not try to wrap other models inside it without checking the definition of the pretrained model first.</p>

FTravi: <p>Alright, thanks for the quick response! I’ll continue to look into it, there’s so much I need to learn.</p>

ptrblck: <p>Could you print the Lua models, which would show all modules, as I’m still unsure, how the small input of <code>28x28</code> could be used in the feature extractor of the VGG model?</p>

FTravi: <p>Sure.</p>
<p>This is how the model is loaded (having been previously downloaded from <a href="https://gist.github.com/ksimonyan/211839e770f7b538e2d8#file-readme-md" rel="noopener nofollow ugc">here</a> and using <a href="https://github.com/szagoruyko/loadcaffe" rel="noopener nofollow ugc">a library for loading Caffe models in Torch7</a>):</p>
<pre><code class="lang-auto">local cmodel = loadcaffe.load('Models/caffevgg16/VGG_ILSVRC_16_layers_deploy_old.prototxt', 'Models/caffevgg16/VGG_ILSVRC_16_layers_old.caffemodel', 'nn')
print(cmodel)
</code></pre>
<p>This is the output:</p>
<pre><code class="lang-auto">conv1_1: 64 3 3 3
conv1_2: 64 64 3 3
conv2_1: 128 64 3 3
conv2_2: 128 128 3 3
conv3_1: 256 128 3 3
conv3_2: 256 256 3 3
conv3_3: 256 256 3 3
conv4_1: 512 256 3 3
conv4_2: 512 512 3 3
conv4_3: 512 512 3 3
conv5_1: 512 512 3 3
conv5_2: 512 512 3 3
conv5_3: 512 512 3 3
fc6: 1 1 25088 4096
fc7: 1 1 4096 4096
fc8: 1 1 4096 1000
nn.Sequential {
  [input -&gt; (1) -&gt; (2) -&gt; (3) -&gt; (4) -&gt; (5) -&gt; (6) -&gt; (7) -&gt; (8) -&gt; (9) -&gt; (10) -&gt; (11) -&gt; (12) -&gt; (13) -&gt; (14) -&gt; (15) -&gt; (16) -&gt; (17) -&gt; (18) -&gt; (19) -&gt; (20) -&gt; (21) -&gt; (22) -&gt; (23) -&gt; (24) -&gt; (25) -&gt; (26) -&gt; (27) -&gt; (28) -&gt; (29) -&gt; (30) -&gt; (31) -&gt; (32) -&gt; (33) -&gt; (34) -&gt; (35) -&gt; (36) -&gt; (37) -&gt; (38) -&gt; (39) -&gt; (40) -&gt; output]
  (1): nn.SpatialConvolution(3 -&gt; 64, 3x3, 1,1, 1,1)
  (2): nn.ReLU
  (3): nn.SpatialConvolution(64 -&gt; 64, 3x3, 1,1, 1,1)
  (4): nn.ReLU
  (5): nn.SpatialMaxPooling(2x2, 2,2)
  (6): nn.SpatialConvolution(64 -&gt; 128, 3x3, 1,1, 1,1)
  (7): nn.ReLU
  (8): nn.SpatialConvolution(128 -&gt; 128, 3x3, 1,1, 1,1)
  (9): nn.ReLU
  (10): nn.SpatialMaxPooling(2x2, 2,2)
  (11): nn.SpatialConvolution(128 -&gt; 256, 3x3, 1,1, 1,1)
  (12): nn.ReLU
  (13): nn.SpatialConvolution(256 -&gt; 256, 3x3, 1,1, 1,1)
  (14): nn.ReLU
  (15): nn.SpatialConvolution(256 -&gt; 256, 3x3, 1,1, 1,1)
  (16): nn.ReLU
  (17): nn.SpatialMaxPooling(2x2, 2,2)
  (18): nn.SpatialConvolution(256 -&gt; 512, 3x3, 1,1, 1,1)
  (19): nn.ReLU
  (20): nn.SpatialConvolution(512 -&gt; 512, 3x3, 1,1, 1,1)
  (21): nn.ReLU
  (22): nn.SpatialConvolution(512 -&gt; 512, 3x3, 1,1, 1,1)
  (23): nn.ReLU
  (24): nn.SpatialMaxPooling(2x2, 2,2)
  (25): nn.SpatialConvolution(512 -&gt; 512, 3x3, 1,1, 1,1)
  (26): nn.ReLU
  (27): nn.SpatialConvolution(512 -&gt; 512, 3x3, 1,1, 1,1)
  (28): nn.ReLU
  (29): nn.SpatialConvolution(512 -&gt; 512, 3x3, 1,1, 1,1)
  (30): nn.ReLU
  (31): nn.SpatialMaxPooling(2x2, 2,2)
  (32): nn.View(-1)
  (33): nn.Linear(25088 -&gt; 4096)
  (34): nn.ReLU
  (35): nn.Dropout(0.500000)
  (36): nn.Linear(4096 -&gt; 4096)
  (37): nn.ReLU
  (38): nn.Dropout(0.500000)
  (39): nn.Linear(4096 -&gt; 1000)
  (40): nn.SoftMax
}
</code></pre>
<p>Once loaded, <em>model_stimuli</em> and <em>model_target</em> are built in the following way:</p>
<pre><code class="lang-auto">model_stimuli = nn.Sequential()
model_target = nn.Sequential()

for i = 1, 30 do    
	model_stimuli:add(cmodel:get(i))
end

for i = 1, 31 do      
	model_target:add(cmodel:get(i))
end
</code></pre>
<p>The output of <em>print(model_target)</em> is: (model_stimuli is precisely the same without layer <span class="hashtag">#31</span>):</p>
<pre><code class="lang-auto">nn.Sequential {
  [input -&gt; (1) -&gt; (2) -&gt; (3) -&gt; (4) -&gt; (5) -&gt; (6) -&gt; (7) -&gt; (8) -&gt; (9) -&gt; (10) -&gt; (11) -&gt; (12) -&gt; (13) -&gt; (14) -&gt; (15) -&gt; (16) -&gt; (17) -&gt; (18) -&gt; (19) -&gt; (20) -&gt; (21) -&gt; (22) -&gt; (23) -&gt; (24) -&gt; (25) -&gt; (26) -&gt; (27) -&gt; (28) -&gt; (29) -&gt; (30) -&gt; (31) -&gt; output]
  (1): nn.SpatialConvolution(3 -&gt; 64, 3x3, 1,1, 1,1)
  (2): nn.ReLU
  (3): nn.SpatialConvolution(64 -&gt; 64, 3x3, 1,1, 1,1)
  (4): nn.ReLU
  (5): nn.SpatialMaxPooling(2x2, 2,2)
  (6): nn.SpatialConvolution(64 -&gt; 128, 3x3, 1,1, 1,1)
  (7): nn.ReLU
  (8): nn.SpatialConvolution(128 -&gt; 128, 3x3, 1,1, 1,1)
  (9): nn.ReLU
  (10): nn.SpatialMaxPooling(2x2, 2,2)
  (11): nn.SpatialConvolution(128 -&gt; 256, 3x3, 1,1, 1,1)
  (12): nn.ReLU
  (13): nn.SpatialConvolution(256 -&gt; 256, 3x3, 1,1, 1,1)
  (14): nn.ReLU
  (15): nn.SpatialConvolution(256 -&gt; 256, 3x3, 1,1, 1,1)
  (16): nn.ReLU
  (17): nn.SpatialMaxPooling(2x2, 2,2)
  (18): nn.SpatialConvolution(256 -&gt; 512, 3x3, 1,1, 1,1)
  (19): nn.ReLU
  (20): nn.SpatialConvolution(512 -&gt; 512, 3x3, 1,1, 1,1)
  (21): nn.ReLU
  (22): nn.SpatialConvolution(512 -&gt; 512, 3x3, 1,1, 1,1)
  (23): nn.ReLU
  (24): nn.SpatialMaxPooling(2x2, 2,2)
  (25): nn.SpatialConvolution(512 -&gt; 512, 3x3, 1,1, 1,1)
  (26): nn.ReLU
  (27): nn.SpatialConvolution(512 -&gt; 512, 3x3, 1,1, 1,1)
  (28): nn.ReLU
  (29): nn.SpatialConvolution(512 -&gt; 512, 3x3, 1,1, 1,1)
  (30): nn.ReLU
  (31): nn.SpatialMaxPooling(2x2, 2,2)
}
</code></pre>
<p>To be honest, I’m still unsure about torch’s <em>get</em> method behaviour (I’m struggling with torch’s documentation); at first glance, I’d have thought it’s the same as doing the following in PyTorch:</p>
<pre><code class="lang-auto">model_stimuli = nn.Sequential(*list(model.features.children())[:-1])
model_target = nn.Sequential(*list(model.features.children()))
</code></pre>
<p>But, evidently, it’s not.<br>
(I’ve searched for more topics on this matter, such as <a href="https://discuss.pytorch.org/t/how-to-extract-features-of-an-image-from-a-trained-model/119/3">this one</a>, and I’m currently trying different variations of it.)</p>

ptrblck: <p>That’s also what I understand from the Lua code. However, since the feature extractor is used in almost the same way as in the original VGG model (the last layer is removed), it should also yield empty activations in the original code given the small input shape.</p>

FTravi: <p>Note that the model which uses 28x28 images as input has <strong>all</strong> the same features as the VGG16 model (all 31 layers of it).</p>
<p>I’m thinking now that the difference may lie in how the model is loaded. Check <em>loadcaffe</em>’s third parameter, which indicates how to load the model. In this case, the value is ‘nn’ (the other possible values are ‘ccn2’ and ‘cudnn’):</p>
<pre><code class="lang-auto">local cmodel = loadcaffe.load('Models/caffevgg16/VGG_ILSVRC_16_layers_deploy_old.prototxt', 'Models/caffevgg16/VGG_ILSVRC_16_layers_old.caffemodel', 'nn')
print(cmodel)
</code></pre>
<p>And, effectively, the whole model is wrapped inside an nn.Sequential, even the classification layers (layers 32-40):</p>
<pre><code class="lang-auto">nn.Sequential {
  [input -&gt; (1) -&gt; (2) -&gt; (3) -&gt; (4) -&gt; (5) -&gt; (6) -&gt; (7) -&gt; (8) -&gt; (9) -&gt; (10) -&gt; (11) -&gt; (12) -&gt; (13) -&gt; (14) -&gt; (15) -&gt; (16) -&gt; (17) -&gt; (18) -&gt; (19) -&gt; (20) -&gt; (21) -&gt; (22) -&gt; (23) -&gt; (24) -&gt; (25) -&gt; (26) -&gt; (27) -&gt; (28) -&gt; (29) -&gt; (30) -&gt; (31) -&gt; (32) -&gt; (33) -&gt; (34) -&gt; (35) -&gt; (36) -&gt; (37) -&gt; (38) -&gt; (39) -&gt; (40) -&gt; output]
  (1): nn.SpatialConvolution(3 -&gt; 64, 3x3, 1,1, 1,1)
  (2): nn.ReLU
  (3): nn.SpatialConvolution(64 -&gt; 64, 3x3, 1,1, 1,1)
  (4): nn.ReLU
  (5): nn.SpatialMaxPooling(2x2, 2,2)
  (6): nn.SpatialConvolution(64 -&gt; 128, 3x3, 1,1, 1,1)
  (7): nn.ReLU
  (8): nn.SpatialConvolution(128 -&gt; 128, 3x3, 1,1, 1,1)
  (9): nn.ReLU
  (10): nn.SpatialMaxPooling(2x2, 2,2)
  (11): nn.SpatialConvolution(128 -&gt; 256, 3x3, 1,1, 1,1)
  (12): nn.ReLU
  (13): nn.SpatialConvolution(256 -&gt; 256, 3x3, 1,1, 1,1)
  (14): nn.ReLU
  (15): nn.SpatialConvolution(256 -&gt; 256, 3x3, 1,1, 1,1)
  (16): nn.ReLU
  (17): nn.SpatialMaxPooling(2x2, 2,2)
  (18): nn.SpatialConvolution(256 -&gt; 512, 3x3, 1,1, 1,1)
  (19): nn.ReLU
  (20): nn.SpatialConvolution(512 -&gt; 512, 3x3, 1,1, 1,1)
  (21): nn.ReLU
  (22): nn.SpatialConvolution(512 -&gt; 512, 3x3, 1,1, 1,1)
  (23): nn.ReLU
  (24): nn.SpatialMaxPooling(2x2, 2,2)
  (25): nn.SpatialConvolution(512 -&gt; 512, 3x3, 1,1, 1,1)
  (26): nn.ReLU
  (27): nn.SpatialConvolution(512 -&gt; 512, 3x3, 1,1, 1,1)
  (28): nn.ReLU
  (29): nn.SpatialConvolution(512 -&gt; 512, 3x3, 1,1, 1,1)
  (30): nn.ReLU
  (31): nn.SpatialMaxPooling(2x2, 2,2)
  (32): nn.View(-1)
  (33): nn.Linear(25088 -&gt; 4096)
  (34): nn.ReLU
  (35): nn.Dropout(0.500000)
  (36): nn.Linear(4096 -&gt; 4096)
  (37): nn.ReLU
  (38): nn.Dropout(0.500000)
  (39): nn.Linear(4096 -&gt; 1000)
  (40): nn.SoftMax
}
</code></pre>
<p>Whereas PyTorch prints the model in the following way:</p>
<pre><code class="lang-auto">VGG(
  (features): Sequential(
    (0): Conv2d(3, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
    (1): ReLU(inplace=True)
    (2): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
    (3): ReLU(inplace=True)
    (4): MaxPool2d(kernel_size=2, stride=2, padding=0, dilation=1, ceil_mode=False)
    (5): Conv2d(64, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
    (6): ReLU(inplace=True)
    (7): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
    (8): ReLU(inplace=True)
    (9): MaxPool2d(kernel_size=2, stride=2, padding=0, dilation=1, ceil_mode=False)
    (10): Conv2d(128, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
    (11): ReLU(inplace=True)
    (12): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
    (13): ReLU(inplace=True)
    (14): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
    (15): ReLU(inplace=True)
    (16): MaxPool2d(kernel_size=2, stride=2, padding=0, dilation=1, ceil_mode=False)
    (17): Conv2d(256, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
    (18): ReLU(inplace=True)
    (19): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
    (20): ReLU(inplace=True)
    (21): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
    (22): ReLU(inplace=True)
    (23): MaxPool2d(kernel_size=2, stride=2, padding=0, dilation=1, ceil_mode=False)
    (24): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
    (25): ReLU(inplace=True)
    (26): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
    (27): ReLU(inplace=True)
    (28): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
    (29): ReLU(inplace=True)
    (30): MaxPool2d(kernel_size=2, stride=2, padding=0, dilation=1, ceil_mode=False)
  )
  (avgpool): AdaptiveAvgPool2d(output_size=(7, 7))
  (classifier): Sequential(
    (0): Linear(in_features=25088, out_features=4096, bias=True)
    (1): ReLU(inplace=True)
    (2): Dropout(p=0.5, inplace=False)
    (3): Linear(in_features=4096, out_features=4096, bias=True)
    (4): ReLU(inplace=True)
    (5): Dropout(p=0.5, inplace=False)
    (6): Linear(in_features=4096, out_features=1000, bias=True)
  )
)
</code></pre>
<p>Maybe you could shed some light on why this would make a difference, if it makes a difference at all. The only other difference I could find between the scripts was the casting of the input tensor to 32-bit, since the Caffe model accepts 64-bit tensors as input.</p>

ptrblck: <aside class="quote no-group" data-username="FTravi" data-post="7" data-topic="111071">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/ftravi/48/30499_2.png" class="avatar"> FTravi:</div>
<blockquote>
<p>Maybe you could shed some light on why this would make a difference, if it makes a difference at all.</p>
</blockquote>
</aside>
<p>I’m not sure what you mean.<br>
My argument is, that the feature extractor should yield an empty activation in both frameworks, if the model is created correctly and all parameters are equal.<br>
I guess Caffe/Lua might use another rounding approach in calculating the output shape, which could yield a <code>1x1</code> activation in the end, and you could check it manually in both frameworks.</p>
<p>I’m unfamiliar with the paper, but find it a bit strange, that the last pooling layer is removed from the stimuli model, which gets the larger inputs, and not from the target model. Feeding the small input to the stimuli model works fine:</p>
<pre data-code-wrap="python"><code class="lang-python">model_stimuli = nn.Sequential(*list(model.features.children())[:-1])
model_target = nn.Sequential(*list(model.features.children()))

out = model_stimuli(torch.randn(1, 3, 28, 28))
print(out.shape)
&gt; torch.Size([1, 512, 1, 1])
</code></pre>

FTravi: <p>You’re absolutely right, the model does work correctly if <em>model_target</em> leaves aside the last pooling layer!<br>
Apparently, it’s no mistake in the code either, as it’s documented in the paper (I_s is the search Image, and I_t is the target Image):<br>
<div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/b/1/b16213f79d2c101bf09a143635a3a8b950ffe3f1.jpeg" data-download-href="https://discuss.pytorch.org/uploads/default/b16213f79d2c101bf09a143635a3a8b950ffe3f1" title="visualsearchzeroshot-layers"><img src="https://discuss.pytorch.org/uploads/default/optimized/3X/b/1/b16213f79d2c101bf09a143635a3a8b950ffe3f1_2_690x190.jpeg" alt="visualsearchzeroshot-layers" data-base62-sha1="pjcDTgnVsyvrySVXJmIFscHUDPb" width="690" height="190" srcset="https://discuss.pytorch.org/uploads/default/optimized/3X/b/1/b16213f79d2c101bf09a143635a3a8b950ffe3f1_2_690x190.jpeg, https://discuss.pytorch.org/uploads/default/original/3X/b/1/b16213f79d2c101bf09a143635a3a8b950ffe3f1.jpeg 1.5x, https://discuss.pytorch.org/uploads/default/original/3X/b/1/b16213f79d2c101bf09a143635a3a8b950ffe3f1.jpeg 2x" data-dominant-color="DEDE61"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">visualsearchzeroshot-layers</span><span class="informations">980×270 103 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>
<p>Here’s Supplementary Figure 13, where it shows this configuration yielded the best results:<br>
<img src="https://discuss.pytorch.org/uploads/default/original/3X/6/6/6621326392053bd16f29c368937d10cb182f8ff9.jpeg" alt="visualsearchzeroshot-layersperformance" data-base62-sha1="eztPcDW6uISw6yc5XwMuKK6BWxr" width="586" height="433"></p>
<p>Maybe the mistake is in resizing the target image to 28x28, instead of 224x224? Even though it’s clearly in the script, I couldn’t find anywhere in the paper where it stated that the target was resized to 28x28.</p>
<p>Anyway, I can’t thank you enough for all the help you’ve given me. I marked your latest post as the solution.</p>