Expected stride to be a single integer value or a list?

pisymbol: <p>I am using ResNet18 to do some transfer learning for a project. I have been able to follow the <a href="https://pytorch.org/tutorials/beginner/transfer_learning_tutorial.html" rel="noopener nofollow ugc">Transfer Learning Tutorial</a> without any real issue (I am using a feed forward DNN for ‘fc’ with a final layer that classifies 102 categoires instead of 1000).</p>
<p>However, after my model is trained, I of course want to perform inference on some test data (images). The problem is when I try to do something like this:</p>
<blockquote>
<p># img is a [3,224,224] Tensor just like all the other images during training<br>
img = process_image(image_path)<br>
model.eval()<br>
with torch.no_grad():<br>
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;outputs = model(img) # Note that wrapping this in a Variable() results in the samething</p>
</blockquote>
<p>It throws a somewhat cryptic exception:</p>
<blockquote>
<p>RuntimeError: expected stride to be a single integer value or a list of 1 values to match the convolution dimensions, but got stride=[2, 2]</p>
</blockquote>
<p>Does anyone know what I’m doing wrong or how I can perform inference using ResNet18?</p>

ptrblck: <p>From your code sample it looks like the batch dimension is missing.<br>
Try to add <code>img.unsqueeze_(0)</code> to your code before passing the image to the model.</p>

pisymbol: <p>That worked! Wow! So can you explain to me the “unsqueeze_” part? Sorry, I’m new to PyTorch.</p>

ptrblck: <p><code>tensor.unsqueeze_()</code> adds a dimension of size one at the specified position.<br>
If you pass the argument <code>dim=0</code> to the function, it will add a new dimension at position 0.<br>
Here is a smal example:</p>
<pre><code class="lang-auto">x = torch.randn(10, 10)
y = torch.unsqueeze(x, 0)
print(y.shape)
&gt; torch.Size([1, 10, 10])
</code></pre>
<p>As you can see <code>y</code> now has an additional dimension.<br>
You can also use the inplace method on <code>x</code> directly. Inplace methods are used with an underscore at the end of the function:</p>
<pre><code class="lang-auto">x = torch.randn(10, 10)
x.unsqueeze_(0)
print(x.shape)
&gt; torch.Size([1, 10, 10])
</code></pre>
<p>In your example, you tried to feed a single image into your model.<br>
The model (or <code>Modules</code> in general) need input with a batch dimension at position 0.<br>
If you use a mini batch of images, it should work:</p>
<pre><code class="lang-auto">x = torch.randn(10, 3, 224, 224)
model = nn.Conv2d(3, 6, 3, 1, 1)
output = model(x)
</code></pre>
<p>However, if you just load a single image, you have to add the batch dimension, if it’s not already there.<br>
Have a look at the following example:</p>
<pre><code class="lang-auto">x_single = torch.randn(3, 224, 224)
output = model(x_single) # RuntimeError

x_single.unsqueeze_(0)
print(x_single.shape)
output = model(x_single) # Works now!
</code></pre>

pisymbol: <p><a class="mention" href="/u/ptrblck">@ptrblck</a> Thank you so you much. I wish the exception thrown was a bit more reflective of “you need to add a batch size” or something like that. But I get it.</p>

ptrblck: <p>Yeah, you are right. The error is quite misleading.</p>

pisymbol: <p>The problem was I was so focused on making sure my image was properly tensorized I didn’t think to actually look at the API and recognize it wants a batch size (would be nice too if the API defaulted to a batch size of one for inference too - I don’t know, I think forcing the squeeze call is a bit strange).</p>
<p>Anyway, thanks <span class="mention">@ptrbick</span>. I’m getting it, really… <img src="https://discuss.pytorch.org/images/emoji/apple/slight_smile.png?v=5" title=":slight_smile:" class="emoji" alt=":slight_smile:"></p>

ptrblck: <p>Could you tell me, which PyTorch version you are using?<br>
You can see it with: <code>print(torch.__version__)</code>.</p>

pisymbol: <p>The answer to your question (for forum word minimums):</p>
<p>0.4.0</p>

mathematics: <p>Hi<br>
I have used just that and printed the result</p>
<p><strong>Image type:  torch.FloatTensor</strong><br>
<strong>shape without unsqueezing:  torch.Size([3, 10, 10])</strong><br>
<strong>After unsqueezing:  torch.Size([1, 3, 10, 10])</strong></p>
<p>I was using VGG16 to detect person face jpg , as<br>
Before unsqueezing it was giving error like</p>
<blockquote>
<p>RuntimeError: expected stride to be a single integer value or a list of 1 values to match the convolution dimensions, but got stride=[1, 1]</p>
</blockquote>
<p>after using  unsqueezeed image to  model as<br>
<strong>vgg16(img.unsqueeze_(0))</strong></p>
<p>again thrown error like this</p>
<blockquote>
<p>RuntimeError: Given input size: (512x1x1). Calculated output size: (512x0x0). Output size is too small at /opt/conda/conda-bld/pytorch_1524584710464/work/aten/src/THNN/generic/SpatialDilatedMaxPooling.c:67</p>
</blockquote>
<p>My code snippet as follow</p>
<details>
<summary>
Summary</summary>
<pre><code class="lang-auto">    img_to_tensor =  transforms.Compose([
        transforms.CenterCrop(10),
        transforms.ToTensor(),
        transforms.Normalize((0.485, 0.456, 0.406), (0.229, 0.224, 0.225))
    ])
    img_pil = Image.open(img_path)
    img = img_to_tensor(img_pil)
#     print("Image type: ", img.type())
#     print("shape without unsqueezing: ", img.shape)
#     print("After unsqueezing: ",img.unsqueeze_(0).shape)
    prediction = VGG16(img.unsqueeze_(0))
    ## Return the *index* of the predicted class for that image
    

    return prediction # predicted class index
</code></pre>
</details>
<p>Please Help!!! .</p>

mathematics: <p><a class="mention" href="/u/ptrblck">@ptrblck</a> , I have found solution. Sorry, if i mentioned same error. <img src="https://discuss.pytorch.org/images/emoji/apple/slight_smile.png?v=9" title=":slight_smile:" class="emoji" alt=":slight_smile:">  .<br>
but only changing to <strong>img.unsqueeze_(0)</strong> to  <strong>img.unsqueeze(0)</strong> , vanishes all errors.</p>

ptrblck: <p>That’s strange, as the last error points to a too small spatial size of your input.<br>
I.e. if you are using convs and pooling layers, which decrease your spatial size, you might end up with an empty tensor at one point, which will raise the issue.</p>

mathematics: <p>But suddenly <strong>img.unsqueeze_(0)</strong> worked, just i tried line. Things are not getting in my head, probably because of my neurons are not trained yet in leanring pytorch.</p>
<p>I was using VGG16 model to predict dogs as assignments. I just tried both it all worked.</p>
<details>
<summary>
Summary</summary>
<p>def VGG16_predict(img_path):<br>
‘’’<br>
Use pre-trained VGG-16 model to obtain index corresponding to<br>
predicted ImageNet class for image at specified path</p>
<pre><code>Args:
    img_path: path to an image
    
Returns:
    Index corresponding to VGG-16 model's prediction
'''

## TODO: Complete the function.
## Load and pre-process an image from the given img_path
img_to_tensor =  transforms.Compose([
    transforms.CenterCrop(224),
    transforms.ToTensor(),
    transforms.Normalize((0.485, 0.456, 0.406), (0.229, 0.224, 0.225))
])
img_pil = Image.open(img_path)
img = img_to_tensor(img_pil)
print("Image type: ", img.type())
print("shape without unsqueezing: ", img.shape)
print("After unsqueezing: ",img.unsqueeze_(0).shape)
prediction = VGG16(img.unsqueeze_(0))
## Return the *index* of the predicted class for that image


return prediction.data.numpy().argmax() # predicted class index
</code></pre>
</details>
<p>Oh sorry, I found tha problem happend previously that i was using<br>
<strong>transforms.CenterCrop(40)</strong> previously , thats why it was giving errr.<br>
When i change to  <strong>transforms.CenterCrop(224)</strong> , It all worked</p>
<p>Thank you for helping <img src="https://discuss.pytorch.org/images/emoji/apple/slight_smile.png?v=9" title=":slight_smile:" class="emoji" alt=":slight_smile:"></p>

parna_eshraghi: <p>Hi<br>
class gdl3d_loss(nn.Module):</p>
<pre><code>def __init__(self, pNorm=2):

    super(gdl3d_loss, self).__init__()

    self.convX = nn.Conv3d(1, 1, kernel_size=(3,3), stride=1, padding=1, bias=False)

    self.convY = nn.Conv3d(1, 1, kernel_size=(3,3), stride=1 ,padding=1, bias=False)

    self.convZ = nn.Conv3d(1, 1, kernel_size=(3,3), stride=1, padding=1 ,bias=False)
</code></pre>
<h1>filterX = torch.FloatTensor([[[[-1, 1]]]])  # 1x2</h1>
<pre><code>    filter_x=np.array([[1, 0, -1],[2,0,-2],[1,0,-1]])

    print(filter_x.shape)

    print(filter_x.size)

    self.convX.weight=nn.Parameter(torch.from_numpy(filter_x).float().unsqueeze(0).unsqueeze(0))

    filter_y=np.array([[1, 2, 1],[0,0,0],[-1,-2,-1]])

    self.convY.weight=nn.Parameter(torch.from_numpy(filter_y).float().unsqueeze(0).unsqueeze(0))

    filter_z=np.array([[-1, -2, 1],[0,0,0],[1,2,1]])

    self.convZ.weight=nn.Parameter(torch.from_numpy(filter_z).float().unsqueeze(0).unsqueeze(0))

    self.pNorm = pNorm

def forward(self, pred, gt):

    assert not gt.requires_grad

    assert pred.size() == 5

    assert gt.size() == 5

    assert pred.size() == gt.size(), "{0} vs {1} ".format(pred.size(), gt.size())

    print(pred.shape)

    print(gt.shape)

    pred_dx = torch.abs(self.convX(pred))

    pred_dy = torch.abs(self.convY(pred))

    pred_dz = torch.abs(self.convZ(pred))

    gt_dx = torch.abs(self.convX(gt))

    gt_dy = torch.abs(self.convY(gt))

    gt_dz = torch.abs(self.convZ(gt))

    

    grad_diff_x = torch.abs(gt_dx - pred_dx)

    grad_diff_y = torch.abs(gt_dy - pred_dy)

    grad_diff_z = torch.abs(gt_dz - pred_dz)

    

    mat_loss_x = grad_diff_x ** self.pNorm

    

    mat_loss_y = grad_diff_y ** self.pNorm  # Batch x Channel x width x height

    mat_loss_z = grad_diff_z ** self.pNorm

    shape = gt.shape

    mean_loss = (torch.sum(mat_loss_x) + torch.sum(mat_loss_y)+torch.sum(mat_loss_z)) / (shape[0] * shape[1] * shape[2] * shape[3]) 

          
    return mean_loss
</code></pre>
<p>I  have this class but I am getting error when I  run  gdl3d(outputG,ct_train_batch)<br>
The torch dimensions are (10,1,64,64,64) I am getting following error<br>
RuntimeError: expected stride to be a single integer value or a list of 2 values to match the convolution dimensions, but got stride=[1, 1, 1]</p>

parna_eshraghi: <p>Could you please help me with that</p>

ptrblck: <p>The error message is a bit misleading, but points to the wrong definition of your kernel size.<br>
Pass the size as a tuple of 3 values <code>kernel_size=(3, 3, 3)</code> or as a single value <code>kernel_size=3</code> and it should work.</p>

parna_eshraghi: <p>Thank you so much , I fixed that , but I still getting same error , do you think the conv.weight size might be wrong ? I just wrote this function  gradient loss function for medical imaging in order to get less blurry image ( I am generating CT from MRI)<br>
I appreciate if you help me</p>

ptrblck: <p>How did you fix it, if you are getting the same error? <img src="https://discuss.pytorch.org/images/emoji/apple/slight_smile.png?v=9" title=":slight_smile:" class="emoji" alt=":slight_smile:"><br>
This code shows the issue and the proposed solution:</p>
<pre><code class="lang-python">
x = torch.randn(1, 1, 24, 24, 24)
conv = nn.Conv3d(1, 1, kernel_size=(3,3), stride=1, padding=1, bias=False)
out = conv(x) # breaks
conv = nn.Conv3d(1, 1, kernel_size=(3, 3, 3), stride=1, padding=1, bias=False)
out = conv(x) # works
</code></pre>

parna_eshraghi: <p>Hi I mean yes I changed it to (3,3,3) and I was still getting same error, I guess it due to conv.weights  dimension<br>
Thnaks</p>

ptrblck: <p>In that case, could you post a minimal, executable code snippet, which yields this error so that we could debug it, please?</p>