Autograd failed after add max_pool1d layer

Nanachi: <h3><a name="update-the-problem-is-solved-1" class="anchor" href="#update-the-problem-is-solved-1"></a>Update, the problem is solved.</h3>
<p>if you encounter with the same issue, try using .contiguous() as following example from <a class="mention" href="/u/ptrblck">@ptrblck</a></p>
<aside class="quote no-group" data-username="ptrblck" data-post="6" data-topic="75812">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/ptrblck/48/1823_2.png" class="avatar"> ptrblck:</div>
<blockquote>
<p>x= F.max_pool1d(x.transpose(1,2).contiguous(), kernel_size=x.size()[1])</p>
</blockquote>
</aside>
<p>Hi all,<br>
I did slight change on a  nn by adding a max_pool1d and get error traceback</p>
<pre><code class="lang-auto">Traceback (most recent call last):
  File "../../learn/training.py", line 212, in train
    loss.backward()
  File "C:\Users\Veid\Anaconda3\envs\caml\lib\site-packages\torch\tensor.py", line 195, in backward
    torch.autograd.backward(self, gradient, retain_graph, create_graph)
  File "C:\Users\Veid\Anaconda3\envs\caml\lib\site-packages\torch\autograd\__init__.py", line 99, in backward
    allow_unreachable=True)  # allow_unreachable flag
RuntimeError: fractional_max_pool2d_backward_out_cuda failed with error code 0
</code></pre>
<p>How can I correct this right?<br>
I’m new with torch so I apologize in advance if this is a stupid question.</p>
<p>The original code snippet that works successfully</p>
<pre data-code-wrap="python"><code class="lang-python">        #x shape is torch.Size([8, k, 400]) where k is an unfixed number
        #U.weight shape is torch.Size([50, 400])
        alpha = F.softmax(self.U.weight.matmul(x.transpose(1,2)), dim=2)
        #alpha shape is torch.Size([8, 50, k])
        m = alpha.matmul(x)
        #m shape is torch.Size([8, 50, 400])
        #final.weight shape is torch.Size([50, 400])
        y = self.final.weight.mul(m).sum(dim=2).add(self.final.bias)
        #y shape is torch.Size([8, 50])
</code></pre>
<p>The code snippet after changing that fails to autograd</p>
<pre><code class="lang-auto">        #x shape is torch.Size([8, k, 400]) where k is an unfixed number, 8 is the batch size
        #U.weight shape is torch.Size([50, 400])
        x= F.max_pool1d(x.transpose(1,2), kernel_size=x.size()[1])
        #after max pooling, x shape is torch.Size([8, 400, 1])
        alpha = self.U.weight.mul(x.transpose(1,2)) 
        #alpha shape is torch.Size([8, 50, 400])
        #final.weight shape is torch.Size([50, 400])
        y = self.final.weight.mul(alpha).sum(dim=2).add(self.final.bias)
        #y shape is torch.Size([8, 50])
</code></pre>
<p>Here is the runable code that I extracted related Variable defination and loss computation part. But the bug can not be reproduced in this<br>
setting. <img src="https://discuss.pytorch.org/images/emoji/apple/frowning.png?v=12" title=":frowning:" class="emoji" alt=":frowning:" loading="lazy" width="20" height="20"></p>
<pre><code class="lang-auto">import torch.nn as nn
import torch
import torch.nn.functional as F
from torch.nn.init import xavier_uniform_
from torch.autograd import Variable
target=Variable(torch.randn(8,50))
U = nn.Linear(400, 50)
xavier_uniform_(U.weight)
final = nn.Linear(400, 50)
xavier_uniform_(final.weight)
x = Variable(torch.randn(8,123,400))
'''
#The code snippet that works successfully
alpha = F.softmax(U.weight.matmul(x.transpose(1,2)), dim=2)
m = alpha.matmul(x)
y = final.weight.mul(m).sum(dim=2).add(final.bias)
'''
'''
#The code snippet that fails to autograd
x= F.max_pool1d(x.transpose(1,2), kernel_size=x.size()[1])
alpha = U.weight.mul(x.transpose(1,2)) 
y = final.weight.mul(alpha).sum(dim=2).add(final.bias)
'''
yhat= y
loss = F.binary_cross_entropy_with_logits(yhat, target)
loss.backward()
print(y.size())
</code></pre>

ptrblck: <p>Could you pose an executable code snippet by wrapping it into three backticks ```, please? <img src="https://discuss.pytorch.org/images/emoji/apple/slight_smile.png?v=9" title=":slight_smile:" class="emoji" alt=":slight_smile:"><br>
This would make debugging much easier.</p>

Nanachi: <p>Thanks for your reply!<br>
I have updated the code snippet.<br>
I also tried to use a runable test code to reproduce the bug but failed.<br>
I also updated the test code, I hope this could be help.<br>
Thanks again for your kind help <img src="https://discuss.pytorch.org/images/emoji/apple/slightly_smiling_face.png?v=9" title=":slightly_smiling_face:" class="emoji" alt=":slightly_smiling_face:"></p>

braindotai: <p>you can use this key ` which is located left to “1”. So if you want to provide some code then just add it like-<br>
<img src="https://discuss.pytorch.org/uploads/default/original/3X/5/0/50bde2362411e827e02d4d5dcca7041ee3c86a9b.png" alt="Annotation 2020-04-08 155857" data-base62-sha1="bwh1ktqAbfSJ4epTgf63wN15Rer" width="173" height="131"><br>
Try it, this would make your code live <img src="https://discuss.pytorch.org/images/emoji/apple/wink.png?v=12" title=":wink:" class="emoji" alt=":wink:" loading="lazy" width="20" height="20"></p>

Nanachi: <p>Thanks for your kind help!<br>
It looks much better now <img src="https://discuss.pytorch.org/images/emoji/apple/blush.png?v=9" title=":blush:" class="emoji" alt=":blush:"></p>

ptrblck: <p>Thanks for the code snippet.<br>
Since it’s not reproducible with the last code, could you try to use:</p>
<pre><code class="lang-python">x= F.max_pool1d(x.transpose(1,2).contiguous(), kernel_size=x.size()[1])
</code></pre>
<p>and rerun the script, please?</p>
<p>Also, which PyTorch version are you using?<br>
Make sure to update to the latest stable release, as <code>Variables</code> are deprecated since <code>0.4</code>.</p>

Nanachi: <p>Thank you! It worked.<br>
I am using PyTorch1.4.0<br>
Seems the bug is coming from misuse of max_pool1d with Variables?</p>

ptrblck: <p>I think it’s coming from passing a non-contiguous tensor to the pooling layer, which was recently fixed, so it should work using the nightly binaries.</p>

Nanachi: <p>I got it. Thanks again for your kind help! <img src="https://discuss.pytorch.org/images/emoji/apple/smiley.png?v=9" title=":smiley:" class="emoji" alt=":smiley:"></p>