Getting unexpected NotImplementedError when running model

seankala: <p>Hello. I’m currently working on debugging a model I wrote using PyTorch.</p>
<p>When I first ran <code>output = model(data)</code> I got the error</p>
<pre><code class="lang-auto">*** RuntimeError: Expected tensor for argument #1 'indices' to have scalar type Long; but got torch.cuda.DoubleTensor instead (while checking arguments for embedding)
</code></pre>
<p>No problem, I simply ran <code>output = model(data.long())</code>. Then I got a:</p>
<pre><code class="lang-auto">*** RuntimeError: Expected object of device type cuda but got device type cpu for argument #1 'self' in call to _th_index_select
</code></pre>
<p>I wasn’t sure what the error message exactly meant, but I recalled that I hadn’t put my model on the GPU device. So I ran:</p>
<pre><code class="lang-auto">model = model.to('cuda')
output = model(data.long())
</code></pre>
<p>but this time I got a:</p>
<pre><code class="lang-auto">*** NotImplementedError
</code></pre>
<p>This was a bit unexpected, and I checked if I had put in a <code>NotImplementedError</code> anywhere in the code but I hadn’t. I’m having some difficulty finding helpful resources, as I don’t even know what the causing piece of code is.</p>
<p>Would anybody have any idea how I should handle this issue? Just an FYI, these errors occurred while I was in the PDB interactive shell.</p>
<p>Thanks in advance.</p>

G.M: <ol>
<li>
<p>The <code>.to()</code> method needs the device code of the device, for gpus, the code is usually something like “cuda:0”. You can also use <code>.cuda()</code>.</p>
</li>
<li>
<p>Idk what device is <code>data</code> on, but calling <code>.long()</code> only changes it to Long type on its original device. If ur model is on gpu, u also have to change the inputs to gpu. In this case, I guess <code>data</code> is not on gpu.</p>
</li>
</ol>

ptrblck: <p>As an addition to what <a class="mention" href="/u/g.m">@G.M</a> said, could you please print the complete stack trace you get from the <code>NotImplementedError</code>?<br>
You would get this error, e.g. if you have a typo in a method name (e.g. <code>forwrad</code> instead of <code>forward</code>) or if you use a wrong indentation.</p>

seankala: <p><a class="mention" href="/u/g.m">@G.M</a> I’ve also put the <code>data</code> on GPU via <code>data.to('cuda')</code>. I’ve never had a problem from not specifying the device and just using <code>'cuda'</code>. Double checking right now <code>data</code> is a PyTorch tensor with <code>device='cuda:0'</code>.</p>

G.M: <p>Then could u please print the whole stack trace as <a class="mention" href="/u/ptrblck">@ptrblck</a> said ?</p>

seankala: <p>Here’s the traceback that I’ve been able to get:</p>
<pre><code class="lang-auto">Traceback (most recent call last):
  File "./main.py", line 27, in &lt;module&gt;
    main()
  File "./main.py", line 20, in main
    output = model(data.long())
  File "C:\Users\user1\Anaconda3\lib\site-packages\torch\nn\modules\module.py", line 541, in __call__
    result = self.forward(*input, **kwargs)
  File "C:\Users\user1\Desktop\github\transformer_implementation\transformer.py", line 25, in forward
    enc_output = self.encoder(x)
  File "C:\Users\user1\Anaconda3\lib\site-packages\torch\nn\modules\module.py", line 541, in __call__
    result = self.forward(*input, **kwargs)
  File "C:\Users\user1\Desktop\github\transformer_implementation\encoder.py", line 54, in forward
    output = self.encoder_stacked(x)
  File "C:\Users\user1\Anaconda3\lib\site-packages\torch\nn\modules\module.py", line 541, in __call__
    result = self.forward(*input, **kwargs)
  File "C:\Users\user1\Anaconda3\lib\site-packages\torch\nn\modules\module.py", line 97, in forward
    raise NotImplementedError
NotImplementedError
</code></pre>
<p>I didn’t initially include too many details here in the original post, but the <code>model</code> that I’m using has a <code>nn.ModuleList</code> inside of it. After going through parts of the code, I found out that this was the problem. More specifically, it looks like:</p>
<pre><code class="lang-auto">self.encoder_stacked = nn.ModuleList([Encoder(self.config) for _ in range(self.N)])
</code></pre>
<p>According to <a href="https://stackoverflow.com/questions/52241680/pytorch-notimplementederror-in-forward" rel="nofollow noopener">this Stack Overflow question</a> it seems that there are cases where the indentation may cause problems, but that’s not the case for me.</p>
<p>The <code>Encoder</code> module looks like this:</p>
<pre><code class="lang-auto">class Encoder(nn.Module):
    def __init__(self, config, dropout=True):
        super().__init__()
        self.config = config
        self.d_model = self.config.d_model

        self.attention = MultiheadAttention(self.config)

        if dropout:
            self.dropout = nn.Dropout(p=self.config.dropout_rate)
        else:
            self.dropout = nn.Dropout(p=0.0)

        self.norm = Normalization(features=self.d_model)
        self.ffnn = FeedforwardNN(self.config)

    def forward(self, x):
        attn_output = self.attention(x, x, x)
        sublayer1_output = self.norm(x + attn_output)
        sublayer1_output = self.dropout(sublayer1_output)

        ffnn_output = self.ffnn(sublayer1_output)
        sublayer2_output = self.norm(sublayer1_output + ffnn_output)
        sublayer2_output = self.dropout(sublayer2_output)

        return sublayer2_output
</code></pre>

seankala: <p>I think I may have found a solution <a href="https://discuss.pytorch.org/t/when-should-i-use-nn-modulelist-and-when-should-i-use-nn-sequential/5463/4">here</a>. I’ll do some more research based off of this answer.</p>

G.M: <p>Yes, ModuleList does not have a <code>forward</code> method. So u definitely can’t call on any ModuleList object as u did.</p>