Issues with Chatbot NLP tutorial

shaun: <p>Hi,<br>
I’m trying to build the <a href="https://pytorch.org/tutorials/beginner/chatbot_tutorial.html" rel="nofollow noopener">Chatbot</a> from the official tutorials and I’m running into two issues. The main issue is the mismatch of dimensions when running through the GRU in the decoder.</p>
<pre><code class="lang-auto">  def forward(self, input_step, last_hidden, encoder_output):
    # we run this one step (word) at a time
    embedded = self.embedding(input_step)
    embedded = self.embedding_dropout(embedded)

    # forward through unidirectional GRU
    set_trace()
    rnn_output, hidden_state = self.gru(embedded, last_hidden)

    # calculate attention weights from the current GRU output
    attn_weights = self.attn(rnn_output, encoder_output)
</code></pre>
<p>The error is occurring at the gru call after <code>set_trace()</code>. Here is the error:</p>
<pre><code class="lang-auto">Traceback (most recent call last):
  File "chatbot.py", line 110, in &lt;module&gt;
    decoder_output, decoder_hidden = decoder(decoder_input, decoder_hidden, encoder_output)
  File "/net/vaosl01/opt/NFS/su0/anaconda3/envs/pyt/lib/python3.7/site-packages/torch/nn/modules/module.py", line 489, in __call__
    result = self.forward(*input, **kwargs)
  File "/home/su0/pytorch-learn/chatbot_tutorial/decoder.py", line 99, in forward
    rnn_output, hidden_state = self.gru(embedded, last_hidden)
  File "/net/vaosl01/opt/NFS/su0/anaconda3/envs/pyt/lib/python3.7/site-packages/torch/nn/modules/module.py", line 489, in __call__
    result = self.forward(*input, **kwargs)
  File "/net/vaosl01/opt/NFS/su0/anaconda3/envs/pyt/lib/python3.7/site-packages/torch/nn/modules/rnn.py", line 175, in forward
    self.check_forward_args(input, hx, batch_sizes)
  File "/net/vaosl01/opt/NFS/su0/anaconda3/envs/pyt/lib/python3.7/site-packages/torch/nn/modules/rnn.py", line 131, in check_forward_args
    expected_input_dim, input.dim()))
RuntimeError: input must have 3 dimensions, got 2
</code></pre>
<p>I’m only working a very small subset to make sure things are good. Here are the parameters:</p>
<pre><code class="lang-auto">  batch_size = 5
  hidden_size = 500
  n_encoder_layers = 2
  n_decoder_layers = 2
  dropout = 0.1
  attn_model = 'dot'
  embedding = nn.Embedding(len(vocab), hidden_size)
</code></pre>
<p>Basically, my <code>embedded</code> shape thats passed as input is of shape <code>(batch_size, hidden_size) = (5, 500)</code> and its asking for a 3-d input. Which makes sense from the <a href="https://pytorch.org/docs/stable/nn.html?highlight=gru#torch.nn.GRU" rel="nofollow noopener">GRU</a> documentation which says input is either of shape <code>(seq_len, batch, input_size)</code> or a <a href="https://pytorch.org/docs/stable/nn.html?highlight=gru#torch.nn.utils.rnn.pack_padded_sequence" rel="nofollow noopener"><code>pad_packed_sequence</code></a>. However, here it is neither of those and I’d like some help fixing it. This code is directly from the tutorial.</p>
<p>There is one more error related to the use of <code>pack_padded_sequence</code> which is also asked <a href="https://discuss.pytorch.org/t/error-with-lengths-in-pack-padded-sequence/35517">here</a>. I’ve temporarily averted that by not setting a default device (and everything is running on the CPU) and will get back to it after I get everything else working.</p>
<p>Let me know if more information is required.<br>
Thanks.</p>

ParticularlyPythonic: <p>Embedding is multidimensional, so it shouldn’t have a dimension problem.<br>
I tried running the jupyter notebook for the chatbot, and it seems to work.<br>
Can you try running that and check the output?<br>
Also what version of pytorch are you using?</p>

shaun: <p>Thanks for your reply. I ran the notebook really quickly and its training (on the GPU) without any problems. I don’t get why when I modularize the (same) code with different files and try to run one iteration, I get issues that I’ve highlighted my first post and the one that you had earlier.</p>
<p>I’m running PyTorch <code>1.0.1.post2</code></p>

ParticularlyPythonic: <p>what is <code>vocab</code> here? There is no variable <code>vocab</code> in the tutorial</p>

shaun: <p>Well I guess I lied a bit about it being completely same code. Instead of <code>voc</code>, I named the instance <code>vocab</code> and I just gave the class a <code>__len__</code> method.</p>

Brando_Miranda: <p>why don’t you write a meaningful title for your question? What issue? Can I know your issue just by reading your question title?</p>