Constantly getting a runtime error: invalid shape

thiziri: <p>I’m trying to use the BERT model “for multiple choice” and I constantly get this error:<br>
here is the code snippet:<br>
defined functions:</p>
<blockquote>
<p>def pad_encode(text, tokenizer, max_length=50):</p>
<p>“”"</p>
<p>creates token ids of a uniform sequence length for a given sentence</p>
<p>“”"</p>
<p>tok_ids = tokenizer.convert_tokens_to_ids(tokenizer.tokenize(text))</p>
<p>tok_ids2 = tokenizer.add_special_tokens_single_sentence(tok_ids)</p>
<p>att_mask = [1 for _ in tok_ids2]</p>
<p>n_spectoks = len(tok_ids2) - len(tok_ids)</p>
<p>if len(tok_ids2) &gt; max_length: # need to truncate</p>
<pre><code>#print('Truncating from', len(tok_ids2))

n_to_trunc = len(tok_ids2) - max_length

tok_ids2 = tokenizer.add_special_tokens_single_sentence(tok_ids[:-n_to_trunc])

att_mask = [1 for _ in tok_ids2]
</code></pre>
<p>elif len(tok_ids2) &lt; max_length: # need to pad</p>
<pre><code>padding = []

for i in range(len(tok_ids2), max_length):

  padding.append(tokenizer.pad_token_id)

att_mask += [0 for _ in padding]

tok_ids2 = tok_ids2 + padding
</code></pre>
<p>assert len(tok_ids2) == max_length</p>
<p>assert len(att_mask) == max_length</p>
<p>return tok_ids2, att_mask</p>
<p>def tokenize_batch(sentences, tok_model, max_len=50, debug=True):</p>
<p>“”"</p>
<p>tokenizes a batch of sentences and produces a tensor that can be fed to the model</p>
<p>“”"</p>
<p>assert type(sentences) == list</p>
<p>encoded = [pad_encode(s, tokenizer=tok_model[‘tokenizer’],</p>
<pre><code>                    max_length=max_len)[0] for s in sentences]
</code></pre>
<p>att_masks = [pad_encode(s, tokenizer=tok_model[‘tokenizer’],</p>
<pre><code>                    max_length=max_len)[1] for s in sentences]
</code></pre>
<p>input_ids = torch.tensor(encoded)</p>
<p>att_masks = torch.tensor(att_masks)</p>
<p>if debug: print(input_ids.shape)</p>
<p>if torch.cuda.is_available():</p>
<pre><code>input_ids = input_ids.cuda()

att_masks = att_masks.cuda()
</code></pre>
<p>return input_ids, att_masks</p>
</blockquote>
<p>when I try to encode the batch data:</p>
<blockquote>
<p>input_ids, att_masks = tokenize_batch(sentences, {“tokenizer”: self.tokenizer,  “model”: self.bert_model}, max_len=self.seq_len, debug=True)<br>
print(input_ids.shape, att_masks.shape)<br>
model_out = bert_model(input_ids, attention_mask=att_masks)</p>
</blockquote>
<p>I get the following:</p>
<blockquote>
<p>torch.Size([64, 50]) torch.Size([64, 50])<br>
…<br>
/usr/local/lib/python3.6/dist-packages/torch/nn/modules/module.py in <strong>call</strong>(self, *input, **kwargs)<br>
530             result = self._slow_forward(*input, **kwargs)<br>
531         else:<br>
 → 532             result = self.forward(*input, **kwargs)<br>
533         for hook in self._forward_hooks.values():<br>
534             hook_result = hook(self, input, result)</p>
<p>/usr/local/lib/python3.6/dist-packages/pytorch_transformers/modeling_bert.py in forward(self, input_ids, token_type_ids, attention_mask, labels, position_ids, head_mask)<br>
1076         pooled_output = self.dropout(pooled_output)<br>
1077         logits = self.classifier(pooled_output)<br>
 → 1078         reshaped_logits = logits.view(-1, num_choices)<br>
1079<br>
1080         outputs = (reshaped_logits,) + outputs[2:]  # add hidden states and attention if they are here</p>
<p>RuntimeError: shape ‘[-1, 50]’ is invalid for input of size 64</p>
</blockquote>