Getting ValueError

username: Jd8111997
cooked: <p>Hey there,<br>
I am trying to use BERT for the multilabel classification task. But I got this unexpected error below.<div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/e/e/ee47352a9164e7fd548a426954ea8d14f868753d.png" data-download-href="https://discuss.pytorch.org/uploads/default/ee47352a9164e7fd548a426954ea8d14f868753d" title="Screenshot%20from%202019-12-10%2011-35-20"><img src="https://discuss.pytorch.org/uploads/default/optimized/3X/e/e/ee47352a9164e7fd548a426954ea8d14f868753d_2_689x251.png" alt="Screenshot%20from%202019-12-10%2011-35-20" data-base62-sha1="xZUcnIIdAhaBEJtEZDcXrdhMVxH" width="689" height="251" srcset="https://discuss.pytorch.org/uploads/default/optimized/3X/e/e/ee47352a9164e7fd548a426954ea8d14f868753d_2_689x251.png, https://discuss.pytorch.org/uploads/default/optimized/3X/e/e/ee47352a9164e7fd548a426954ea8d14f868753d_2_1033x376.png 1.5x, https://discuss.pytorch.org/uploads/default/original/3X/e/e/ee47352a9164e7fd548a426954ea8d14f868753d.png 2x" data-dominant-color="F4F4F6"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">Screenshot%20from%202019-12-10%2011-35-20</span><span class="informations">1240×452 48.3 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>
<p>I set the batch_size to 32 in train and validation data loader<br>
Here is the code below.<br>
train_loss_set = <span class="chcklst-box fa fa-square-o fa-fw"></span><br>
epochs = 4<br>
for _ in trange(epochs, desc=“Epoch”):<br>
model.train()<br>
tr_loss = 0<br>
nb_tr_examples, nb_tr_steps = 0, 0<br>
for step, batch in enumerate(train_dataloader):<br>
batch = tuple(t.to(device) for t in batch)<br>
b_input_ids, b_input_mask, b_labels = batch<br>
optimizer.zero_grad()<br>
loss = model(b_input_ids, token_type_ids=None, attention_mask=b_input_mask, labels=b_labels)<br>
train_loss_set.append(loss.item())<br>
loss.backward()<br>
optimizer.step()<br>
tr_loss += loss.item()<br>
nb_tr_examples += b_input_ids.size(0)<br>
nb_tr_steps += 1<br>
print(“Train loss: {}”.format(tr_loss/nb_tr_steps))<br>
model.eval()<br>
eval_loss, eval_accuracy = 0, 0<br>
nb_eval_steps, nb_eval_examples = 0, 0</p>
<p>for batch in validation_dataloader:<br>
batch = tuple(t.to(device) for t in batch)<br>
b_input_ids, b_input_mask, b_labels = batch<br>
with torch.no_grad():<br>
logits = model(b_input_ids, token_type_ids=None, attention_mask=b_input_mask)<br>
logits = logits.detach().cpu().numpy()<br>
label_ids = b_labels.to(‘cpu’).numpy()<br>
tmp_eval_accuracy = flat_accuracy(logits, label_ids)<br>
eval_accuracy += tmp_eval_accuracy<br>
nb_eval_steps += 1<br>
print(“Validation Accuracy: {}”.format(eval_accuracy/nb_eval_steps))</p>

username: albanD
cooked: <p>Hi,</p>
<p>You can use triple backticks to format your code nicely:</p>
<pre><code class="lang-auto">```
# Your code
```
</code></pre>
<p>For your specific question, it looks like the label Tensor you give your loss has a first dimension of size 192. While it should be the same as the input 32.<br>
You might want to add some prints to make sure it has the right size.</p>