Training BERT for multi-classfication: ValueError: Expected input batch_size (1) to match target batch_size (512)

abadrawy: <p>I have been getting the shape error, and I am not sure where the problem is. I have tried reshaping and it still does not work. Any help would greatly be appreciated. The batch size is 1 and the target labels are 512.</p>
<p>Here is the error log<br>
<div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/3/3/339281151e80057514f6c69d71ed29d0244081cd.png" data-download-href="https://discuss.pytorch.org/uploads/default/339281151e80057514f6c69d71ed29d0244081cd" title="Future Group USA IRS 2020-06-27 at 16.26.05"><img src="https://discuss.pytorch.org/uploads/default/optimized/3X/3/3/339281151e80057514f6c69d71ed29d0244081cd_2_690x207.png" alt="Future Group USA IRS 2020-06-27 at 16.26.05" data-base62-sha1="7meeyMucdolX9aosptLkYt5qj2B" width="690" height="207" srcset="https://discuss.pytorch.org/uploads/default/optimized/3X/3/3/339281151e80057514f6c69d71ed29d0244081cd_2_690x207.png, https://discuss.pytorch.org/uploads/default/optimized/3X/3/3/339281151e80057514f6c69d71ed29d0244081cd_2_1035x310.png 1.5x, https://discuss.pytorch.org/uploads/default/optimized/3X/3/3/339281151e80057514f6c69d71ed29d0244081cd_2_1380x414.png 2x" data-dominant-color="F6F6F8"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">Future Group USA IRS 2020-06-27 at 16.26.05</span><span class="informations">2690×810 261 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>
<p>Here us the training code.</p>
<pre><code class="lang-auto">t = [] 

# Store our loss and accuracy for plotting
train_loss_set = []

# Number of training epochs (authors recommend between 2 and 4)
epochs = 1


# trange is a tqdm wrapper around the normal python range
for _ in trange(epochs, desc="Epoch"):
  
  
  # Training
  
  # Set our model to training mode (as opposed to evaluation mode)
  model.train()
  
  # Tracking variables
  tr_loss = 0
  nb_tr_examples, nb_tr_steps = 0, 0
  
  # Train the data for one epoch
  
  for step, batch in enumerate(train_dataloader):
    # Add batch to GPU
    batch = tuple(t.to(device) for t in batch)
    # Unpack the inputs from our dataloader
    b_input_ids, b_input_mask, b_labels = batch
    # Clear out the gradients (by default they accumulate)
    optimizer.zero_grad()
    # Forward pass


  
    #b_input_ids=b_input_ids.resize(MAX_LEN,batch_size)
    #b_input_mask=b_input_mask.resize(MAX_LEN,batch_size)
    #b_labels=b_labels.resize(batch_size,MAX_LEN)
    b_labels = b_labels.view(MAX_LEN,batch_size)

    
    print(b_input_ids.shape)
    print(b_input_mask.shape)
    print(b_labels.shape)

  

    loss = model(b_input_ids, token_type_ids=None, attention_mask=b_input_mask, labels=b_labels)
   # print("loss",loss)
   # print("loss",loss.item)

    train_loss_set.append(loss.item())    
    # Backward pass
    loss.backward()
    # Update parameters and take a step using the computed gradient
    optimizer.step()
    
    
    # Update tracking variables
    tr_loss += loss.item()
    nb_tr_examples += b_input_ids.size(0)
    nb_tr_steps += 1

  print("Train loss: {}".format(tr_loss/nb_tr_steps))
    
    
  # Validation

  # Put model in evaluation mode to evaluate loss on the validation set
  model.eval()

  # Tracking variables 
  eval_loss, eval_accuracy = 0, 0
  nb_eval_steps, nb_eval_examples = 0, 0

  # Evaluate data for one epoch
  for batch in validation_dataloader:
    # Add batch to GPU
    batch = tuple(t.to(device) for t in batch)
    # Unpack the inputs from our dataloader
    b_input_ids, b_input_mask, b_labels = batch
    # Telling the model not to compute or store gradients, saving memory and speeding up validation
    b_labels = b_labels.view(MAX_LEN,batch_size)

    print(b_input_ids.shape)
    print(b_input_mask.shape)
    print(b_labels.shape)

    #b_input_ids=b_input_ids.resize(MAX_LEN,batch_size)
    #b_input_mask=b_input_mask.resize(MAX_LEN,batch_size)
    #b_labels=b_labels.resize(batch_size,MAX_LEN)




    with torch.no_grad():
      # Forward pass, calculate logit predictions
      logits = model(b_input_ids, token_type_ids=None, attention_mask=b_input_mask)
    
    # Move logits and labels to CPU
    logits = logits.detach().cpu().numpy()
    label_ids = b_labels.to('cpu').numpy()

    tmp_eval_accuracy = flat_accuracy(logits, label_ids)
    
    eval_accuracy += tmp_eval_accuracy
    nb_eval_steps += 1

  print("Validation Accuracy: {}".format(eval_accuracy/nb_eval_steps))```</code></pre>

harsha_g: <p>You are using <code>NLLLoss</code> and the targets <code>b_labels</code> are rightly shaped <code>512x1</code>. So, make sure that, wherever you are calculating the loss the input is shaped <code>512xC</code> where C is the number of classes.</p>

abadrawy: <p>the batch size is 1, the input’s len is 512 and the number of classes/output is 512 as well. So, do I shape the input to be 512x512? I think I’m missing something</p>

harsha_g: <p>Will this help? <a href="https://discuss.pytorch.org/t/loss-functions-for-batches/20488/8" class="inline-onebox">Loss functions for batches</a></p>

abadrawy: <p>I read through it, but I still can’t seem to get it to work. my input len is 512 and the batch size is, in this case 1. So, I don’t get why I need to introduce a new dimension. My output is the same size as the input and is also 512.</p>
<p>Here is the model, BertForSequenceClassification(<br>
(bert): BertModel(<br>
(embeddings): BertEmbeddings(<br>
(word_embeddings): Embedding(30522, 768, padding_idx=0)<br>
(position_embeddings): Embedding(512, 768)<br>
(token_type_embeddings): Embedding(2, 768)<br>
(LayerNorm): BertLayerNorm()<br>
(dropout): Dropout(p=0.1, inplace=False)<br>
)<br>
(encoder): BertEncoder(<br>
(layer): ModuleList(<br>
(0): BertLayer(<br>
(attention): BertAttention(<br>
(self): BertSelfAttention(<br>
(query): Linear(in_features=768, out_features=768, bias=True)<br>
(key): Linear(in_features=768, out_features=768, bias=True)<br>
(value): Linear(in_features=768, out_features=768, bias=True)<br>
(dropout): Dropout(p=0.1, inplace=False)<br>
)<br>
(output): BertSelfOutput(<br>
(dense): Linear(in_features=768, out_features=768, bias=True)<br>
(LayerNorm): BertLayerNorm()<br>
(dropout): Dropout(p=0.1, inplace=False)<br>
)<br>
)<br>
(intermediate): BertIntermediate(<br>
(dense): Linear(in_features=768, out_features=3072, bias=True)<br>
)<br>
(output): BertOutput(<br>
(dense): Linear(in_features=3072, out_features=768, bias=True)<br>
(LayerNorm): BertLayerNorm()<br>
(dropout): Dropout(p=0.1, inplace=False)<br>
)<br>
)<br>
(1): BertLayer(<br>
(attention): BertAttention(<br>
(self): BertSelfAttention(<br>
(query): Linear(in_features=768, out_features=768, bias=True)<br>
(key): Linear(in_features=768, out_features=768, bias=True)<br>
(value): Linear(in_features=768, out_features=768, bias=True)<br>
(dropout): Dropout(p=0.1, inplace=False)<br>
)<br>
(output): BertSelfOutput(<br>
(dense): Linear(in_features=768, out_features=768, bias=True)<br>
(LayerNorm): BertLayerNorm()<br>
(dropout): Dropout(p=0.1, inplace=False)<br>
)<br>
)<br>
(intermediate): BertIntermediate(<br>
(dense): Linear(in_features=768, out_features=3072, bias=True)<br>
)<br>
(output): BertOutput(<br>
(dense): Linear(in_features=3072, out_features=768, bias=True)<br>
(LayerNorm): BertLayerNorm()<br>
(dropout): Dropout(p=0.1, inplace=False)<br>
)<br>
)<br>
(2): BertLayer(<br>
(attention): BertAttention(<br>
(self): BertSelfAttention(<br>
(query): Linear(in_features=768, out_features=768, bias=True)<br>
(key): Linear(in_features=768, out_features=768, bias=True)<br>
(value): Linear(in_features=768, out_features=768, bias=True)<br>
(dropout): Dropout(p=0.1, inplace=False)<br>
)<br>
(output): BertSelfOutput(<br>
(dense): Linear(in_features=768, out_features=768, bias=True)<br>
(LayerNorm): BertLayerNorm()<br>
(dropout): Dropout(p=0.1, inplace=False)<br>
)<br>
)<br>
(intermediate): BertIntermediate(<br>
(dense): Linear(in_features=768, out_features=3072, bias=True)<br>
)<br>
(output): BertOutput(<br>
(dense): Linear(in_features=3072, out_features=768, bias=True)<br>
(LayerNorm): BertLayerNorm()<br>
(dropout): Dropout(p=0.1, inplace=False)<br>
)<br>
)<br>
(3): BertLayer(<br>
(attention): BertAttention(<br>
(self): BertSelfAttention(<br>
(query): Linear(in_features=768, out_features=768, bias=True)<br>
(key): Linear(in_features=768, out_features=768, bias=True)<br>
(value): Linear(in_features=768, out_features=768, bias=True)<br>
(dropout): Dropout(p=0.1, inplace=False)<br>
)<br>
(output): BertSelfOutput(<br>
(dense): Linear(in_features=768, out_features=768, bias=True)<br>
(LayerNorm): BertLayerNorm()<br>
(dropout): Dropout(p=0.1, inplace=False)<br>
)<br>
)<br>
(intermediate): BertIntermediate(<br>
(dense): Linear(in_features=768, out_features=3072, bias=True)<br>
)<br>
(output): BertOutput(<br>
(dense): Linear(in_features=3072, out_features=768, bias=True)<br>
(LayerNorm): BertLayerNorm()<br>
(dropout): Dropout(p=0.1, inplace=False)<br>
)<br>
)<br>
(4): BertLayer(<br>
(attention): BertAttention(<br>
(self): BertSelfAttention(<br>
(query): Linear(in_features=768, out_features=768, bias=True)<br>
(key): Linear(in_features=768, out_features=768, bias=True)<br>
(value): Linear(in_features=768, out_features=768, bias=True)<br>
(dropout): Dropout(p=0.1, inplace=False)<br>
)<br>
(output): BertSelfOutput(<br>
(dense): Linear(in_features=768, out_features=768, bias=True)<br>
(LayerNorm): BertLayerNorm()<br>
(dropout): Dropout(p=0.1, inplace=False)<br>
)<br>
)<br>
(intermediate): BertIntermediate(<br>
(dense): Linear(in_features=768, out_features=3072, bias=True)<br>
)<br>
(output): BertOutput(<br>
(dense): Linear(in_features=3072, out_features=768, bias=True)<br>
(LayerNorm): BertLayerNorm()<br>
(dropout): Dropout(p=0.1, inplace=False)<br>
)<br>
)<br>
(5): BertLayer(<br>
(attention): BertAttention(<br>
(self): BertSelfAttention(<br>
(query): Linear(in_features=768, out_features=768, bias=True)<br>
(key): Linear(in_features=768, out_features=768, bias=True)<br>
(value): Linear(in_features=768, out_features=768, bias=True)<br>
(dropout): Dropout(p=0.1, inplace=False)<br>
)<br>
(output): BertSelfOutput(<br>
(dense): Linear(in_features=768, out_features=768, bias=True)<br>
(LayerNorm): BertLayerNorm()<br>
(dropout): Dropout(p=0.1, inplace=False)<br>
)<br>
)<br>
(intermediate): BertIntermediate(<br>
(dense): Linear(in_features=768, out_features=3072, bias=True)<br>
)<br>
(output): BertOutput(<br>
(dense): Linear(in_features=3072, out_features=768, bias=True)<br>
(LayerNorm): BertLayerNorm()<br>
(dropout): Dropout(p=0.1, inplace=False)<br>
)<br>
)<br>
(6): BertLayer(<br>
(attention): BertAttention(<br>
(self): BertSelfAttention(<br>
(query): Linear(in_features=768, out_features=768, bias=True)<br>
(key): Linear(in_features=768, out_features=768, bias=True)<br>
(value): Linear(in_features=768, out_features=768, bias=True)<br>
(dropout): Dropout(p=0.1, inplace=False)<br>
)<br>
(output): BertSelfOutput(<br>
(dense): Linear(in_features=768, out_features=768, bias=True)<br>
(LayerNorm): BertLayerNorm()<br>
(dropout): Dropout(p=0.1, inplace=False)<br>
)<br>
)<br>
(intermediate): BertIntermediate(<br>
(dense): Linear(in_features=768, out_features=3072, bias=True)<br>
)<br>
(output): BertOutput(<br>
(dense): Linear(in_features=3072, out_features=768, bias=True)<br>
(LayerNorm): BertLayerNorm()<br>
(dropout): Dropout(p=0.1, inplace=False)<br>
)<br>
)<br>
(7): BertLayer(<br>
(attention): BertAttention(<br>
(self): BertSelfAttention(<br>
(query): Linear(in_features=768, out_features=768, bias=True)<br>
(key): Linear(in_features=768, out_features=768, bias=True)<br>
(value): Linear(in_features=768, out_features=768, bias=True)<br>
(dropout): Dropout(p=0.1, inplace=False)<br>
)<br>
(output): BertSelfOutput(<br>
(dense): Linear(in_features=768, out_features=768, bias=True)<br>
(LayerNorm): BertLayerNorm()<br>
(dropout): Dropout(p=0.1, inplace=False)<br>
)<br>
)<br>
(intermediate): BertIntermediate(<br>
(dense): Linear(in_features=768, out_features=3072, bias=True)<br>
)<br>
(output): BertOutput(<br>
(dense): Linear(in_features=3072, out_features=768, bias=True)<br>
(LayerNorm): BertLayerNorm()<br>
(dropout): Dropout(p=0.1, inplace=False)<br>
)<br>
)<br>
(8): BertLayer(<br>
(attention): BertAttention(<br>
(self): BertSelfAttention(<br>
(query): Linear(in_features=768, out_features=768, bias=True)<br>
(key): Linear(in_features=768, out_features=768, bias=True)<br>
(value): Linear(in_features=768, out_features=768, bias=True)<br>
(dropout): Dropout(p=0.1, inplace=False)<br>
)<br>
(output): BertSelfOutput(<br>
(dense): Linear(in_features=768, out_features=768, bias=True)<br>
(LayerNorm): BertLayerNorm()<br>
(dropout): Dropout(p=0.1, inplace=False)<br>
)<br>
)<br>
(intermediate): BertIntermediate(<br>
(dense): Linear(in_features=768, out_features=3072, bias=True)<br>
)<br>
(output): BertOutput(<br>
(dense): Linear(in_features=3072, out_features=768, bias=True)<br>
(LayerNorm): BertLayerNorm()<br>
(dropout): Dropout(p=0.1, inplace=False)<br>
)<br>
)<br>
(9): BertLayer(<br>
(attention): BertAttention(<br>
(self): BertSelfAttention(<br>
(query): Linear(in_features=768, out_features=768, bias=True)<br>
(key): Linear(in_features=768, out_features=768, bias=True)<br>
(value): Linear(in_features=768, out_features=768, bias=True)<br>
(dropout): Dropout(p=0.1, inplace=False)<br>
)<br>
(output): BertSelfOutput(<br>
(dense): Linear(in_features=768, out_features=768, bias=True)<br>
(LayerNorm): BertLayerNorm()<br>
(dropout): Dropout(p=0.1, inplace=False)<br>
)<br>
)<br>
(intermediate): BertIntermediate(<br>
(dense): Linear(in_features=768, out_features=3072, bias=True)<br>
)<br>
(output): BertOutput(<br>
(dense): Linear(in_features=3072, out_features=768, bias=True)<br>
(LayerNorm): BertLayerNorm()<br>
(dropout): Dropout(p=0.1, inplace=False)<br>
)<br>
)<br>
(10): BertLayer(<br>
(attention): BertAttention(<br>
(self): BertSelfAttention(<br>
(query): Linear(in_features=768, out_features=768, bias=True)<br>
(key): Linear(in_features=768, out_features=768, bias=True)<br>
(value): Linear(in_features=768, out_features=768, bias=True)<br>
(dropout): Dropout(p=0.1, inplace=False)<br>
)<br>
(output): BertSelfOutput(<br>
(dense): Linear(in_features=768, out_features=768, bias=True)<br>
(LayerNorm): BertLayerNorm()<br>
(dropout): Dropout(p=0.1, inplace=False)<br>
)<br>
)<br>
(intermediate): BertIntermediate(<br>
(dense): Linear(in_features=768, out_features=3072, bias=True)<br>
)<br>
(output): BertOutput(<br>
(dense): Linear(in_features=3072, out_features=768, bias=True)<br>
(LayerNorm): BertLayerNorm()<br>
(dropout): Dropout(p=0.1, inplace=False)<br>
)<br>
)<br>
(11): BertLayer(<br>
(attention): BertAttention(<br>
(self): BertSelfAttention(<br>
(query): Linear(in_features=768, out_features=768, bias=True)<br>
(key): Linear(in_features=768, out_features=768, bias=True)<br>
(value): Linear(in_features=768, out_features=768, bias=True)<br>
(dropout): Dropout(p=0.1, inplace=False)<br>
)<br>
(output): BertSelfOutput(<br>
(dense): Linear(in_features=768, out_features=768, bias=True)<br>
(LayerNorm): BertLayerNorm()<br>
(dropout): Dropout(p=0.1, inplace=False)<br>
)<br>
)<br>
(intermediate): BertIntermediate(<br>
(dense): Linear(in_features=768, out_features=3072, bias=True)<br>
)<br>
(output): BertOutput(<br>
(dense): Linear(in_features=3072, out_features=768, bias=True)<br>
(LayerNorm): BertLayerNorm()<br>
(dropout): Dropout(p=0.1, inplace=False)<br>
)<br>
)<br>
)<br>
)<br>
(pooler): BertPooler(<br>
(dense): Linear(in_features=768, out_features=768, bias=True)<br>
(activation): Tanh()<br>
)<br>
)<br>
(dropout): Dropout(p=0.1, inplace=False)<br>
(classifier): Linear(in_features=768, out_features=512, bias=True)<br>
)</p>

harsha_g: <aside class="quote no-group quote-modified" data-username="abadrawy" data-post="1" data-topic="87117">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/abadrawy/48/26097_2.png" class="avatar"> abadrawy:</div>
<blockquote>
<pre><code class="lang-auto">    #b_input_ids=b_input_ids.resize(MAX_LEN,batch_size)    
    print(b_input_ids.shape)
</code></pre>
</blockquote>
</aside>
<p>The shape of <code>b_input_ids</code> should be <code>512x1</code> but what you have is <code>1x512</code>. Am I right?</p>

abadrawy: <p>right now, the b_input_ids shape is 512x1, I think I’m having trouble with the labels, I tried reshaping them to 512x1 but I still get the same error</p>

harsha_g: <p>I apologize for the misunderstanding on my part. The size of labels should be <code>512</code> and not <code>512x1</code> as I noted earlier.</p>

abadrawy: <p>it’s okay. But I just tried it and still got the same error. I had the labels to be of shape 512.</p>

Kushagra_Bhatia: <p>Since your batch_size is 1 and the number of tokens is 512, your input will be of the shape <code>1*512</code>. The size of label will be equal to <code>batch_size</code> i.e equal to <code>1</code> in your case.</p>

Kushagra_Bhatia: <p>The target label is always of the shape <code>batch_size</code>. Irrespective of the <code>number of classes</code>.</p>

abadrawy: <p>okay, but how could I reshape target labels to be of just size 1, is this possible?</p>

Kushagra_Bhatia: <p>Here is an example from huggingface’s BERT documentation.</p>
<pre><code class="lang-auto">from transformers import BertTokenizer, BertForSequenceClassification
import torch

tokenizer = BertTokenizer.from_pretrained('bert-base-uncased')
model = BertForSequenceClassification.from_pretrained('bert-base-uncased')

input_ids = torch.tensor(tokenizer.encode("Hello, my dog is cute", add_special_tokens=True)).unsqueeze(0)  # Batch size 1
labels = torch.tensor([1]).unsqueeze(0)  # Batch size 1
outputs = model(input_ids, labels=labels)

loss, logits = outputs[:2]
</code></pre>
<p>So, here the <code>batch_size</code> is <code>1</code> and as you can see the label has shape <code>1</code>. The line <code>labels=torch.tensor([1])</code> means the correct label is 1. It could be in range <code>[0,...,num_labels-1]</code>.</p>

abadrawy: <p>I am so sorry, but I am still not able to fully understand. My labels are currently of size 512, how is it possible to just map them to a single value. Could you elaborate more?</p>

Kushagra_Bhatia: <p>Can you give an example of one label? They must be one hot encoded.</p>

abadrawy: <p>here is the label/batch tensor:<br>
The labels are token id’s<br>
labels tensor([[ 1037,  2843,  1997, 13649,  3747,  1012,  2027,  6719,  2145,  2360,<br>
1000,  8038,  1000,  2738,  2084,  1000,  3398,  1000,  1012,  1012,<br>
1012,  2004,  1999,  1010,  1996, 15139,  1000, 14855,  1012,  1000,<br>
1006,  2054,  2003,  2115,  8239,  3291,  1010, 21591, 22052,  2015,<br>
1029,  1045,  4660,  1996,  3160,  1012,  1007,     0,     0,     0,<br>
0,     0,     0,     0,     0,     0,     0,     0,     0,     0,<br>
0,     0,     0,     0,     0,     0,     0,     0,     0,     0,<br>
0,     0,     0,     0,     0,     0,     0,     0,     0,     0,<br>
0,     0,     0,     0,     0,     0,     0,     0,     0,     0,<br>
0,     0,     0,     0,     0,     0,     0,     0,     0,     0,<br>
0,     0,     0,     0,     0,     0,     0,     0,     0,     0,<br>
0,     0,     0,     0,     0,     0,     0,     0,     0,     0,<br>
0,     0,     0,     0,     0,     0,     0,     0,     0,     0,<br>
0,     0,     0,     0,     0,     0,     0,     0,     0,     0,<br>
0,     0,     0,     0,     0,     0,     0,     0,     0,     0,<br>
0,     0,     0,     0,     0,     0,     0,     0,     0,     0,<br>
0,     0,     0,     0,     0,     0,     0,     0,     0,     0,<br>
0,     0,     0,     0,     0,     0,     0,     0,     0,     0,<br>
0,     0,     0,     0,     0,     0,     0,     0,     0,     0,<br>
0,     0,     0,     0,     0,     0,     0,     0,     0,     0,<br>
0,     0,     0,     0,     0,     0,     0,     0,     0,     0,<br>
0,     0,     0,     0,     0,     0,     0,     0,     0,     0,<br>
0,     0,     0,     0,     0,     0,     0,     0,     0,     0,<br>
0,     0,     0,     0,     0,     0,     0,     0,     0,     0,<br>
0,     0,     0,     0,     0,     0,     0,     0,     0,     0,<br>
0,     0,     0,     0,     0,     0,     0,     0,     0,     0,<br>
0,     0,     0,     0,     0,     0,     0,     0,     0,     0,<br>
0,     0,     0,     0,     0,     0,     0,     0,     0,     0,<br>
0,     0,     0,     0,     0,     0,     0,     0,     0,     0,<br>
0,     0,     0,     0,     0,     0,     0,     0,     0,     0,<br>
0,     0,     0,     0,     0,     0,     0,     0,     0,     0,<br>
0,     0,     0,     0,     0,     0,     0,     0,     0,     0,<br>
0,     0,     0,     0,     0,     0,     0,     0,     0,     0,<br>
0,     0,     0,     0,     0,     0,     0,     0,     0,     0,<br>
0,     0,     0,     0,     0,     0,     0,     0,     0,     0,<br>
0,     0,     0,     0,     0,     0,     0,     0,     0,     0,<br>
0,     0,     0,     0,     0,     0,     0,     0,     0,     0,<br>
0,     0,     0,     0,     0,     0,     0,     0,     0,     0,<br>
0,     0,     0,     0,     0,     0,     0,     0,     0,     0,<br>
0,     0,     0,     0,     0,     0,     0,     0,     0,     0,<br>
0,     0,     0,     0,     0,     0,     0,     0,     0,     0,<br>
0,     0,     0,     0,     0,     0,     0,     0,     0,     0,<br>
0,     0,     0,     0,     0,     0,     0,     0,     0,     0,<br>
0,     0,     0,     0,     0,     0,     0,     0,     0,     0,<br>
0,     0,     0,     0,     0,     0,     0,     0,     0,     0,<br>
0,     0,     0,     0,     0,     0,     0,     0,     0,     0,<br>
0,     0,     0,     0,     0,     0,     0,     0,     0,     0,<br>
0,     0,     0,     0,     0,     0,     0,     0,     0,     0,<br>
0,     0,     0,     0,     0,     0,     0,     0,     0,     0,<br>
0,     0,     0,     0,     0,     0,     0,     0,     0,     0,<br>
0,     0,     0,     0,     0,     0,     0,     0,     0,     0,<br>
0,     0]])</p>

harsha_g: <p>Think of this: If you have a batch of 10 sentences, you will have 10 labels (one for each sentence). If you have 512 sentences, you will have 512 labels. If you have a batch of 1 sentence, you will have 1 label. You don’t have to reshape the labels from 512 to 1. It’s only indexing. The <code>random_split</code> or or <code>DataLoader</code> (assuming you are using either one of those two) would have taken care of that for you unless you are doing this in a manual fashion.</p>
<p>Thanks <a class="mention" href="/u/kushagra_bhatia">@Kushagra_Bhatia</a> for chipping in. At one point, I was so engrossed in the batch size being 512 because it didn’t occur to me that the training (in this post) is being done one sample at a time.</p>

Kushagra_Bhatia: <p>So, what you have shown here is a vector of shape <code>512</code>. Are you calling the <em>labels given to the tokens in your sentence</em> the <strong>label</strong> tensor?</p>
<p>In sentence classification, the problem is like we have a sentence like:<br>
“Hello, my dog is cute” we convert it to token_ids that look like [1996, 15139, 1000, 14855,…0,0] i.e of shape <code>1*512</code> i.e <code>batch_size*num_tokens</code>. This tensor is called the <code>input_ids</code>. We give another tensor called the label. For each sentence, there is a label corresponding to it. Fore example in case of sentiment classification we have label Neutral(0), Negative(1) and Positive(2). So each sentence in our batch will have a label which is nothing but a number like 0,1 or 2. So, the label tensor is of shape <code>batch_size</code>.</p>

abadrawy: <p>I am not sure, this is my first time dealing with a bert trained model. But here is my colab notebook, if you’d like to take a look <a href="https://colab.research.google.com/drive/1hfkGaZvSLkxB4BqUm24E5RiF3V098u4m?usp=sharing" rel="nofollow noopener">https://colab.research.google.com/drive/1hfkGaZvSLkxB4BqUm24E5RiF3V098u4m?usp=sharing</a>.</p>

abadrawy: <p>I understand but in my case here, each sentence  does not have a single label, instead I am assuming that each sentence has 512 labels ( this is the max len output for bert) as my input is question and answers and I output is the answer. so my labels are word ids.</p>