Batch size keeps on changin, throwing `Pytorch Value Error Expected: input batch size does not match target batch size`

pregenRobot: <p>I am working on a multilabel text classification task with Bert.</p>
<p>The following is the code for generating an iterable Dataset.</p>
<pre><code class="lang-auto">from torch.utils.data import TensorDataset, DataLoader, RandomSampler, SequentialSampler

train_set = TensorDataset(X_train_id,X_train_attention, y_train)
test_set = TensorDataset(X_test_id,X_test_attention,y_test)

train_dataloader = DataLoader(
    train_set,
    sampler = RandomSampler(train_set),
    drop_last=True,
    batch_size=13
)

test_dataloader = DataLoader(
    test_set,
    sampler = SequentialSampler(test_set),
    drop_last=True,
    batch_size=13
)
</code></pre>
<p>The following are the the dimensions of the training set:</p>
<p>In<span class="chcklst-box fa fa-square-o fa-fw"></span></p>
<pre><code class="lang-auto">print(X_train_id.shape)
print(X_train_attention.shape)
print(y_train.shape)
</code></pre>
<p>Out<span class="chcklst-box fa fa-square-o fa-fw"></span></p>
<pre><code class="lang-auto">torch.Size([262754, 512])
torch.Size([262754, 512])
torch.Size([262754, 34])
</code></pre>
<p>There should be 262754 rows each with 512 columns. The output should predict the values from 34 possible labels. I am breaking them down into batches of 13.</p>
<p>Training code</p>
<pre><code class="lang-auto">optimizer = AdamW(model.parameters(), lr=2e-5)
# Training
def train(model):
    model.train()
    train_loss = 0
    for batch in train_dataloader:
        b_input_ids = batch[0].to(device)
        b_input_mask = batch[1].to(device)
        b_labels = batch[2].to(device)
        optimizer.zero_grad()
        loss, logits = model(b_input_ids, 
                             token_type_ids=None, 
                             attention_mask=b_input_mask, 
                             labels=b_labels)
        loss.backward()
        torch.nn.utils.clip_grad_norm_(model.parameters(), 1.0)
        optimizer.step()
        train_loss += loss.item()
    return train_loss


# Testing
def test(model):
    model.eval()
    val_loss = 0
    with torch.no_grad():
        for batch in test_dataloader:
            b_input_ids = batch[0].to(device)
            b_input_mask = batch[1].to(device)
            b_labels = batch[2].to(device)
            with torch.no_grad():        
                (loss, logits) = model(b_input_ids, 
                                    token_type_ids=None, 
                                    attention_mask=b_input_mask,
                                    labels=b_labels)
            val_loss += loss.item()
    return val_loss

# Train task
max_epoch = 1
train_loss_ = []
test_loss_ = []

for epoch in range(max_epoch):
    train_ = train(model)
    test_ = test(model)
    train_loss_.append(train_)
    test_loss_.append(test_)

</code></pre>
<p>Out<span class="chcklst-box fa fa-square-o fa-fw"></span></p>
<pre><code class="lang-auto">Expected input batch_size (13) to match target batch_size (442).
</code></pre>
<p>This is the description of my model:</p>
<pre><code class="lang-auto">from transformers import BertForSequenceClassification, AdamW, BertConfig

model = BertForSequenceClassification.from_pretrained(
    "cl-tohoku/bert-base-japanese-whole-word-masking", # 日本語Pre trainedモデル
    num_labels = 34, 
    output_attentions = False,
    output_hidden_states = False,
)
</code></pre>
<p>I have clearly stated that I want the batch size to be 13. However, during the training process pytorch is throwing a Runtime Error</p>
<p>Where is the number 442 even coming from? I have clearly stated that I want each batch to have a size of 13 rows.</p>
<p>I have already confirmed that each batch has input_id with dimensions [13,512], attention tensor with dimensions [13,512], and labels with dimensions [13,34].</p>
<p>I have tried caving in and using a batch size of 442 when initializing the DataLoader, but after a single batch iteration, it throws another <code>Pytorch Value Error Expected:  input batch size does not match target batch size</code>, this time showing:</p>
<pre><code class="lang-auto">ValueError: Expected input batch_size (442) to match target batch_size (15028).
</code></pre>
<p>Why does the batch size keep on changing? Where is the number 15028 even coming from?</p>
<p>The following are some of the answers I have looked through, but had no luck on applying to my source code because none of them talk about batch sizes changing:</p>
<aside class="quote quote-modified" data-post="1" data-topic="24498">
  <div class="title">
    <div class="quote-controls"></div>
    <img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/mujeeb_arshad/48/6502_2.png" class="avatar">
    <a href="https://discuss.pytorch.org/t/valueerror-expected-input-batch-size-324-to-match-target-batch-size-4/24498">ValueError: Expected input batch_size (324) to match target batch_size (4)</a> 
  </div>
  <blockquote>
    I’m getting the following error. I have tried every of the solution provided on any platform but nothings working. My dataset is of facial expressions and all the images are in grayscale. The code is pasted below: 

import torch
import torchvision
import torchvision.transforms as transforms
from torch.autograd import Variable
 
transform = transforms.Compose(
    [transforms.ToTensor(),
     transforms.Normalize((0.5, 0.5, 0.5), (0.5, 0.5, 0.5))])


trainset = torchvision.datasets.ImageFolder(ro…
  </blockquote>
</aside>

<aside class="quote quote-modified" data-post="1" data-topic="43071">
  <div class="title">
    <div class="quote-controls"></div>
    <img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/eadomenech/48/9626_2.png" class="avatar">
    <a href="https://discuss.pytorch.org/t/valueerror-expected-input-batch-size-1-to-match-target-batch-size-64/43071">ValueError: Expected input batch_size (1) to match target batch_size (64)</a> 
  </div>
  <blockquote>
    from __future__ import print_function
import argparse
import torch
import torch.nn as nn
import torch.nn.functional as F
import torch.optim as optim
from torchvision import datasets, transforms


class Net(nn.Module):
    def __init__(self):
        super(Net, self).__init__()
        self.fc1 = nn.Linear(64 * 3 * 8 * 8, 1000)
        self.fc2 = nn.Linear(1000, 200)
        self.fc3 = nn.Linear(200, 10)

    def forward(self, x):
        x = F.relu(self.fc1(x))
        x = F.relu(self.fc2(x))
  …
  </blockquote>
</aside>

<aside class="onebox stackexchange" data-onebox-src="https://stackoverflow.com/questions/54928638/pytorch-cnn-error-expected-input-batch-size-4-to-match-target-batch-size-64">
  <header class="source">

      <a href="https://stackoverflow.com/questions/54928638/pytorch-cnn-error-expected-input-batch-size-4-to-match-target-batch-size-64" target="_blank" rel="noopener nofollow ugc">stackoverflow.com</a>
  </header>

  <article class="onebox-body">
      <a href="https://stackoverflow.com/users/10615705/morpheus" target="_blank" rel="noopener nofollow ugc">
    <img alt="Morpheus" src="https://www.gravatar.com/avatar/2ba0a9d8efc5c4616ee69a4264855a24?s=256&amp;d=identicon&amp;r=PG&amp;f=y&amp;so-version=2" class="thumbnail onebox-avatar" width="256" height="256">
  </a>

<h4>
  <a href="https://stackoverflow.com/questions/54928638/pytorch-cnn-error-expected-input-batch-size-4-to-match-target-batch-size-64" target="_blank" rel="noopener nofollow ugc">Pytorch CNN error: Expected input batch_size (4) to match target batch_size (64)</a>
</h4>

<div class="tags">
  <strong>python, machine-learning, computer-vision, conv-neural-network</strong>
</div>

<div class="date">
  asked by
  
  <a href="https://stackoverflow.com/users/10615705/morpheus" target="_blank" rel="noopener nofollow ugc">
    Morpheus
  </a>
  on <a href="https://stackoverflow.com/questions/54928638/pytorch-cnn-error-expected-input-batch-size-4-to-match-target-batch-size-64" target="_blank" rel="noopener nofollow ugc">03:05PM - 28 Feb 19 UTC</a>
</div>

  </article>

  <div class="onebox-metadata">
    
    
  </div>

  <div style="clear: both"></div>
</aside>

<p>Thanks in advance. Your support is truly appreciated <img src="https://discuss.pytorch.org/images/emoji/apple/slight_smile.png?v=12" title=":slight_smile:" class="emoji" alt=":slight_smile:" loading="lazy" width="20" height="20"></p>

ptrblck: <aside class="quote no-group" data-username="pregenRobot" data-post="1" data-topic="98085">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/pregenrobot/48/29431_2.png" class="avatar"> pregenRobot:</div>
<blockquote>
<p>Where is the number 442 even coming from? I have clearly stated that I want each batch to have a size of 13 rows.</p>
</blockquote>
</aside>
<p>I think it’s the flattened target tensor: <code>[13*43=442]</code>.<br>
Based on the <a href="https://huggingface.co/transformers/model_doc/bert.html#transformers.BertForSequenceClassification">docs</a> for <code>BertForSequenceClassification</code>, the targets are expected as <code>[batch_size]</code> so they are most likely internally flattened.</p>
<p>I’m not familiar with the implementation, but since it’s a classification use case I guess the targets have the same (or similar) requirements as the targets for <code>nn.CrossEntropyLoss</code>.<br>
I.e. they are expected in the shape <code>[batch_size]</code> as <code>LongTensor</code>s containing the class indices in the range <code>[0, nb_classes-1]</code> for a multi-class classification use case.</p>

pregenRobot: <aside class="quote no-group" data-username="ptrblck" data-post="2" data-topic="98085">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/ptrblck/48/1823_2.png" class="avatar"> ptrblck:</div>
<blockquote>
<p><code>LongTensor</code> s</p>
</blockquote>
</aside>
<p>Thanks for the feedback.</p>
<p>However, I have confirmed that during each iteration during the training, my dimensions are:</p>
<p>input_ids: [13,512]<br>
input_masks: [13,512]<br>
y_labels: [13,34]</p>
<p>where 34 is the number of possible labels (as one hot vectors).<br>
Ex:<br>
[0,0,0,1,1,0,0,… 0,1]</p>
<p>Doesn’t this meet the requirement for calculating Cross EntropyLoss ?</p>
<p><img src="https://discuss.pytorch.org/uploads/default/original/3X/d/1/d1878f665435cd94b2dc84df6e2b22f890279e47.png" alt="Screen Shot 2020-10-06 at 10.32.50" data-base62-sha1="tTAe6Xsz8JSLhR60wKX5ZdDGF5J" width="408" height="203"></p>

ptrblck: <aside class="quote no-group" data-username="pregenRobot" data-post="3" data-topic="98085">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/pregenrobot/48/29431_2.png" class="avatar"> pregenRobot:</div>
<blockquote>
<p>Doesn’t this meet the requirement for calculating Cross EntropyLoss ?</p>
</blockquote>
</aside>
<p>No, since class labels are expected.<br>
Use <code>target = torch.argmax(target, dim=1)</code> to create the expected target tensor.</p>