DataParallel, Expected input batch_size (64) to match target batch_size (32)

zeng: <p>model = nn.DataParallel(model, device_ids=[0, 1])</p>
<p>context, ctx_length = batch.context<br>
response, rsp_length = batch.response<br>
label = batch.label</p>
<p>prediction = self.model(context, response)<br>
loss = self.criterion(prediction, label)</p>
<p>the batch size is 32, the size of <strong>prediction</strong> is 32 * 2 =  64, but the size of the <strong>label</strong> is still 32, which cause the <strong>criterion</strong> to raise Error.</p>
<p>What’s the problem?</p>

Shani_Gamrian: <p>How come the predictions doesn’t match the labels? you should set the output of the prediction to 1 (will be 32 if the batch size is 32).</p>

zeng: <p>Acctually, the size of prediction is [64, 2], however, the size of label is [32]. That’s what I am confused. When I change <strong>device_ids=[0]</strong>, the size of predication become [32, 2].</p>

Shani_Gamrian: <p>I don’t know what model you’re using so it’s hard for me to help you, but to find the problem I would probably change the batch size to 1 and see what happens, just to make sure that the model’s output size is really 1.</p>

Aayushee_Gupta: <p>Hi. Were you able to resolve this?<br>
I am also using DataParallel to use multiple GPUs in Pytorch script but face a similar error with batch size 64 and 4 GPUs:</p>
<p>ValueError: Expected input batch_size (256) to match target batch_size (64)</p>

sdeva14: <p>It is too late to answer, but I leave the answer here for other people.<br>
I recently had the same problem for multi-gpu usage.</p>
<p><strong>Problem:</strong> It seems like the problem is that we need to pass Tensor-type variable to model.forward for DataParallel.<br>
Otherwise, it is not going to be scattered, but copied as the number of gpu.</p>
<p><strong>Solution:</strong> easy solution is to pass Tensor variable to model.forward, instead of type casting in the forward function.<br>
Another solution is to return loss from the forward function, then normalize loss by the batch size.</p>
<p>Please see here for detail about DataParallel<br>
<aside class="onebox githubblob">
  <header class="source">
      <a href="https://github.com/pytorch/pytorch/blob/master/torch/nn/parallel/data_parallel.py" target="_blank" rel="nofollow noopener">github.com</a>
  </header>
  <article class="onebox-body">
    <h4><a href="https://github.com/pytorch/pytorch/blob/master/torch/nn/parallel/data_parallel.py" target="_blank" rel="nofollow noopener">pytorch/pytorch/blob/master/torch/nn/parallel/data_parallel.py</a></h4>
<pre><code class="lang-py">import operator
import torch
import warnings
from itertools import chain
from ..modules import Module
from .scatter_gather import scatter_kwargs, gather
from .replicate import replicate
from .parallel_apply import parallel_apply
from torch.cuda._utils import _get_device_index


def _check_balance(device_ids):
    imbalance_warn = """
    There is an imbalance between your GPUs. You may want to exclude GPU {} which
    has less than 75% of the memory or cores of GPU {}. You can do so by setting
    the device_ids argument to DataParallel, or by setting the CUDA_VISIBLE_DEVICES
    environment variable."""
    device_ids = list(map(lambda x: _get_device_index(x, True), device_ids))
    dev_props = [torch.cuda.get_device_properties(i) for i in device_ids]

</code></pre>

  This file has been truncated. <a href="https://github.com/pytorch/pytorch/blob/master/torch/nn/parallel/data_parallel.py" target="_blank" rel="nofollow noopener">show original</a>

  </article>
  <div class="onebox-metadata">
    
    
  </div>
  <div style="clear: both"></div>
</aside>
</p>

harsh_grover: <p><a class="mention" href="/u/sdeva14">@sdeva14</a> Hey can you please post the first solution how to do that, it is not becoming clear to me.</p>
<p>Thanks in advance <img src="https://discuss.pytorch.org/images/emoji/apple/slight_smile.png?v=9" title=":slight_smile:" class="emoji" alt=":slight_smile:"></p>

harsh_grover: <p><a class="mention" href="/u/aayushee_gupta">@Aayushee_Gupta</a>  Hey, did you resolve this?</p>

harsh_grover: <p><a class="mention" href="/u/zeng">@zeng</a> Hey, how did you resolve this?</p>