How to fetch full batch of output when using 2 GPUs

username: marathon
cooked: <p>Hi, I am a newbie of Pytorch. Does anyone help me on this problem. Thanks a lot~<br>
I just find the batch_size of output  is 1/2  of the input when running my model on two GPUs. <strong>So, how to get the whole batch of output?</strong><br>
part of my code is as following:</p>
<pre><code class="lang-auto">device_ids = [0,1]
model = nn.DataParallel(model, device_ids=device_ids)
model = model.cuda()

print('start')
start = time.time()
for i in range(1):
    inputs = torch.zeros(100, 64, 256).cuda()
    outputs = model(inputs)
    print(outputs.shape)
end = time.time()
print('time cost:',end-start)
</code></pre>
<p><strong>where the second dim ‘64’ represents the batch_size.</strong><br>
Here is the results:</p>
<pre><code class="lang-auto">torch.Size([260, 50, 10000])
time cost: 3.447650671005249
</code></pre>
<p>Obviously the printed  dimension of batch is 1/2 of the input.</p>
<p>I am sure the two GPUs are both used in this situation.</p>

username: marathon
cooked: <p>I find tutorial here:<a href="https://pytorch.org/tutorials/beginner/blitz/data_parallel_tutorial.html" rel="noopener nofollow ugc">https://pytorch.org/tutorials/beginner/blitz/data_parallel_tutorial.html</a><br>
<div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/0/8/089beb3dea2a24478bfc36952597fd37a71bd117.png" data-download-href="https://discuss.pytorch.org/uploads/default/089beb3dea2a24478bfc36952597fd37a71bd117" title="image"><img src="https://discuss.pytorch.org/uploads/default/original/3X/0/8/089beb3dea2a24478bfc36952597fd37a71bd117.png" alt="image" data-base62-sha1="1e9SdyFy3veULkkew0qodG4HIz5" width="690" height="432" data-dominant-color="F0F1F3"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">image</span><span class="informations">890×558 36.1 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>
<p>So, I guess  there are no particularly serious errors in my situation, right?  But I don’t know how to retrieve the output  completely…<br>
p.s.: The inputs in my case are mel-spectrograms  of  speech clips with different durations. And I don’t know how to use dataloader  with data of different dimension when global padding is not feasible.</p>

username: ptrblck
cooked: <p><code>nn.DataParallel</code> will chunk the input data in dim0, so you should make sure to permute your data and adapt the model if necessary.</p>

username: marathon
cooked: <p><a class="mention" href="/u/ptrblck">@ptrblck</a>  Thank you very much for your reply.</p>
<aside class="quote no-group" data-username="ptrblck" data-post="3" data-topic="57854" data-full="true">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/ptrblck/48/1823_2.png" class="avatar"> ptrblck:</div>
<blockquote>
<p><code>nn.DataParallel</code> will chunk the input data in dim0, so you should make sure to permute your data and adapt the model if necessary.</p>
</blockquote>
</aside>
<p>Is dim0 the dimension of batch？ If so, the inputs in my code no longer need to be permuted, right? because ‘100’ in dim0 indicates the batch_size.</p>
<aside class="quote no-group" data-username="marathon" data-post="1" data-topic="57854">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/marathon/48/16548_2.png" class="avatar"> marathon:</div>
<blockquote>
<p>inputs = torch.zeros(100, 64, 256).cuda()</p>
</blockquote>
</aside>

username: ptrblck
cooked: <p>I thought dim1 represents the batch size:</p>
<aside class="quote no-group" data-username="marathon" data-post="1" data-topic="57854">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/marathon/48/16548_2.png" class="avatar"> marathon:</div>
<blockquote>
<p>where the second dim ‘64’ represents the batch_size</p>
</blockquote>
</aside>