Backward fails when the batch size is less than 5

username: kranti
cooked: <p>I am trying to run a simple model</p>
<pre><code class="lang-auto">import torch
import torchvision.models as models

model = models.resnet18(pretrained=True)
device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
# model = torch.nn.DataParallel(model)
model.to(device)

loss_criterion = torch.nn.CrossEntropyLoss()
inp = torch.randn(5, 3, 224, 224).to(device)
target = torch.empty(5, dtype=torch.long).random_(1000).to(device)
output = model(inp)
loss = loss_criterion(output, target)
loss.backward()
</code></pre>
<p>I observed that backward fails when I give the input with batch size less than 5. Further if I am using <code>nn.dataparallel</code>, the model also requires there are atleaset 5 samples per batch other wise it gives an error.<br>
I am attaching the screenshot of the error below.<br>
<div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/4/d/4dac53d1fd5efb395f289f1a5d5f04cbee5cc0b4.png" data-download-href="https://discuss.pytorch.org/uploads/default/4dac53d1fd5efb395f289f1a5d5f04cbee5cc0b4" title="Screenshot 2021-05-01 at 10.11.01 PM"><img src="https://discuss.pytorch.org/uploads/default/original/3X/4/d/4dac53d1fd5efb395f289f1a5d5f04cbee5cc0b4.png" alt="Screenshot 2021-05-01 at 10.11.01 PM" data-base62-sha1="b57YrZRBCT5VDF3o9dTs3L1nD3C" width="690" height="77" data-dominant-color="161616"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">Screenshot 2021-05-01 at 10.11.01 PM</span><span class="informations">1249×141 13.6 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>
<p>My system:<br>
Ubuntu: 20.04<br>
Pytorch:1.8.1<br>
CUDA: 11.1</p>

username: Shima_Shahfar
cooked: <p>May I know how many devices you are running your script on? Or should I say what is the number of GPUs you are using?</p>

username: ptrblck
cooked: <p>Could you post the output of <code>python -m torch.utils.collect_env</code>? Also, how did you install PyTorch (pip wheel, conda binary, source build)?</p>

username: kranti
cooked: <p>I have tested with both cases, using a single GPUs and multiple GPUs. In both the cases, I have found that the minimum data points in each GPU to be 5 for correctly backpropagating.</p>

username: kranti
cooked: <p>Collecting environment information…<br>
PyTorch version: 1.8.1+cu111<br>
Is debug build: False<br>
CUDA used to build PyTorch: 11.1<br>
ROCM used to build PyTorch: N/A</p>
<p>OS: Ubuntu 20.04.2 LTS (x86_64)<br>
GCC version: (Ubuntu 9.3.0-17ubuntu1~20.04) 9.3.0<br>
Clang version: Could not collect<br>
CMake version: version 3.16.3</p>
<p>Python version: 3.8 (64-bit runtime)<br>
Is CUDA available: True<br>
CUDA runtime version: Could not collect<br>
GPU models and configuration:<br>
GPU 0: GeForce GTX 1080 Ti<br>
GPU 1: GeForce GTX 1080 Ti<br>
GPU 2: GeForce GTX 1080 Ti<br>
GPU 3: GeForce GTX 1080 Ti</p>
<p>Nvidia driver version: 455.32.00<br>
cuDNN version: Probably one of the following:<br>
/usr/lib/x86_64-linux-gnu/libcudnn.so.7.0.5<br>
/usr/local/cuda-10.1/targets/x86_64-linux/lib/libcudnn.so.7<br>
/usr/local/cuda-9.1/targets/x86_64-linux/lib/libcudnn.so.7<br>
HIP runtime version: N/A<br>
MIOpen runtime version: N/A</p>
<p>Versions of relevant libraries:<br>
[pip3] numpy==1.20.2<br>
[pip3] torch==1.8.1+cu111<br>
[pip3] torchaudio==0.8.1<br>
[pip3] torchsummary==1.5.1<br>
[pip3] torchvision==0.9.1+cu111<br>
[conda] blas                      1.0                         mkl<br>
[conda] mkl                       2020.1                      217<br>
[conda] mkl-service               2.3.0            py38he904b0f_0<br>
[conda] mkl_fft                   1.1.0            py38h23d657b_0<br>
[conda] mkl_random                1.1.1            py38h0573a6f_0<br>
[conda] numpy                     1.19.4                   pypi_0    pypi<br>
[conda] numpydoc                  1.1.0                      py_0<br>
[conda] torch                     1.7.0                    pypi_0    pypi<br>
[conda] torchaudio                0.7.0                    pypi_0    pypi<br>
[conda] torchvision               0.8.1                    pypi_0    pypi</p>

username: kranti
cooked: <aside class="quote no-group" data-username="ptrblck" data-post="3" data-topic="120008">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/ptrblck/48/1823_2.png" class="avatar"> ptrblck:</div>
<blockquote>
<p>PyTorch (pip wheel,</p>
</blockquote>
</aside>
<p>I installed it using pip wheel.</p>

username: ptrblck
cooked: <p>Thanks, could you create a new virtual environment and install the conda binaries, as the pip wheels in <code>1.8.1</code> currently fail for <code>sm_61</code>. <a href="https://github.com/pytorch/pytorch/issues/56747">Related issue</a></p>

username: kranti
cooked: <p>Thanks for the reply. I tested with conda installation and it works fine.</p>