Repository: pytorch
Issue: Channels last mode doesn't work with inputs with 0 batch dimension
Labels: module: nn,triaged,module: memory format

## 🐛 Bug

 2d convolution layers do not work with inputs with zero batch dimension when used with the `channels_last` format. I couldn't tell if this is meant to be supported from the docs. I understand from the [memory format tutorial](https://pytorch.org/tutorials/intermediate/memory_format_tutorial.html) there is some ambiguity when the channels  or width/height = 1, but I'm not sure if that applies here or not.
Running conv layers on tensors with an empty batch dimension is a fairly common use case in architectures with conditional convolutions, e.g two-stage object detectors.  

## To Reproduce

Running this code
```python
import torch
x = torch.randn(0,3,100,50)
conv = torch.nn.Conv2d(3,16,(3,3))
conv = conv.to(memory_format=torch.channels_last)
preds = conv(x)
```
produces this error:
```
Traceback (most recent call last):
  File "/usr/local/lib/python3.6/dist-packages/IPython/core/interactiveshell.py", line 3296, in run_code
    exec(code_obj, self.user_global_ns, self.user_ns)
  File "<ipython-input-43-c70bce315b67>", line 1, in <module>
    cnn(empty_img)
  File "/home/soren/.local/lib/python3.6/site-packages/torch/nn/modules/module.py", line 1051, in _call_impl
    return forward_call(*input, **kwargs)
  File "/home/soren/.local/lib/python3.6/site-packages/torch/nn/modules/conv.py", line 443, in forward
    return self._conv_forward(input, self.weight, self.bias)
  File "/home/soren/.local/lib/python3.6/site-packages/torch/nn/modules/conv.py", line 440, in _conv_forward
    self.padding, self.dilation, self.groups)
RuntimeError: view size is not compatible with input tensor's size and stride (at least one dimension spans across two contiguous subspaces). Use .reshape(...) instead.
```
This happens in both inference mode and normally, and regardless of calling `.to(memory_format=torch.channels_last)` on the input beforehand.

## Expected behavior
`preds` should be a tensor with output shape (0,16,98,48), as happens when the layer runs with contiguous format.

## Environment

```
Collecting environment information...
PyTorch version: 1.9.0+cu111
Is debug build: False
CUDA used to build PyTorch: 11.1
ROCM used to build PyTorch: N/A

OS: Ubuntu 18.04.5 LTS (x86_64)
GCC version: (Ubuntu 7.5.0-3ubuntu1~18.04) 7.5.0
Clang version: 6.0.0-1ubuntu2 (tags/RELEASE_600/final)
CMake version: version 3.10.2
Libc version: glibc-2.25

Python version: 3.6 (64-bit runtime)
Python platform: Linux-5.4.0-80-generic-x86_64-with-Ubuntu-18.04-bionic
Is CUDA available: True
CUDA runtime version: 11.1.105
GPU models and configuration: GPU 0: GeForce GTX 1060
Nvidia driver version: 460.56
cuDNN version: /usr/lib/x86_64-linux-gnu/libcudnn.so.7.5.0
HIP runtime version: N/A
MIOpen runtime version: N/A

Versions of relevant libraries:
[pip3] msgpack-numpy==0.4.4.3
[pip3] numpy==1.19.0
[pip3] torch==1.9.0+cu111
[pip3] torch-model-archiver==0.1.0b20200409
[pip3] torchaudio==0.9.0
[pip3] torchserve==0.0.1b20200409
[pip3] torchvision==0.10.0+cu111
[conda] Could not collect

```



cc @albanD @mruberry @jbschlosser @VitalyFedyunin @jamesr66a @ppwwyyxx