Repository: pytorch
Issue: [JIT] Zero-channel conv2d cannot be applied with `optimize_for_inference`
Labels: oncall: jit

### 🐛 Describe the bug

```python
import torch

class MyModule(torch.nn.Module):
    def __init__(self):
        super(MyModule, self).__init__()
        self.conv2d = torch.nn.Conv2d(0, 1, kernel_size=(1, 1))

    def forward(self, v0):
        return self.conv2d(v0)

# jit ~ trace
m = MyModule()
m.eval()
inp = torch.randn(1, 0, 1, 1)
print(f"{m(inp).shape = }") # [1, 0, 1, 1]

jit_m = torch.jit.trace(m, torch.randn(1, 0, 1, 1))
print(f"{jit_m(inp).shape = }") # [1, 0, 1, 1]
# Zero-channel conv2d failed when doing `optimize_for_inference`.
opt_m = torch.jit.optimize_for_inference(jit_m) # ❌ compiler failure!
print(f"{opt_m(inp).shape = }")
```

Zero-channel conv2d layer, which is legal for both eager and jit and basically returns an empty tensor ~ is however invalid (i.e., cannot be compiled) with `torch.jit.optimize_for_inference`.

```
/miniconda3/lib/python3.9/site-packages/torch/nn/init.py:405: UserWarning: Initializing zero-element tensors is a no-op
  warnings.warn("Initializing zero-element tensors is a no-op")
m(inp).shape = torch.Size([1, 0, 1, 1])
tensor([], size=(1, 0, 1, 1), grad_fn=<ConvolutionBackward0>)
jit_m(inp).shape = torch.Size([1, 0, 1, 1])
Traceback (most recent call last):
  File "test.py", line 54, in <module>
    opt_m = torch.jit.optimize_for_inference(jit_m)
  File "/miniconda3/lib/python3.9/site-packages/torch/jit/_freeze.py", line 217, in optimize_for_inference
    torch._C._jit_pass_optimize_for_inference(mod._c, other_methods)
RuntimeError: could not create a descriptor for a dilated convolution forward propagation primitive
```

This looks like a dispatch bug that should be fixed by falling back to some default implementation.

*Of course this zero-channel conv2d bug is not user-facing for being automatically found by a fuzzer -- but fixing it could improve PyTorch compiler's robustness and keep code compatibility between interpreter and compiler.

### Versions


<details><summary><b>Env </b> <i>[click to expand]</i></summary>
<div>

```python
"""
Collecting environment information...
PyTorch version: 1.14.0.dev20221202+cu117
Is debug build: False
CUDA used to build PyTorch: 11.7
ROCM used to build PyTorch: N/A

OS: Ubuntu 22.04.1 LTS (x86_64)
GCC version: (Ubuntu 11.3.0-1ubuntu1~22.04) 11.3.0
Clang version: Could not collect
CMake version: version 3.25.0
Libc version: glibc-2.35

Python version: 3.9.12 (main, Apr  5 2022, 06:56:58)  [GCC 7.5.0] (64-bit runtime)
Python platform: Linux-5.15.0-56-generic-x86_64-with-glibc2.35
Is CUDA available: True
CUDA runtime version: 11.6.124
CUDA_MODULE_LOADING set to: LAZY
GPU models and configuration: 
GPU 0: NVIDIA GeForce RTX 3090
GPU 1: NVIDIA GeForce RTX 3090
GPU 2: NVIDIA GeForce RTX 3090

Nvidia driver version: 515.86.01
cuDNN version: Probably one of the following:
/usr/lib/x86_64-linux-gnu/libcudnn.so.8.4.1
/usr/lib/x86_64-linux-gnu/libcudnn_adv_infer.so.8.4.1
/usr/lib/x86_64-linux-gnu/libcudnn_adv_train.so.8.4.1
/usr/lib/x86_64-linux-gnu/libcudnn_cnn_infer.so.8.4.1
/usr/lib/x86_64-linux-gnu/libcudnn_cnn_train.so.8.4.1
/usr/lib/x86_64-linux-gnu/libcudnn_ops_infer.so.8.4.1
/usr/lib/x86_64-linux-gnu/libcudnn_ops_train.so.8.4.1
HIP runtime version: N/A
MIOpen runtime version: N/A
Is XNNPACK available: True

Versions of relevant libraries:
[pip3] mypy-extensions==0.4.3
[pip3] numpy==1.23.3
[pip3] onnx2torch==1.5.3
[pip3] torch==1.14.0.dev20221202+cu117
[pip3] torchaudio==0.14.0.dev20221203+cu117
[pip3] torchtriton==2.0.0+0d7e753227
[pip3] torchvision==0.15.0.dev20221203+cpu
[conda] numpy                     1.23.3                   pypi_0    pypi
[conda] onnx2torch                1.5.3                    pypi_0    pypi
[conda] torch                     1.14.0.dev20221202+cu117          pypi_0    pypi
[conda] torchaudio                0.14.0.dev20221203+cu117          pypi_0    pypi
[conda] torchtriton               2.0.0+0d7e753227          pypi_0    pypi
[conda] torchvision               0.15.0.dev20221203+cpu          pypi_0    pypi
"""
```

</div>
</details>


cc @EikanWang @jgong5 @wenzhe-nrv @sanchitintel