Repository: pytorch
Issue: Operator with only `Tensor[][]` args unsupported by dispatcher
Labels: triaged,module: custom-operators,module: dispatch,oncall: pt2,module: pt2-dispatcher

### 🐛 Describe the bug

```py
In [1]: import torch

In [2]: def return_first(l):
   ...:     return l[0][0]
   ...:

In [3]: lib = torch.library.Library("my_lib", "DEF")

In [4]: lib.define("return_first(Tensor(a)[][] l) -> Tensor(a)")
Out[4]: 'return_first'

In [5]: lib.impl("return_first", return_first, "CUDA")

In [6]: torch.ops.my_lib.return_first([[torch.empty((1,), device="cuda")]])
---------------------------------------------------------------------------
NotImplementedError                       Traceback (most recent call last)
Cell In[6], line 1
----> 1 torch.ops.my_lib.return_first([[torch.empty((1,), device="cuda")]])

File /.../lib/python3.10/site-packages/torch/_ops.py:760, in OpOverloadPacket.__call__(self, *args, **kwargs)
    755 def __call__(self, *args, **kwargs):
    756     # overloading __call__ to ensure torch.ops.foo.bar()
    757     # is still callable from JIT
    758     # We save the function ptr as the `op` attribute on
    759     # OpOverloadPacket to access it here.
--> 760     return self._op(*args, **kwargs or {})

NotImplementedError: There were no tensor arguments to this function (e.g., you passed an empty list of Tensors), but no fallback function is registered for schema my_lib::return_first.  This usually means that this function requires a non-empty list of Tensors, or that you (the operator writer) forgot to register a fallback function.  Available functions are [CUDA, Meta, BackendSelect, Python, FuncTorchDynamicLayerBackMode, Functionalize, Named, Conjugate, Negative, ZeroTensor, ADInplaceOrView, AutogradOther, AutogradCPU, AutogradCUDA, AutogradXLA, AutogradMPS, AutogradXPU, AutogradHPU, AutogradLazy, AutogradMeta, Tracer, AutocastCPU, AutocastCUDA, FuncTorchBatched, BatchedNestedTensor, FuncTorchVmapMode, Batched, VmapMode, FuncTorchGradWrapper, PythonTLSSnapshot, FuncTorchDynamicLayerFrontMode, PreDispatch, PythonDispatcher].

CUDA: registered at /dev/null:3526 [kernel]
Meta: registered at /opt/conda/conda-bld/pytorch_1698219967775/work/aten/src/ATen/core/MetaFallbackKernel.cpp:23 [backend fallback]
BackendSelect: fallthrough registered at /opt/conda/conda-bld/pytorch_1698219967775/work/aten/src/ATen/core/BackendSelectFallbackKernel.cpp:3 [backend fallback]
Python: registered at /opt/conda/conda-bld/pytorch_1698219967775/work/aten/src/ATen/core/PythonFallbackKernel.cpp:153 [backend fallback]
FuncTorchDynamicLayerBackMode: registered at /opt/conda/conda-bld/pytorch_1698219967775/work/aten/src/ATen/functorch/DynamicLayer.cpp:498 [backend fallback]
Functionalize: registered at /opt/conda/conda-bld/pytorch_1698219967775/work/aten/src/ATen/FunctionalizeFallbackKernel.cpp:302 [backend fallback]
Named: registered at /opt/conda/conda-bld/pytorch_1698219967775/work/aten/src/ATen/core/NamedRegistrations.cpp:7 [backend fallback]
Conjugate: registered at /opt/conda/conda-bld/pytorch_1698219967775/work/aten/src/ATen/ConjugateFallback.cpp:17 [backend fallback]
Negative: registered at /opt/conda/conda-bld/pytorch_1698219967775/work/aten/src/ATen/native/NegateFallback.cpp:19 [backend fallback]
ZeroTensor: registered at /opt/conda/conda-bld/pytorch_1698219967775/work/aten/src/ATen/ZeroTensorFallback.cpp:86 [backend fallback]
ADInplaceOrView: fallthrough registered at /opt/conda/conda-bld/pytorch_1698219967775/work/aten/src/ATen/core/VariableFallbackKernel.cpp:86 [backend fallback]
AutogradOther: registered at /opt/conda/conda-bld/pytorch_1698219967775/work/aten/src/ATen/core/VariableFallbackKernel.cpp:53 [backend fallback]
AutogradCPU: registered at /opt/conda/conda-bld/pytorch_1698219967775/work/aten/src/ATen/core/VariableFallbackKernel.cpp:57 [backend fallback]
AutogradCUDA: registered at /opt/conda/conda-bld/pytorch_1698219967775/work/aten/src/ATen/core/VariableFallbackKernel.cpp:65 [backend fallback]
AutogradXLA: registered at /opt/conda/conda-bld/pytorch_1698219967775/work/aten/src/ATen/core/VariableFallbackKernel.cpp:69 [backend fallback]
AutogradMPS: registered at /opt/conda/conda-bld/pytorch_1698219967775/work/aten/src/ATen/core/VariableFallbackKernel.cpp:77 [backend fallback]
AutogradXPU: registered at /opt/conda/conda-bld/pytorch_1698219967775/work/aten/src/ATen/core/VariableFallbackKernel.cpp:61 [backend fallback]
AutogradHPU: registered at /opt/conda/conda-bld/pytorch_1698219967775/work/aten/src/ATen/core/VariableFallbackKernel.cpp:90 [backend fallback]
AutogradLazy: registered at /opt/conda/conda-bld/pytorch_1698219967775/work/aten/src/ATen/core/VariableFallbackKernel.cpp:73 [backend fallback]
AutogradMeta: registered at /opt/conda/conda-bld/pytorch_1698219967775/work/aten/src/ATen/core/VariableFallbackKernel.cpp:81 [backend fallback]
Tracer: registered at /opt/conda/conda-bld/pytorch_1698219967775/work/torch/csrc/autograd/TraceTypeManual.cpp:297 [backend fallback]
AutocastCPU: fallthrough registered at /opt/conda/conda-bld/pytorch_1698219967775/work/aten/src/ATen/autocast_mode.cpp:378 [backend fallback]
AutocastCUDA: fallthrough registered at /opt/conda/conda-bld/pytorch_1698219967775/work/aten/src/ATen/autocast_mode.cpp:244 [backend fallback]
FuncTorchBatched: registered at /opt/conda/conda-bld/pytorch_1698219967775/work/aten/src/ATen/functorch/LegacyBatchingRegistrations.cpp:720 [backend fallback]
BatchedNestedTensor: registered at /opt/conda/conda-bld/pytorch_1698219967775/work/aten/src/ATen/functorch/LegacyBatchingRegistrations.cpp:746 [backend fallback]
FuncTorchVmapMode: fallthrough registered at /opt/conda/conda-bld/pytorch_1698219967775/work/aten/src/ATen/functorch/VmapModeRegistrations.cpp:28 [backend fallback]
Batched: registered at /opt/conda/conda-bld/pytorch_1698219967775/work/aten/src/ATen/LegacyBatchingRegistrations.cpp:1075 [backend fallback]
VmapMode: fallthrough registered at /opt/conda/conda-bld/pytorch_1698219967775/work/aten/src/ATen/VmapModeRegistrations.cpp:33 [backend fallback]
FuncTorchGradWrapper: registered at /opt/conda/conda-bld/pytorch_1698219967775/work/aten/src/ATen/functorch/TensorWrapper.cpp:203 [backend fallback]
PythonTLSSnapshot: registered at /opt/conda/conda-bld/pytorch_1698219967775/work/aten/src/ATen/core/PythonFallbackKernel.cpp:161 [backend fallback]
FuncTorchDynamicLayerFrontMode: registered at /opt/conda/conda-bld/pytorch_1698219967775/work/aten/src/ATen/functorch/DynamicLayer.cpp:494 [backend fallback]
PreDispatch: registered at /opt/conda/conda-bld/pytorch_1698219967775/work/aten/src/ATen/core/PythonFallbackKernel.cpp:165 [backend fallback]
PythonDispatcher: registered at /opt/conda/conda-bld/pytorch_1698219967775/work/aten/src/ATen/core/PythonFallbackKernel.cpp:157 [backend fallback]
```

### Versions

PyTorch version: 2.2.0.dev20231025

cc @ezyang @chauhang @penguinwu @zou3519 @bdhirsh @yf225