Repository: pytorch
Issue: Add nesting of nested Tensor
Labels: triaged,module: nestedtensor

### 🚀 The feature, motivation and pitch

Currently the nested tensor implementation isn't very nested. The `nested_tensor` function allows for a list of `Tensor`s but not a list of `NestedTensor`:

```
>>> import torch
>>> torch.__version__
'1.12.0.dev20220406+cu113'
>>> a = torch.rand([10])
>>> b = torch.rand([8])
>>> nested = torch.nested_tensor([a, b])
>>> c = torch.rand([4])
>>> d = torch.rand([7])
>>> nested2 = torch.nested_tensor([c, d])
>>> double_nested = torch.nested_tensor([nested, nested2])
/usr/lib/python3/dist-packages/requests/__init__.py:89: RequestsDependencyWarning: urllib3 (1.26.9) or chardet (3.0.4) doesn't match a supported version!
  warnings.warn("urllib3 ({}) or chardet ({}) doesn't match a supported "
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
NotImplementedError: Could not run 'aten::nested_tensor' with arguments from the 'NestedTensor' backend. This could be because the operator doesn't exist for this backend, or was omitted during the selective/custom build process (if using custom build). If you are a Facebook employee using PyTorch on mobile, please visit https://fburl.com/ptmfixes for possible resolutions. 'aten::nested_tensor' is only available for these backends: [Dense, FPGA, ORT, Vulkan, Metal, Meta, Quantized, CustomRNGKeyId, MkldnnCPU, Sparse, SparseCsrCPU, SparseCsrCUDA, NestedTensor, BackendSelect, Python, Named, Conjugate, Negative, ZeroTensor, FuncTorchDynamicLayerBackMode, ADInplaceOrView, AutogradOther, AutogradFunctionality, AutogradNestedTensor, Tracer, AutocastCPU, Autocast, FuncTorchBatched, FuncTorchVmapMode, Batched, VmapMode, FuncTorchGradWrapper, Functionalize, PythonTLSSnapshot, FuncTorchDynamicLayerFrontMode, TESTING_ONLY_GenericWrapper, TESTING_ONLY_GenericMode, UNKNOWN_TENSOR_TYPE_ID, UNKNOWN_TENSOR_TYPE_ID, CPU, CUDA, HIP, XLA, MLC, XPU, VE, Lazy, PrivateUse1, PrivateUse2, PrivateUse3, UNKNOWN_TENSOR_TYPE_ID, QuantizedCUDA, UNKNOWN_TENSOR_TYPE_ID, UNKNOWN_TENSOR_TYPE_ID, UNKNOWN_TENSOR_TYPE_ID, QuantizedXPU, UNKNOWN_TENSOR_TYPE_ID, UNKNOWN_TENSOR_TYPE_ID, UNKNOWN_TENSOR_TYPE_ID, UNKNOWN_TENSOR_TYPE_ID, UNKNOWN_TENSOR_TYPE_ID, UNKNOWN_TENSOR_TYPE_ID, UNKNOWN_TENSOR_TYPE_ID, SparseCPU, SparseCUDA, UNKNOWN_TENSOR_TYPE_ID, UNKNOWN_TENSOR_TYPE_ID, SparseXPU, UNKNOWN_TENSOR_TYPE_ID, UNKNOWN_TENSOR_TYPE_ID, UNKNOWN_TENSOR_TYPE_ID, UNKNOWN_TENSOR_TYPE_ID].

Undefined: registered at aten/src/ATen/RegisterCompositeImplicitAutograd.cpp:10423 [math kernel]
CPU: registered at aten/src/ATen/RegisterCompositeImplicitAutograd.cpp:10423 [math kernel]
CUDA: registered at aten/src/ATen/RegisterCompositeImplicitAutograd.cpp:10423 [math kernel]
HIP: registered at aten/src/ATen/RegisterCompositeImplicitAutograd.cpp:10423 [math kernel]
XLA: registered at aten/src/ATen/RegisterCompositeImplicitAutograd.cpp:10423 [math kernel]
MLC: registered at aten/src/ATen/RegisterCompositeImplicitAutograd.cpp:10423 [math kernel]
XPU: registered at aten/src/ATen/RegisterCompositeImplicitAutograd.cpp:10423 [math kernel]
HPU: registered at aten/src/ATen/RegisterCompositeImplicitAutograd.cpp:10423 [math kernel]
VE: registered at aten/src/ATen/RegisterCompositeImplicitAutograd.cpp:10423 [math kernel]
Lazy: registered at aten/src/ATen/RegisterCompositeImplicitAutograd.cpp:10423 [math kernel]
PrivateUse1: registered at aten/src/ATen/RegisterCompositeImplicitAutograd.cpp:10423 [math kernel]
PrivateUse2: registered at aten/src/ATen/RegisterCompositeImplicitAutograd.cpp:10423 [math kernel]
PrivateUse3: registered at aten/src/ATen/RegisterCompositeImplicitAutograd.cpp:10423 [math kernel]
FPGA: registered at aten/src/ATen/RegisterCompositeImplicitAutograd.cpp:10423 [math kernel]
ORT: registered at aten/src/ATen/RegisterCompositeImplicitAutograd.cpp:10423 [math kernel]
Vulkan: registered at aten/src/ATen/RegisterCompositeImplicitAutograd.cpp:10423 [math kernel]
Metal: registered at aten/src/ATen/RegisterCompositeImplicitAutograd.cpp:10423 [math kernel]
Meta: registered at aten/src/ATen/RegisterCompositeImplicitAutograd.cpp:10423 [math kernel]
QuantizedCPU: registered at aten/src/ATen/RegisterCompositeImplicitAutograd.cpp:10423 [math kernel]
QuantizedCUDA: registered at aten/src/ATen/RegisterCompositeImplicitAutograd.cpp:10423 [math kernel]
UNKNOWN_TENSOR_TYPE_ID: registered at aten/src/ATen/RegisterCompositeImplicitAutograd.cpp:10423 [math kernel]
UNKNOWN_TENSOR_TYPE_ID: registered at aten/src/ATen/RegisterCompositeImplicitAutograd.cpp:10423 [math kernel]
UNKNOWN_TENSOR_TYPE_ID: registered at aten/src/ATen/RegisterCompositeImplicitAutograd.cpp:10423 [math kernel]
QuantizedXPU: registered at aten/src/ATen/RegisterCompositeImplicitAutograd.cpp:10423 [math kernel]
UNKNOWN_TENSOR_TYPE_ID: registered at aten/src/ATen/RegisterCompositeImplicitAutograd.cpp:10423 [math kernel]
UNKNOWN_TENSOR_TYPE_ID: registered at aten/src/ATen/RegisterCompositeImplicitAutograd.cpp:10423 [math kernel]
UNKNOWN_TENSOR_TYPE_ID: registered at aten/src/ATen/RegisterCompositeImplicitAutograd.cpp:10423 [math kernel]
UNKNOWN_TENSOR_TYPE_ID: registered at aten/src/ATen/RegisterCompositeImplicitAutograd.cpp:10423 [math kernel]
UNKNOWN_TENSOR_TYPE_ID: registered at aten/src/ATen/RegisterCompositeImplicitAutograd.cpp:10423 [math kernel]
UNKNOWN_TENSOR_TYPE_ID: registered at aten/src/ATen/RegisterCompositeImplicitAutograd.cpp:10423 [math kernel]
CustomRNGKeyId: registered at aten/src/ATen/RegisterCompositeImplicitAutograd.cpp:10423 [math kernel]
MkldnnCPU: registered at aten/src/ATen/RegisterCompositeImplicitAutograd.cpp:10423 [math kernel]
SparseCPU: registered at aten/src/ATen/RegisterCompositeImplicitAutograd.cpp:10423 [math kernel]
SparseCUDA: registered at aten/src/ATen/RegisterCompositeImplicitAutograd.cpp:10423 [math kernel]
SparseHIP: registered at aten/src/ATen/RegisterCompositeImplicitAutograd.cpp:10423 [math kernel]
UNKNOWN_TENSOR_TYPE_ID: registered at aten/src/ATen/RegisterCompositeImplicitAutograd.cpp:10423 [math kernel]
UNKNOWN_TENSOR_TYPE_ID: registered at aten/src/ATen/RegisterCompositeImplicitAutograd.cpp:10423 [math kernel]
SparseXPU: registered at aten/src/ATen/RegisterCompositeImplicitAutograd.cpp:10423 [math kernel]
UNKNOWN_TENSOR_TYPE_ID: registered at aten/src/ATen/RegisterCompositeImplicitAutograd.cpp:10423 [math kernel]
SparseVE: registered at aten/src/ATen/RegisterCompositeImplicitAutograd.cpp:10423 [math kernel]
UNKNOWN_TENSOR_TYPE_ID: registered at aten/src/ATen/RegisterCompositeImplicitAutograd.cpp:10423 [math kernel]
UNKNOWN_TENSOR_TYPE_ID: registered at aten/src/ATen/RegisterCompositeImplicitAutograd.cpp:10423 [math kernel]
UNKNOWN_TENSOR_TYPE_ID: registered at aten/src/ATen/RegisterCompositeImplicitAutograd.cpp:10423 [math kernel]
UNKNOWN_TENSOR_TYPE_ID: registered at aten/src/ATen/RegisterCompositeImplicitAutograd.cpp:10423 [math kernel]
SparseCsrCPU: registered at aten/src/ATen/RegisterCompositeImplicitAutograd.cpp:10423 [math kernel]
SparseCsrCUDA: registered at aten/src/ATen/RegisterCompositeImplicitAutograd.cpp:10423 [math kernel]
BackendSelect: fallthrough registered at ../aten/src/ATen/core/BackendSelectFallbackKernel.cpp:3 [backend fallback]
Python: registered at ../aten/src/ATen/core/PythonFallbackKernel.cpp:113 [backend fallback]
Named: registered at ../aten/src/ATen/core/NamedRegistrations.cpp:7 [backend fallback]
Conjugate: registered at ../aten/src/ATen/ConjugateFallback.cpp:18 [backend fallback]
Negative: registered at ../aten/src/ATen/native/NegateFallback.cpp:18 [backend fallback]
ZeroTensor: registered at ../aten/src/ATen/ZeroTensorFallback.cpp:86 [backend fallback]
ADInplaceOrView: fallthrough registered at ../aten/src/ATen/core/VariableFallbackKernel.cpp:64 [backend fallback]
AutogradOther: registered at aten/src/ATen/RegisterCompositeImplicitAutograd.cpp:10423 [math kernel]
AutogradCPU: registered at aten/src/ATen/RegisterCompositeImplicitAutograd.cpp:10423 [math kernel]
AutogradCUDA: registered at aten/src/ATen/RegisterCompositeImplicitAutograd.cpp:10423 [math kernel]
UNKNOWN_TENSOR_TYPE_ID: registered at aten/src/ATen/RegisterCompositeImplicitAutograd.cpp:10423 [math kernel]
AutogradXLA: registered at aten/src/ATen/RegisterCompositeImplicitAutograd.cpp:10423 [math kernel]
AutogradMLC: registered at aten/src/ATen/RegisterCompositeImplicitAutograd.cpp:10423 [math kernel]
AutogradXPU: registered at aten/src/ATen/RegisterCompositeImplicitAutograd.cpp:10423 [math kernel]
AutogradHPU: registered at aten/src/ATen/RegisterCompositeImplicitAutograd.cpp:10423 [math kernel]
UNKNOWN_TENSOR_TYPE_ID: registered at aten/src/ATen/RegisterCompositeImplicitAutograd.cpp:10423 [math kernel]
AutogradLazy: registered at aten/src/ATen/RegisterCompositeImplicitAutograd.cpp:10423 [math kernel]
AutogradPrivateUse1: registered at aten/src/ATen/RegisterCompositeImplicitAutograd.cpp:10423 [math kernel]
AutogradPrivateUse2: registered at aten/src/ATen/RegisterCompositeImplicitAutograd.cpp:10423 [math kernel]
AutogradPrivateUse3: registered at aten/src/ATen/RegisterCompositeImplicitAutograd.cpp:10423 [math kernel]
Tracer: registered at ../torch/csrc/autograd/generated/TraceType_1.cpp:11468 [kernel]
AutocastCPU: fallthrough registered at ../aten/src/ATen/autocast_mode.cpp:462 [backend fallback]
Autocast: fallthrough registered at ../aten/src/ATen/autocast_mode.cpp:305 [backend fallback]
Batched: registered at ../aten/src/ATen/BatchingRegistrations.cpp:1059 [backend fallback]
VmapMode: fallthrough registered at ../aten/src/ATen/VmapModeRegistrations.cpp:33 [backend fallback]
Functionalize: registered at ../aten/src/ATen/FunctionalizeFallbackKernel.cpp:65 [backend fallback]
PythonTLSSnapshot: registered at ../aten/src/ATen/core/PythonFallbackKernel.cpp:117 [backend fallback]
```

This feature request is to add the ability to arbitrarly nest `NestedTensor`s. My usecase for multiple nested tensors is to have the ability to load data that doesn't have the same shape. Most manipulations of the values can be done on the unbound (not nested) tensors. Having a sinlge return value for labels with different shapes in a data loader can drastically simplify my pipeline. Where as most data pipelines have nested tuples of values that all have to conform to the same shape.

### Alternatives

_No response_

### Additional context

_No response_

cc @cpuhrsch