Nested Tensors with PyG dataset class

vkovac2: <p>Hello everyone,</p>
<p>I am trying to implement a custom dataset class using PyTorch geometric. The initial issue was that in order the method “<strong>inc</strong>” to work we must have tensors. After being directed to the function “torch.nested.nested_tensor”, I realized that I could use that instead of padding the lists with negative values so I can make them a tensor later. However, when implementing it, I get the following error:</p>
<p>Exception has occurred: RuntimeError<br>
Expected both self and other to be nested, but got a nested self and non-nested other.<br>
File “/Users/try.py”, line 38, in <br>
batch = Batch.from_data_list([data, data])</p>
<p>The batching techniques of PyG is introduced here: <a href="https://pytorch-geometric.readthedocs.io/en/latest/advanced/batching.html" class="inline-onebox" rel="noopener nofollow ugc">Advanced Mini-Batching — pytorch_geometric documentation</a></p>
<p>Do you have any suggestions of how to solve my issue?</p>
<pre><code class="lang-py">
import torch
from torch_geometric.data import Batch, Data
from typing import Any

class MyData(Data):
    def __inc__(self, key, value, *args, **kwargs):
        if 'adj' in key: 
            return torch.tensor([[getattr(self, 'x').size(0)], [getattr(self, 'x').size(0)]])
        elif "list" in key:
            return torch.tensor([getattr(self, 'x').size(0)])
        else:
            return super().__inc__(key, value, *args, **kwargs)
            
    def __cat_dim__(self, key: str, value: Any, *args, **kwargs) -&gt; Any:
        if 'adj' in key:
            return 1
        else:
            return 0
            

adj = torch.tensor([[0,  1,  1,  2,  2,  2,  3,  3,  4,  4,  5,  5,  5,  6,  6,  7,  7,  8,
          8,  8,  9, 10, 10, 10, 11, 11, 12, 12, 12, 13, 13, 14, 14, 15, 15, 15,
         16, 16, 16, 16, 17, 18, 19, 19, 19, 20, 20, 21, 21, 21, 22, 23, 23, 24,
         24, 25, 25, 26, 26, 27, 27, 27, 28, 28], 
         [ 1,  0,  2,  1,  3, 28,  2,  4,  3,  5,  4,  6, 27,  5,  7,  6,  8,  7,
          9, 10,  8,  8, 11, 27, 10, 12, 11, 13, 26, 12, 14, 13, 15, 14, 16, 25,
         15, 17, 18, 19, 16, 16, 16, 20, 24, 19, 21, 20, 22, 23, 21, 21, 24, 19,
         23, 15, 26, 12, 25,  5, 10, 28,  2, 27]])

relations = {0:torch.tensor([2, 28, 27, 10, 8, 7, 6, 5, 4, 3]), 1:torch.tensor([2, 28, 27, 5, 4, 3]), 2:torch.tensor([2, 3, 4, 5, 27, 28])}
list_relations = torch.nested.nested_tensor(list(relations.values()))

data = MyData(x = torch.randn([29,5]), adj = adj, list_relations = list_relations)

batch = Batch.from_data_list([data, data])
print(batch)

</code></pre>
<p>When deleting the following lines:<br>
elif “list” in key:<br>
return torch.tensor([getattr(self, ‘x’).size(0)])</p>
<p>I get the following error:</p>
<p>Could not run ‘aten::cat’ with arguments from the ‘NestedTensorCPU’ backend. This could be because the operator doesn’t exist for this backend, or was omitted during the selective/custom build process (if using custom build). If you are a Facebook employee using PyTorch on mobile, please visit <a href="https://fburl.com/ptmfixes" class="inline-onebox" rel="noopener nofollow ugc">Internal Login</a> for possible resolutions. ‘aten::cat’ is only available for these backends: [CPU, CUDA, HIP, MPS, IPU, XPU, HPU, VE, Meta, MTIA, PrivateUse1, PrivateUse2, PrivateUse3, FPGA, ORT, Vulkan, Metal, QuantizedCPU, QuantizedCUDA, QuantizedHIP, QuantizedMPS, QuantizedIPU, QuantizedXPU, QuantizedHPU, QuantizedVE, QuantizedMeta, QuantizedMTIA, QuantizedPrivateUse1, QuantizedPrivateUse2, QuantizedPrivateUse3, CustomRNGKeyId, MkldnnCPU, SparseCPU, SparseCsrCPU, SparseCsrCUDA, BackendSelect, Python, FuncTorchDynamicLayerBackMode, Functionalize, Named, Conjugate, Negative, ZeroTensor, ADInplaceOrView, AutogradOther, AutogradCPU, AutogradCUDA, AutogradHIP, AutogradXLA, AutogradMPS, AutogradIPU, AutogradXPU, AutogradHPU, AutogradVE, AutogradLazy, AutogradMeta, AutogradMTIA, AutogradPrivateUse1, AutogradPrivateUse2, AutogradPrivateUse3, AutogradNestedTensor, Tracer, AutocastCPU, AutocastCUDA, FuncTorchBatched, FuncTorchVmapMode, Batched, VmapMode, FuncTorchGradWrapper, PythonTLSSnapshot, FuncTorchDynamicLayerFrontMode, PythonDispatcher].</p>
<p>Thanks a lot.</p>
<h3>
<a name="versions-1" class="anchor" href="#versions-1"></a>Versions</h3>
<p>PyTorch version: 2.0.1<br>
Is debug build: False<br>
CUDA used to build PyTorch: None<br>
ROCM used to build PyTorch: N/A</p>
<p>OS: macOS 12.0.1 (x86_64)<br>
GCC version: Could not collect<br>
Clang version: 13.0.0 (clang-1300.0.29.3)<br>
CMake version: Could not collect<br>
Libc version: N/A</p>
<p>Python version: 3.9.16 (main, Mar  8 2023, 04:29:44)  [Clang 14.0.6 ] (64-bit runtime)<br>
Python platform: macOS-10.16-x86_64-i386-64bit<br>
Is CUDA available: False<br>
CUDA runtime version: No CUDA<br>
CUDA_MODULE_LOADING set to: N/A<br>
GPU models and configuration: No CUDA<br>
Nvidia driver version: No CUDA<br>
cuDNN version: No CUDA<br>
HIP runtime version: N/A<br>
MIOpen runtime version: N/A<br>
Is XNNPACK available: True</p>
<p>CPU:<br>
Apple M1 Pro</p>
<p>Versions of relevant libraries:<br>
[pip3] numpy==1.24.3<br>
[pip3] pytorch-lightning==2.0.3<br>
[pip3] torch==2.0.1<br>
[pip3] torch-cluster==1.6.1<br>
[pip3] torch-geometric==2.3.1<br>
[pip3] torch-scatter==2.1.1<br>
[pip3] torch-sparse==0.6.17<br>
[pip3] torch-spline-conv==1.2.2<br>
[pip3] torchmetrics==0.11.4<br>
[conda] blas                      1.0                         mkl<br>
[conda] cpuonly                   2.0                           0    pytorch<br>
[conda] mkl                       2023.1.0         h59209a4_43558<br>
[conda] numpy                     1.24.3                   pypi_0    pypi<br>
[conda] pytorch                   2.0.1                   py3.9_0    pytorch<br>
[conda] pytorch-lightning         2.0.3                    pypi_0    pypi<br>
[conda] pytorch-mutex             1.0                         cpu    pytorch<br>
[conda] torch-cluster             1.6.1                    pypi_0    pypi<br>
[conda] torch-geometric           2.3.1                    pypi_0    pypi<br>
[conda] torch-scatter             2.1.1                    pypi_0    pypi<br>
[conda] torch-sparse              0.6.17                   pypi_0    pypi<br>
[conda] torch-spline-conv         1.2.2                    pypi_0    pypi<br>
[conda] torchmetrics              0.11.4                   pypi_0    pypi</p>