RuntimeError: Expected canUse32BitIndexMath(input)

username: rokopi
cooked: <p>Hi,<br>
I’m trying to use this <a href="https://github.com/ChunmingHe/Reti-Diff" rel="noopener nofollow ugc">repo</a> on a nvidia A6000 48Gb. I get the following error:</p>
<p>RuntimeError: Expected canUse32BitIndexMath(input) &amp;&amp; canUse32BitIndexMath(output) to be true, but got false.<br>
(Could this error message be improved? If so, please report an enhancement request to PyTorch.)</p>
<p>After some investigation, I understand that this issue occurs because large images can result in an output tensor exceeding 2^31 elements, which is not supported by cuDNN versions earlier than 9.3 (<a href="https://github.com/pytorch/pytorch/issues/80020" rel="noopener nofollow ugc">related PyTorch issue</a> and <a href="https://github.com/pytorch/pytorch/pull/134890" rel="noopener nofollow ugc">PR</a>). Since I am performing inference, I cannot split the batch to work around this limitation.</p>
<p>The problem is that starting from today I’m using pytorch 2.7 which comes with cuDNN nvidia-cudnn-cu12==9.5.1.17 and should not have this issue.</p>
<p>Any hints ?</p>

username: ptrblck
cooked: <p>Were you able to create a stacktrace pointing to the failing call? The linked source file does not use <code>canUse32BitIndexMath</code> directly, so I’m unsure if your conv fails or another kernel.</p>

username: rokopi
cooked: <p>Here you are:</p>
<pre><code class="lang-auto">  Traceback (most recent call last):
    File "/home/vrai/Reti-Diff/Reti-Diff/test.py", line 60, in &lt;module&gt;
      test_pipeline(root_path)
    File "/home/vrai/Reti-Diff/Reti-Diff/test.py", line 53, in test_pipeline
      model.validation(test_loader, current_iter=opt['name'], tb_logger=None, save_img=opt['val']['save_img'])
    File "/home/vrai/Reti-Diff/venv/lib/python3.10/site-packages/basicsr/models/base_model.py", line 48, in validation
      self.nondist_validation(dataloader, current_iter, tb_logger, save_img)
    File "/home/vrai/Reti-Diff/Reti-Diff/models/S2_Interface_model.py", line 310, in nondist_validation
      self.test()
    File "/home/vrai/Reti-Diff/Reti-Diff/models/S2_Interface_model.py", line 409, in test
      self.output = self.net_g(lq, retinex_lq)
    File "/home/vrai/Reti-Diff/venv/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1751, in _wrapped_call_impl
      return self._call_impl(*args, **kwargs)
    File "/home/vrai/Reti-Diff/venv/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1762, in _call_impl
      return forward_call(*args, **kwargs)
    File "/home/vrai/Reti-Diff/Reti-Diff/archs/S2_interface_arch.py", line 562, in forward
      sr = self.G(img, IPRS2_rex, IPRS2_img)
    File "/home/vrai/Reti-Diff/venv/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1751, in _wrapped_call_impl
      return self._call_impl(*args, **kwargs)
    File "/home/vrai/Reti-Diff/venv/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1762, in _call_impl
      return forward_call(*args, **kwargs)
    File "/home/vrai/Reti-Diff/Reti-Diff/archs/S2_interface_arch.py", line 352, in forward
      out_dec_level1, _ = self.decoder_level1([inp_dec_level1, k_v])
    File "/home/vrai/Reti-Diff/venv/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1751, in _wrapped_call_impl
      return self._call_impl(*args, **kwargs)
    File "/home/vrai/Reti-Diff/venv/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1762, in _call_impl
      return forward_call(*args, **kwargs)
    File "/home/vrai/Reti-Diff/venv/lib/python3.10/site-packages/torch/nn/modules/container.py", line 240, in forward
      input = module(input)
    File "/home/vrai/Reti-Diff/venv/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1751, in _wrapped_call_impl
      return self._call_impl(*args, **kwargs)
    File "/home/vrai/Reti-Diff/venv/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1762, in _call_impl
      return forward_call(*args, **kwargs)
    File "/home/vrai/Reti-Diff/Reti-Diff/archs/S2_interface_arch.py", line 217, in forward
      x = x + self.ffn(self.norm2(x), k_v)
    File "/home/vrai/Reti-Diff/venv/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1751, in _wrapped_call_impl
      return self._call_impl(*args, **kwargs)
    File "/home/vrai/Reti-Diff/venv/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1762, in _call_impl
      return forward_call(*args, **kwargs)
    File "/home/vrai/Reti-Diff/Reti-Diff/archs/S2_interface_arch.py", line 89, in forward
      x1, x2 = self.dwconv(x).chunk(2, dim=1)
    File "/home/vrai/Reti-Diff/venv/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1751, in _wrapped_call_impl
      return self._call_impl(*args, **kwargs)
    File "/home/vrai/Reti-Diff/venv/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1762, in _call_impl
      return forward_call(*args, **kwargs)
    File "/home/vrai/Reti-Diff/venv/lib/python3.10/site-packages/torch/nn/modules/conv.py", line 554, in forward
      return self._conv_forward(input, self.weight, self.bias)
    File "/home/vrai/Reti-Diff/venv/lib/python3.10/site-packages/torch/nn/modules/conv.py", line 549, in _conv_forward
      return F.conv2d(
  RuntimeError: Expected canUse32BitIndexMath(input) &amp;&amp; canUse32BitIndexMath(output) to be true, but got false.  (Could this error message be improved?  If so, please report an enhancement request to PyTorch.)
</code></pre>

username: ptrblck
cooked: <p>No issue reproducible with <code>torch==2.7.0+cu126</code> using:</p>
<pre data-code-wrap="python"><code class="lang-python">import torch

device = "cuda"
x = torch.randn(1024, 1, 1024*8, 1024, device=device)
weight = torch.randn(1, 1, 1, 1, device=device)

print(x.numel() &gt; 2**31 - 1)

out = torch.nn.functional.conv2d(x, weight, stride=1)
print(out.shape)
</code></pre>
<pre data-code-wrap="python"><code class="lang-python">True
torch.Size([1024, 1, 8192, 1024])
</code></pre>

username: rokopi
cooked: <p>I tried your code and I don’t have the issue. I have OOM, but that’s normal.<br>
Thanks for the reply, I’m not an expert and I’d like to dig deeper into this..<br>
Since the code is calling the original conv2d function (last row of the stacktrace).. where could the problem located ?</p>

username: ptrblck
cooked: <p>A quick search points to <a href="https://github.com/search?q=repo%3Apytorch%2Fpytorch%20canUse32BitIndexMath&amp;type=code">these methods</a> so maybe you are using a depthwise conv?</p>

username: lely475
cooked: <p>I am running into the same error (num elements exceeding 2^31) when using latest pytorch 2.7.0+ cudnn &gt; 9.3 and depthwise conv.</p>
<pre data-code-wrap="python"><code class="lang-python">import torch
import torch.nn as nn

device = torch.device("cuda")

# Define an extremely large input tensor (exceeding 2**31 elements for a single sample), use grouped (depthwise separable) convolutions
# For example: Batch size = 1, Channels = 2, Height = 32,800, Width = 32,800
# Total elements = 1 * 2 * 32,800 * 32,800 = 2,151,680,000 &gt; 2**31 (2,147,483,648)
num_channels=2
input_tensor = torch.randn(1, num_channels, 32800, 32800, device=device)
conv_layer = nn.Conv2d(num_channels, num_channels, kernel_size=3, stride=1, padding=1, groups=num_channels).to(device)
output_tensor = conv_layer(input_tensor)
</code></pre>
<p>What would be needed to extend the support of tensors with &gt; 2^31 to groups&gt;1 (depthwise separable convs)? Are there any plans to support this in the future?</p>

username: lely475
cooked: <p>I have also opened an issue related to this in the PyTorch Github: <a href="https://github.com/pytorch/pytorch/issues/152816" class="inline-onebox" rel="noopener nofollow ugc">Depthwise Separable Convolutions with Large Tensors (&gt; 2**31) Elements) Fail Despite cuDNN 64-bit Indexing Support · Issue #152816 · pytorch/pytorch · GitHub</a></p>