Error in torch.where: "Trying to create tensor with negative dimension -2133151913: [-2133151913]"

Algomorph: <p>Hi, all!</p>
<p>Trying to circumvent <a href="https://discuss.pytorch.org/t/invalid-shape-dimension-huge-negative-number-on-tensor-masking-operation/114606">another</a> <em>apparent</em> bug in PyTorch.<br>
This might be related to <a href="https://discuss.pytorch.org/t/combine-histograms-histogram-with-output-range-torch-zeros-nbins-downsample-rate-device-orig-hist-device-runtimeerror-trying-to-create-tensor-with-negative-dimension-4398046511104-4398046511104/89086/6">this post</a> (which was never fully resolved).</p>
<p>Relevant code fragment (with print statements for debugging, <code>mask</code> is also made for debugging):</p>
<pre data-code-wrap="Python"><code class="lang-Python">mask = torch.where(graph_clusters_i == cluster_id, torch.ones_like(graph_clusters_i), torch.zeros_like(graph_clusters_i))
index_equals_cluster_id = torch.where(graph_clusters_i == cluster_id)
print("mask.sum()......................: ", mask.sum())
print("type(graph_clusters_i)..........: ", type(graph_clusters_i))
print("graph_clusters_i.shape..........: ", graph_clusters_i.shape)
print("graph_clusters_i.device.........: ", graph_clusters_i.device)
print("type(cluster_id)................: ", type(cluster_id))
print("cluster_id......................: ", cluster_id)
print("len(index_equals_cluster_id)....: ", len(index_equals_cluster_id))
print("type(index_equals_cluster_id)...: ", type(index_equals_cluster_id))
print("type(index_equals_cluster_id[0]): ", type(index_equals_cluster_id[0]))
print("index_equals_cluster_id[0].shape: ", index_equals_cluster_id[0].shape)

x = index_equals_cluster_id[0].tolist()  # &lt;-- error here!
</code></pre>
<p>The output is:</p>
<blockquote>
<p>mask.sum()…:  tensor(939, device=‘cuda:0’)<br>
type(graph_clusters_i)…:  &lt;class ‘torch.Tensor’&gt;<br>
graph_clusters_i.shape…:  torch.Size([1165, 1])<br>
graph_clusters_i.device…:  cuda:0<br>
type(cluster_id)…:  &lt;class ‘int’&gt;<br>
cluster_id…:  0<br>
len(index_equals_cluster_id)…:  2<br>
type(index_equals_cluster_id)…:  &lt;class ‘tuple’&gt;<br>
type(index_equals_cluster_id[0]):  &lt;class ‘torch.Tensor’&gt;<br>
index_equals_cluster_id[0].shape:  torch.Size([-2133151913])<br>
Traceback (most recent call last):<br>
(A few lines omitted for brevity&gt;)<br>
File “/home/algomorph/.local/lib/python3.8/site-packages/torch/nn/modules/module.py”, line 889, in _call_impl<br>
result = self.forward(*input, **kwargs)<br>
File “/home/algomorph/Workbench/NeuralTracking/model/model.py”, line 437, in forward<br>
x = index_equals_cluster_id[0].tolist()  # ← error here!<br>
RuntimeError: Trying to create tensor with negative dimension -2133151913: [-2133151913]</p>
</blockquote>
<p>Any idea about how to fix or avoid this?</p>

Algomorph: <p>The answer to “how to circumvent” was staring me right in the face: just use <code>torch.nonzero</code> on the mask. Here’s the unelegant/dirty solution, which involves an extra sum operation that shouldn’t really be there to circumvent the bug:</p>
<pre><code class="lang-auto">mask = torch.where(graph_clusters_i == cluster_id, torch.ones_like(graph_clusters_i), torch.zeros_like(graph_clusters_i))
dummy = mask.sum()  # TODO: figure out why there is some kind of lazy-evaluation bug in PyTorch without this line...
del dummy
x = torch.nonzero(mask, as_tuple=True)[0].tolist()
```</code></pre>

Algomorph: <p>Finally, I got PyTorch itself to admit there is some kind of bug here. Not sure how to report it though, because it’s a hell of a lot of code to get down to a minimal reproducible example.</p>
<p>The code:</p>
<pre data-code-wrap="Python"><code class="lang-Python">index_equals_cluster_id = torch.where(graph_clusters_i == cluster_id)
dummy = index_equals_cluster_id[0].sum()  # This triggers some kind of bug
</code></pre>
<p>Output:</p>
<blockquote>
<p>Traceback (most recent call last):<br>
(Some intermediate calls omitted for brevity).<br>
File “/home/algomorph/.local/lib/python3.8/site-packages/torch/nn/modules/module.py”, line 889, in _call_impl<br>
result = self.forward(*input, **kwargs)<br>
File “/home/algomorph/Workbench/NeuralTracking/model/model.py”, line 442, in forward<br>
dummy = index_equals_cluster_id[0].sum()   # This triggers some kind of bug<br>
RuntimeError: iter.numel() &gt; 0 &amp;&amp; iter.ntensors() - iter.noutputs() == 1 &amp;&amp; iter.noutputs() &gt;= 1 INTERNAL ASSERT FAILED at “/pytorch/aten/src/ATen/native/cuda/Reduce.cuh”:896, please report a bug to PyTorch.</p>
</blockquote>