Strange error when doing loss.backward()

andrster: <p>Hello, I am having a very strange issue. When doing backwards, the following error code pops up:<br>
<div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/9/b/9b2f1e41fdf5e9b1a29d90bc6d3d3b6570eb53d6.png" data-download-href="https://discuss.pytorch.org/uploads/default/9b2f1e41fdf5e9b1a29d90bc6d3d3b6570eb53d6" title="error"><img src="https://discuss.pytorch.org/uploads/default/optimized/3X/9/b/9b2f1e41fdf5e9b1a29d90bc6d3d3b6570eb53d6_2_690x286.png" alt="error" data-base62-sha1="m8OXpgt9OkQ94i4XksxHXKIhbRs" width="690" height="286" srcset="https://discuss.pytorch.org/uploads/default/optimized/3X/9/b/9b2f1e41fdf5e9b1a29d90bc6d3d3b6570eb53d6_2_690x286.png, https://discuss.pytorch.org/uploads/default/optimized/3X/9/b/9b2f1e41fdf5e9b1a29d90bc6d3d3b6570eb53d6_2_1035x429.png 1.5x, https://discuss.pytorch.org/uploads/default/optimized/3X/9/b/9b2f1e41fdf5e9b1a29d90bc6d3d3b6570eb53d6_2_1380x572.png 2x" data-dominant-color="3F1B32"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">error</span><span class="informations">1884×781 228 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>
<p>Here is a code snippet to reproduce the error.</p>
<pre><code class="lang-auto">import torch

def geometric_line_distances(joints):
    batch, _, _ = joints.size(0), joints.size(1), joints.size(2)
    distances = torch.cdist(joints, joints).to(joints.device)
    line_dists2 = torch.zeros(batch, 21, 210).to(joints.device)

    for k in range(21):
        cnt = 0
        for m in range(21):
            n = 21 - (m + 1)
            if m != k:
                size = distances[:, m, m+1:21].size(-1)
                s1 = .5*(distances[:, m, m+1:21] +
                         distances[:, k, m].unsqueeze(-1).repeat(1, size) +
                         distances[:, m+1:21, k])
                x1 = (s1 - distances[:, k, m].unsqueeze(-1).repeat(1, size)) * \
                     (s1 - distances[:, m, m+1:21]) * \
                     (s1 - distances[:, m+1:21, k])

                x1[x1 &lt; 0] = 1e-10
                inter = 2 * torch.sqrt(s1 * x1) / distances[:, m, m+1:21]
                line_dists2[:, k, cnt:cnt + inter.size(-1)] = inter
                cnt += inter.size(-1)
    return line_dists2.contiguous()

""" To reproduce error """
x = torch.randn((4, 21, 3), requires_grad=True)
mse = torch.nn.MSELoss()
dist = geometric_line_distances(x)
loss = mse(dist, .5*dist.ditach())
loss.backward()
</code></pre>
<p>Any idea what’s going on?</p>

albanD: <p>Hey,</p>
<p>The stack trace points to the backward of the <code>repeat()</code> operation.<br>
What are the arguments you give them? Any special case there where some inputs are of size 0?</p>

andrster: <p>Yup, that was it! In some cases, the “size” argument in repeat becomes zero. Fixed that and everything runs smoothly.</p>
<p>Thanks for the answer!</p>

albanD: <p>Perfect.<br>
I opened an issue to make sure we fix this to behave properly: <a href="https://github.com/pytorch/pytorch/issues/45201">https://github.com/pytorch/pytorch/issues/45201</a></p>