Invalid index in scatter in `ArcFaceLoss`

kombucha: <p>Hi guys,</p>
<p>I am working with <code>ArcFaceLoss</code>, code taken from: <a href="https://github.com/ronghuaiyang/arcface-pytorch/blob/master/models/metrics.py" rel="nofollow noopener">https://github.com/ronghuaiyang/arcface-pytorch/blob/master/models/metrics.py</a>.</p>
<p>Here is the following snippet:</p>
<pre><code class="lang-auto">class ArcMarginProduct(nn.Module):
    r"""Implement of large margin arc distance: :
        Args:
            in_features: size of each input sample
            out_features: size of each output sample
            s: norm of input feature
            m: margin
            cos(theta + m)
        """
    def __init__(self, in_features, out_features, s=30.0, m=0.50, easy_margin=False):
        super(ArcMarginProduct, self).__init__()
        self.in_features = in_features
        self.out_features = out_features
        self.s = s
        self.m = m
        self.weight = nn.Parameter(torch.FloatTensor(out_features, in_features))
        nn.init.xavier_uniform_(self.weight)

        self.easy_margin = easy_margin
        self.cos_m = math.cos(m)
        self.sin_m = math.sin(m)
        self.th = math.cos(math.pi - m)
        self.mm = math.sin(math.pi - m) * m

    def forward(self, input, label):
        # --------------------------- cos(theta) &amp; phi(theta) ---------------------------
        cosine = F.linear(F.normalize(input), F.normalize(self.weight))
        sine = torch.sqrt((1.0 - torch.pow(cosine, 2)).clamp(0, 1))
        phi = cosine * self.cos_m - sine * self.sin_m
        if self.easy_margin:
            phi = torch.where(cosine &gt; 0, phi, cosine)
        else:
            phi = torch.where(cosine &gt; self.th, phi, cosine - self.mm)
        # --------------------------- convert label to one-hot ---------------------------
        # one_hot = torch.zeros(cosine.size(), requires_grad=True, device='cuda')
        one_hot = torch.zeros(cosine.size())
        
        print(one_hot.size(), label.size(), label.view(-1, 1).long().size())
        # torch.Size([32, 1108]) torch.Size([32]) torch.Size([32, 1])
        
        one_hot.scatter_(1, label.view(-1, 1).long(), 1) # ERROR HERE
        # -------------torch.where(out_i = {x_i if condition_i else y_i) -------------
        output = (one_hot * phi) + ((1.0 - one_hot) * cosine)  # you can use torch.where if your torch.__version__ is 0.4
        output *= self.s
        # print(output)

        return output
</code></pre>
<p>After trying the above loss function,</p>
<pre><code class="lang-auto">metric_fc = ArcMarginProduct(512, NUM_CLASSES)

inputs = torch.randn(32, 512, 1, 1)
labels = torch.randn(32)

metric_fc(inputs.squeeze(), labels)
</code></pre>
<p>I am getting the error of <code>Invalid index in scatter at /opt/conda/conda-bld/pytorch_1556653099582/work/aten/src/TH/generic/THTensorEvenMoreMath.cpp:551</code>.</p>
<p>I check the dimensions by printing out and it seems that nothing is wrong. Any help is appreciated. Thank you.</p>

ptrblck: <p><code>label.view(-1, 1).long()</code> contains invalid indices for the <code>scatter_</code> operation.<br>
Since <code>one_hot</code> has the shape <code>[32, 1108]</code>, and you are trying to scatter into dim1, <code>label</code> should be clamped at <code>[0, 1108]</code>:</p>
<pre><code class="lang-python">label = label.clamp(0, 1108)
one_hot.scatter_(1, label.view(-1, 1).long(), 1)
</code></pre>