ArcFace loss error

sbht_sbht: <p>My ArcFace loss:</p>
<pre><code class="lang-auto">import math

class ArcFaceLoss(nn.Module):
    def __init__(self, num_classes, embedding_size=64, margin=0.5, scale=64):
        super(ArcFaceLoss, self).__init__()
        self.num_classes = num_classes
        self.embedding_size = embedding_size
        self.margin = margin
        self.scale = scale
        self.cos_m = math.cos(margin)
        self.sin_m = math.sin(margin)
        self.threshold = math.cos(math.pi - margin)
        self.mm = self.sin_m * margin
        self.weight = nn.Parameter(torch.FloatTensor(num_classes, embedding_size))
        nn.init.xavier_uniform_(self.weight)

    def forward(self, x, labels):
        cosine = F.linear(F.normalize(x), F.normalize(self.weight))
        sine = torch.sqrt(1.0 - torch.pow(cosine, 2))
        phi = cosine * self.cos_m - sine * self.sin_m
        phi = torch.where(cosine &gt; self.threshold, phi, cosine - self.mm)
        one_hot = torch.zeros(cosine.size(), device=x.device)
        one_hot.scatter_(1, labels.view(-1, 1).long(), 1)
        output = (one_hot * phi) + ((1.0 - one_hot) * cosine)
        output *= self.scale
        return output
</code></pre>
<p>Getting this error:</p>
<blockquote>
<p>Input In [89], in ArcFaceLoss.forward(self, x, labels)<br>
17 def forward(self, x, labels):<br>
—&gt; 18     cosine = F.linear(F.normalize(x), F.normalize(self.weight))<br>
19     sine = torch.sqrt(1.0 - torch.pow(cosine, 2))<br>
20     phi = cosine * self.cos_m - sine * self.sin_m</p>
<p>RuntimeError: mat1 and mat2 shapes cannot be multiplied (64x10171 and 64x10171)</p>
</blockquote>
<p>mat1 and mat2 have same shape. If I use transpose it work, but I dont know is it right and after this I get this error:</p>
<blockquote>
<p>Input In [136], in ArcFaceLoss.forward(self, x, labels)<br>
21 phi = torch.where(cosine &gt; self.threshold, phi, cosine - self.mm)<br>
22 one_hot = torch.zeros(cosine.size(), device=x.device)<br>
—&gt; 23 one_hot.scatter_(1, labels.view(-1, 1).long(), 1)<br>
24 output = (one_hot * phi) + ((1.0 - one_hot) * cosine)<br>
25 output *= self.scale</p>
<p>RuntimeError: index 4601 is out of bounds for dimension 1 with size 64<br>
I know how to fix it, but i’m afraid that it’s somehow ruind my ArcFace loss.<br>
When writing the function I relied on this notebook <a href="https://github.com/egcode/pytorch-losses/blob/master/mnist-visualize-arcface-loss.ipynb" rel="noopener nofollow ugc">pytorch-losses</a></p>
</blockquote>

ptrblck: <aside class="quote no-group" data-username="sbht_sbht" data-post="1" data-topic="178608">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/sbht_sbht/48/46999_2.png" class="avatar"> sbht_sbht:</div>
<blockquote>
<p>mat1 and mat2 have same shape. If I use transpose it work,</p>
</blockquote>
</aside>
<p>A matrix multiplication expects inputs as <code>[M, K]</code> and <code>[K, N]</code> so transposing should be the correct approach.</p>
<aside class="quote no-group" data-username="sbht_sbht" data-post="1" data-topic="178608">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/sbht_sbht/48/46999_2.png" class="avatar"> sbht_sbht:</div>
<blockquote>
<p>RuntimeError: index 4601 is out of bounds for dimension 1 with size 64<br>
I know how to fix it, but i’m afraid that it’s somehow ruind my ArcFace loss.</p>
</blockquote>
</aside>
<p>Could you describe this concern a bit more, please?<br>
The error is raised since you are trying to use an out-of-bounds index in the <code>scatter_</code> operation. You explained you would know how to fix it (I assume by either increasing the <code>one_hot</code> size of making sure only valid indices are used, but I don’t understand how it could invalidate the loss.</p>

sbht_sbht: <p>I got 10171 different people. Of each of them I have set of photo. I need to train my model to recognize faces and get embedding feature vector.</p>
<p>My resnet50 last layer has size of 10171. Each point for each class. Embedding vector size equals 256.</p>
<p>I use this ArcFace loss (I commented size of each variable):</p>
<pre><code class="lang-auto">class ArcFaceLoss(nn.Module):
    def __init__(self, num_classes, embedding_size=256, margin=0.5, scale=64):
        super(ArcFaceLoss, self).__init__()
        self.num_classes = num_classes
        self.embedding_size = embedding_size
        self.margin = margin
        self.scale = scale
        self.cos_m = math.cos(margin)
        self.sin_m = math.sin(margin)
        self.threshold = math.cos(math.pi - margin)
        self.mm = self.sin_m * margin
        self.weight = nn.Parameter(torch.FloatTensor(num_classes, embedding_size))
        nn.init.xavier_uniform_(self.weight)

    def forward(self, x, labels):
        # labels.shape = torch.Size([64])
        # F.normalize(x).shape = torch.Size([64, 10171])
        # F.normalize(self.weight).T.shape = torch.Size([256, 10171])
        cosine = F.linear(F.normalize(x), F.normalize(self.weight).T)
        # cosine.shape = torch.Size([64, 256])
        sine = torch.sqrt(1.0 - torch.pow(cosine, 2))
        # sine.shape = torch.Size([64, 256])
        phi = cosine * self.cos_m - sine * self.sin_m
        phi = torch.where(cosine &gt; self.threshold, phi, cosine - self.mm)
        # phi.shape = torch.Size([64, 256])
        one_hot = torch.zeros(cosine.size(), device=x.device)
        # one_hot.shape = torch.Size([64, 256])
        # labels.view(-1, 1).long().shape = torch.Size([64, 1])
        one_hot.scatter_(1, labels.view(-1, 1).long(), 1)
        output = (one_hot * phi) + ((1.0 - one_hot) * cosine)
        output *= self.scale
        return output
</code></pre>
<p>The problem is that in the example that I threw off in the previous post, and judging by the arcface loss articles, we create a one-hot encoded vector based on cosine. But the number of classes significantly exceeds the size of one-hot.</p>
<blockquote>
<p>Input In [31], in ArcFaceLoss.forward(self, x, labels)<br>
29 print(one_hot.shape)<br>
30 print(labels.view(-1, 1).long().shape)<br>
—&gt; 31 one_hot.scatter_(1, labels.view(-1, 1).long(), 1)<br>
32 output = (one_hot * phi) + ((1.0 - one_hot) * cosine)<br>
33 output *= self.scale</p>
<p>RuntimeError: index 4458 is out of bounds for dimension 1 with size 256</p>
</blockquote>
<p>I guess, that in this <a href="https://github.com/egcode/pytorch-losses/blob/master/mnist-visualize-arcface-loss.ipynb" rel="noopener nofollow ugc">example of implementation ArcFace loss </a> somehow didn’t used transpose operation, so this code doesnt raise error of mismatch of size of one-hot and number of classes.</p>
<p>So my question is how can I should implement this function right? Maby there is allredi implemented arcface loss in pytorch that I can’t find?</p>

ptrblck: <aside class="quote no-group" data-username="sbht_sbht" data-post="3" data-topic="178608">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/sbht_sbht/48/46999_2.png" class="avatar"> sbht_sbht:</div>
<blockquote>
<p>I guess, that in this <a href="https://github.com/egcode/pytorch-losses/blob/master/mnist-visualize-arcface-loss.ipynb">example of implementation ArcFace loss </a> somehow didn’t used transpose operation,</p>
</blockquote>
</aside>
<p>Yes, since the linked example initialized the <code>self.weight</code> as:</p>
<pre data-code-wrap="python"><code class="lang-python">self.weight = nn.Parameter(torch.FloatTensor(out_features, in_features))
</code></pre>
<p>while you are using <code>self.weight = nn.Parameter(torch.FloatTensor(num_classes, embedding_size))</code> where <code>num_classes</code> corresponds to the <code>in_features</code> and <code>embedding_size</code> to the <code>out_features</code>.</p>
<aside class="quote no-group" data-username="sbht_sbht" data-post="3" data-topic="178608">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/sbht_sbht/48/46999_2.png" class="avatar"> sbht_sbht:</div>
<blockquote>
<p>So my question is how can I should implement this function right?</p>
</blockquote>
</aside>
<p>I don’t know what the <code>label</code> represents but since it’s used to index the <code>one_hot</code> tensor, which has the size of the <code>embedding_size</code> I would assume <code>label</code> contains some indices of this embedding dimension?</p>

sbht_sbht: <p>I understand where am I was wrong. I should make number of classes equal with number of embedding size. I guesse, I just did not fully understand the principle of arcface loss. Sorry for wasting time and thanks for help!</p>