What is +1/-1 encoding?

xolotl18: <p>Hi, I found this in a paper about loss functions, but I cannot seem to find anything about it elsewhere.<br>
the paper is this one: <a href="https://arxiv.org/pdf/1702.05659.pdf" rel="nofollow noopener">https://arxiv.org/pdf/1702.05659.pdf</a><br>
On page 2 you can see that in order to define hinge loss and its variants, it uses a -1/+1 encoded label, instead of the one hot encoded label used for the cross entropy loss.<br>
Can anyone help me understand how this works and maybe suggest a way of implementing it in pytorch?<br>
Thank you in advance</p>

harsha_g: <p>One-hot encoding is <code>0/1</code>-encoding (0 for negative and 1 for positive), likewise <code>-1/+1</code> encoding. Replace 0’s with -1s. You will see this: <em>Measures the loss given an input tensor xx and a labels tensor yy (containing 1 or -1)</em> in the <a href="https://pytorch.org/docs/stable/nn.html#hingeembeddingloss" rel="nofollow noopener">docs</a>.</p>

xolotl18: <p>thank you! Is there a way to perform this encoding in pytorch?</p>

harsha_g: <p>Not directly. But this <a href="https://discuss.pytorch.org/t/convert-int-into-one-hot-format/507/4">post</a> should help. Use <code>ones</code> instead of <code>zeros</code>, multiply with <code>-1</code>, and then <code>scatter</code>. Let me know if this is unclear.</p>

xolotl18: <p>thank you for the link. I do not understand where I need to use ones insteas of zeros</p>

harsha_g: <p>My apologies! I should have been more specific. <code>zero_</code> and not <code>zeros</code>.</p>
<pre><code class="lang-auto">import torch

batch_size = 5
nb_digits = 10
# Dummy input that HAS to be 2D for the scatter (you can use view(-1,1) if needed)
y = torch.LongTensor(batch_size,1).random_() % nb_digits
# One hot encoding buffer that you create out of the loop and just keep reusing
# y_onehot = torch.FloatTensor(batch_size, nb_digits)

# In your for loop
# y_onehot.zero_()
y_onehot = -1 * torch.ones(batch_size, nb_digits).long()
y_onehot.scatter_(1, y, 1)

print(y)
print(y_onehot)
</code></pre>

xolotl18: <p>thank you! I thought there was a tensor.ones that I didn’t know about.</p>

xolotl18: <p>hi, I have tried your implementation on its own and it works perfectly.<br>
But now that I tried integrating it in my program, I get the following error:</p>
<pre><code class="lang-auto">--&gt; 707         y_plusone.scatter_(1, targets, 1)
    708 
    709         return y_plusone.to(self.device)

RuntimeError: invalid argument 3: Index tensor must be either empty or have same dimensions as output tensor at /pytorch/aten/src/THC/generic/THCTensorScatterGather.cu:318

</code></pre>
<p>the function is the following</p>
<pre><code class="lang-auto">def to_plusone(self, targets):
        """ Convert targets to +1/-1 encoding for hinge loss """

        num_classes = self.net.fc.out_features
        batch_size = targets.size()[0]
        y_plusone = -1 * torch.ones(batch_size, num_classes).long().cuda()
        y_plusone.scatter_(1, targets, 1)

        return y_plusone.to(self.device)
</code></pre>
<p>the target arguments are a batch of labels from the dataloader of cifar100</p>
<p>I also checked the size of the targets :</p>
<pre><code class="lang-auto">torch.Size([64])
</code></pre>
<p>while the size of the y_plusone tensor is :</p>
<pre><code class="lang-auto">torch.Size([64, 10])
</code></pre>
<p>can you see something wrong? Thank you and sorry for bothering you</p>

harsha_g: <p>The problem is with the shape of your <code>targets</code>. <code>targets = targets.unsqueeze(1)</code> will do.</p>