PyTocrh way for one-hot-encoding multiclass target variable

al314: <p>Hey,</p>
<p>Sorry for maybe super basic question but could not find it.</p>
<p>What is a correct Pytorch way to encode multi-class target variable?</p>
<p>I have <code>&gt; 30</code> target classes for target variable - like <code>AA, AB, BB, BA, BC ...</code>.</p>
<p>Should I use ScikitLearn tools and then convert numpy arrays into torch tensors?</p>
<p>Or there is built-in functionality?</p>
<p>Thanks!</p>

ptrblck: <p>You could use this code snippet to transform your class indices into a one-hot encoded target:</p>
<pre><code class="lang-python">target = torch.randint(0, 10, (10,))
one_hot = torch.nn.functional.one_hot(target)
</code></pre>

al314: <p>Thanks!<br>
How do I map rows in one hot torch tensor to original labels and back?<br>
Is it built-in functionality or should I care about this myself by creating custom dictionaries?<br>
Trying not to invent a wheel )</p>

ptrblck: <p>To get the target containing class indices back, you could use <code>torch.argmax(one_hot, dim=1)</code>.</p>

stalhabukhari: <p>Apart from <code>.scatter_</code>, is there a better way to create a one-hot tensor in the (N,C,H,W) format without transposing?</p>

ptrblck: <p>Which dimension should be used for the one-hot encoding?<br>
<code>.scatter_</code> should not involve any transposing. Could you post a code snippet which would need it?</p>

stalhabukhari: <p>I’m sorry I didn’t elaborate on my question better.<br>
To the best of my knowledge, there are two ways of creating one-hot encoded tensors in PyTorch:</p>
<ol>
<li>
<code>.scatter_</code>: with this I can create a one-hot encoded tensor from a given tensor (for me, usually a label map). But for this, I need to define a <code>torch.FloatTensor</code>, and use <code>.scatter_</code> on it. A <code>torch.FloatTensor</code> needs all its dimensions preemptively defined and does not accept <code>None</code> values. This will usually be an issue when the final mini-batch in a training/validation session is smaller than the other mini-batches. I could alternatively define a new <code>torch.FloatTensor</code> of appropriate shape on each iteration, but I wish to skip this extra operation.</li>
<li>
<code>torch.nn.functional.one_hot</code>: with this I can directly yield a one-hot encoded tensor from a given tensor, but the output is in the <em>channels-last</em> format or (N,H,W,C). Hence, <code>torch.transpose</code> is required to convert it into PyTorch’s (N,C,H,W) format. I wish to skip this extra operation as well. (It would be awesome if we have the option in <code>torch.nn.functional.one_hot</code>, to specify which dimension will be the channel dimension.)</li>
</ol>
<p>Is there a better way? I wish to improve the following <em>rough</em> implementation:</p>
<pre><code class="lang-auto">class MyLoss(torch.nn.Module):
    def __init__(self, batch_size, classes):
        super(MyLoss, self).__init__()
        # define some attributes
        self.y_true_one_hot = torch.FloatTensor(batch_size, classes, 240, 240)

    def forward(self, y_pred, y_true):
        with torch.no_grad():
            self.y_true_one_hot.zero_().scatter_(1, y_true, 1)
        # do some operations
        return loss
</code></pre>

ptrblck: <p>Thanks for the update.<br>
I would personally stick to the <code>scatter_</code> approach and either check for a different batch size or just recreate the tensor e.g. via <code>torch.zeros_like</code>. <code>F.one_hot</code> would also recreate the tensor additionally to the permutation, so you wouldn’t avoid this operation.</p>
<p>Regarding the <code>dim</code> argument for <code>one_hot</code>: could you create a feature request on GitHub and explain your use case to start the discussion please?</p>

stalhabukhari: <p><a href="https://github.com/pytorch/pytorch/issues/40790#issue-648342219" rel="nofollow noopener">Feature request</a> has been created. Thank you for your time.</p>

Jaideep_Valani: <p><a class="mention" href="/u/ptrblck">@ptrblck</a><br>
i have got tensor of size   bs,seq len, list of targets  for each length</p>
<pre><code class="lang-auto">eg.` bs *1* list of target  [132,15,42,24]` 
       bs *2*  list of target [ 132,12,43,49],[1,3,2]
</code></pre>
<p>each element of  seq length may have  different length of target list. max value of target assume 100</p>
<p>how can i build Hot encode in the forward method of Model. and then multiply the absolute value of target to generated hot encode vector.</p>

ptrblck: <p>I don’t quite understand the use case. Could you provide pseudo-code for the method you would like to implement?</p>

Jaideep_Valani: <ol>
<li>Hot encode tensor [1,10,11,12] assume  batch size of 10<br>
2  multiply respective hot encode vector with absolute value  like 1,10 11,12 .  00 1 10 11 12 000…</li>
</ol>
<p>I m generating 128 dim size  embeddings  of each encode vector   through linear layer  with in feature as length of encode vector .</p>
<p>I dont get desired results if I directly pass hot encode of 0 1 to linear layer so want to try with absolute values</p>

talhaanwarch: <p>This helped me<br>
<code>y_pred.shape,label.shape</code><br>
(torch.Size([5, 2]), torch.Size([5, 2]))<br>
<code>loss_fn(y_pred,torch.argmax(label, dim=1))</code><br>
where as<br>
<code>loss_fn = torch.nn.CrossEntropyLoss()</code><br>
The last layer is as follow</p>
<pre><code class="lang-auto">self.dropout = torch.nn.Dropout(0.1)
self.fc1 = torch.nn.Linear(768, 256)
self.fc2 = torch.nn.Linear(256, 2)
</code></pre>