Set Max value to 1, others to 0

Keverino: <p>Hello guys,</p>
<p>I’m probably just bad at searching. So pardon me if this is a repost.</p>
<p>I have a tensor with [batch_size, 4]</p>
<p>Andi Want the value of the 2nd dimension to be somehting like  [0,0,1,0], where the one corresponds to the max value in this tensor.<br>
Any ideas?</p>
<p>thanks in advance!</p>

KFrank: <p>Hello Kevin!</p>
<aside class="quote no-group" data-username="Keverino" data-post="1" data-topic="44350" data-full="true">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/keverino/48/11749_2.png" class="avatar"> Keverino:</div>
<blockquote>
<p>I have a tensor with [batch_size, 4]</p>
<p>Andi Want the value of the 2nd dimension to be somehting like  [0,0,1,0], where the one corresponds to the max value in this tensor.<br>
Any ideas?</p>
</blockquote>
</aside>
<p>I am assuming that you want your result tensor to have the<br>
same shape as your input tensor, <code>[batch_size, 4]</code>, with<br>
the largest value in each row replaced by <code>1</code> and the lesser<br>
values replaced by <code>0</code>.</p>
<p>How about <code>(1.e6 * t).softmax (1)</code>?</p>
<p>(No, no, no …   Bad idea!  Don’t listen to me!!!)</p>
<p>I don’t know of any good, one-step way of doing this.  You have<br>
to call <code>argmax()</code> (or do something equivalent) and then “one-hot”<br>
your result.  The commonly-suggested way to “one-hot” (that as<br>
far as I know is the best way) is to use <code>scatter()</code>.</p>
<p>Try this:</p>
<pre data-code-wrap="python"><code class="lang-python">import torch
torch.manual_seed (1414)
t = torch.randn (8, 4)
a = t.argmax (1)
m = torch.zeros (t.shape).scatter (1, a.unsqueeze (1), 1.0)
print ('\n', t, '\n\n', a, '\n\n', m)
</code></pre>
<p>Running this, I get</p>
<pre><code class="lang-plaintext">tensor([[ 0.1868,  1.2265,  1.0181,  0.6943],
       [ 0.1075, -0.3540,  0.1766, -0.4940],
       [ 3.0013,  1.8697, -0.0673, -1.3875],
       [-0.0560, -0.6007, -0.0410, -0.6681],
       [-0.2963,  0.1108,  0.2250, -0.5483],
       [-0.1541, -1.1390,  0.4984, -0.4016],
       [-0.3887, -0.1821,  0.4817,  0.4268],
       [ 0.8756,  2.0755,  0.2577,  0.8941]])

tensor([1, 2, 0, 2, 2, 2, 2, 1])

tensor([[0., 1., 0., 0.],
       [0., 0., 1., 0.],
       [1., 0., 0., 0.],
       [0., 0., 1., 0.],
       [0., 0., 1., 0.],
       [0., 0., 1., 0.],
       [0., 0., 1., 0.],
       [0., 1., 0., 0.]])
</code></pre>
<p>(This is a sufficiently commonly requested manipulation that<br>
I’m surprised that there isn’t a built-in function for it – at<br>
least I haven’t found one.)</p>
<p>Happy One-Hotting!</p>
<p>K. Frank</p>

ptrblck: <aside class="quote no-group" data-username="KFrank" data-post="2" data-topic="44350">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/k/ecb155/48.png" class="avatar"> KFrank:</div>
<blockquote>
<p>This is a sufficiently commonly requested manipulation that<br>
I’m surprised that there isn’t a built-in function for it – at<br>
least I haven’t found one.</p>
</blockquote>
</aside>
<p><code>F.one_hot</code> should yield the same result as the <code>scatter</code> operation <img src="https://discuss.pytorch.org/images/emoji/apple/wink.png?v=12" title=":wink:" class="emoji" alt=":wink:" loading="lazy" width="20" height="20"> :</p>
<pre data-code-wrap="python"><code class="lang-python">x = torch.tensor([1, 2, 0, 2, 2, 2, 2, 1])
F.one_hot(x, num_classes=4)
</code></pre>

KFrank: <p>Hi Peter!</p>
<aside class="quote no-group" data-username="ptrblck" data-post="3" data-topic="44350" data-full="true">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/ptrblck/48/1823_2.png" class="avatar"> ptrblck:</div>
<blockquote>
<p><code>F.one_hot</code> should yield the same result as the <code>scatter</code> operation <img src="https://discuss.pytorch.org/images/emoji/apple/wink.png?v=12" title=":wink:" class="emoji" alt=":wink:" loading="lazy" width="20" height="20"> :</p>
<pre data-code-wrap="python"><code class="lang-python">x = torch.tensor([1, 2, 0, 2, 2, 2, 2, 1])
F.one_hot(x, num_classes=4)
</code></pre>
</blockquote>
</aside>
<p>Thanks <a class="mention" href="/u/ptrblck">@ptrblck</a>.  <code>one_hot()</code> is just the ticket to replace the<br>
<code>scatter()</code> / <code>unsqueeze()</code> nonsene.</p>
<p>Best.</p>
<p>K. Frank</p>

KFrank: <aside class="quote no-group" data-username="KFrank" data-post="4" data-topic="44350" data-full="true">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/k/ecb155/48.png" class="avatar"> KFrank:</div>
<blockquote>
<p>Thanks <a class="mention" href="/u/ptrblck">@ptrblck</a>.  <code>one_hot()</code> is just the ticket to replace the<br>
<code>scatter()</code> / <code>unsqueeze()</code> nonsene.</p>
</blockquote>
</aside>
<p>Oops!  Spoke too soon.</p>
<p>This github issue:</p>
<p><a href="https://github.com/pytorch/pytorch/issues/15060" rel="noopener nofollow ugc">https://github.com/pytorch/pytorch/issues/15060</a></p>
<p>suggests that <code>one_hot</code> is there, but is recent, as does this entry<br>
in the documentation:</p>
<p><a href="https://pytorch.org/docs/master/nn.html#one-hot" rel="noopener nofollow ugc">https://pytorch.org/docs/master/nn.html#one-hot</a></p>
<p>However, this  analogous url for the “stable” documentation<br>
suggests that <code>one_hot</code> hasn’t yet made it to prime time:</p>
<p>“https ://pytorch.org/docs/stable/nn.html#one_hot”</p>
<p>In any event, my 1.0.1 version of pytorch doesn’t have it yet:</p>
<pre><code class="lang-plaintext">&gt;&gt;&gt; import torch
&gt;&gt;&gt; torch.version.__version__
'1.0.1'
&gt;&gt;&gt; torch.nn.functional.one_hot (torch.tensor ([1, 2, 3, 2, 1]))
Traceback (most recent call last):
  File "&lt;stdin&gt;", line 1, in &lt;module&gt;
AttributeError: module 'torch.nn.functional' has no attribute 'one_hot'
</code></pre>
<p>(I don’t understand github or the documentation well enough to<br>
be able to figure out in which version <code>one_hot</code> first appeared.)</p>
<p>Best.</p>
<p>K. Frank</p>

Keverino: <aside class="quote no-group" data-username="KFrank" data-post="4" data-topic="44350">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/k/ecb155/48.png" class="avatar"> KFrank:</div>
<blockquote>
<p>F.one_hot(x, num_classes=4)</p>
</blockquote>
</aside>
<p>Thank you, this is exactly what I was looking for!</p>
<p>I have a follow up question though:<br>
When I use this approach in my custom loss, I get the following error:</p>
<pre><code class="lang-auto">RuntimeError: element 0 of tensors does not require grad and does not have a grad_fn
</code></pre>
<p>I guess the auto_grad function has troubles with the one_hot function.<br>
any ideas on how to solve this?</p>
<p>Thanks in advance!</p>

Javier: <p>Hi,<br>
I’m not sure if the F.one_hot answer is extendable to the n-dimensional case, I’m also not sure how to do it with reshape operations. <a class="mention" href="/u/ptrblck">@ptrblck</a></p>
<p>Assume I have a tensor where the first two dims are batch and channel, and the last three correspond to xyz space:</p>
<p>A = torch.randn(b,c,32,32,32)</p>
<p>What I would like to do is to binarize along the x dimension (dim=2) for any batch or channel, i.e for every yz location I want to set the maximum value along x-axis to 1 and the rest to zero. Is there a way of doing this?</p>
<p>Thank you!</p>

ptrblck: <aside class="quote no-group" data-username="Javier" data-post="7" data-topic="44350">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/javier/48/12526_2.png" class="avatar"> Javier:</div>
<blockquote>
<p>What I would like to do is to binarize along the x dimension (dim=2) for any batch or channel, i.e for every yz location I want to set the maximum value along x-axis to 1 and the rest to zero.</p>
</blockquote>
</aside>
<p>If I understand this use case correctly, this should work:</p>
<pre data-code-wrap="python"><code class="lang-python">x = torch.randn(2, 3, 4, 4, 4)
idx = torch.argmax(x, dim=2, keepdims=True)
ret = torch.zeros_like(x).scatter_(2, idx, 1.)
</code></pre>