Argmax with PyTorch

username: MatthewKleinsmith
cooked: <p>How can I use argmax with PyTorch?</p>
<p>Meta:</p>
<ul>
<li>How could I have answered my own question? I searched the PyTorch docs and the PyTorch repo for “argmax” but got no results.</li>
<li>Does it make sense to use argmax with a GPU?</li>
</ul>

username: fmassa
cooked: <p><code>torch.max</code> returns both the max values as well as the indices.<br>
so you can do</p>
<pre><code class="lang-python">values, indices = tensor.max(0)
values, indices = torch.max(tensor, 0)
</code></pre>

username: platero
cooked: <p>I know that the <code>values</code> are differentiable (e.g., global max-pooling). Are the <code>indices</code> also differentiable? Thank you very much!</p>

username: Andre_Holzner
cooked: <p>(hard) <code>argmax</code> is not differentiable in general (this has nothing to do with PyTorch), i.e. one can not use gradient based methods with argmax. See e.g. <a href="https://www.reddit.com/r/MachineLearning/comments/4e2get/argmax_differentiable/" rel="nofollow noopener">https://www.reddit.com/r/MachineLearning/comments/4e2get/argmax_differentiable/</a> on how to train models involving argmax functions. One potential alternative suggested there is to use  softmax instead.</p>

username: platero
cooked: <p>Thank you very much!</p>

username: bgenchel
cooked: <p>Any ideas for how to use this max function in a differentiable way? A custom loss function i’m writing has to do with the indices of max values. Not sure how to redo the loss function such that it uses differentiable components.</p>

username: dpernes
cooked: <p>Hi <a class="mention" href="/u/bgenchel">@bgenchel</a>,</p>
<p>I am not sure if I understand your problem. Do you want to have gradients with respect to indices? Well, indices are integers by definition and you cannot take derivatives of a function with respect to a variable that is defined over the integers only…</p>

username: Brando_Miranda
cooked: <p>what does the 0 do in the indexing?</p>

username: BlakeWest
cooked: <p>Yeah I found the zero to be confusing too. It’s the dimension along which you want to find the max. I was getting confused because in my case, the thing I wanted to find the max of had shape (1, 49), which meant when I did <code>torch.max(preds, 0)</code>, I would just get back the whole array, and it didn’t make any sense. I needed to do <code>torch.max(preds, 1)</code>, and indeed that returned (max value, index)</p>

username: jchillin
cooked: <p>you can now do torch.argmax(preds, dim=1) in version 0.4.0</p>
<p><a class="mention" href="/u/blakewest">@BlakeWest</a> dimension 0 is the batch and dimension 1 is the class probabilities (assuming you use softmax on your final output). Therefore you would want to to do an argmax along dimension 1 ie. the class with the highest probabilities</p>

username: chenchr
cooked: <p>hello. How can I manipulate the <code>tensor</code> by the <code>indice</code>, for example what should I do if I want to change the value of <code>tensor</code> elements that corresponding to the <code>indice</code>.</p>

username: Brando_Miranda
cooked: <p>what about argmax from the tensor itself like:</p>
<pre><code class="lang-auto">import torch

x = torch.randn(3)
x.argmax(-1)
</code></pre>
<p>meta:<br>
- where does one find the docs for this? googling takes you to the <code>torch.argmax</code> function and not the one for tensors…ans: seems this is where it is: <a href="https://pytorch.org/docs/stable/tensors.html" rel="nofollow noopener">https://pytorch.org/docs/stable/tensors.html</a></p>
<hr>
<p>probably useful:</p>
<ul>
<li><a href="https://discuss.pytorch.org/t/calculating-accuracy-of-the-current-minibatch/4308/11" class="inline-onebox">Calculating accuracy of the current minibatch?</a></li>
<li><a href="https://stackoverflow.com/questions/51503851/calculate-the-accuracy-every-epoch-in-pytorch" rel="nofollow noopener">https://stackoverflow.com/questions/51503851/calculate-the-accuracy-every-epoch-in-pytorch</a></li>
<li><a href="https://towardsdatascience.com/understanding-dimensions-in-pytorch-6edf9972d3be" rel="nofollow noopener">https://towardsdatascience.com/understanding-dimensions-in-pytorch-6edf9972d3be</a></li>
<li><a href="https://discuss.pytorch.org/t/how-does-one-get-the-predicted-classification-label-from-a-pytorch-model/91649/4" class="inline-onebox">How does one get the predicted classification label from a pytorch model?</a></li>
</ul>

username: Brando_Miranda
cooked: <p>what do negative indices do? <a class="mention" href="/u/fmassa">@fmassa</a></p>