Gradient Ascent and Gradient Modification/Modifying Optimizer instead of Grad_weight

username: steveo
cooked: <p>Hi All,<br>
I have a few questions related to the topic of modifying gradients and the optimizer. I’m wondering if there is an easy way to perform gradient ascent instead of gradient descent. For example, this would correspond to replacing grad_weight by -grad_weight in linear layer definition as seen in class LinearFunction(Function): from the Extending PyTorch page. My concern here is that this will mess up a downstream function that requires grad_weight instead of -grad_weight, or is this not a concern at all? A suggestion made to me was to try to modify the optimizer. Is there a simple way to go about doing W + dW instead of W - dW in the optimizer? I can’t really tell from the source code for SGD or ADAM.</p>
<p>Thanks for reading!</p>

username: albanD
cooked: <p>Hi,</p>
<p>The simplest way to do gradient ascent on a loss <code>L</code> is to do gradient descent on <code>-L</code> . <img src="https://discuss.pytorch.org/images/emoji/apple/smiley.png?v=9" title=":smiley:" class="emoji" alt=":smiley:"></p>

username: steveo
cooked: <p>That is an interesting solution. I think I need to further clarify my original question. I would like to include a negative sign on the updates to the weights, and this corresponds to changing grad_weight to -grad_weight, while grad_input and grad_bias are left untouched.  However, I am wary of unintended consequences of doing something like this to the gradients, and was wondering if there was an easy way to change the optimizer such that it performed gradient ascent(W + dW) for the non last layer weights specifically, but left the other parameters alone?</p>

username: albanD
cooked: <p>In that case I guess you will have to create your custom optimizer to handle that. With one group for the descent part and one group for the ascent part for example.</p>

username: Tamal_Chowdhury
cooked: <p>Continuing the discussion from <a href="https://discuss.pytorch.org/t/gradient-ascent-and-gradient-modification-modifying-optimizer-instead-of-grad-weight/62777/4">Gradient Ascent and Gradient Modification/Modifying Optimizer instead of Grad_weight</a>:</p>
<p>I’m working on a similar problem where I need to optimize the following loss function:<br>
<div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/e/1/e14f14574d5538911e496e16e0cc68ec1fa9b88a.jpeg" data-download-href="https://discuss.pytorch.org/uploads/default/e14f14574d5538911e496e16e0cc68ec1fa9b88a" title="IMG_20200926_222955"><img src="https://discuss.pytorch.org/uploads/default/optimized/3X/e/1/e14f14574d5538911e496e16e0cc68ec1fa9b88a_2_379x500.jpeg" alt="IMG_20200926_222955" data-base62-sha1="w9aRbpjBwVatnsBpRygHzmG2otk" width="379" height="500" srcset="https://discuss.pytorch.org/uploads/default/optimized/3X/e/1/e14f14574d5538911e496e16e0cc68ec1fa9b88a_2_379x500.jpeg, https://discuss.pytorch.org/uploads/default/optimized/3X/e/1/e14f14574d5538911e496e16e0cc68ec1fa9b88a_2_568x750.jpeg 1.5x, https://discuss.pytorch.org/uploads/default/optimized/3X/e/1/e14f14574d5538911e496e16e0cc68ec1fa9b88a_2_758x1000.jpeg 2x" data-dominant-color="FBFBFB"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">IMG_20200926_222955</span><span class="informations">3106×4096 176 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div><br>
Here <code>w</code> (omega) is model parameter and <code>Lamdas</code> are Lagrange Multipliers. I need to perform gradient descent wrt. <code>omega</code> and simultaneously gradient ascent wrt. <code>lambda</code>. <code>lambda</code> is not a model parameter and only included in the loss term.<br>
Will your solution of updating lambda using gradient descent on -L work in this case? If it does then taking negative learning rate for <code>lambdas</code> in gradient descent should also be equivalent. And if it doesn’t then what should be the pytorch solution for this(without changing the optimizer source code)? Or should I need to creat a custom optimizer?</p>

username: t_naumenko
cooked: <p>I think that this is a bit too late, but the solution I came up with is to use a custom autograd function, which reverses gradient direction. As like as <a class="mention" href="/u/tamal_chowdhury">@Tamal_Chowdhury</a> , I have a lagrangian optimization problem, for which this function works perfectly. A small working example would be:</p>
<pre><code class="lang-auto">import torch


class AscentFunction(torch.autograd.Function):
    @staticmethod
    def forward(ctx, input):
        return input

    @staticmethod
    def backward(ctx, grad_input):
        return -grad_input


def make_ascent(loss):
    return AscentFunction.apply(loss)


x = torch.normal(10, 3, size=(10,))
w = torch.ones_like(x, requires_grad=True)

loss = (x * w).sum()
print(f'descent loss: {loss.item():.2f}')

loss.backward()
print(w.grad)

w.grad = None

loss = (x * w).sum()
m_loss = make_ascent(loss)
print(f'ascent loss: {m_loss.item():.2f}')

m_loss.backward()
print(w.grad)
</code></pre>
<p>It’s output:</p>
<pre><code class="lang-auto">descent loss: 96.13
tensor([12.7093, 11.2243,  6.4265,  7.6572, 14.2737, 15.1144,  8.0099,  6.2517,
         7.6352,  6.8274])
ascent loss: 96.13
tensor([-12.7093, -11.2243,  -6.4265,  -7.6572, -14.2737, -15.1144,  -8.0099,
         -6.2517,  -7.6352,  -6.8274])
</code></pre>

username: BraveDrXuTF
cooked: <aside class="quote no-group" data-username="Tamal_Chowdhury" data-post="5" data-topic="62777">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/tamal_chowdhury/48/29176_2.png" class="avatar"> Tamal_Chowdhury:</div>
<blockquote>
<p>king negative learning rate for <code>lambdas</code> in gradient descent should also be equivalent. And if it doesn’t then what should be the pytorch solution for this(without changing the optimizer source code)? Or should I need to creat a custom optimizer?</p>
</blockquote>
</aside>
<p>I got the same problem.</p>