Not sure how to implement GAN optimizer

username: import-antigravity
cooked: <p>I’m trying to implement <a href="https://arxiv.org/abs/1705.10461" rel="noopener nofollow ugc">the ConOpt optimizer</a> for GANs, which combines stochastic gradient descent for both the generator and discriminator with a gradient penalty regularizer. I know that standard practice is usually to give the generator and discriminator separate optimizers, but for this algorithm it is necessary to add the squared norm of the gradients from both models to <em>each</em> loss. Therefore, it is necessary to have something like the <code>closure</code> argument in the <code>Optimizer.step()</code> method, but somehow have two closures, one for each model. Does anyone have a suggested way of doing this with only one closure?</p>

username: import-antigravity
cooked: <p>I don’t know if the only way to do it would be to, for example, make a new class <code>GANOptimizer</code> where the <code>step()</code> method requires two closures, one for each loss function. That’s pretty messy but I’m not sure of another solution</p>

username: Hmrishav_Bandyopadhy
cooked: <p>If you want to implement a min-max loss using a single optimizer, have you tried reverse grad ? Not sure if that would help…</p>

username: import-antigravity
cooked: <p>I’m not sure I understand your reply. Ideally, I’d like to have separate optimizers for the generator and discriminator, but the <em>loss</em> depends on both for ConOpt, even if the optimizer is only optimizing one set of parameters.</p>

username: Hmrishav_Bandyopadhy
cooked: <p>Sorry i misinterpreted the question. So as i understand you are trying to make the Consensus Optimizer, but in pytorch. The code would be something like this:</p>
<pre><code class="lang-auto">class ConsensusOptimizer(object):
    def __init__(self, learning_rate, alpha=0.1, beta=0.9, eps=1e-8):
        self.optimizer = torch.optim.RMSPropOptimizer(learning_rate)
        self._eps = eps
        self._alpha = alpha
        self._beta = beta

    def conciliate(self, d_loss, g_loss, d_vars, g_vars, global_step=None):
        alpha = self._alpha
        beta = self._beta

        # Compute gradients
        d_grads = torch.autograd.grad(d_loss, d_vars)
        g_grads = torch.autograd.grad(g_loss, g_vars)

        # Merge variable and gradient lists
        variables = d_vars + g_vars
        grads = d_grads + g_grads

        # Reguliarizer
        

        reg = 0.5 * sum(
            torch.sum(torch.square(g)) for g in grads
        )

        # Jacobian times gradiant
        Jgrads = torch.autograd.grad(reg, variables)
       
        for param in variables:
               param.grad+=self._alpha*Jg
        optimizer.step() 
</code></pre>
<p>I assume you call this after you call loss.backward(). This is untested code and you prob need to make some changes.<br>
See more info <a href="https://discuss.pytorch.org/t/how-to-modify-the-gradient-manually/7483/5">here</a></p>

username: import-antigravity
cooked: <p>That’s more or less what I’ve tried already but it doesn’t work. My understanding is that the best way to implement this is as an optimizer, hence my original post.</p>

username: rsk97
cooked: <p><a class="mention" href="/u/import-antigravity">@import-antigravity</a> - did you by any chance solve this? I am also trying to implement this optimizer and not very sure about the way to go forward.</p>