One of the variables needed for gradient computation has been modified by an inplace operation:

Kevinkevin189: <p>I’m training a GAN<br>
the traininng code is here</p>
<pre><code class="lang-auto">            while images is not None:
                outputs=self.model['G'](images)
                self.model['D'].freeze(requires_grad=True)
                po=self.model['D'](outputs.detach())
                pg=self.model['D'](images)
                loss1=self.criteron['adv'](po,False)
                loss2=self.criteron['adv'](pg,True)


</code></pre>
<p>But I got this error message which makes me confused.<br>
one of the variables needed for gradient computation has been modified by an inplace operation: [torch.cuda.FloatTensor [256]] is at version 4; expected version 3 instead.</p>
<p>I tried several times,and I removed all inplace tag in RELU.<br>
The code works fine while only loss1 is calculated and backward, or loss2 is calculated and backward.But It reported this error when I write them both into the training code.<br>
And I found that I put gt and output both into the discriminator, then loss1 and loss2 is ok to be calculated.The critical code makes loss1 unable to backward is this line</p>
<pre><code class="lang-auto">pg=self.model['D'](images)
</code></pre>
<p>Removing this,my loss1 works fine.<br>
But my dis network just conv and maxpool the image,and there is no in-place op.<br>
I also replace this line:</p>
<pre><code class="lang-auto">pg=self.model['D'](images.clone().detach())
</code></pre>
<p>it still can’t work.<br>
what’s wrong?</p>

Kevinkevin189: <p>The new point I found is below:<br>
I changed the line order,</p>
<pre><code class="lang-auto">pg=self.model['D'](images)
po=self.model['D'](outputs.detach())
</code></pre>
<p>then the loss1 works.But now loss2 can’t backward.<br>
Seems the two are confilcts</p>
<p>Here is my Dis network code:</p>
<pre><code class="lang-auto">class Discriminator(nn.Module):
    def __init__(self,in_dim=3,inner_dim=64,norm=True):
        super(Discriminator, self).__init__()
        def getSpectralNormedModule(module,norm):
            if norm:
                 return nn.utils.spectral_norm(module)
            return module
        self.conv1=nn.Sequential(
            getSpectralNormedModule(nn.Conv2d(in_dim,inner_dim,4,2,1),norm),
            nn.BatchNorm2d(inner_dim),
            nn.LeakyReLU(0.2)
        )
        self.conv2=nn.Sequential(
            getSpectralNormedModule(nn.Conv2d(inner_dim,2*inner_dim,4,2,1),norm),
            nn.BatchNorm2d(2*inner_dim),
            nn.LeakyReLU(0.2)
        )
        self.conv3=nn.Sequential(
            getSpectralNormedModule(nn.Conv2d(2*inner_dim,4*inner_dim,4,2,1),norm),
            nn.BatchNorm2d(4*inner_dim),
            nn.LeakyReLU(0.2)
        )
        self.conv4=nn.Sequential(
            getSpectralNormedModule(nn.Conv2d(4 * inner_dim, 4 * inner_dim, 4, 1, 1), norm),
            nn.BatchNorm2d(4 * inner_dim),
            nn.LeakyReLU(0.2)
        )
        self.conv5 = nn.Sequential(
            getSpectralNormedModule(nn.Conv2d(4 * inner_dim, 4 * inner_dim, 4, 1, 1), norm),
            nn.BatchNorm2d(4 * inner_dim),
            nn.LeakyReLU(0.2)
        )
    def forward(self,x):
        x1=self.conv1(x)
        x2=self.conv2(x1)
        x3=self.conv3(x2)
        x4=self.conv4(x3)
        x5=self.conv5(x4)
        return x5
</code></pre>
<p>I still can’t figure out the key point.Someone Plz help me.I’m new to GAN.</p>

ptrblck: <p>I guess you might be facing a similar issue as described <a href="https://discuss.pytorch.org/t/struggling-with-a-runtime-error-related-to-in-place-operations/102222/6">here</a> and <a href="https://discuss.pytorch.org/t/error-running-multiple-models-in-torch-1-7-1-but-works-in-torch-1-0/104146/8">here</a>.<br>
Could you check if you are indeed trying to use stale gradients for already updated parameters?</p>