Two loss criterions simple Auto encoder

username: Kasi_Chennupati
cooked: <p>Hi</p>
<p>i have created a simple Linear AE and using adam optimizer<br>
i have two functions to minimize</p>
<p>loss1 = reconstruction loss<br>
loss 2 = centroids and encoded space euclidean distance<br>
so I created something like below   pytorch version 1.9.0+cu102</p>
<pre><code class="lang-auto">loss1 = 0.8*criterion(decoded, image)
loss2 = torch.sum(torch.cdist(encoded, dist_matrix.to(device))**2)

loss = loss1 + loss2
loss.backward(retain_graph=True)
optimizer.step()
optimizer.zero_grad()
</code></pre>
<p>Auto encoder code</p>
<pre><code class="lang-auto">class LAE3d(nn.Module):
    def __init__(self):
        # N, 784(28*28)
        super(LAE3d, self).__init__()
        self.encoder = nn.Sequential(
            nn.Linear(1*28 * 28, 1000),
            nn.ReLU(),
            nn.Linear(1000, 250),
            nn.ReLU(),
            nn.Linear(250, 50),
            nn.ReLU()
        )
        self.decoder = nn.Sequential(
            nn.Linear(50, 250),
            nn.ReLU(),
            nn.Linear(250, 1000),
            nn.ReLU(),
            nn.Linear(1000, 1*28*28),
            nn.ReLU()
        )

    def forward(self, x):
        encoded = self.encoder(x)
        decoded = self.decoder(encoded)
        return encoded, decoded


model = LAE3d()
model.to(device)
criterion = nn.MSELoss()
optimizer = torch.optim.Adam(model.parameters(), lr=0.001)

print(summary(model, input_size=(1, 1*28*28)))
</code></pre>
<p>basically trying to recreate <a href="https://link.springer.com/epdf/10.1007/978-3-642-41822-8_15?sharing_token=wGJAk4mZ6aTNhERspWEsZve4RwlQNchNByi7wbcMAY6rWZxw-I2NiNrqbR37q3i_NyvNy0iFaVD199Ak-qJJR8DMI2ftqEvUVLM2L19--N5BS0ZU-xCIEPXNr5IMBpydWGo1w__iHtyYg7UGs4brAufblvhH0JsntWQxtLwubm8%3D" rel="noopener nofollow ugc">Paper</a></p>
<p>I get the following error</p>
<p>RuntimeError: one of the variables needed for gradient computation has been modified by an inplace operation: [torch.cuda.FloatTensor [250, 50]], which is output 0 of TBackward, is at version 305; expected version 304 instead. Hint: the backtrace further above shows the operation that failed to compute its gradient. The variable in question was changed in there or anywhere later. Good luck!</p>

username: ptrblck
cooked: <p>The <code>retain_graph=True</code> option might be the root cause for this error and is often not needed.<br>
Could you explain why you are using it and if your model would work without this argument?</p>

username: Kasi_Chennupati
cooked: <p>Hi</p>
<p>No, this gives the following error.</p>
<p>i use the retain graph =true  because to avoid freeing up the grad applied to the encoder weights and decoder weights… so to avoid the mismatch of weight data when the second loss is applied</p>
<p><code>RuntimeError: Trying to backward through the graph a second time (or directly access saved variables after they have already been freed). Saved intermediate values of the graph are freed when you call .backward() or autograd.grad(). Specify retain_graph=True if you need to backward through the graph a second time or if you need to access saved variables after calling backward.</code></p>

username: ptrblck
cooked: <aside class="quote no-group" data-username="Kasi_Chennupati" data-post="3" data-topic="131430">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/kasi_chennupati/48/41940_2.png" class="avatar"> Kasi_Chennupati:</div>
<blockquote>
<p>so to avoid the mismatch of weight data when the second loss is applied</p>
</blockquote>
</aside>
<p>I don’t quite understand the use case, since you are adding both losses in your initial post and call <code>loss.backward()</code> on the accumulated loss.</p>

username: Kasi_Chennupati
cooked: <p>use case:<br>
i want to cluster the encoded data into n clusters while its training so loss1 is the reconstruction error and loss 2 is euclidean distance between the centroids and encoded data.</p>
<p>i want to minimize both the losses so I can improve clustering efficiency.</p>
<p>so I did loss1+loss2    and loss.backward()</p>

username: ptrblck
cooked: <p>That sounds valid and I don’t know why <code>retain_graph=True</code> would be needed.<br>
Based on your description you are executing forward passes through the models, calculate losses, accumulate them, and call <code>backward</code> on the final loss.<br>
I don’t see why the graph should be kept alive and where the “backward through the graph a second time” issue would be raised.</p>