Unable to understand why I am repeatedly getting this RunTime error

imRP26: <pre><code class="lang-auto">RuntimeError: Trying to backward through the graph a second time (or directly access saved tensors after they have already been freed). Saved intermediate values of the graph are freed when you call .backward() or autograd.grad(). Specify retain_graph=True if you need to backward through the graph a second time or if you need to access saved tensors after calling backward.
</code></pre>
<p>This is the model that I have defined :-</p>
<pre><code class="lang-auto">class SentimentAnalysis(nn.Module):

    def __init__(self, vocab_size, embedding_dim):
        super(SentimentAnalysis, self).__init__()
        self.fc = nn.Linear(embedding_dim, 2)
        self.sigmoid = nn.Sigmoid()

    def forward(self, x):
        out1 = self.fc(x)
        out2 = self.sigmoid(out2)
        out3 = out3.mean(dim=1)
        return out3
</code></pre>
<p>And this is my model training step :-</p>
<pre><code class="lang-auto">device = torch.device('cuda' if torch.cuda.is_available() else 'cpu')
sentiment_analysis_model = SentimentAnalysis(len(train_vocabulary), 100)
sentiment_analysis_model = sentiment_analysis_model.to(device)
loss_fn = nn.BCELoss()
optimizer = torch.optim.Adam(sentiment_analysis_model.parameters(), lr=1e-3)

epochs = 5
losses = []

for epoch in range(epochs):
    sentiment_analysis_model.train()
    for batch_idx, batch in enumerate(train_loader):
        input = batch[0].to(device)
        target = batch[1].to(device)
        optimizer.zero_grad()
        out = sentiment_analysis_model(input)
        target1 = []
        for i in target:
            if i == 0:
                target1.append(torch.tensor([1.0, 0.0], requires_grad=True))
            else:
                target1.append(torch.tensor([0.0, 1.0], requires_grad=True))
        target1 = torch.stack(target1)
        loss = loss_fn(out, target1)
        loss.backward()
        optimizer.step()
        
    losses.append(loss.item())
</code></pre>
<p>I’ve been stuck at this for more than a couple of days now, but unable to get any solution here. Any help would be so very much appreciated!</p>

imRP26: <p>I’ve even tried this :-</p>
<pre><code class="lang-auto">loss.backward(retain_graph=True)
</code></pre>
<p>but still getting the above RuntimeError.</p>

srishti-git1110: <p>Hey, could you please try removing <code>requires_grad=True</code> from the training code here -</p>
<aside class="quote no-group" data-username="imRP26" data-post="1" data-topic="177803">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/i/ee59a6/48.png" class="avatar"> imRP26:</div>
<blockquote>
<pre><code class="lang-auto">for i in target:
            if i == 0:
                target1.append(torch.tensor([1.0, 0.0], requires_grad=True))
            else:
                target1.append(torch.tensor([0.0, 1.0], requires_grad=True))
        target1 = torch.stack(target1)
</code></pre>
</blockquote>
</aside>
<p>Please let me know if it doesn’t work. I’ll try to dig deeper then.</p>

imRP26: <p>Hello, thanks for your reply. Unfortunately, it didn’t work! <img src="https://discuss.pytorch.org/images/emoji/apple/frowning.png?v=12" title=":frowning:" class="emoji" alt=":frowning:" loading="lazy" width="20" height="20"></p>

imRP26: <p>Hello, I got the solution finally. Basically, input had requires_grad set to True, since it was basically an ELMo embedding, so after doing this :-</p>
<pre><code class="lang-auto">input = batch[0].to(device)
input = input.detach()
</code></pre>
<p>The code is running fine now. So very sorry for not mentioning that input was an ELMo embedding! But having said that, thanks a lot for your earlier reply!</p>