Gradient is None for some Parameters

milan_kalkenings: <p>For adversial training, I want to perturbate word embeddings. Oversimplified, the problematic part of my code looks like this:</p>
<pre><code class="lang-auto">optimizer.zero_grad()
emb = embedder(token_ids)
emb = Variable(emb, requires_grad=True)
logits = cls_head(emb)
loss = loss_func(logits, target) 
loss.backward()
optimizer.step()
print(emb.weight.grad) &gt;&gt;&gt; None
print(cls_head.weight.grad) &gt;&gt;&gt; Tensor
</code></pre>
<p>gradient for  <code>cls_head</code> is calculated, but gradient for <code>embedder</code> is <code>None</code>. Why is the gradient of <code>embedder</code> None and how can  I calculate the gradient w.r.t. <code>emb</code> without ending in this problem?</p>
<p>Kind Regards,<br>
Milan</p>

ptrblck: <p>You are detaching <code>emb</code> from the computation graph by re-wrapping it into the deprecated <code>Variable</code>:</p>
<pre><code class="lang-python">emb = Variable(emb, requires_grad=True)
</code></pre>
<p>Remove this line of code to keep the computation graph alive.</p>

milan_kalkenings: <p>Is there aby other way to get the gradient with respect to emb?<br>
I need to get the gradient to both, <code>emb</code> and the parametere within the <code>embedder</code> (for some perturbation).</p>

ptrblck: <p>I don’t quite understand the question. In your current code you are implicitly detaching the computation graph by recreating the tensor. If you remove this line of code, it should work without any other changes.</p>

srishti-git1110: <p>Since you mentioned this</p>
<aside class="quote no-group" data-username="milan_kalkenings" data-post="1" data-topic="162715">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/milan_kalkenings/48/42748_2.png" class="avatar"> milan_kalkenings:</div>
<blockquote>
<p><code>print(emb.weight.grad) &gt;&gt;&gt; None</code></p>
</blockquote>
</aside>
<p>I created a short simulation like this:</p>
<pre><code class="lang-auto">model = nn.Sequential(nn.Linear(2,1))
emb = model(torch.tensor([1.0, 2]))
emb = torch.autograd.Variable(emb, requires_grad=True)
loss_fn = nn.MSELoss()
loss = loss_fn(emb, torch.tensor([1.0]))
loss.backward()
print(emb.grad)
print(model[0].weight.grad)
</code></pre>
<p>gives</p>
<pre><code class="lang-auto">tensor([0.0333])
None
</code></pre>
<p>and according to this <code>emb</code> should have its grad attribute populated, but as pointed out by <a class="mention" href="/u/ptrblck">@ptrblck</a> you explicitly detach <code>emb</code> from the graph and hence the model (embedder) parameters’ grad shan’t be populated.</p>

milan_kalkenings: <p>My goal is to transfer “fast gradient sign method (FGSM)” to nlp. FGSM calculates the gradient w.r.t. input images and later uses that in an adversial update step.</p>
<p>Calculating the gradient w.r.t. a sequence of (int-valued) token sequences doesn’t work, so my idea was to instead calculate the gradient w.r.t. the token embeddings, so that the token embeddings can later on be perturbated for the adversial uodate step.</p>
<p>That’s why i need to calculate the gradients to both, the embedder weights (for updating them) and the embeddings (for the perturbation). Is there any way to achieve this?</p>

ptrblck: <p>Is removing the line of code, mentioned a few times already, which recreates the tensor not working? If not, why?</p>

milan_kalkenings: <p>But then it doesn’t require a grad, i.e. I can’t calculate the gradient with respect to emb then</p>

ptrblck: <p>That’s not the case unless you already detached the computation graph before or disabled the gradient calculation through any context manager such as <code>with torch.no_grad()</code>.<br>
This code works properly:</p>
<pre><code class="lang-python">embedder = nn.Embedding(10, 10)
cls_head = nn.Linear(10, 10)
token_ids = torch.randint(0, 10, (10,))

loss_func = nn.MSELoss()

emb = embedder(token_ids)
logits = cls_head(emb)

loss = loss_func(logits, torch.randn_like(logits)) 
loss.backward()

print(embedder.weight.grad)
print(cls_head.weight.grad)
</code></pre>
<p>as it’s not detaching the computation graph and prints valid gradients for both modules.</p>

milan_kalkenings: <p>But <code>emb.grad</code> doesn’t exist then</p>

ptrblck: <p><code>emb</code> is an intermediate forward activation and doesn’t have a <code>weight</code> attribute.<br>
If you want to check the gradient from this activation, call <code>.retain_grad()</code> on it:</p>
<pre><code class="lang-python">embedder = nn.Embedding(10, 10)
cls_head = nn.Linear(10, 10)
token_ids = torch.randint(0, 10, (10,))

loss_func = nn.MSELoss()

emb = embedder(token_ids)
emb.retain_grad()
logits = cls_head(emb)

loss = loss_func(logits, torch.randn_like(logits)) 
loss.backward()

print(embedder.weight.grad)
print(cls_head.weight.grad)
print(emb.grad)
</code></pre>