Model not training, gradients are None

otatopeht: <p>Hey everyone!<br>
I’m currently finetuning a pretrained sentence transformer with in-domain data.</p>
<pre><code class="lang-auto">SentenceTransformer(
  (0): Transformer({'max_seq_length': 512, 'do_lower_case': False}) with Transformer model: XLMRobertaModel 
  (1): Pooling({'word_embedding_dimension': 768, 'pooling_mode_cls_token': False, 'pooling_mode_mean_tokens': True, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False})
)
</code></pre>
<p>Below you can find a snippet of the code:</p>
<pre><code class="lang-auto">
from sentence_transformers import SentenceTransformer
import torch
activation = {}

def hook(name, output):
    activation[name] = output[0].detach()

model = SentenceTransformer('T-Systems-onsite/cross-en-de-roberta-sentence-transformer')
model.train()
optimizer = torch.optim.Adam(model.parameters(), lr=0.01)
cos_sim = torch.nn.CosineSimilarity(dim=0)
optimizer.zero_grad()

query_prediction = model.encode("a man is cutting up a tomato", convert_to_tensor=True)
positive_prediction = model.encode("a man is slicing a tomato", convert_to_tensor=True)
negative_prediction = model.encode("she's brushing her hair", convert_to_tensor=True)

dist_pos = 1 - cos_sim(query_prediction, positive_prediction)
dist_neg = 1 - cos_sim(query_prediction, negative_prediction)

loss = torch.max(torch.tensor(0.), 0.7 + dist_pos - dist_neg)

if loss != torch.tensor(0.):
    loss.backward()
for p in model.parameters():
    print(p.grad)

for name, layer in model.named_modules():
    layer.register_forward_hook(hook(name,query_prediction))
print(activation)
</code></pre>
<p>The loss is being calculated, but the gradients are None. Therefore, the model is not training.</p>
<p>When going through the <a href="https://github.com/UKPLab/sentence-transformers/blob/master/sentence_transformers/SentenceTransformer.py" rel="noopener nofollow ugc">sentence transformer</a> code, within the <strong>encode</strong> method the forward seems to be calculated with no grad. Might that be the problem?</p>
<p>Any tips or ideas on why the gradients are None would be much appreciated.</p>

ptrblck: <p>Did you verify that <code>loss.backward()</code> was called?<br>
If so, do you see valid <code>.grad_fn</code> attributes for <code>loss</code>, <code>dist_pos</code>, <code>dist_neg</code>, <code>query_***</code>, <code>***_preciction</code>?</p>

otatopeht: <p>Thank you for replying.<br>
The loss.backward() is in fact called when the batch loss is not 0.<br>
The output of .grad_fn attributes is:</p>
<pre><code class="lang-auto">&gt;  loss &lt;MaximumBackward object at 0x7f20ea3fb3d0&gt;
&gt;  query_prediction &lt;SelectBackward object at 0x7f20ea3fb3d0&gt;
&gt;  positive_prediction &lt;SelectBackward object at 0x7f20ea3fb3d0&gt;
&gt;  negative_prediction &lt;SelectBackward object at 0x7f20ea3fb3d0&gt;
&gt;  dist_neg &lt;RsubBackward1 object at 0x7f20ea3fb3d0&gt;
&gt;  dist_neg &lt;RsubBackward1 object at 0x7f20ea3fb3d0&gt;
</code></pre>

ptrblck: <p>This would mean that at least the model output is attached to the graph, so you could check the <code>grad_fn</code> attributes of previous activations and check, if any yields a <code>None</code>.</p>

otatopeht: <p>Thanks for the explanation!</p>
<aside class="quote no-group" data-username="ptrblck" data-post="4" data-topic="131236">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/ptrblck/48/1823_2.png" class="avatar"> ptrblck:</div>
<blockquote>
<p>so you could check the <code>grad_fn</code> attributes of previous activations and check, if any yields a <code>None</code>.</p>
</blockquote>
</aside>
<p>How can I do that?</p>
<p>When checking the parameters like so:</p>
<pre><code class="lang-auto">for p in model.parameters():
    print(p.grad_fn)
</code></pre>
<p>all I get is None.</p>

ptrblck: <p>The parameters don’t have any <code>grad_fn</code>, as they are leaf nodes, so you would need to check the forward activations either directly in the <code>forward</code> method, e.g. via:</p>
<pre><code class="lang-python">def forward(self, x):
    x = self.layer(x)
    print(x.grad_fn)
    ...
</code></pre>
<p>or via forward hooks.</p>

otatopeht: <p>Maybe something similar?</p>
<pre><code class="lang-auto">
def hook(name, output):
    activation[name] = output[0].detach()

query_prediction = model.encode("a man is cutting up a tomato", convert_to_tensor=True)



for name, layer in model.named_modules():
    layer.register_forward_hook(hook(name,query_prediction))
print(activation)
</code></pre>
<p>The output tensors are all of the same vaue.</p>
<pre><code class="lang-auto">{'': tensor(-0.0094), 
'0': tensor(-0.0094),
 '0.auto_model': tensor(-0.0094), 
'0.auto_model.embeddings': tensor(-0.0094), 
'0.auto_model.embeddings.word_embeddings': tensor(-0.0094), 
.
.
.
'1': tensor(-0.0094)}
</code></pre>

sprakashdash: <p>I have got the same issue, and I am getting <code>&lt;UnsafeViewBackward0 object at 0x2b2ac389e220&gt;</code> when I access <code>grad_fn</code>. Is this something I should not be expecting?</p>

ptrblck: <p>Yes, <code>UnsafeViewBackward</code> is expected and <code>_unsafe_view</code> is described <a href="https://github.com/pytorch/pytorch/blob/b7df3bba623541f33e0a780e21e858ae2247f8b1/aten/src/ATen/native/TensorShape.cpp#L3305-L3332">here</a>.</p>