Sum is zero gradient still flowing when multiplied with one hot vec

barakb: <p>In order to explain my issue, I’ll just show step by step:<br>
First let’s build a simple model:</p>
<pre><code class="lang-auto">class Model(nn.Module):
    def __init__(self):
      super(Model,self).__init__()
      self.fc = nn.Linear(10,2)
    def forward(self,x):
      return self.fc(x)
</code></pre>
<p>Let’s declare the model ,optimizer and a random input:</p>
<pre><code class="lang-auto">model = Model()
optimizer = torch.optim.SGD(model.parameters(),lr=1)
x = torch.rand(10)
</code></pre>
<p>Let’s run the model:</p>
<pre><code class="lang-auto">out = model(x)
</code></pre>
<p>We are not going to use any known loss function but we will do the following:</p>
<pre><code class="lang-auto">prob1 = F.softmax(out,dim=-1)
prob2 = F.softmax(out,dim=-1)
loss = prob1 - prob2.detach()
loss = torch.sum(loss)
loss.backward()
</code></pre>
<p>The loss suppose to be zero, the gradients should flow through one of the softmax.<br>
Let’s print the loss and the gradient of fc.w:</p>
<blockquote>
<p>loss: tensor(0., grad_fn=)<br>
gradient: tensor([[0., 0., 0., 0., 0., 0., 0., 0., 0., 0.],<br>
[0., 0., 0., 0., 0., 0., 0., 0., 0., 0.]])</p>
</blockquote>
<p>At least for me, it  make sense.<br>
<strong>But</strong>, Let’s make a small change, we’ll add a target as follows:</p>
<pre><code class="lang-auto">target = torch.LongTensor([1])
target_onehot = F.one_hot(target,2)
prob1 = F.softmax(out,dim=-1)
prob2 = F.softmax(out,dim=-1)
loss = prob1 - prob2.detach()
loss = torch.sum(loss*target_onehot)
loss.backward()
</code></pre>
<p>Now let’s run and print again:</p>
<blockquote>
<p>sum_loss: tensor(0., grad_fn=)<br>
gradient: tensor([[-0.1637, -0.1643, -0.2050, -0.2044, -0.1300, -0.0278, -0.2135, -0.0859,<br>
-0.2186, -0.0658],<br>
[ 0.1637,  0.1643,  0.2050,  0.2044,  0.1300,  0.0278,  0.2135,  0.0859,<br>
0.2186,  0.0658]])</p>
</blockquote>
<p>The target is multiplied by loss which is a vector of zeros, how can it be that now we have gradients?<br>
even if… why the multiplication with “target” helps? he is not an important part of the backward graph?<br>
What am I missing?<br>
Thanks a lot!!!</p>

albanD: <p>Hi,</p>
<p>The fact that the loss is 0 does not mean the gradient is 0.<br>
The gradient that flows back from the backward() call is actually 1 (because it computes vector jacobian product. And so when the vector is just 1, you get the jacobian).<br>
The big difference between the two is that in the first case, the gradient that flows back to the softmax is a vector full of <code>1</code> while in the second case, that vector has been multiplied by <code>target_onehot</code> (backward of the multiplication) and so contains way less <code>1</code>s and is not symetric anymore. I guess this is where these non-<code>0</code> values come from.</p>

barakb: <p>I see your point but my calculations are just not agreeing with that.<br>
I’ll make things a bit clearer<br>
I’ll change the notations to:</p>
<pre><code class="lang-auto">loss1 = torch.sum(loss*target_onehot)
</code></pre>
<p>And let’s denote:<br>
S(out) = softmax(out)</p>
<p>Now:<br>
dloss1/dloss = [ target_onehot[0] ,  target_onehot[1]]<br>
dloss/dout=[[S(out[0])S(out[0]),S(out[0])(1-S(out[1])) ],<br>
[S(out[1])(1-S(out[0])),S(out[1])S(out[1])]]<br>
dout/db = [[1 , 1] , [1, 1]]<br>
dloss1/db = dloss1/dloss * dloss/dout * dout/db<br>
If dloss1/dloss is:[1,1] -&gt; I don’t see how it make the grad equal zero. (unless for very specific S() values.)</p>

albanD: <p>You can simply look at <code>dloss1/dloss * dloss/dout</code><br>
Here you do a product of matrices: <code>[1, 1] * [[S0(1-S0), -S0*S1], [-S1*S0, S1(1-S1)]]</code>.<br>
And so the result is: <code>[S0(1 - S0 - S1), S1(1-S1-S0)]</code>. But because your softmax has only two entries, we know that <code>S0 + S1 = 1</code>. So the result for the gradient is <code>[0, 0]</code>.</p>

barakb: <p>Thanks I forgot that detail, and I had a big mistake in the softmax derivative.<br>
Thanks again.</p>