Why is my parameter not changing and its gradient 0?

charlieparker: <p>I am building a really simple model to learn the parameter in a Poisson model and I am not sure where I am going wrong. I am using pytorch.nn and doing the following.</p>
<p>I made some really simple fake data</p>
<pre><code class="lang-auto"># This is the value I am trying to estimate

x = torch.tensor(2.0)


# This is a value drawn from the Poisson(x) distribution 
# In this example it is 4

y = torch.poisson(x).reshape(1)
</code></pre>
<p>Then I just set up a really simple model</p>
<pre><code class="lang-auto"># I initialised the parameter that is going to estimate x with a random value (0.2) 
# and set that it requires a gradient

a = torch.tensor([0.2], requires_grad = True)


# I define the loss function with log_input set to false

loss_function = torch.nn.PoissonNLLLoss(log_input = False)


# Defined the model

def model(a):
    return torch.poisson(a)

# And the parameter to be optimised 
# I chose SGD arbitrarily, maybe this is the problem?

optimizer = torch.optim.SGD([a], lr = 0.1)
</code></pre>
<p>Then I do iterations to update a</p>
<pre><code class="lang-auto">for i in range(2000):
    
    # Forward pass

    y_pred = model(a)


    # Compute the loss

    loss = loss_function(y_pred, y)
    

    # Backprop

    optimizer.zero_grad()
    
    loss.backward()
    
    
    # Update parameters

    optimizer.step()
</code></pre>
<p>The problem is after this the a is still 0.2 and if I call a.grad it is 0. Where am I going wrong?</p>
<p>I have also tried instead to initiate a class for the model inheriting a nn.Module. However the same problem persists :</p>
<pre><code class="lang-auto">class learning_model(nn.Module):
    
    def __init__(self):
        super().__init__()
        self.a = nn.Parameter(torch.rand(1))
        self.a.requires_grad = True
        
    def forward(self):
        return torch.poisson(self.a)
</code></pre>
<pre><code class="lang-auto">model = learning_model()

loss_function = nn.PoissonNLLLoss(log_input = False)

optimizer = torch.optim.SGD(model.parameters(), lr = 0.1)

print(model.a)
</code></pre>
<p>Outputs:</p>
<pre><code class="lang-auto">Parameter containing:
tensor([0.1402], requires_grad=True)
</code></pre>
<p>Then:</p>
<pre><code class="lang-auto">for i in range(20):
    
    # Forward pass
    y_pred = model()
    
    # Compute the loss
    loss = loss_function(y_pred, y)
    
    # Backprop
    optimizer.zero_grad()
    
    loss.backward()    
    
    # Update parameters
    optimizer.step()
    
print(model.a, '\n gradient:', model.a.grad)
</code></pre>
<p>Outputs:</p>
<pre><code class="lang-auto">Parameter containing:
tensor([0.1402], requires_grad=True) 
 gradient: tensor([0.])
</code></pre>

KFrank: <p>Hi Charlie!</p>
<aside class="quote no-group" data-username="charlieparker" data-post="1" data-topic="113116" data-full="true">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/c/f17d59/48.png" class="avatar"> charlieparker:</div>
<blockquote>
<p>The problem is after this the a is still 0.2 and if I call a.grad it is 0.</p>
</blockquote>
</aside>
<p>The is because that although <code>torch.poisson()</code> returns floating-point<br>
numbers and attaches a <code>grad_fn=&lt;PoissonBackward&gt;</code> to the<br>
computation graph, it gives, by design, zero gradients.</p>
<p>First is the question of what the derivative of a random sample<br>
with respect to some distribution parameter ought to be.  Pytorch<br>
chooses zero.  See the discussion of “gradient estimators” in the<br>
<a href="https://pytorch.org/docs/1.7.1/distributions.html" rel="noopener nofollow ugc">torch.distributions</a> documentation.</p>
<p>Second is the fact that <code>torch.poisson()</code> returns values that are<br>
discrete integers (even though they are packaged as floating-point<br>
numbers).  Since the result of <code>torch.poisson()</code> is stuck at an integer,<br>
it can’t change by “just a little bit,” so it makes sense to define its<br>
derivative as zero.</p>
<p>Best.</p>
<p>K. Frank</p>

charlieparker: <p>Hi Kirk,</p>
<p>Thanks so much for the great reply! Is there any alternative you could give instead then?<br>
I really need to use a Poisson distribution for my model.</p>
<p>Thanks,<br>
Charlie</p>

KFrank: <p>Hi Charlie!</p>
<aside class="quote no-group" data-username="charlieparker" data-post="3" data-topic="113116" data-full="true">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/c/f17d59/48.png" class="avatar"> charlieparker:</div>
<blockquote>
<p>Is there any alternative you could give instead then?<br>
I really need to use a Poisson distribution for my model.</p>
</blockquote>
</aside>
<p>The best I can think of is:</p>
<p>Find a continuous approximation to the Poisson distribution that<br>
adequately meets you needs.  (Search for “cumulative Poisson<br>
distribution.”)  This won’t be exactly the Poisson distribution, so<br>
you won’t get exactly the same results as you would have with the<br>
Poisson distribution.</p>
<p>Implement the inverse cumulative probability distribution for the<br>
approximation you chose.  Let’s call this function <code>f (u, a)</code>, where<br>
<code>u</code> is understood to be a random value sampled uniformly between<br>
zero and one, and <code>a</code> is your Poisson-distribution parameter<br>
(sometimes called the Poisson “rate”).  Holding <code>a</code> fixed, for a uniform<br>
deviate <code>u</code>, <code>f (u, a)</code> is a random deviate sampled from your<br>
(approximate) Poisson distribution.</p>
<p>Package <code>f()</code> as a pytorch “custom autograd function” class.  It will<br>
have a <code>forward()</code> function that calculates <code>f (u, a)</code>.  <code>u</code> will be a<br>
tensor with <code>requires_grad = False</code> that consists of one or more<br>
uniform deviates , while <code>a</code> will be a tensor that wraps the rate<br>
parameter and has <code>requires_grad = True</code>.</p>
<p>Now the important part:  Your custom autograd function will also have<br>
a <code>backward()</code> function that calculates the derivative of <code>f (u, a)</code> with<br>
respect to <code>a</code>.  (You have to work out the math for this and implement<br>
it.)</p>
<p>You can now feed random approximate-Poisson deviates into the<br>
calculations that lead to your loss function, and then use pytorch’s<br>
autograd system to backpropagate and optimize your loss with respect<br>
to your rate parameter <code>a</code> (as well as any other parameters you like).</p>
<p>Of course, any single random deviate is “noisy,” so your loss could<br>
also be noisy, jump around, and not optimize well.  But if any given<br>
optimization step involves averaging over many deviates, that noise<br>
will smooth out, and you will be able to optimize in a sensible and<br>
mathematically-legitimate way with respect to <code>a</code>.</p>
<p>Best.</p>
<p>K. Frank</p>