Poisson loss function

chilango: <p>I want to predict count data using a simple fully connected network. I could in principle frame it as a classification problem where each class corresponds to the event count, but I would like to do it properly using a Poisson loss function. Two quick questions:</p>
<ol>
<li>I can’t seem to find the implementation of this loss function, am I missing anything?</li>
<li>I also cannot seem to find any examples of using pytorch to predict count data, any pointers?</li>
</ol>
<p>Thank you.</p>

Tony-Y: <aside class="onebox whitelistedgeneric">
  <header class="source">
      <img src="https://isaacchanghau.github.io/img/icon.png" class="site-icon" width="16" height="16">
      <a href="https://isaacchanghau.github.io/post/loss_functions/" target="_blank" title="12:00AM - 07 June 2017" rel="nofollow noopener">Isaac Changhau – 7 Jun 17</a>
  </header>
  <article class="onebox-body">
    

<h3><a href="https://isaacchanghau.github.io/post/loss_functions/" target="_blank" rel="nofollow noopener">Loss Functions in Neural Networks | Isaac Changhau</a></h3>

<p>Loss function is an important part in artificial neural networks, which is used to measure the inconsistency between predicted value ($\hat{y}$) and actual label ($y$). It is a non-negative value, where the robustness of model increases along with...</p>


  </article>
  <div class="onebox-metadata">
    
    
  </div>
  <div style="clear: both"></div>
</aside>

<p>This article provides the definition of the Poisson loss. You can calculate it using a few pytorch functions.</p>

chilango: <p>Super simple question: since lambda &gt; 0 [because of the log(lambda_hat)] how could I go about defining the loss function to guarantee proper lambdas? In principle implementing it with pytorch functions is straightforward:</p>
<pre><code class="lang-auto">def poissonLoss(predicted, observed):
    """Custom loss function for Poisson model."""
    loss=torch.mean(predicted-observed*torch.log(predicted))
    return loss
</code></pre>
<p>But I obviously need to force the output to be strictly positive otherwise I’ll get -inf and nans. What sort of correction for the last layer could I use? Like ReLU with a floor&gt;0 of sorts?</p>

Tony-Y: <aside class="onebox wikipedia">
  <header class="source">
      <a href="https://en.m.wikipedia.org/wiki/Poisson_regression" target="_blank" rel="nofollow noopener">en.m.wikipedia.org</a>
  </header>
  <article class="onebox-body">
    <img src="//upload.wikimedia.org/wikipedia/commons/thumb/3/3a/Linear_regression.svg/220px-Linear_regression.svg.png" class="thumbnail" width="" height="">

<h3><a href="https://en.m.wikipedia.org/wiki/Poisson_regression" target="_blank" rel="nofollow noopener">Poisson regression</a></h3>

<p>In statistics, Poisson regression is a generalized linear model form of regression analysis used to model count data and contingency tables. Poisson regression assumes the response variable Y has a Poisson distribution, and assumes the logarithm of its expected value can be modeled by a linear combination of unknown parameters. A Poisson regression model is sometimes known as a log-linear model, especially when used to model contingency tables.
 Negative binomial regression is a popular generaliz...</p>

  </article>
  <div class="onebox-metadata">
    
    
  </div>
  <div style="clear: both"></div>
</aside>

<p>Lambda should be exp(theta * x).</p>

Intel_Novel: <p>I think that Poisson loss function is not supported by default in PyTorch. You may try this:</p><aside class="quote quote-modified" data-post="6" data-topic="27009">
  <div class="title">
    <div class="quote-controls"></div>
    <img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/y/7c8e57/48.png" class="avatar">
    <a href="https://discuss.pytorch.org/t/solved-how-to-implement-custom-loss-function/27009/6">[Solved] How to implement custom loss function?</a> 
  </div>
  <blockquote>
    I’ve read your loss function 
but I can’t see what is the difference except you use torch and F’s operation (like torch.mean(), F.softplus()) at the end of each function 
This is code I’m writing 
https://github.com/youngminpark2559/test/tree/master/update_CNN

Would you tell me what is the difference? 
Generating network 
https://github.com/youngminpark2559/test/blob/master/update_CNN/train/train.py#L65

Obtaining prediction 
https://github.com/youngminpark2559/test/blob/master/update_CNN/train…
  </blockquote>
</aside>

<p>BTW, I think that one should focus on loss functions that can manage learning rates self adaption instead of probabilities.</p>

chilango: <p>I’ve implemented the following poisson loss function using the first suggested link:</p>
<pre><code class="lang-auto">def poissonLoss(xbeta, y):
    """Custom loss function for Poisson model."""
    loss=torch.mean(torch.exp(xbeta)-y*xbeta)
    return loss
</code></pre>
<p>Training:</p>
<pre><code class="lang-auto">optimizer=torch.optim.SGD(net.parameters())
lossFn=poissonLoss

output_ = net(input_)
loss = lossFn(output_, target_)
net.zero_grad()
loss.backward()
optimizer.step()
</code></pre>
<p>Since I’m using pytorch internals, is this the right way to train/backprop/optimize? Not sure about this because my model is not learning.</p>
<p>Thanks,</p>

Tony-Y: <aside class="quote no-group" data-username="chilango" data-post="6" data-topic="44301">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/c/e47c2d/48.png" class="avatar"> chilango:</div>
<blockquote>
<p>net.zero_grad()</p>
</blockquote>
</aside>
<p>You should use optimizer.zero_grad() instead of it.</p>
<p><a href="https://pytorch.org/tutorials/beginner/pytorch_with_examples.html#pytorch-optim" class="onebox" target="_blank" rel="noopener nofollow ugc">https://pytorch.org/tutorials/beginner/pytorch_with_examples.html#pytorch-optim</a></p>