From MSE to Poisson loss - Car Demand Prediction

gshdk: <p>Hi,</p>
<p>I’m working on a project where we wish to predict demand for a free-floating car-sharing service using Conv-LSTM’s.</p>
<p>The problem is formulated as aggregate demand in grid-cells, where each grid cell is around 1 km^2 and the value contained in this cell is the number of successful rentals. Using this formulation we have a time series of 60-minute windows as our data. This gives an array of slices as input and a single slice as output - example shown below<br>
<img src="https://discuss.pytorch.org/uploads/default/original/3X/e/9/e9d2a2b323bf4888f6033e2b629c39bf61b14601.png" alt="Screen Shot 2020-12-27 at 17.15.19" data-base62-sha1="xmuxbgT97zizBHKtkDcjILCEAb7" width="319" height="163"></p>
<p>We have achieved mediocre performance with this setup and the use of the MSE-loss but would like to give the Poisson-loss a try.</p>
<p>Our problem is that we end up with a negative loss after a few iterations of training and are unsure about how to identify the problem or if there is something we are unaware of.</p>
<p>We use the Adam optimizer and with the following settings for the criterion:</p>
<pre><code class="lang-auto">criterion = nn.PoissonNLLLoss(log_input=False)
</code></pre>
<p>Using the default log_input=True did not help.</p>
<p>Our traning loop looks a bit like this where HORIZONTAL_LINES and VERTICAL_LINES are related to the grid sizing.</p>
<pre><code class="lang-auto">model.train()
    # For grid in traning set
    for i, (inputs, targets) in enumerate(train_loader):
        # Add channel
        x = inputs.unsqueeze(2)
        y = targets.view(1,1,1,HORIZONTAL_LINES+1,VERTICAL_LINES+1)

        # Forward pass to get output/logits
        y_hat = model.forward(x, future_seq=1)
        
        loss = criterion(y,y_hat)
        
        # Clear gradients w.r.t. parameters
        optimizer.zero_grad()
        
        # Getting gradients w.r.t. parameters
        loss.backward()

        # Updating parameters
        optimizer.step()

        # Update loss
        epoch_training_loss += loss.cpu().detach().numpy()
</code></pre>
<p>We were considering if our problem is because we have a very sparse dataset (average cell value is around 1) and with 0 a lot of the time, or the fact that our model predicts in the continuous range instead of discrete values.</p>
<p>The architecture of the model is based on the following model, but with slight such as nn.ReLU as the activation fuction.</p><aside class="onebox githubblob" data-onebox-src="https://github.com/holmdk/Video-Prediction-using-PyTorch/blob/master/models/seq2seq_ConvLSTM.py">
  <header class="source">

      <a href="https://github.com/holmdk/Video-Prediction-using-PyTorch/blob/master/models/seq2seq_ConvLSTM.py" target="_blank" rel="noopener nofollow ugc">github.com</a>
  </header>

  <article class="onebox-body">
    <h4><a href="https://github.com/holmdk/Video-Prediction-using-PyTorch/blob/master/models/seq2seq_ConvLSTM.py" target="_blank" rel="noopener nofollow ugc">holmdk/Video-Prediction-using-PyTorch/blob/master/models/seq2seq_ConvLSTM.py</a></h4>


      <pre><code class="lang-py">import torch
import torch.nn as nn

from models.ConvLSTMCell import ConvLSTMCell

class EncoderDecoderConvLSTM(nn.Module):
    def __init__(self, nf, in_chan):
        super(EncoderDecoderConvLSTM, self).__init__()

        """ ARCHITECTURE 

        # Encoder (ConvLSTM)
        # Encoder Vector (final hidden state of encoder)
        # Decoder (ConvLSTM) - takes Encoder Vector as input
        # Decoder (3D CNN) - produces regression predictions for our model

        """
        self.encoder_1_convlstm = ConvLSTMCell(input_dim=in_chan,
                                               hidden_dim=nf,
                                               kernel_size=(3, 3),
</code></pre>



  This file has been truncated. <a href="https://github.com/holmdk/Video-Prediction-using-PyTorch/blob/master/models/seq2seq_ConvLSTM.py" target="_blank" rel="noopener nofollow ugc">show original</a>

  </article>

  <div class="onebox-metadata">
    
    
  </div>

  <div style="clear: both"></div>
</aside>

<p>Any help would be much appreciated</p>

googlebot: <p>Your y,y_hat order is reversed from the correct one.</p>