LSTM sequence to label

Linkan: <p>I’m trying to do occupancy detection with LSTM based on temperature and humidity data as the image shows.<br>
<div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/f/d/fd79117d6b7b8680b297ab744e73c2da37c6ccd0.png" data-download-href="https://discuss.pytorch.org/uploads/default/fd79117d6b7b8680b297ab744e73c2da37c6ccd0" title="fake prediction"><img src="https://discuss.pytorch.org/uploads/default/optimized/3X/f/d/fd79117d6b7b8680b297ab744e73c2da37c6ccd0_2_389x500.png" alt="fake prediction" data-base62-sha1="AakbDwRGicCYJ0KIp04fm2KhbiM" width="389" height="500" srcset="https://discuss.pytorch.org/uploads/default/optimized/3X/f/d/fd79117d6b7b8680b297ab744e73c2da37c6ccd0_2_389x500.png, https://discuss.pytorch.org/uploads/default/optimized/3X/f/d/fd79117d6b7b8680b297ab744e73c2da37c6ccd0_2_583x750.png 1.5x, https://discuss.pytorch.org/uploads/default/optimized/3X/f/d/fd79117d6b7b8680b297ab744e73c2da37c6ccd0_2_778x1000.png 2x" data-dominant-color="EEEDED"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">fake prediction</span><span class="informations">1112×1426 72.8 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>
<p>My problem is that I’m getting around 50% accuracy on both of my training and validation dataset under the training. I have tried with different hyperparameters and normalized the dataset etc but I think that the problem is that I don’t really know if my model is implemented correctly. One batch is for example [1, 400, 2] (batch, timesteps, features).</p>
<pre><code class="lang-auto">class RNN(nn.Module):
  def __init__(self, input_size, hidden_layer_size, num_classes):
    super(RNN, self).__init__()
    self.hidden_layer_size = hidden_layer_size
    self.lstm = nn.LSTM(input_size, hidden_layer_size, batch_first=True)                                          
    self.fc = nn.Linear(hidden_layer_size, num_classes)
 
  def forward(self, input_seq):
    h0 = torch.zeros(1, input_seq.size(0), self.hidden_layer_size).to(device)
    c0 = torch.zeros(1, input_seq.size(0), self.hidden_layer_size).to(device)
 
    out, (h_out, _) = self.lstm(input_seq, (h0,c0)) 
    h_out = h_out.view(-1, self.hidden_layer_size)
    out = self.fc(h_out)
    
    return out
</code></pre>

vdw: <p>Do I understand correctly that your target is three classes, i.e., occupancy levels: 0, 1, and 2? Because this would have been my initial suggestion after reading the first sentence.</p>
<p>In general, though, this seems to be a very tricky task. Temperature and humidity are physical phenomena that generally change rather slowly over time – compared to a light or noise sensor, for example. With, say, a noise sensor, you probably wouldn’t even need a RNN.</p>
<p>Sorry, for not really helping here, but I would be curious if you could solve this. At the moment, I see principle limitations with this approach.</p>

Linkan: <p>Yes, I‘m using 3 target classes (number of occupants 0, 1 or 2) and later under the training I’m using CrossEntropyLoss.<br>
I’m aware of the limitations that you have mentioned but I want to test if it’s a suitable idea for an IoT device.</p>

Linkan: <p>The image shows the model predictions for the test dataset <img src="https://discuss.pytorch.org/images/emoji/apple/sweat_smile.png?v=12" title=":sweat_smile:" class="emoji" alt=":sweat_smile:" loading="lazy" width="20" height="20"></p>
<p><img src="https://discuss.pytorch.org/uploads/default/original/3X/b/3/b3e9bc1948cb0c67353bc131aed03ffe139a3c18.png" alt="LSTM preds" data-base62-sha1="pFAeYOcYQdfC5WRWZLPmONYcVio" width="599" height="424"></p>

vdw: <p><a class="mention" href="/u/linkan">@Linkan</a> Can you overlay the last plot this with the sensor values?</p>
<p>If I understand correctly, a single data sample is a sequence 400 real values as input and 1 target variable (0, 1, or 2), right?</p>

Linkan: <p>This is 15 epochs and the accuracy is around 60% (doesn’t look like that from the image).<br>
<img src="https://discuss.pytorch.org/uploads/default/original/3X/3/3/3357bb2b9b997e7c821e321af5fdf1d7955bb57c.png" alt="dataframe" data-base62-sha1="7kcjtkhBczRpro3L4oLEzl8dRms" width="607" height="424"></p>
<aside class="quote no-group" data-username="vdw" data-post="6" data-topic="106182">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/vdw/48/10074_2.png" class="avatar"> vdw:</div>
<blockquote>
<p>If I understand correctly, a single data sample is a sequence 400 real values as input and 1 target variable (0, 1, or 2), right?</p>
</blockquote>
</aside>
<p>Yes, correct.</p>
<p>I found this function that I used to create the look back sequence for each sample.</p>
<pre><code class="lang-auto">def TimeSteps(X_data, Y_data, seq_length):
    x = []
    y = []

    for i in range(len(X_data)-seq_length-1):
        _x = X_data[i:(i+seq_length)]
        _y = Y_data[i+seq_length]
        x.append(_x)
        y.append(_y)

    return np.array(x),np.array(y)
</code></pre>