RNN for time-series anomaly detection

artrujillo: <p>Hello, I am trying to create an RNN that will be able to detect anomalies in time-series data. In particular, looking for glitches in voltage/time plots. I currently am trying to implement a very simple version of this to just make sure that it is doable, but I continue to run into issues when trying to create and train the model.</p>
<p>Unlike other anomaly detection rnn’s that I have come across, I am particularly interested in not just being able to predict if a plot has an anomaly in it, but the time stamps that correlate to the anomaly. So, for each datapoint per plot in my training data, there is a corresponding label of 0 if there is no glitch at the current point, and a 1 if there is a glitch occuring.</p>
<p>Below is the code for my model:</p>
<pre><code class="lang-auto">class MorphModel(Module):
     def __init__(self, n_layers):
       # Define LSTM layer
       self.lstm = nn.LSTM(2, 3 * 2 * 4, n_layers, batch_first=True)
       # Define Linear layer
       self.l_s = nn.Linear(3 * 2 * 4, 1)
       # Define Sigmoid function (Output Layer)
       self.sigmoid = nn.Sigmoid() # TODO look into best way of initializing this
      
     def forward(self, input):
       print("forward inputs: ", list(input[0].size())[0])
       print("lstm in")
       # lstm input: sequence length, batch size, 1
       lstm_out, _ = self.lstm(torch.Tensor(list(input[0].size())[0],2,2)) # we want the hidden outputs (h_n)
       print("lstm output")
       # do some manipulation to be able to feed into linear?
       print("lstm out: ", lstm_out.size())
       lstm_out.reshape(-1, 2 * 24)
       print("lstm out after size: ", lstm_out.size())
       sig_input = self.l_s(lstm_out.view(13,2,-1))
       print("linear layer out")
       print(sig_input.size())
       # will be batch size * len of input
       # may need two outputs
       output = self.sigmoid(sig_input)
       print("sigmout out")
       print(output.size())
       return output
</code></pre>
<p>I set up my learner as such:</p>
<pre><code class="lang-auto">learn = Learner(dls, MorphModel(2), loss_func=CrossEntropyLossFlat())
                #metrics=accuracy, cbs=ModelResetter)
learn.summary()
</code></pre>
<p>And, the summary shows:</p>
<pre><code class="lang-auto">forward inputs:  13
lstm in
lstm output
lstm out:  torch.Size([13, 2, 24])
lstm out after size:  torch.Size([13, 2, 24])
linear layer out
torch.Size([13, 2, 1])
sigmout out
torch.Size([13, 2, 1])
MorphModel (Input shape: 2)
============================================================================
Layer (type)         Output Shape         Param #    Trainable 
============================================================================
                     []                  
LSTM                                                           
____________________________________________________________________________
                     2 x 2 x 1           
Linear                                    25         True      
Sigmoid                                                        
____________________________________________________________________________

Total params: 25
Total trainable params: 25
Total non-trainable params: 0

Optimizer used: &lt;function Adam at 0x7fa0f2e31710&gt;
Loss function: FlattenedLoss of CrossEntropyLoss()

Callbacks:
  - TrainEvalCallback
  - Recorder
  - ProgressCallback
</code></pre>
<p>However, when I run <code>fine_tune()</code> on my learner, I get the following output:</p>
<pre><code class="lang-auto">forward inputs:  13
lstm in
lstm output
lstm out:  torch.Size([13, 2, 24])
lstm out after size:  torch.Size([13, 2, 24])
linear layer out
torch.Size([13, 2, 1])
sigmout out
torch.Size([13, 2, 1])
---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
&lt;ipython-input-12-ca61b3aa75fc&gt; in &lt;module&gt;()
----&gt; 1 learn.fine_tune(1)

21 frames
/usr/local/lib/python3.7/dist-packages/torch/nn/functional.py in nll_loss(input, target, weight, size_average, ignore_index, reduce, reduction)
   2260     if input.size(0) != target.size(0):
   2261         raise ValueError('Expected input batch_size ({}) to match target batch_size ({}).'
-&gt; 2262                          .format(input.size(0), target.size(0)))
   2263     if dim == 2:
   2264         ret = torch._C._nn.nll_loss(input, target, weight, _Reduction.get_enum(reduction), ignore_index)

ValueError: Expected input batch_size (26) to match target batch_size (4).
</code></pre>
<p>For my “proof-of-concept” data, each consists of 13 datapoints, which is why there are 13 forward inputs.</p>
<p>However, I really am not sure how to fix this ValueError. I also am not too confident in this approach as RNN’s are a very new concept to me. So, any advice is incredibly helpful. thanks!</p>