## 3 The Proposed IDS Framework
In the first step, \(h(t)\) is calculated based on the previously hidden state and the input at the current step: \[h(t)=f(Ux(t)+Wh(t-1))\] (1) where \(f\) is a nonlinear function such as \(tanh\) or \(ReLU\). \(h(t-1)\) is required to calculate the first hidden state which is typically initialized to all zeroes. In the second step, \(y(t)\) is the output at step \(t\) is calculated following the formula: \[y(t)=f(Vh(t))\] (2) In RNNs, \(h_{j}(t)\) and \(y_{e}(t)\) are calculated for recurrent networks as follows: \[h_{j}(t)=f(\Sigma_{i}^{m}x_{i}(t)u_{ij}+\Sigma_{s}^{n}h_{s}(t-1)w_{sj})\] (3) \[y_{e}(t)=f(\Sigma_{j=1}^{k}h_{j}(t)v_{je})\] (4) RNN training is similar to train a traditional neural network. This model also used the backpropagation algorithm but with a slight difference. Because the parameters are shared by all time steps in the network, the gradient at each output depends not only on the calculations of the current time steps but also the previous time steps. This is called backpropagation through time (BPTT) [31]. Unfortunately, vanilla RNNs trained with BPTT have difficulty learning long-term dependencies because of the so-called vanishing/exploding gradient problem [32]. There are some machines that deal with these problems, and certain types of RNNs (like LSTMs, GRUs) were specifically created to get around them. * LSTM model LSTM is proposed by [PERSON] and [PERSON] [33]. This model is more capable to learn long-term dependences model than traditional RNN model. Thus the network overcomes the vanishing gradient problem. Furthermore, this network was designed to be better at storing and accessing information compared to standard RNNs. The memory cells replaced the hidden notes in traditional RNNs. A memory cell includes three gates: an input gate \(i_{t}\), a forget gate \(f_{t}\)  
Figure 3: RNN architecture by time. and an output gate \(o_{t}\). The memory cell also are known as a cell state \(c_{t}\). The incoming signals can alter the cell state or block it by the input gate. The cell state has an effect on other neurons or prevents it from doing so by the output gate. Modulate the cell state of the memory cell can allow the cell to remember or forget its previous state as need by the forget gate. In LSTM networks, the hidden units are replaced by LSTM cells. Figure 4 shows the architecture of this model which has two input units, three LSTM cells as hidden units, and three output units. Assumption that \(X_{t}=[x_{t}^{1},x_{t}^{2},\ldots,x_{t}^{n_{x}}]\) is an input vector. \(H_{t}=[h_{t}^{1},h_{t}^{2},\ldots,h_{t}^{n_{t}}]\) is hidden vector. \(Y_{t}=[y_{t}^{1},y_{t}^{2},\ldots,y_{t}^{n_{y}}]\) is output vector. \(C_{t}=[c_{t}^{1},c_{t}^{2},\ldots,c_{t}^{n_{t}}]\) is cell vector. The elements of each vector are corresponding units for each layer of the LSTM model. \(n_{x}\), \(n_{h}\), \(n_{c}\), and \(n_{y}\) are the numbers of input, hidden, cell, and output units, respectively. \(\sigma\) is the _sigmoid_ function. \(W_{t}^{xi},W_{t}^{xf},W_{t}^{xc}\), and \(W_{t}^{xo}\) are weight metrics that connect from the input node to the input gate, forget gate, cell state, and output gate, respectively. \(W_{t}^{hi}\) is a weight matrix that connects from the hidden node (LSTM cell) to the input gate. \(W_{t}^{hf}\) is a weight matrix that connects from the LSTM cell to the hidden gate. \(W_{t}^{hc}\) is a weight matrix that connects from the LSTM cell to the cell gate. \(W_{t}^{ho}\) is a weight matrix that connects from the LSTM cell to the output gate. \(W_{t}^{co}\) is a weight matrix that connects from the cell state to the output node. \(W_{t}^{hy}\) is a weight matrix from the LSTM cell to the output node. There are several steps to calculate values of each layers. The first step, the decision is what information is to be thrown away from the cell state by a _sigmoid_ (\(\sigma\)) layer called the forget gate.