# Sequential Movie Genre Prediction Using Average Transition Probability with Clustering
## 3 Genre Prediction Algorithm
### Model Training
#### 3.4.2 Training Models
The cell input activation vector \(\tilde{c}_{t}\) is computed by: \[\tilde{c}_{t}=\tanh(W_{c}\cdot[h_{t-1},x_{t}]+b_{c}),\] (10) where \(W_{c}\) and \(b_{c}\) are weight matrix and bias vector parameter and tanh\((\cdot)\) is a hyperbolic tangential function as a sigmoid function. (iii) _Cell State Update._ Next, LSTM performs a cell state update procedure to update the old cell state, \(c_{t-1}\), into the new cell state \(c_{t}\). The previous steps already decided what to do; it just needs to actually do it. Then, it multiplies the old state to \(f_{t}\), forgetting the things it decided to forget earlier. Then, it adds \(i_{t}\cdot\tilde{c}_{t}\). This is the new candidate value, scaled by how much it decided to update each state value. The update of cell state \(t\) is computed by: \[c_{t}=f_{t}*c_{t-1}+i_{t}*\tilde{c}_{t}.\] (11) (iv) _Output Gate Layer._ Finally, in the output gate layer, LSTM decides what information going to be output. This output will be based on the cell state, but it will be a filtered version. First, it runs a sigmoid layer that decides what parts of the cell state going to output. Then, it puts the cell state through tanh and multiplies it by the output of the sigmoid gate, so that it only output the parts it decided to output. The output gate vector \(o_{t}\) is given by: \[o_{t}=\sigma(W_{o}\cdot[h_{t-1},x_{t}]+b_{o}),\] (12)  
\[h_{t}=\tanh(Ux_{t}+Wh_{t-1}+b), \tag{6}\]0 Long Short Term Memory [32]. where \(W_{o}\) and \(b_{o}\) are weight matrix and bias vector parameter of the output gate layer. Here, \(h_{t}\) is computed by:  
\[h_{t}=\tanh(Ux_{t}+Wh_{t-1}+b), \tag{6}\]1  
**(3) GRU. ** [PERSON] et al. [33] first introduced a slight variation on the LSTM named GRU. It uses the forget and input gates as a single update gate. Furthermore, it also combines the cell state and hidden state. It is known that the GRU is simpler than the LSTM model. The detailed structure of GRU as shown in Figure 9 is in what follows:  
**(i)**: _Update Gate._ In GRU, it first begins with computing the update gate \(z_{t}\) for time step \(t\) by:  
\[h_{t}=\tanh(Ux_{t}+Wh_{t-1}+b), \tag{6}\]2  
where \(W_{z}\) is a weight matrix. When the input \(x_{t}\) is generated into the network, it is multiplied by its own weight \(W_{z}\). The previous \(h_{t-1}\) also multiplied by the current input \(x_{t}\). As an activation function, a sigmoid is commonly used. The update gate is used to determine how much of the past information (from previous time steps) needs to be passed along to the next. The most useful fact is that the model can control to copy all the information from the past and eliminate the risk of vanishing gradient problems. (ii)**: _Reset Gate._ Next, a reset gate is applied from the model to decide how much of the past information to forget by:  
\[h_{t}=\tanh(Ux_{t}+Wh_{t-1}+b), \tag{6}\]3  
The difference between this from the update gate is the weights and the gate's usage. Like similar steps in the update gate, it plugs in \(h_{t-1}\) and \(x_{t}\), multiplies them with their corresponding weights, sums the results and applies the sigmoid function. (iii)**: _Current memory content._ The current memory content is then used for the reset gate to store the relevant information from the past. It is computed as:  
\[h_{t}=\tanh(Ux_{t}+Wh_{t-1}+b), \tag{6}\]4  
where \(W\) is a weight matrix and the operator \(*\) denotes the Hadamard elementwise product. Then, the result determines what to remove from the previous time steps. In this step, it uses tanh as the nonlinear activation function. (iv)**: _Final memory at current time step._ As the last step, the network needs to calculate \(h_{t}\), which is a vector that holds information for the current unit and passes it down to the network. In order to do that, the update gate is needed. It determines what to collect  
\[h_{t}=\tanh(Ux_{t}+Wh_{t-1}+b), \tag{6}\]5 Gated Recurrent Unit [34].