## 4 Methodology
### Sentiment Analysis
#### 4.2.3 Recurrent Neural Network (RNN)  
Model  
We have used tools provided by Keras and Tensorflow to build the model. We created a sequential model by passing a list of layer instances. A sequential model is appropriate for a plain stack of layers, where each layer has exactly one input tensor and one output tensor. The first layer was the embedding layer, which can be used for neural networks on text data. The embedding layer enables us to convert each word into a fixed length vector of defined size. It requires the input data to be integer-encoded, so that each word is represented by a unique integer. Then, we used bidirectional layer ([[https://keras.io/api/layers/recurrent_layers/bidirectional/](https://keras.io/api/layers/recurrent_layers/bidirectional/)]([https://keras.io/api/layers/recurrent_layers/bidirectional/](https://keras.io/api/layers/recurrent_layers/bidirectional/)) accessed on 1 October 2021), which is a layer wrapper. This wrapper takes a recurrent layer as an argument. It also allows us to specify the merge mode, which is how the forward and backward outputs should be combined before being passed onto the next layer. The default mode is to concatenate, and this is the method often used in studies of bidirectional LSTMs. We used the default mode. We used LSTM layers with the bidirectional layers. The long short-term memory (LSTM) is an RNN 'architecture'; these networks constitute a type of recurrent neural network, and are capable of learning order dependence in sequence prediction problems. Next are the dense and dropout layers. A dense layer is a classic fully connected neural network layer, and each input node is connected to each output node. A dropout layer is similar, except that when the layer is used, the activations are set to zero for some random nodes. This is a way to prevent overfitting. We used a dense layer with'relu' activation, then a dropout layer, and again a dense layer with'sigmoid' activation. Difference between RNN and LSTM  
All RNN has a feedback loop in the recurrent layer. This allows them to maintain information in \"memory\" over time. However, it can be difficult to train standard RNNs to solve problems that require learning long-term temporal dependencies. This is because the gradient of the loss function decays exponentially over time; this is called the disappearing gradient problem. LSTM networks are a type of RNN that use special units in addition to standard units. LSTM units contain a \"memory cell\" that can maintain information in memory for a long time. A set of gates is used to control when information is written into memory, when it is output, and when it is forgotten. This architecture allows them to learn longer-term dependencies. Trained Model Information  
The RNN model was trained based on an IMDB review dataset ([[https://www.tensorflow.org/datasets/catalog/imdb_reviews](https://www.tensorflow.org/datasets/catalog/imdb_reviews)]([https://www.tensorflow.org/datasets/catalog/imdb_reviews](https://www.tensorflow.org/datasets/catalog/imdb_reviews)) accessed on 1 October 2021). The dataset comes from the official tensorflow catalog, which provides 25,000 highly polar reviews for training, and 25,000 for testing. We used the \"subwords8k\" option with a vocab size of 8185. The data consist of labels and texts. In the test and train dataset sections, we used the shuffle method as well. The accuracy of our model was 84.7% on the test dataset. The model is not overfitting, and it is more generalized and can make good predictions for new data. Furthermore, we can mention that the buffer size was 10,000, and the batch size was 64. In the 'compile', the loss argument was \"binary crossentropy\" with the \"Adam\" optimizer. We also save our trained models in '.h5' format. This previously mentioned model was used to analyze further tweets. The 'positive', 'neutral', 'negative' labels were expanded in this case as well, just like in the previous models with'strongly positive and negative' and with 'weakly positive and negative' labels.