# Recurrent Embedded Topic Model
## 5 Results
### Discussion  
Pairing the results of the train data set, it showed an improvement of at least 15 times better perplexity values over CTM, Zeroshot-CTM, ETM and LDA, meaning that every time a text-based dataset is needed for data analysis, data organization or any task where no prediction is needed over new data, this model proved to heavily outperform current models. On the other hand, the perplexity calculated in test dataset, although, still better than all models, proved to not be as high as the training dataset, being at least 10% better than the other models.  
In terms of topic coherence, the RETM showed slightly worse results for the books (RETM = 0.1602, ETM = 0.1630), movies (RETM = 0.0789, ETM = 0.0828) and news (RETM = 0.1587, ETM = 0.1816) datasets, but better for the email (RETM = 0.3188, ETM = 0.2364) dataset; however, it still severely outperformed CTM and Zeroshot-CTM, showing a possible hypothesis that since the email dataset has a much lower vocabulary, this could have affected the predictions for the RETM, since \(p\) has less coordinates in \(V\) which could increase the convergence in \(\alpha_{w}\). Unfortunately, there is not sufficient data to prove this hypothesis, and so it will be highly encouraged as future work.  
Outside of previously stated results, it is important to mention that one of the biggest limitations to the RETM is represented by the high computational cost it needs to fit to a dataset. As the recurrence of the model is based on the order of the words per text, coupling this with the embedding layer, if using the 300-dimensional space (as it was the case for these experiments), the amount of V-RAM that the GPU (Nvidia GeForce RTX 3070 Laptop) needs to process the model represents a high cost. To reduce the cost of V-RAM, the batch size can be lowered, but it can penalize the final scores. Once the batch sizes are lowered, it is normal to expect a higher time to compute the model, although it is faster than the traditional LDA calculations on big vocabularies. Also, to reduce computational power, the average amount of words in the dataset was used as the size of the vector for the topic-document distribution calculation, and the door is still open for running the model using the full size vector, given the hypothesis that this can also prove beneficial for the results of the model. Running the current model on a CPU seemed to be a never-ending task. To provide an example, the movie dataset (which contains 30,000 rows worth of text  
Figure 5: Two-dimensional representation of the embedding space where the top 20 words for a topic are shown. Each word is represented by a color and the size of the circle corresponds to the importance of that word at describing the topic.  
data), using a batch size of six, with eight neurons inside the LSTM, cost 5 GB of RAM and 4 GB of VRAM from the GPU and took 28 min to fit the model; in comparison, the ETM, for the same dataset, took 1.5 GB of RAM and 2.3 GB of VRAM and only spent 1 min fitting the model.  
LSTM has opened the door for several applications in the topic modelling area. Given that the aggregation level for the LSTM on this model was based on max this created clear-cut topics and improved the quality in almost all datasets; however, there is still room for experimentation and different uses cases for the current RETM (different aggregations) that can benefit from the current implementation, making a less static model compared to the rest.  
The RETM was shown to be outstanding at topic modelling tasks: any database that uses text-based data can benefit from the implementation of this model, although it may be penalized with small vocabularies. Since the RETM can calculate its own CBOW, it can provide useful topic modelling in domain-specific documents similar to the ETM. In the end, the RETM heavily outperformed previous methodologies and proved to be a state-of-the-art model for topic-modelling problems.