## 2 Related Work
Thus, LDA had been reported to suffer from the data sparsity problem. Improved extensions of LDA have been proposed such as the correlated topic model [14], the hierarchical Dirichlet process model [43], constrained-LDA [44], MaxEnt-LDA [45], automated knowledge LDA [46], LDADE [12], and ontology-LDA [42]. To overcome the data sparsity problem, [PERSON] et al. [47] proposed the Bi-term Topic Model (BTM) for topic modelling, specifically for short text. BTM learns topics by modelling word pairs in the whole corpus. For example, _\"my sure bank\"_ have biterms such as _\"my sure, sure bank, my bank\"_. This means global occurrences of biterms are captured. BTM was developed with the assumption that two words will be assigned the same topic label if they have co-occurred [48]. However, it is worth mentioning that this study considers that assumption too strong. For example, a tweet such as _\"My atom card could not work with bank X machine, not sure if it's the atm network or card issue\"_. In the tweet, the key words are _\"atm, network, machine, card, issue\"_ and can have equal global co-occurrence. Unfortunately, topic modelling of tweets such as this suggest that the assumption may be too strong and does not apply in all cases. BTM has been shown to outperform traditional topic models such as LDA. For example, Discriminative-BTM [48], Twitter-BTM [17], and GraphBTM [18]. Due to the instability in the performance of the conventional TM approaches, the use of language models is evolving. Bidirectional Encoder Representations from Transformers (BERT) is a deep bidirectional unsupervised language representation model developed by Google [19] and has shown good result in topic extraction. For example, [PERSON] and [PERSON] [49] employed BERT for aspect extraction. They collected Indonesian tourism reviews from TripAdvisor and pretrained BERT with 4220 review sentences (using train batch size 32, max sequence length 128, number of epoch 16 learning rate 3 \(\times\) 10\({}^{-5}\), and Adam epsilon 3 \(\times\) 10\({}^{-8}\)). Their approach used 501 Indonesian amusement park reviews for testing and reported an accuracy of 79.9% and F1 score of 73.8%. [PERSON] and [PERSON] [50] compared variants of BERT against bidirectional long short-term memory (BiLSTM) and conditional random field (CRF) for aspect extraction using 2265 Arabic news posts (retrieved from Facebook about the 2014 Gaza attack). They showed that the combination of BERT, bidirectional gated recurrent unit (BiGRU), and CRF achieved the highest performance for aspect term extraction with an F1 score (88%). However, despite the good performances shown by BERT, [PERSON] et al. [51] stated BERT was undertrained and, thus, proposed RoBERTa, a robustly optimized BERT pretraining approach for topic extraction. They showed RoBERTa performed better than BERT on GLUE and SQuAD. [PERSON] et al. [52] inserted topic layers into fine-tuned RoBERTa and, thus, proposed a topic augmented language model for topic extraction. [PERSON] [53] presented BERTopic by leveraging clustering techniques and a class-based variation of TF-IDF to generate coherent topic representations. The author created document embeddings using a pretrained transformer-based language model to obtain document-level information. Thereafter, they applied dimension reduction in document embeddings and created semantically similar document clusters. Finally, the class-based version of TF-IDF was used to extract the topic representation from each topic. [PERSON] [53] showed the approach produced coherent topics in three different datasets namely, 20 Newsgroups (16,309 news articles across 20 categories), BBC News (contains 2225 documents between 2004 and 2005), and [PERSON]'s tweets. [PERSON] and [PERSON] [54] compared BERTopic, LDA, and Non-Negative Matrix Factorization (NMF) in three different Arabic newspapers, namely, Assabah, Hespress, and Akhbarona. They showed BERTopic outperformed other models. [PERSON] et al.