## 3 Materials and Methods
### Topic Modeling for Identification of Usability Issues in ERP Systems  
Topic modeling is a clustering-based machine learning approach where text analysis is performed without the initial labeling of data. The topic model provides an automated procedure of coding a large chunk of text into a set of significantly meaningful coding categories that are called \"topics\" [27]. At the beginning, the researchers specify the number of topics to the algorithms to find in the provided text. The program is developed to identify the specified number of topics in the input text. The output is the probabilities of words being used in the topic and the distribution of those topics across the corpus of texts. Latent Dirichlet allocation (LDA), first introduced by [PERSON] et al. [27], is based on a generative probabilistic model of a corpus. This method uses the idea of distributing different documents as random mixtures over latent topics, and each topic is categorized by its distribution over words. The output consists of the words list in the groups of topics that provide the context of use [28]. The words with a higher probability provide the title of that topic. [PERSON] et al. [29] compared the methods of semantic analysis, LDA, non-negative matrix factorization, random projection, and principal component analysis on short textual social data to investigate their benefits in detecting important topics. The results showed that the LDA defined the best and clearest meanings of Facebook conversation data compared to the other methods. Moreover, the study compared the outcomes of various topic modeling methods reported in the previous literature and identified the LDA as the most popular and highly studied model in many domains. It is also available in many toolkits such as machine learning toolkits of MALLET ([[https://mimo.github.io/Mallet/](https://mimo.github.io/Mallet/)]([https://mimo.github.io/Mallet/](https://mimo.github.io/Mallet/)), accessed on 23 December 2021), Genism ([[https://radimrehurek.com/gensim/](https://radimrehurek.com/gensim/)]([https://radimrehurek.com/gensim/](https://radimrehurek.com/gensim/)), accessed on 5 February 2022), and Stanford TM toolbox (TMT) ([[https://downloads.cs.stanford.edu/nlp/software/tmt/tmt-0.4/](https://downloads.cs.stanford.edu/nlp/software/tmt/tmt-0.4/)]([https://downloads.cs.stanford.edu/nlp/software/tmt/tmt-0.4/](https://downloads.cs.stanford.edu/nlp/software/tmt/tmt-0.4/)), accessed on 5 February 2022). Therefore, in this paper, we have applied LDA over the collection of the list of problems reported by the ERP system users in the previous research. The reason for applying topic modeling on the collection of ERP usability problem data is to identify the critical and recurrent usability issues in interfaces. Consequently, the developers can use them as a reference and avoid them during the design and development activities. As a first step in applying LDA, it is vital to calculate the optimal number of topics for the input textual dataset. For this purpose, we have calculated the Jaccard similarity and coherence. Firstly, the Jaccard similarity coefficient [30] is applied to measure the similarity and diversity of sample textual input. Jaccard similarity coefficient is  
\[J(A,B)=\frac{(A\cap B)}{(A\cup B)} \tag{1}\]In Equation (1), \(A\) and \(B\) are two sets/topics; the common words in \(A\) and \(B\) are divided by all words in \(A\) and \(B\). Furthermore, the measure of coherence is used to find the degree of semantic similarity between a single topic and high-scoring words in the topic [31]. Figure 2 presents six topics that are optimal for the input dataset of usability issues. The ideal number of topics is based on maximizing coherence and minimizing the topic overlap based on Jaccard similarity.