## 4 Data Modeling
### Text Mining
#### 4.1.2 Text Topic Modeling  
To understand the types of information included in the pre-bid RFI documents, text topic modeling, which is frequently used in text mining, was performed. Text topic modeling is based on the idea that documents are mixtures of topics, where a topic is a probability distribution over words [23]. Text topic modeling is a way to identify topics that cover each document by grouping the documents \begin{table}
\begin{tabular}{c c} \hline Number of documents & 243 documents \\ Number of terms (before preprocessing) & 5009 terms \\ Number of terms (after preprocessing) & 3948 terms \\ Minimum word length & 3 characters \\ Sparsity & 98\% \\ \hline \end{tabular}
\end{table} 
Table 1: Preprocessing of text data. according to the occurrence probability of words. Text topic modeling is distinguished from the text clustering method since one document can contain several topics. The Latent Dirichlet Allocation (LDA) method was applied as a topic modeling algorithm. LDA is an unsupervised generative probabilistic model for collections of discrete data, such as text corpora. LDA is a three-level hierarchical Bayesian model, in which each item of a collection is modeled as a finite mixture over an underlying set of topics [24]. Assuming that several topics are mixed in each document, the results of learning can be applied to the new document to analyze the topic of the new document. An inference process is required to analyze the topic of given documents through the LDA model. Gibbs sampling, which is easy to provide a relatively efficient method of extracting a set of topics from large corpus [25; 26; 27], was used for the inference in this study. 11 topics were derived from the topic modeling of the pre-bid RFI document using the LDA algorithm, as shown in Figure 4. In topic modeling, trial and error is repeated several times in the process of determining the number of topics. In this study, 11 topics are finally obtained using the harmonic mean method [28] to determine the optimal number of topics. The scalar value of the Dirichlet hyper-parameter for topic proportions, alpha, was set at 0.02 and the number of iterations was 2000. Figure 4 shows a bar-chart of the probability of the topic terms representing 11 topics. Figure 5 shows a visualization of the topic modeling results using LDAvis, one of the R packages. LDAvis gives a global view of the topics (and the way in which they differ from each other), while at the same time allowing for a deep inspection of the terms most strongly associated with each individual topic [29]. The left side of Figure 5 shows the distribution of each topic, and the right side shows a bar chart of the key terms that represent the selected topic (see topic 5 in Figure 5). In particular, the blue bar refers to the global frequency of each word and the red bar refers to the local frequency in the bar chart of the right view. For example, the first word 'asphal' in the bar chart of Figure 5 shows that most of the blue bar is occupied by the red bar, implying that the word is rarely used in other topics except topic 5, and the word can thus be considered as a representative key word of topic 5. The 11 topics that were found as a result of the topic modeling were examined using the LDAvis visualization tool, and the title of each topic could be deduced as follows:  
* Topic 1: Surface roughness requirements
* Topic 2: Disadvantaged Business Enterprise (DBE) work category  
Figure 4: Results of pre-bid Request for Information (RFI) topic modeling. * Topic 3: Guardrail system detail
* Topic 4: Excavation quantity and payment
* Topic 5: Highway planting and irrigation
* Topic 6: Hot Mixed Asphalt (HMA) spec.