## 4 Extraction Module
### Machine Learning-Based Classification  
In contrast to rule-based classification, the alternate classification approach used in the present study was supervised ML models, with the learning process driven by previous knowledge of the data [28]. In ML algorithms, a general inductive process automatically builds a classification model for each class by observing the characteristics of a set of manually classified statements. The ML-based text classification approach is summarized in Figure 7. \begin{table}
\begin{tabular}{c c c c c c c c} \hline \hline  & & \multicolumn{4}{c}{**Scenario 1:**} & \multicolumn{4}{c}{**Scenario 2:**} \\  & & \multicolumn{4}{c}{**without Lemmatization**} & \multicolumn{4}{c}{**with Lemmatization**} \\  & & \multicolumn{4}{c}{**Stop-Words Retained**} & \multicolumn{4}{c}{**Stop-Words Removed**} \\ \hline
**N-Gram** & **Class** & **Precision** & **Recall** & **F1-Score** & **Precision** & **Recall** & **F1-Score** \\
**Set** & **Label** & **(\%)** & **(\%)** & **(\%)** & **(\%)** & **(\%)** & **(\%)** & **(\%)** \\ \hline \multirow{2}{*}{Uni} & True & 91 & 56 & 69 & 93 & 55 & 69 \\  & False & 98 & 100 & 99 & 98 & 100 & 99 \\ \hline \multirow{2}{*}{Bi} & True & 100 & 86 & 92 & 96 & 88 & 92 \\  & False & 99 & 100 & 100 & 99 & 100 & 100 \\ \hline \multirow{2}{*}{Tri} & True & 99 & 86 & 92 & 98 & 71 & 83 \\  & False & 99 & 100 & 100 & 98 & 100 & 99 \\ \hline \multirow{2}{*}{Quad} & True & 100 & 74 & 85 & 100 & 49 & 49 \\  & False & 99 & 100 & 99 & 97 & 100 & 99 \\ \hline \hline \end{tabular}
\end{table} 
Table 2: Effect of n-gram sets and data pre-processing on performance of rule-based classification for two experimental scenarios. To ensure compatibility with computer processors, words were first converted into a numeric format using feature engineering. Here, raw text data were transformed into feature vectors, and new features were created using the dataset. Different methods were used to create relevant dataset features prior to input into the text classification algorithm [37]. Two approaches for constructing representation vectors, namely count vectors and term frequency-inverse document frequency (TF-IDF) vectors, were implemented. Count vectors are a matrix representation of the dataset, where every row represents a statement, every column represents a word, and every cell represents the frequency count of a particular word (i.e., either zero or a real number) in a particular statement [38]. Words that appear in many textual statements are considered less meaningful and, therefore, each vector component (i.e., a word) can be weighed based on the number of statements in which the word appears. Another approach for constructing representation vectors is TF-IDF, which is a technique designed to identify important terms in a dataset by weighing a term's frequency (TF) together with its inverse document frequency (IDF), which weighs down high-frequency domain-specific terms while scaling up rare terms [38]. In TF-IDF vectors, terms can be extended to include characters and n-gram-level models, such as uni-gram (i.e., words), bi-grams (i.e., pairs of words), as well as tri and quad-grams (i.e., phrases). The TF-IDF of terms are calculated using Equation (5) [38],  
\[\text{TF}-\text{IDF}=\frac{tf_{i}}{T}\times\left(1+log\bigg{(}\frac{N}{N_{i}} \bigg{)}\right) \tag{5}\]  
where \(tf_{i}\) is the frequency of term \(i\) in the statement, \(T\) is the total number of words in the statement, \(N\) is the total number of statements, and \(N_{i}\) is the number of statements containing term \(i\). For the proposed method to be feasible in practice, retraining and prediction [39] must be completed within a relatively short period of time. As such, models that required longer than an hour to be fine-tuned and retrained (e.g., deep learning algorithms) were excluded from this study to ensure applicability of this research.