## 5 Experiments
### Model Compilation  
In this section, we review the models used in the experiments. For the ML models, the default parameters were adjusted to ensure equality between all tested models (Table 4). On the other hand, DL parameters had to be fine-tuned to achieve robust results. To compare the performance of each model using various feature extraction strategies equally, we ensured that all the models were equitably adjusted. Fine-tuning is performed by adding a fully connected layer on top of the pretrained model. Each model is fine-tuned for the specific task. We unify the essential parameters (Table 5) when creating deep learning models, which are as follows:  
* The batch size is the number of examples to be taken into account prior to updating the model's parameters. Batch sizes of 32, 64, and 128 were examined in this study. * The number of epochs is determined by how frequently the algorithm will execute the training dataset. Here, we experimented with epoch numbers ranging from 1 to 15. To avoid wasting time and storage, we used the stop accuracy method. It allowed us to terminate the training when we reached the highest accuracy to make up for the discrepancy between the loss function and the updating of model parameters. The Adam optimizer was adjusted with a learning rate of 0.001. * Dropout improves the model's generalization and reduces the likelihood of overfitting. To constrain the weight of layers, the dropout rate was modified to 0.2. * The classifier is the final layer that converts all the input into predicted classes. Thus, choosing this layer included Conv1D layer, MaxPooling1D layer, and Dense layer with a sigmoid activation function for binary classification. \begin{table}
\begin{tabular}{l l} \hline \hline
**Model** & **Parameters** \\ \hline SVM & batchSize 100 kernel linear \\ NB & batchSize 100 \\ LR & batchSize 100, maxBoosting-Iterations 500 \\ RF & batchSize 100, bagging with num-Iteraions 100, and number of trees 100 \\ \hline \hline \end{tabular}
\end{table} 
Table 4: Machine learning classifier parameters. \begin{table}
\begin{tabular}{l l} \hline \hline
**Technique** & **Description** \\ \hline N-Grams & Contiguous sequences of n tokens that can be words, characters, or other units extracted from a given text. \\ Textual Features & Refer to various characteristics or properties of text that can be extracted or analyzed to gain insights into the text \\ Word Embeddings & Capture semantic relationships between words by representing each word as a point in the embedding space, where similar words are closer to each other. \\ \hline \hline \end{tabular}
\end{table} 
Table 3: Summary of feature extraction techniques. A detailed description of the compilation of each deep learning and transformer model can be found below. * **CNN**: The architecture consists of several layers. First, the Conv1D layer, which approximately applies 64 filters with a kernel size of 3 to the input data, and ReLU was added for an applied activation function. Second, the MaxPooling1D layer, which was used to perform max pooling with a pool size of two. Third, the Conv1D layer, which applied 64 filters with a kernel size of three to the output of the previous layer. MaxPooling1D layer was then applied to perform max pooling with a pool size of two. Then, the Flatten layer, which flattens the output of the previous layer, was applied. Finally, the dense layer, which has two neurons with a sigmoid activation function, outputs a probability distribution over the output classes. * **Bi-LSTM**: its architecture uses the Keras library. The model has a single bidirectional-LSTM layer with 64 units, followed by a dense layer with a sigmoid activation function that outputs two values for binary classification. The loss function used is categorical cross-entropy, and the optimizer was Adam. The model was trained for two epochs; the input shape was 768.1, which is the size of the BERT embeddings after reshaping.