## 4 Experiment and Result
### Embedding and Classification  
We imported the BERT tokenizer and BERT base uncased from Huggingface's transformers library. The original and augmented datasets were then prepared with a BERT tokenizer in the format required by the BERT model. The encoder in the base model comprises 12 layers and 110 million parameters. We used the BERT base uncased model by frozen and unfrozen weights for training with our news corpora. We named our proposed model AugFake-BERT. The suggested AugFake-BERT classifier model was assigned a number of parameter values, which are provided in Table 2, to achieve optimized hyperparameters. For AugFAKE-BERT, we adapted a set of values for diverse parameters. The maximum sequence length was from 128 to 50, the learning rate from \(1\times 10^{-3}\) to \(2\times 10^{-3}\), and the batch size from 5 to 64. The batch size describes how many samples are sent to the algorithm for extraction at a specific iteration. The classifier performance seen in Figure 5(a) is significantly impacted by the batch-wise approach, which boosts GPU performance. For the frozen model, the pre-trained BERT base uncased model's weights were frozen in all the layers except the output layer, which was fine-tuned for news classification. For the unfrozen model, the model was trained from scratch with our news corpora. The data were classified using a feed-forward neural network layer utilizing a softmax activation function. As our optimizer, we chose AdamW, a more effective variation of the Adam optimizer. As more epochs results in overfitting issues and requires a longer training time for the algorithm, the number of epochs was fixed to four. \begin{table}
\begin{tabular}{c c c c} \hline \hline
**Dataset** & **True News** & **Fake News** & **Total** \\ \hline BanFakeNews & 48,678 & 1299 & 49,977 \\ \hline Augmented Dataset & 4000 & 4000 & 8000 \\ \hline \hline \end{tabular}
\end{table} 
Table 1: Description of the datasets. \begin{table}
\begin{tabular}{c c} \hline \hline
**Parameters** & **Parameters Value** \\ \hline Maximum Sequence Length & 128 \\ Epochs & 2 \\ Batch Size & 32 \\ Activation & ReLU, SoftMax \\ Learning Rate & \(1\times 10^{-3}\) \\ Optimizer & AdamW \\ \hline \hline \end{tabular}
\end{table} 
Table 2: Initial hyperparameters adapted for AugFake-BERT. Large batch sizes slow down training by consuming more GPU memory. In other words, GPU memory plays a significant role in the processing of diverse batch sizes. Because of the reasonable accuracy observed, the batch size was set at fifteen for all assessments. A small batch size enhances the outcomes, while a large batch size uses more memory, takes longer to analyze data, and reduces overall accuracy. Dropout is a simple and direct mathematical operation used to protect the classification algorithm from noisy features and overfitting. Because of the small dataset used for training, the classifier may be prone to overfitting. This issue can be fixed by increasing the dataset quantity or reducing the number of hidden layers. This issue was resolved by our enhanced dataset. Dropout removes or disables detached units in the hidden neuron that do not participate in subsequent iterations of the computation. The learning rate determines the best termination time for classification models. A high learning rate causes the model to be terminated quickly, whereas a low learning rate causes the model to be terminated slowly. At a learning rate of \(1\times 10^{-3}\), the accuracy goes up to 85.25%, which is the maximum. The optimized parameters that achieved maximum accuracy are illustrated in Table 3.  
To compute the loss of the AugFake-BERT model, the cross-entropy loss function was used. The loss shows the total number of errors in our model to evaluate how well (or poorly) the model performs.