## 4 Materials and Methods
### Modeling and Evaluation
#### 4.4.2 Fine-Tuning the BERT Model for Classifying Infringing Sites
At this stage of development, BERT + DNN has been configured to receive samples and to initiate the training process. This phase involves the integration of DNNs [97], classified as a category within DL algorithms, distinguished by their proficiency in a range of activities including synthesis, classification, and outlining of concepts in unstructured data, such as those found on websites. The DNNs have solidified their robustness, particularly in operations related to the analysis of information from the IoE, as a case in point, the detection of Denial of Service (DoS) attacks, the identification of internal/external cybersecurity threats, the discernment of web attacks through false data injection, the neutralization of cyber assaults, the containment of phishing attempts, the discovery of piracy activities, and the prevention of the proliferation of malicious software, to name a few. Various architectures of DNNs exist, distinguished by the quantity of dense hidden layers, the types of activation functions utilized, the optimizers employed to reduce error, and the number of units present in the output layer. Despite the plethora of configurations, the architecture delineated in [98] has shown exceptional efficacy in the detection of cryptojacking patterns within website payloads when juxtaposed with other neural network models like Convolutional Neural Networks (CNNs), RNNs, and LSTMs. The BERT + DNN model starts with 776 units for processing the input and 5 for the output, each output representing a different class (\(y=MS,M,S,B\)). In this method, adjustments are made to several hyperparameters, including the number of epochs (\(\mathcal{E}\)), the learning rate (\(\eta\)), the batch size (\(\mathcal{B}\)), and the number of layers (\(H\)), along with the number of units in each, forming these adjustments into a series of subsets that integrate into the set \(\mathcal{P}\). The process is iterative and seeks to optimize these parameters through a function that maximizes the Expected Improvement (\(EI\)), with the goal of achieving the best possible accuracy, namely the \(Best\,Performance\). The identification of this optimum, \(Best\,Performance\), is carried out by evaluating \(X_{conalty}\) along with \(y_{V}\) using the best subset of \(\mathcal{P}^{+}\) at the conclusion of the training process, resulting in the selection of the most effective model (\(BestModel\)). To find the best set of hyperparameters for BERT + DNN in Table 6, the parameters and ranges used are summarized. It is crucial to highlight that the ReLu activation function is used both in the input layer, \(I\), and in the hidden layers, \(H_{M}\), while the Sigmoid function is employed in the output layer, \(O\). For stable operation, a BERT + DNN requires the establishment of several foundational configurations, detailed as follows:  
* _Batch normalization layer_: this component ensures the stabilization of the dense layer's (\(H_{M}\)) output by normalizing the values, aiming for a standard deviation near one and a mean approaching zero. * _ADAM (Adaptive Moment Estimation)_-type optimizer: this mechanism efficiently manages the adjustment of each unit's weights, utilizing a learning coefficient in conjunc \begin{table}
\begin{tabular}{c c} \hline \hline
**Parameter** & **Range** \\ \hline \(\mathcal{E}\) & \(\{75,100,150\}\) \\ \hline \(\mathcal{B}\) & \(\{16,32,64,128\}\) \\ \hline \(\eta\) & \(\{1\times 10^{-6},1\times 10^{-5},1\times 10^{-4},1\times 10^{-3},1\times 10^{-2},1 \times 10^{-1}\}\) \\ \hline \(I\) (Input Layer) & \(776\) \\ \hline \(H_{M}\) (Hidden Layer) & \(M=\{1,2,3,4,5\}\); \(U=\{32,64,128,256,512\}\) \\ \hline \(\mathcal{O}\) (Output) & 5 \\ \hline \hline \end{tabular}
\end{table} 
Table 6: Summary of the proposed parameters for BERT + DNN, detailing the ranges used for its validation. tion with stochastic gradient descent. It anticipates the estimation of first- and second-order moments, maintaining low computational complexity throughout the process.