## 4 Materials and Methods
### Modeling and Evaluation
#### 4.4.2 Fine-Tuning the BERT Model for Classifying Infringing Sites
* _Categorical Cross Entropy loss function (\(\mathcal{L}\))_: for multi-class classification, where the output layer \(O\) employs a Sigmoid function, the loss can be determined by assessing the difference between the target classes \(y_{V}\) and the predicted classes \(\hat{y}_{V}\) from the validation set \(X_{V_{\text{concat}}}\), during the training step. The goal is to minimize this discrepancy by using the divergence between the probability distributions of each class \(C=5\), as shown in Equation (7). \[\mathcal{L}(y_{V},\hat{y}_{V})=-\sum_{i=1}^{C}y_{V_{i}}\text{log}(\hat{y}_{V_{ i}}) \tag{7}\]  
The formulation of the optimization problem is presented as follows: Let \(\mathcal{P}=\{\mathcal{E},B,\eta,H_{M}\}\) be the set of hyperparameters and \(f(\mathcal{P})\) be the function aimed at maximizing accuracy. \(f(\mathcal{P})\) is modeled through a Gaussian process that calculates the mean of (\(\mu(\mathcal{P})\)) and its covariance function (\(k(\mathcal{P},\mathcal{P}^{{}^{\prime}})\)), where \(f(\mathcal{P})\leftarrow[\mu(\mathcal{P}),k(\mathcal{P},\mathcal{P}^{{}^{ \prime}})]\). Subsequently, \(f(\mathcal{P})\) is subjected to \(EI\), as detailed in Equation (8). \[EI(\mathcal{P})=\mathbb{E}[max(f(\mathcal{P})-f(\mathcal{P}^{+}))], \tag{8}\]  
where \(\mathbb{E}\) represents the expected value of the outcome of the set exhibiting the highest accuracy, calculated from the difference between the current set \(f(\mathcal{P})\) and the one that achieved the best value \(f(\mathcal{P}^{+})\). The training process will continue until all hyperparameters \(\mathcal{P}\) next have been optimized to maximize the \(EI(\mathcal{P})\) function, with validation conducted using the tuple \(XV_{\text{concat}},y_{V}\). Consequently, the \(\mathit{BestModel}(\mathit{BERT}+DNN)\) will be acquired and subsequently employed to evaluate the final performance with \(\hat{y}_{P_{\text{concat}}}=\mathit{BestModel}(\mathit{BERT}+DNN)\leftarrow(X_{P_ {\text{concat}}},y_{P_{\text{concat}}})\). In the context of multi-label classification, Figure 10 displays a confusion matrix that evaluates the predictive performance of the resultant model \(\mathit{BestModel}(\mathit{BERT}+DNN)\) across the selected classes: non-infringing (\(NI\)), movies and series (\(MS\)), music (\(M\)), software (\(S\)), and books (\(B\)). The matrix reveals two significant values: \(TP_{k}\) (true positives), indicating the number of correct predictions for the \(C\)-th class compared to the ground truth, as well as \(FP_{k}\) (false positives). The multi-class performance metrics used to evaluate the BERT + DNN model are described in Table 7. \begin{table}
\begin{tabular}{p{113.8 pt} p{113.8 pt} p{113.8 pt}} \hline \hline
**Metric** & **Description** & **Mathematical Definition** \\ \hline Precision & It is the fraction of relevant predicted observations for each class with the total number of classified instances with respect to the others. & \(\mathit{Precision}_{i=1}^{k}=\frac{TP_{i=1}}{TP_{i=1}+\sum_{i=1}^{k}FP_{i}}\) \\ \hline Recall & It is the fraction of relevant samples (true positives) predicted for each class concerning the total number of relevant instances. & \(\mathit{Recall}_{i=1}^{k}=\frac{TP_{i=1}}{\sum_{i=1}^{k}TP_{i}}\) \\ \hline \hline \end{tabular}
\begin{tabular}{p{113.8 pt} p{113.8 pt}} \hline \hline \(F_{1}\)-score & The \(F_{1}\)-score represents the harmonic mean of precision and recall, offering a balanced measure of a model’s performance by assessing its equilibrium between \(TP_{i=1}^{k}\) and \(FP_{i=1}^{k}\). & \(F_{i=1}^{k}=2\cdot\frac{\mathit{Precision}-\mathit{Recall}}{\mathit{ Precision}+\mathit{Recall}}\) \\ \hline \hline \end{tabular}
\end{table} 
Table 7: Multi-class performance metrics used to evaluate the DNN’s efficacy, where \(i=1\) denotes the index ranging from the first to the \(k\)-th class.