# XTTS-Based Data Augmentation for Profanity Keyword Recognition in Low-Resource Speech Scenarios
## 3 Experimental Results and Comparisons of Various Algorithms
### Evaluation Metrics  
To comprehensively assess the classification performance of the model, this study adopts the F1 Score, Area Under the Receiver Operating Characteristic Curve (AUROC), and mean Average Precision (mAP) as the primary evaluation metrics. The metrics are described in the following sections.  
In classification tasks, model predictions can be categorized into four outcomes: True Positive (TP), True Negative (TN), False Positive (FP), and False Negative (FN). Based on these fundamental measures, we calculate Precision, Recall, and F1 Score. Precision quantifies the proportion of samples predicted as positive that are actually positive, while Recall measures the proportion of actual positive samples correctly identified by the model. The formulas for these metrics are provided in (5) and (6).  
\[Precision=\frac{TP}{TP+FP} \tag{5}\]  
\[Recall=TPR=\frac{TP}{TP+FN} \tag{6}\]  
There is typically a trade-off between Precision and Recall. For instance, in medical diagnosis applications, Recall is prioritized, whereas in spam email classification, Precision is of greater concern. To balance these two metrics comprehensively, we employ the F1  
\begin{table}
\begin{tabular}{c c c c c} \hline \hline
**RHS Dataset** & \multicolumn{2}{c}{**RHS vs.**} & \multirow{2}{*}{**Loss**} & \multirow{2}{*}{**Accuracy**} & \multirow{2}{*}{**Improvement**} \\
**Ratio** & & & & \\ \hline \multirow{5}{*}{7:1:2} & 1:7 & 2.253 & 62.05\% & 6.70\% \\ \cline{2-5}  & 1:8 & 2.244 & 63.09\% & 7.74\% \\ \cline{2-5}  & 1:9 & 2.194 & 67.86\% & 12.51\% \\ \cline{2-5}  & 1:10 & 2.192 & 68.15\% & 12.80\% \\ \cline{2-5}  & 1:11 & 2.219 & 65.17\% & 9.82\% \\ \hline \hline \end{tabular}
\end{table}
Table 6: _Cont._  
\begin{table}
\begin{tabular}{c c c c c c} \hline \hline \multirow{2}{*}{**TTS Method**} & \multicolumn{5}{c}{**RHS vs. Synthetic Ratio**} \\ \cline{2-5}  & **1:1** & **1:2** & **1:3** & **1:4** & **1:5** \\ \hline OpenVoice [5] & 60.57\% & 61.61\% & 59.82\% & 60.42\% & 59.67\% \\ \hline XTTS [6] & 62.50\% & 66.37\% & 64.88\% & 65.92\% & 66.97\% \\ \hline Fish-Speech [7] & 57.74\% & 55.80\% & 57.29\% & 57.89\% & 56.55\% \\ \hline \hline \end{tabular}
\end{table}
Table 7: Comparison of TTS methods on test set average accuracy for Proposed-I.  
Score, calculated as shown in (7). The F1 Score ranges between 0 and 1, with higher values indicating a better balance between Precision and Recall.  
\[F1=2\times\frac{\textit{Precision}\times\textit{Recall}}{\textit{Precision}+ \textit{Recall}} \tag{7}\]  
AUROC evaluates the model's ability to distinguish between classes, reflecting its effectiveness in classification tasks. The Receiver Operating Characteristic (ROC) curve, which plots the True Positive Rate (TPR) (6) against the False Positive Rate (FPR) (8), is used to compute the area under the curve. The Area Under the ROC Curve (AUROC) is typically calculated using numerical methods, such as the trapezoidal rule, which approximates the area by summing the areas of trapezoids formed between consecutive points \((FPR_{i},TPR_{i})\) and \((FPR_{i+1},TPR_{i+1})\) on the ROC curve. The formula for this approximation is given in (9), where n is the number of points on the ROC curve. An AUROC value closer to 1 indicates superior keyword discrimination capability, while a value near 0.5 suggests that the model's discriminative performance is comparable to random guessing.  
\[FPR=\frac{FP}{FP+TN} \tag{8}\]  
\[AUROC\approx\sum_{i=1}^{n}\frac{(FPR_{i+1}-FPR_{i})\cdot(TPR_{i}+TPR_{i+1})}{2} \tag{9}\]  
mAP serves as an evaluation metric in multi-class classification problems, commonly applied to object detection and classification models. It is calculated by integrating Precision across different Recall levels, as shown in (10) and (11). The mAP is then derived by averaging the Average Precision (AP) across all categories \(C\). A higher mAP value indicates better predictive performance across all classes.  
\[AP=\int_{0}^{1}P(R),dR \tag{10}\]  
\[mAP=\frac{1}{C}\sum_{n=1}^{C}AP_{n} \tag{11}\]