# Data Augmentation Method Using Room Transfer Function for Monitoring of Domestic Activities
## 4 Simulation
### Evaluation Results
#### 4.2.2 Comparison of Performance with Unknown Microphones  
In the previous section, we observed that our algorithm enhanced model performance similarly to conventional data augmentation techniques. In this section, we examine the performance of microphones in both known and unknown locations within the evaluation dataset to determine if the model has been robustly trained with respect to location, as intended.  
From the results in Table 5, it is evident that for known microphones in the evaluation dataset, the highest performance was achieved in terms of micro and weighted averages when specaugment [14] and the proposed method were used together. For unknown microphones, the highest performance was achieved in the micro and weighted averages when mixup [27] and the proposed method were combined.  
For known microphones, using specaugment [14] together with the proposed method was effective. However, for unknown microphones, combining mixup [27] with the proposed method resulted in enhanced performance improvements. This result indicated that depending on the situation, it could be more effective to use our algorithm in conjunction with conventional techniques.  
It was observed that using mixup alone resulted in the highest performance improvement in terms of the macro average. Since macro averaging entails F1-Score calculation by applying equal weight to each class, it is more sensitive to the performance of classes with fewer samples. As shown in Figure 1, classes with fewer samples and relatively lower baseline performances, such as other, dishwashing, and eating, may have experienced greater improvements with mixup. This could be due to mixup's tendency to enhance performance in ambiguous or overlapping classes. Specifically, the dishwashing and eating classes, both of which contain dish sounds, appear to have benefited from this effect, although this conclusion remains speculative based on the macro average results.  
Although, while mixup showed improvements in specific cases, the proposed method outperforms mixup when considering the overall performance across multiple metrics. The proposed method consistently achieves higher F1-Score in both the micro and weighted averages regardless of whether mixup is used alone or in combination with other techniques, demonstrating greater robustness and effectiveness across a broader range of classes. Thus, although mixup offers localized benefits in certain scenarios, our approach is more advantageous for achieving comprehensive performance improvements.  
\begin{table}
\begin{tabular}{l c c c c c c c c} \hline \hline  & \multicolumn{2}{c}{**Evaluation Dataset**} & \multicolumn{2}{c}{**Known Mic**} & \multicolumn{2}{c}{**Unknown Mic**} & \\ \hline
**F1-Score** & **Micro** & **Macro** & **Weighted** & **Micro** & **Macro** & **Weighted** & **Micro** & **Macro** & **Weighted** \\ \hline Without augmentation & 0.8861 & 0.8397 & 0.8848 & 0.9072 & 0.8541 & 0.9053 & 0.8581 & 0.8201 & 0.8573 \\ \hline mix\&shuffle [13] & 0.8874 & 0.8417 & 0.8873 & 0.9091 & 0.8562 & 0.9083 & 0.8585 & 0.8223 & 0.8591 \\ \hline mixup [27] & 0.8905 & **0.8460** & 0.8890 & 0.9116 & **0.8610** & 0.9095 & 0.8625 & **0.8257** & 0.8616 \\ \hline Specaugment [14] & 0.8897 & 0.8445 & 0.8887 & 0.9110 & 0.8597 & 0.9095 & 0.8614 & 0.8240 & 0.8608 \\ \hline RTFs (proposed) & **0.8939** & 0.8433 & **0.8922** & **0.9143** & 0.8577 & **0.9119** & **0.8668** & **0.8241** & **0.8658** \\ \hline mix\&shuffle \& RTFs & 0.8854 & 0.8397 & 0.886 & 0.9106 & **0.8629** & 0.9096 & 0.8517 & 0.8110 & 0.8546 \\ \hline mixup \& RTFs & **0.8935** & **0.8445** & **0.8917** & **0.9133** & 0.8606 & **0.9110** & **0.8672** & 0.8227 & **0.8659** \\ \hline Specaugment \& RTFs & **0.8920** & **0.8476** & **0.891** & **0.9138** & **0.8641** & **0.9124** & **0.8629** & **0.8254** & **0.8625** \\ \hline \hline \end{tabular}
\end{table}
Table 5: F1-Score comparison of known and unknown mics. The three highest scores are bold, and the highest scores are underlined.