# Open-Set Recognition of Pansori Rhythm Patterns Based on Audio Segmentation
## 6 Results
### Dataset Acquisition
#### 6.1.2 The Evaluation Performance in Various Encoder Networks  
It is feasible to integrate various backbone networks into the encoder component of our architecture. Table 5 presents the AUROC scores for six convolutional networks, all evaluated under identical hyperparameter settings. Notably, as the complexity of the network increases, there is a corresponding enhancement in the network's capacity to detect the unknown class, culminating in a peak AUROC of 91.8%, achieved by HRNet64. This improvement suggests that more complex networks, with their broader perceptual domains, are capable of capturing a more extensive array of potential information. However, the enhancement in detection capability does not scale linearly with network complexity. This nonlinear improvement is primarily because unknown classes are not represented in the training data, and more complex, deeper networks may increasingly suffer from overfitting on the known classes, thereby diminishing the overall performance. Despite these challenges, our proposed methodology significantly enhances the detection of unknown classes. This improvement ranges from 1 to 10 percent across different networks, attributable to the specialized loss functions implemented. These losses effectively organize the embedded feature space so that intra-class samples are tightly grouped, and inter-class samples are distinctly separated. Consequently, when encountering an unknown class, it tends to be distinctly isolated from all known classes in the feature space due to its \begin{table}
\begin{tabular}{c c c c c} \hline \hline  & **Backbone** & \begin{tabular}{c} **MSP** \\ **(Baseline/Our)** \\ \end{tabular} & \
\begin{tabular}{c} **MaxLogit** \\ **(Baseline/Our)** \\ \end{tabular} &
\begin{tabular}{c} **OpenMax** \\ **(Baseline/Our)** \\ \end{tabular} \\ \hline
**AUROC\(\uparrow\)** & HRNet64 & 0.721/0.877 & 0.754/0.888 & 0.526/0.545 \\
**AUPRC\(\uparrow\)** & HRNet64 & 0.426/0.509 & 0.435/0.531 & 0.209/0.210 \\ \hline \hline \end{tabular}
\end{table} 
Table 4: The performance of various open-set-detection techniques in terms of AUROC/AUPRC. Figure 5: The process overview of audio-segmentation-data preparation. divergent feature distribution, thereby potentially elevating the network's accuracy in recognizing unknown classes. The prediction results are contingent upon the unnormalized threshold logit value, \(\rho\), as specified in Equation (6). Finding an appropriate threshold proved challenging; thus, to facilitate threshold determination, we constrained the value range from 0 to 1 by applying the softmax function and selected an arbitrary threshold of 0.86. Under this setting, a maximum probability of normalized logits less than 0.86 indicates an unknown event, whereas probabilities above this threshold signify known events. Generally, higher accuracy for known classes correlates with a poorer detection performance for unknowns, making threshold selection a critical and subjective decision by the designer. Following this, the label for a known event was determined by identifying the indices of the maximum softmax probability value. At the established normalized threshold, the accuracy for known classes reached 88.9%, while the detection precision and recall for unknowns were 0.20 and 0.38, respectively. Figure 6 illustrates the segmentation results achieved by our proposed method. The unknown class is represented in dark green, highlighting that our approach effectively distinguishes known classes in almost all instances and successfully detects unknowns. However, similar to findings reported by Ref. [15], achieving fine-grained segmentation boundaries for the unknown class remains a challenge. In audio-segmentation tasks, features are temporally correlated, and when semantic information of the unknown class is absent from the training data, the network tends to minimize the distinctions between known and unknown classes. Additionally, accurately labeling the precise boundary mask of the Passori rhythm pattern presents another layer of complexity.