# Uncertainty Detection: A Multi-View Decision Boundary Approach Against Healthcare Unknown Intents
## 5 Performance Evaluation and Result Analysis
### Comparative Experiment  
We follow the experiment design by [PERSON] and [PERSON] [11] and [PERSON], [PERSON], [PERSON], and [PERSON] [13] to conduct laboratory-level experiments and compare model performance. The detection results of different models under the unknown intent ratio of 25%, 50%, and 75% are illustrated in Tables 3-5. Every group of results is an average of ten times running results using different random seeds to select categories as the unknown intents. The evaluation metrics include the accuracy score (Accuracy), macro F1-score (F1), and macro F1-scores over known intent classes (F1-known) and over the unknown intent class (F1-unknown). Accuracy mainly serves as a baseline measure of overall prediction correctness but proves insufficient for imbalanced class distributions. F1 compensates for this limitation by equally weighting all classes through the mean of macro precision and macro recall. Most critically, the separate calculation of F1-scores for known and unknown intent classes provides granular insight into model performance. F1-known evaluates the stability in classifying permitted queries amidst interference from unknown intent samples, while F1-unknown directly quantifies the detection capability for queries without known intents. \begin{table}
\begin{tabular}{c c} \hline \hline
**Data Field** & **Example** \\ \hline Query & “[PERSON] [11] [11] [11] [11] [11] [11] [11]  
In terms of accuracy and F1, our method achieves the highest values across all unknown intent ratios, as shown in Figure 3. For example, at a 50% ratio, our method achieves an accuracy of 0.7457 and an F1 of 0.7279, outperforming the second-best method, ADB (0.7346 and 0.7161), demonstrating its superior ability to detect unknown intents. This advantage persists in both the 25% and 75% scenarios, highlighting our method's robustness under extreme data distributions. As for the LOF, DOC, and DeepUnk methods, they demonstrate comparable performance at the 25% unknown intent ratio, particularly in terms of accuracy. However, as the proportion of unknown intents increases, their performance gap with our method widens sharply. The F1-unknown metric provides a more specific measure of unknown intent detection performance. Our method outperforms all baselines across all three ratios, particularly at the 25% ratio (0.4082 vs. ADB's 0.3799, a 7.4% improvement), indicating that our method captures unknown intent features more effectively. Interestingly, when the unknown intent ratio increases to 75%, all methods exhibit a significant drop in F1 and F1-known, while F1-unknown improves. This suggests that a high unknown intent ratio harms known intent recognition, although the model can compensate by refining its decision boundaries to enhance unknown intent detection. \begin{table}
\begin{tabular}{c c c c c} \hline \hline
**Method** & **Accuracy** & **F1** & **F1-Known** & **F1-Unknown** \\ \hline LOF & 0.4535 \(\dagger\) & 0.4259 \(\dagger\) & 0.4942 \(\dagger\) & 0.0850 \(\dagger\) \\ \hline DOC & 0.5320 \(\dagger\) & 0.4940 \(\dagger\) & 0.5284 \(\dagger\) & 0.3227 \(\dagger\) \\ \hline DeepUnk & 0.4871 \(\dagger\) & 0.4597 \(\dagger\) & 0.5137 \(\dagger\) & 0.1903 \(\dagger\) \\ \hline ADB & 0.7346 & 0.7161 & 0.7242 & 0.6757 \\ \hline Ours & **0.7457** & **0.7279** & **0.7362** & **0.6864** \\ \hline \hline \end{tabular} Notes: \(t\)-test between our proposed method and benchmarks: \(\dagger\)\(p<\) 0.05.  
\end{table} 
Table 4: Detection results under the unknown intent ratio of 50%.