# Uncertainty-Aware Active Meta-Learning for Few-Shot Text Classification
## 5 Experiments
### Out of Scope Intent Detection  
The CLINC150 dataset is a comprehensive benchmark designed to evaluate the performance of intent classification systems, particularly in the presence of out-of-scope (OOS) queries--queries that do not belong to any system-supported intent class. Unlike traditional datasets that only contain in-scope data, CLINC150 includes both 22,500 in-scope queries across 150 intent classes grouped into 10 domains and 1200 out-of-scope queries. Additionally, three variations in the dataset are available: Small, which limits in-scope training queries to 50 per intent instead of 100; Imbalanced, with a varying number of training queries (25, 50, 75, or 100 per intent); and a version where out-of-scope training data is expanded to 250 examples instead of 100. These variations simulate practical production scenarios where training data are often imbalanced or limited, making CLINC150 an essential benchmark for low-resource natural language understanding (NLU) tasks. In the experiments, a BERT-based model with the SNGP methodology was used for out-of-scope classification, trained exclusively on in-scope data while uncertainty estimation techniques were applied. Comparative models included MC Dropout, Deep Ensemble, MCD-GP, DUQ, and other uncertainty-aware methods. Table 4 highlights that while most methods performed similarly on in-domain accuracy, Deep Ensemble achieved the highest performance but incurred high computational costs due to multi-model training and inference requirements. For out-of-scope detection, the proposed UA-AML SNGP model, enhanced with USB loss, outperformed other approaches with an AUROC of 0.972  
Figure 3: Few-shot sentiment classification accuracy by Uncertainty Applying Rate (UAR). \begin{table}
\begin{tabular}{c c} \hline \hline
**Method** & **Accuracy \% (2-Way 5-Shot)** \\ \hline Matching Networks & 65.73 \\ Prototypical Networks & 68.17 \\ Graph Network & 82.61 \\ Relation Network & 83.07 \\ SNAIL & 82.57 \\ ROBUSTTC-FSL & 83.12 \\ BERT & 79.80 \\ Induction Networks & 84.52 \\ (Ours) UA-AML + Induction Networks & 85.17 \\ \hline \hline \end{tabular}
\end{table} 
Table 3: Few-shot sentiment classification accuracy results on Amazon Review Sentiment Classification dataset. and an AUPR of 0.884. Given the dataset imbalance of 4500 in-domain to 1000 OOS queries, AUPR was used to ensure balanced evaluation. As shown in Figure 4, the Uncertainty Applying Rate (UAR) significantly impacted out-of-scope prediction. The best performance, with an ROC-AUC of 0.972, was recorded at an 80% UAR, reflecting an optimal balance between uncertainty estimation and prediction robustness. These results indicate that UA-AML SNGP effectively enhances out-of-scope intent detection while maintaining computational efficiency and robust uncertainty estimation. This makes it a compelling approach for practical applications where imbalanced and limited training data are common. On the other hand, Figure 5 shows how calibrated the output confidence of the proposed model is. In the case of the existing SNGP, it shows very high calibration performance as a model for estimating uncertainty properly. In contrast, the method applying the proposed UA-AML SNGP model with USB loss has a higher ratio of actual accuracy, and the model underestimates the results. This can be inferred that the model has been trained to underestimate the overall model by training mainly on samples with high uncertainty and \begin{table}
\begin{tabular}{c c c c} \hline \hline \multirow{2}{*}{**Method**} & \multicolumn{2}{c}{**Accuracy \%**} & \multicolumn{2}{c}{**Out-of-Domain**} \\ \cline{2-4}  & **(In-Domain)** & **AUROC** & **AUPR** \\ \hline Deterministic & 96.5 & 0.897 & 0.757 \\ MC Dropout & 96.1 & 0.938 & 0.799 \\ Deep Ensemble & 97.5 & 0.964 & 0.862 \\ MCD-GP & 95.9 & 0.906 & 0.803 \\ DUQ & 96.0 & 0.917 & 0.806 \\ SNGP & 96.6 & 0.969 & 0.880 \\ (Ours) UA-AML + SNGP & 96.9 & 0.972 & 0.884 \\ \hline \hline \end{tabular}
\end{table} 
Table 4: Out-of-scope detection results on CLINC150 dataset.