# Margin and Shared Proxies: Advanced Proxy Anchor Loss for Out-of-Domain Intent Classification
## 3 Proposed Method
### Advanced Proxy-Anchor Loss  
As implied by the name, Advanced Proxy-Anchor loss is designed to overcome the limitations of previous OOD intent classification studies by achieving advancements over Proxy-Anchor loss [15]. Our proposed method utilizes three types of proxies: a centroid proxy, a margin proxy, and a shared proxy. Centroid proxy \(p\) is a proxy representing each class, as is used in the Proxy-Anchor loss [15]. \(p_{k}\in\mathcal{R}^{H}\) is an embedding vector that can be learned and which represents the centroid proxy of the k-th IND intent. Margin proxy \(m_{k}\in\mathcal{R}^{H}\) determines the decision boundary by calculating the distance to the corresponding centroid proxy \(p_{k}\). The concept of the margin also exists in Proxy-Anchor loss. However, since the margin of the proxy-anchor loss is a hyperparameter, the researchers set the margin, and there is a limitation in that several experiments must be conducted to find the optimal value [17]. Moreover, there have been various studies aiming to learn the margin that determines the decision boundary in the OOD Detectiontask [10; 34; 35], but since the low-dimensional distance itself is set as a learning parameter, these approaches have been limited in terms of their ability to capture the features of the high-dimensional embedding vector. Since each margin proxy of our method is a vector of the same dimension as the data embedding, our method captures the semantic features of the sample more effectively than previous studies. Only one shared proxy exists in the embedding space, and one margin proxy is assigned the same as the centroid proxy. The shared proxy \(p_{shared}\in\mathcal{R}^{H}\) is treated as a positive proxy for all IND samples in the training process. That is, all samples are in a multi-label state belonging to both the original centroid proxy and the shared proxy. [PERSON] et al. [36] use the method of intentionally overfitting the model when training data are sparse. Inspired by this method, we overfit the model so that all samples are located inside the decision boundary of the shared proxy. However, since the model is trained using multi-label samples, it is possible to overfit with a shared proxy and learn the semantic features of the intent to which each sample originally belongs. In other words, the model is weakly overfitted. With this intentional overfitting method, IND samples are located inside the decision boundaries of both the shared proxy and the original proxy. Further, since OOD samples do not include the semantic features of IND intents, they are only located only inside the decision boundary of the shared proxy. Advanced Proxy-Anchor loss is formulated as follows:  
\[\mathcal{L}=\frac{1}{|P^{+}|}\sum_{p\in P^{+}}\text{log}\bigg{(}1+\sum_{x\in X _{p}^{+}}e^{\alpha M_{\text{pos}}}\bigg{)}+\frac{1}{|P|}\sum_{p\in P}\text{ log}\bigg{(}1+\sum_{x\in X_{p}^{-}}e^{\alpha M_{\text{neg}}}\bigg{)}, \tag{3}\]  
where the notation of each variable is the same as that of the Proxy-Anchor loss, and \(M_{\text{pos}}\) and \(M_{\text{neg}}\) are, respectively, defined as follows:  
\[M_{pos}=\left\{ \begin{array}{ll}-s(x,m),&\text{if }d>0\\ -d,&\text{if }d\leq 0,\end{array} \right. \tag{4}\]  
\[M_{neg}=\left\{ \begin{array}{ll}d,&\text{if }d>0\\ s(x,m),&\text{if }d\leq 0,\end{array} \right. \tag{5}\]  
where \(m\) is a margin proxy corresponding to the centroid proxy \(p\). The \(d\) is defined as:  
\[d=s(x,p)-s(m,p), \tag{6}\]  
where \(d\) is the difference of the distance between the sample and the centroid proxy and the distance between the sample and the margin proxy. This value determines whether the sample is located inside the decision boundary. Advanced Proxy-Anchor loss not only makes it so that each sample is located inside the decision boundary of the positive proxy, but it also limits the space in which OOD samples can be located inside by making it smaller. Here, the decision boundary is made small but still large enough to contain the samples inside.