## 5 Proposed Model Architecture
### Proposed U-IntentCapsNet
#### 5.2.3 Dynamic Routing between Capsules  
Traditionally, the learned hidden state \(h_{T}\) for each word \(w_{T}\) is used as the logit to predict the intent. Using the hierarchal modelling capability of the capsule networks, IntentCaps uses the output of WordCaps to learn the query level vector representations for each intent label \(y\) through the routing by-agreement mechanism [7] explained in Algorithm 1. This algorithm replaces the pooling operation of a CNN that loses the spatial information, which is critical for feature clustering in prediction. A word representation from WordCaps is sent to the higher level capsule, the IntentCaps, only if the word representation has a strong agreement with the intent representation. This agreement value for query words may vary for different intent representations. Thus, a prediction vector \(\mathbf{p}_{k|t}\) is estimated between the two capsule layers, denoted as follows for the \(t\)-th word when being recognized as the \(k\)-th intent. This is given in Equation (3):  
\[\mathbf{p}_{k|t}=\sigma\Big{(}W_{k}\,h_{t}^{T}+b_{k}\;\Big{)} \tag{3}\]  
where \(k\in\{1,2,3\}\) denotes the intent label and \(t\in\{1,2,\ldots,T\}\) words in a query. \(\sigma\) is an activation function, i.e., tanh. \(W_{k}\;\in R^{\;2D_{H}\times D_{P}}\) and \(b_{k}\;\in R^{\;D_{P}\times 1}\;\) are the weight and bias matrices for the \(k\)-th capsule in the IntentCaps and \(D_{P}\;\in R^{\;D_{K}\times 1}\) is the dimension of the prediction vector. We propose to detect the intent of the input query \(x\) by dynamically routing the prediction vectors \(\mathbf{p}_{k|t}\) obtained from the WordCaps to obtain the IntentCaps output vectors \(v_{k}\) as shown in Equation (4):  
\[v_{k}=\text{DYNAMICROUTING}\Big{(}\mathbf{p}_{k|t},iter_{intents}\Big{)} \tag{4}\]  
The dynamic routing by agreement Algorithm [7] is briefly explained in Algorithm 1.  
```
1:procedureDYNAMIC ROUTING (\(\mathbf{p}_{k|t}\), iter)
2:for each WordCaps \(t\) and IntentCaps \(k\): \(b_{kt}\gets 0\).
3:for iter iterations do
4:for all WordCaps \(t\): \(c_{t}\leftarrow\) softmax (\(b_{t}\))
5:for all IntentCaps \(k\): \(s_{k}\leftarrow\frac{T}{t}c_{kt}\mathbf{p}_{k|t}\)
6:for all IntentCaps \(k\): \(v_{k}=\) squash (\(s_{k}\))
7:for all WordCaps \(t\) and IntentCaps \(k\): \(b_{kt}\;\leftarrow\;b_{kt}\;+\;\mathbf{p}_{k|t}\). \(v_{k}\)
8:endfor
9: Return \(v_{k}\)
10:endprocedure
```  
**Algorithm 1**Dynamic Routing by Agreement.  
This algorithm computes the coupling co-efficient \(c_{kt}\) that determines how much contribution the \(t\)-th word, \(w_{t}\) has with the \(k\)-th intent, \(y_{k}\)\(c_{t}\) is a vector that consists of all \(c_{kt}\) where \(k\in K.\) and \(b_{kt}\) is the initial logit representing the log prior probability (initialized to 0) that the \(t\)-th word in the WordCaps agrees to be routed to the \(k\)-th intent capsule in the IntentCaps. During each iteration, an intent representation \(s_{k}\) is calculated by a weighted sum over all its prediction vectors as given in Equation (5):  
\[s_{k}=\sum_{t}^{T}c_{kt}\mathbf{p}_{k|t} \tag{5}\]  
The dynamic routing-by-agreement algorithm assigns a low \(c_{kt}\) when there is an inconsistency between \(\mathbf{p}_{k|t}\), and \(v_{k}\), which ensures the outputs of the WordCaps are sent to the appropriate subsequent IntentCaps. A squashing function squash (\(\cdot\)) is applied on \(s_{k}\) to obtain an activation vector \(v_{k}\) for each intent \(k\) given in Equation (6):  
\[v_{k}=squash\;(s_{k})=\frac{\|s_{k}\|^{2}}{1+\|s_{k}\|^{2}}\;\frac{s_{k}}{\|s _{k}\|} \tag{6}\]  
The orientation of the activation vector \(v_{k}\) represents the intent properties while its norm indicates the probability of activation.