# DE-CapsNet: A Diverse Enhanced Capsule Network with Disperse Dynamic Routing
## 3 Diverse Enhanced Capsule Network  
### Enhanced Capsules  
We drew inspiration from the Spatial Group-wise Enhance module [9] in our architecture to enhance the representation power of capsules. Figure 1 shows the procedure of capsule enhancement.  
A capsule is a group of neurons. In the Spatial Group-wise Enhance module [9], the operations are based on each group of neurons, which can learn diversifying entity representations and learn the group-wise similarity. In view of this, we arranged \(\mathcal{C}\) channels and \(f=\mathrm{H}\times\mathrm{W}\) feature maps in groups. The quantity of feature maps in each set was equal to the dimension of each capsule. Therefore, \(\mathcal{G}\) groups of feature maps are able to be viewed as \(\mathcal{G}\) channels of \(\kappa=\frac{C}{\mathcal{G}}\) dimension capsules. \(p_{i}\) is a vector that represents the capsule, \(p_{i}\in\mathbb{R}^{\kappa}\). Different sets of feature maps constitute a space containing several capsules. The space is named \(\Gamma=\left\{\mathrm{p}_{1},\cdots\mathrm{p}_{\mathrm{f}}\right\}\).  
First, we obtained the global feature \(\mathbf{g}\) of each set of grouped capsules by computing the spatial average, as in Equation (3) [9].  
\[\mathbf{g}\ =\ \frac{1}{f}{\sum}_{i=1}^{f}p_{i} \tag{3}\]  
After that, the simple dot product was used to compare the resemblance between the global \(\mathbf{g}\) feature and the capsule \(p_{i}\). This can be simply seen as the projection of capsule \(p_{i}\) onto the global feature vector \(\mathbf{g}\). \(\theta_{i}\) is the angle between the two vectors, as in Equation (4) [9].  
\[r_{i}\ =\mathbf{g}p_{i}=\left\|\,\mathbf{g}\,\right\|\,\left\|p_{i}\right\| \cos(\theta_{i}) \tag{4}\]  
Figure 1: Illustration of capsule enhancement by dot product for group feature learning from Spatial Group-wise Enhance.  
Normalizing \(r_{i}\), which is shown in Equation (5), can offset the bias size between different samples [24].  
\[\hat{r}_{i}=\ \frac{r_{i}-\mu_{r}}{\sigma_{r}+\epsilon} \tag{5}\]  
Here, \(\epsilon\) is a constant for numerical stability [24], \(\mu_{r}\) is the expectation of \(\mathcal{R}=\{r_{1},\ldots r_{i}\}\), and \(\sigma_{r}\) is the variance of \(\mathcal{R}\), with [9; 25; 26]  
\[\mu_{r}=\ \frac{1}{f}\sum\
olimits_{j}f_{j},\sigma_{r}^{2}=\frac{1}{f}\sum \
olimits_{j}f_{j}\left(r_{j}-\mu_{r}\right)^{2}.\]  
Parameters \(\gamma\) and \(\beta\) corresponding to each coefficient of \(\hat{r}_{i}\) scale and divert the normalized value to represent the characteristic transform, as indicated in Equation (6) [9].  
\[a_{i}=\gamma\ \hat{r}_{i}+\beta \tag{6}\]  
Finally, we adopt the Sigmoid function \(\sigma\) to scale the transforming space, and \(\hat{p}_{i}\) is the enhanced capsule, as shown in Equation (7) [9].  
\[\hat{p}_{i}=p_{i}\sigma(a_{i}) \tag{7}\]  
The group of enhanced capsules is named \(\hat{f}=\left\{\hat{p}_{1},\ldots\hat{p}_{f}\right\}\). The enhanced capsule blocks are inserted in between the residual blocks in enhanced capsule residual convolutional subnetworks. Furthermore, the enhancement is introduced after convolution on the quasi-primary capsule layers.