# DE-CapsNet: A Diverse Enhanced Capsule Network with Disperse Dynamic Routing
## 1 Introduction  
Deep networks have been successful in the tasks of image classification and object recognition. Increasing the depth of a Convolutional Neural Network (CNN) provides a substantial improvement in the performance [1]. However, if the CNN goes too deep, it can also lead to the challenges of vanishing gradient and saturated accuracy. The degradation problem can be countered by adopting Residual Networks (ResNets) [2], adding connections from the initial layers to the later layers, and by adopting Densely Connected Convolutional Networks (DenseNets) [3], adding dense connections between every other layer. However, CNNs are not robust enough to affine transformations and cannot reserve spatial relationships between features in an image. Diversion in the position of an object in the image may lead the CNN to an incorrect prediction. To overcome the abovementioned weakness, [PERSON] et al. [4] proposed the Capsule Network (CapsNet), which has shown huge potential compared to the conventional CNNs on multiple datasets. A capsule is a group of neurons whose activity vector can represent an object or a part of an object to extract structured features, while keeping the information of the spatial relationship at the same time. The architecture comprises one convolution layer and one fully connected capsule layer, using routing-by-agreement to achieve state-of-the-art accuracy on the Modified National Institute of Standards and Technology database (MNIST) benchmark data set and detecting overlapping digits by using reconstruction regularization. However, the performance of CapsNet on complex benchmark datasets, such as Canadian Institute For Advanced Research(CIFAR-10), is still not on the same level as CNNs. One reason for this is that the implementation of CapsNet is susceptible to background information [5]. The shallow network structure of CapsNets lacks the ability of drawing features. It is not an ideal solution to simply stack the fully connected capsule layers, which may generate useless middle layers [6]. CapsNet passes the information of each capsule to the next layers at the full magnitude of its activation value yet lacks a suitable mechanism of selecting discriminative information from the outputs of each layer [7]. The initial logit \(b_{ij}\) of \"routing Softmax\" is the log prior probability that depends on the location and type of the two capsules, which determines how tightly capsule \(i\) should be coupled to capsule \(j\). However, the Softmax function converts the logits of the coupling coefficients into a set of concentrative values, which may mistakenly send the background information to the next capsule layers with too large a coefficient, resulting in the wrong summation of the prediction vectors with larger values. This may affect the final result of the classification. In addition, the number of computational parameters for dynamic routing used in stacking capsules is huge, which leads to higher cost of training time. In order to strengthen the anti-interference ability of the CapsNet [4] and discernibility of the output vector of each class and to improve the classification accuracy of images with complicated data and background, this paper presents the following contributions:  
1. Drawing from Diverse Capsule Network (DCNet++) [8], we propose a novel architecture called Diverse Enhanced Capsule Network (DE-CapsNet). Multiple-layer residual blocks, instead of one convolutional layer, are used in a residual convolutional subnetwork to extract features from complicated data such as CIFAR-10. The features are input into different levels of primary capsules. DE-CapsNet utilizes a two-level primary capsules hierarchical model to represent different scales of images. Furthermore, the output from the primary capsule is assigned to digit capsules (DigitCaps) by a routing algorithm, and DE-CapsNet fuses the features of the two-level primary capsules together to identify the instantiation.