  Capsule networks are a type of neural network that have recently gained
increased popularity. They consist of groups of neurons, called capsules, which
encode properties of objects or object parts. The connections between capsules
encrypt part-whole relationships between objects through routing algorithms
which route the output of capsules from lower level layers to upper level
layers. Capsule networks can reach state-of-the-art results on many challenging
computer vision tasks, such as MNIST, Fashion-MNIST, and Small-NORB. However,
most capsule network implementations use two to three capsule layers, which
limits their applicability as expressivity grows exponentially with depth. One
approach to overcome such limitations would be to train deeper network
architectures, as it has been done for convolutional neural networks with much
increased success. In this paper, we propose a methodology to train deeper
capsule networks using residual connections, which is evaluated on four
datasets and three different routing algorithms. Our experimental results show
that in fact, performance increases when training deeper capsule networks. The
source code is available on https://github.com/moejoe95/res-capsnet.
