  The Capsule Network is widely believed to be more robust than Convolutional
Networks. However, there are no comprehensive comparisons between these two
networks, and it is also unknown which components in the CapsNet affect its
robustness. In this paper, we first carefully examine the special designs in
CapsNet that differ from that of a ConvNet commonly used for image
classification. The examination reveals five major new/different components in
CapsNet: a transformation process, a dynamic routing layer, a squashing
function, a marginal loss other than cross-entropy loss, and an additional
class-conditional reconstruction loss for regularization. Along with these
major differences, we conduct comprehensive ablation studies on three kinds of
robustness, including affine transformation, overlapping digits, and semantic
representation. The study reveals that some designs, which are thought critical
to CapsNet, actually can harm its robustness, i.e., the dynamic routing layer
and the transformation process, while others are beneficial for the robustness.
Based on these findings, we propose enhanced ConvNets simply by introducing the
essential components behind the CapsNet's success. The proposed simple ConvNets
can achieve better robustness than the CapsNet.
