# Towards Feasible Capsule Network for Vision Tasks
## 1 Introduction  
**State-of-the-art Networks:** Convolutional neural networks (CNNs) have dominated the field of computer vision for a decade, solving a wide range of tasks such as image classification [1, 2] and segmentation [3, 4]. Their success can be attributed to several factors. Firstly, CNNs leverage the translational invariance inductive bias [5], making them adopt at recognizing objects under different translations. Secondly, extensive research has focused on developing efficient, high-performance CNN architectures [6, 7, 8, 9]. Additionally, the availability of pre-trained models has facilitated the practical adoption of CNNs in both research and enterprise applications. While transformer models, with their self-attention mechanism [10], have gained attention for their flexibility, it is worth noting that they often require substantial amounts of data and computational resources for effective training [11]. Despite this, CNNs remain the preferred choice for smaller datasets due to their ability to quickly adapt to new experiments. **Invariance vs. Equivariance:** However, CNN models possess inherent limitations that impede their effectiveness in recognizing features across different orientations and encoding comprehensive spatial relationships. The pooling operations employed in CNNs discard essential pose information, resulting in the loss of internal properties such as shape, location, pose, and orientation [12, 13]. Furthermore, CNNs rely on large, labeled datasets, face challenges in encoding deformation information, and are vulnerable to unpredictable shifts during testing. Moreover, CNNs tend to rely on memorization rather than comprehension, lacking the ability to capture intricate feature relationships. In contrast, capsule networks (CapsNets) [14] overcome these limitations by incorporating pose, relationships,and robustness to affine transformations. CapsNets built on the concept of inverse graphics [15] offer superior explanations compared to CNNs. CapsNets provide benefits such as reduced parameter requirements, improved resilience against adversarial attacks [16; 17; 18], enhanced interpretability [19], and the capability to handle affine transformation [20; 21; 22]. **Challenges:** Despite the promising advancements in CapsNets, several challenges that need to be addressed remain. Training CapsNets continues to pose difficulties, as they have yet to surpass the state-of-the-art benchmarks in a fair manner [20; 23; 24; 25; 26]. One notable concern is the computational complexity associated with CapsNets, particularly due to the routing algorithm, which necessitates additional memory space and leads to a multiplicative increase in the number of computations, even for relatively small input sizes. Another limitation lies in the specific structure of CapsNets, which inherently limits their applicability primarily to vision-related tasks [27]. While the routing-by-agreement mechanism [28] is strong in theory, its unsupervised clustering nature may not hold up well under heavy input noise. Moreover, unoptimized implementations and a lack of comprehensive architectural understanding further impede researchers from fully harnessing the potential of capsule networks. Additionally, CapsNets struggle with the absence of a pooling mechanism commonly found in traditional CNNs, which results in attempts to account for all aspects of an image, including background noise [29]. Addressing these challenges and developing further improvements will be key to enhancing the robustness and efficiency of capsule networks. **Contributions:** This study is motivated by the observation that most existing studies in the field of CapsNets have predominantly utilized basic architecture comprising a single convolutional layer for low-level feature extraction, a primary capsule layer for mapping from a feature space to the capsule space, and a class capsule layer for routing to predict class activation and corresponding pose vectors [30; 31; 32; 33].