# Towards Feasible Capsule Network for Vision Tasks
## 4 Experiments
### Results
#### 4.3.4 Ablation Study  
In essence, the strength of the pre-trained backbone model provides a solid foundation, enabling us to concentrate on the hyperparameters specific to our design. In this section, we focus on investigating the type of routing method and the structure of the primary capsule layer, among other parameters, across four different settings. These experiments allow us to gain insights into the impact of these architectural choices on the overall performance of the CapsNet. During this experiment, unless specified otherwise, we adhered to the default setting as follows: the first (1) architecture was utilized, where capsules are directly  
Figure 3: Comparison of accuracy for different backbones from scratch and pre-trained on the CIFAR10 and CIFAR100 datasets. \begin{table}
\begin{tabular}{c c c c} \hline \hline
**Dataset** & **Backbone** & **Scratch** & **With Pre-Trained** \\ \hline \multirow{2}{*}{CIFAR 10} & ResNet18 & 83.7 & 94.08 \\  & DenseNet & 86.03 & **94.97** \\ \hline \multirow{2}{*}{CIFAR 100} & ResNet18 & 54.29 & 73.03 \\  & DenseNet & 55.08 & **79.91** \\ \hline \hline \end{tabular}
\end{table} 
Table 3: Performance of CapsHead (setting (1), dynamic routing) with a pre-trained backbone and from scratch. constructed from the feature map of the ResNet18 backbone model, and the routing method employed was dynamic routing. Table 4 presents an ablation study on the capsule head's performance for the classification task using the CIFAR10 dataset. In the primary capsule configuration, four designs are investigated: architecture (1) with a single routing layer with 200K parameters, achieving an accuracy of 83.7%; method (2) with the projection of two hidden layers with 800 K parameters, resulting in an accuracy of 83.21%; method (3) with two hidden layers with 5.1 M parameters and keeping spatial dimension, yielding an accuracy of 84.02%; and (4) capsule styles are constructed from feature maps, with 500 K parameters, achieving the highest accuracy of 84.26%. The accuracy scores for each configuration demonstrate the trade-offs between parameter counts and model performance. Additionally, we evaluate four routing methods: dynamic with an accuracy of 83.7%, max-min with an accuracy of 80.64%, EM with an accuracy of 80.1%, and Fuzzy with an accuracy of 82.18%. Figure 4 illustrates the outcomes of our ablation studies. We observe that there is no distinct trend or significant variation between different primary capsule architectures and routing types. Nevertheless, approaches employing cosine similarity and vector length, such as dynamic and max-min routing, appear to yield better results compared to those utilizing clustering assumptions, such as EM and fuzzy routing. Furthermore, methods (3) and (4), which leverage more spatial information to construct capsules, demonstrate higher effectiveness compared to approaches solely relying on feature vectors. These findings shed light on the strengths and limitations of various capsule configurations and routing techniques, guiding future developments in the CapsNet architecture design. \begin{table}
\begin{tabular}{c c c c} \hline \hline
**Tunning** & **Value** & **Accuracy** &
\begin{tabular}{c} **Params of Capsule** \\ **Head** \\ \end{tabular} \\ \hline \multirow{4}{*}{Primary Capsule} & (1) & 83.7 & 700 K \\  & (2)—2 hidden layers & 83.21 & 1.5 M \\  & (3)—2 hidden layers & 84.02 & 5.6 M \\  & (4)—2 hidden layers & **84.26** & 1 M \\ \hline \multirow{4}{*}{Routing Method} & Dynamic & **83.7** & \\  & Max–Min & 80.64 & \\ \cline{1-1}  & EM & 80.1 & \\ \cline{1-1}  & Fuzzy & 82.18 & \\ \hline \hline \end{tabular}
\end{table} 
Table 4: Ablation study on CapsHead with classification tasks on the CIFAR10 dataset. Figure 4: CapsHead ablation studies. Table 5 present a comprehensive comparison of the performance of CapsHead, our proposed method, against several prior works on both the CIFAR10 and CIFAR100 datasets. For the CIFAR10 dataset, the baseline model accuracies range from 73.3% (HitNet) to 95.45% (Encapsulation).