# Non-Iterative Cluster Routing: Analysis and Implementation Strategies
## 3 Methods
### Original Non-Iterative Cluster Routing  
Cluster routing [7] generates clusters of votes through a non-iterative approach, where the standard deviation is employed as the fundamental indicator of agreement and confidence for the information encoded within the vote cluster. Clusters with higher variance indicate lower confidence in their encoded information or decision; consequently, they are weighted less and passed to the next layer. The agreement vector is computed based on the standard deviation of the cluster's individual votes. The centroids or means of the vote clusters are weighted with their corresponding agreement vectors. The weighted sum of these centroids integrates and aggregates the information from all vote clusters across the capsules in the current layer, which is then normalized and transmitted to the next layer. By leveraging agreement and weighted summation, cluster routing can efficiently and dynamically route information flow between each layer of capsules in a single pass. A vote cluster \(\hat{\mathbf{u}}_{i}^{\{1,\cdots,K\}}\) can be generated by multiplying a capsule \(\mathbf{u}_{i}\) with each weight matrix **W** of a weight cluster \(\mathbf{W}_{i}^{\{1,\cdots,K\}}\),  
\[\hat{\mathbf{u}}_{i}^{k}=\mathbf{W}_{i}^{k}\mathbf{u}_{i}. \tag{1}\]  
To enhance the receptive fields of each vote, the neighborhood of \(\mathbf{u}_{i}\) in Equation (1) is expanded to \(3\times 3\) in practice. \(\mathbf{u}_{i}\) becomes the concatenation \(N(\mathbf{u}_{i})\in\mathbb{R}^{9D}\) of its neighborhood. Then, the \(k\)th vote in the cluster is produced as follows,  
\[\hat{\mathbf{u}}_{i}^{k}=\mathbf{W}_{i}^{k}N(\mathbf{u}_{i}). \tag{2}\]  
A vote cluster \(\hat{\mathbf{u}}_{i}^{\{1,\cdots,K\}}\) will send its centroid or mean \(\mathbf{m}_{i}=\frac{1}{K}\sum_{k=1}^{K}\hat{\mathbf{u}}_{i}^{k}\) to the next layer, with weighting determined by the agreement vector \(\mathbf{a}_{i}\). The computation of \(\mathbf{a}_{i}\) is outlined as follows,  
\[\mathbf{a}_{i}=-\log(\sigma_{i}), \tag{3}\]where  
\[\sigma_{i}=\sqrt{\frac{1}{K}\sum_{k=1}^{K}(\hat{\mathbf{a}}_{i}^{k}-\mathbf{m}_{i} ^{k})\times(\hat{\mathbf{a}}_{i}^{k}-\mathbf{m}_{i}^{k})} \tag{4}\]  
The centroids (or means) of these vote clusters are weighted summed as follows,  
\[\mathbf{s}=\sum_{i=1}^{C}\mathbf{c}_{i}\times\mathbf{m}_{i}, \tag{5}\]  
where  
\[\mathbf{c}_{i}=\frac{\exp(\mathbf{a}_{i})}{\sum_{j=1}^{C}\exp(\mathbf{a}_{j})} \tag{6}\]  
Each of the \(\mathbf{C}\) capsule channels contributes a vote cluster to the capsules in the next layer, making the number of vote clusters (\(\mathbf{N}\)) equivalent to the number of channels (\(\mathbf{C}\)) in the multiple-channel version. Layer normalization [6] is applied to \(\mathbf{s}\) to produce the output of the capsule layer. The computational task in Equation (2) can be efficiently executed through utilization of PyTorch's pre-existing optimized convolutional operations [24], similar to the technique employed in [21].  
Notably, this mechanism utilizes significantly fewer parameters than both attention routing [21] and dynamic routing [1]. The absence of iterations in the process enhances its computational efficiency and reduces training resource requirements.