## 5 Discussion and Analysis
### Why ConvProtoNet Works: Some Comparisons
#### 5.2.3 With Induction Network  
Induction Network [14] tries to improve prototype reduction using dynamic routing algorithm derived from Capsule Network [54] to replace the mean operation, where the class-level representation  
Figure 8: An illustration of distribution mismatch using mean as prototype when data satisfy multimodal distribution.  
Figure 10: An illustration of shrinking embedding space.  
Figure 9: Average KL divergence between the fitted distribution generated by two models and distribution of embedded data. The distribution of prototypes from ConvProtoNet and ProtoNet are estimated by gaussian KDE on over 1000 sample points and the average KL divergence is computed for 100 epoches.  
is calculated dynamically based on support set in a non-parametric manner. To be specific, first, each sample \(e_{i,j}^{s}\) in support set is transformed by transformation weights \(W_{s}\):  
\[\tilde{e}_{i,j}^{s}=W_{s}e_{i,j}^{s}. \tag{13}\]  
At each training episode, the connection strength \(b_{i}\) of each sample in support set \(S\) is amended for several times and SoftMax is always used to make sure that coupling coefficients \(d_{i}\) computed by \(b_{i}\) sum to 1:  
\[d_{i}=softmax(b_{i})=\frac{\exp(b_{i})}{\sum_{j}\exp(b_{j})}. \tag{14}\]  
\(d_{i}\) is used to weight the importance of each sample in \(S\) and new prototype is computed by performing linear combination:  
\[\hat{c_{i}}=\sum_{j}d_{i,j}\cdot\hat{e}_{i,j}^{s}. \tag{15}\]  
Squashing operation is then applied to the new prototype:  
\[c_{i}=\frac{\left\|\hat{c_{i}}\right\|^{2}}{1+\left\|\hat{c_{i}}\right\|^{2}} \frac{\hat{c_{i}}}{\left\|\hat{c_{i}}\right\|}. \tag{16}\]  
At last, update the connection strength according to the new prototype:  
\[b_{i,j}=b_{i,j}+\hat{e}_{i,j}^{s}\cdot c_{i}. \tag{17}\]  
Repeating above operations for several iterations to get the final class prototype for \(i\)-th class. From (13)-(17), We can get the updating rule for connection strength at step \(t\):  
\[b_{i,j}^{t+1}=b_{i,j}^{t}+a\cdot\frac{W_{s}^{2}\cdot e_{i,j}\cdot\sum_{k}(\exp (b_{i,k}^{t})e_{i,k})}{\sum_{p}\exp(b_{i,p}^{t})}. \tag{18}\]  
where \(a=\frac{\left\|\hat{c_{i}}\right\|}{1+\left\|\hat{c_{i}}\right\|^{2}}\) is the squashing coefficient. One more step, we can get the following equation using (18):  
\[\frac{\partial b_{i,j}^{t+1}}{\partial b_{i,j}^{t}}\approx 1+a\cdot\frac{W_{s} ^{2}\cdot e_{i,j}^{2}\cdot\exp(b_{i,j}^{t})}{\sum_{p}\exp(b_{i,p}^{t})}. \tag{19}\]  
Obviously, this formula points out that bigger connection strength will grow at exponentially faster speed than other points in \(S\). Initially, all connection strengths are set to 0 and resulted prototype is equal to mean vector of support set. After that, the generated prototype is gradually pulled to the sample point that is closest to the mean vector rapidly resulting prototypes converging at the initially closest point, as shown in Figure 11. However, in most cases, the closest point is a bad choice for class prototype. In short, InductionNet does not consider the distribution of data when computing prototype and rigid reduction yields poor performance in experiments.  
For ConvProtoNet, the class prototypes are produced by stacked convolution layers where more complex mapping from samples to prototypes are performed to fit the implicit data distribution. It takes data distribution into consideration towards better reduction of prototype.