# Empirical Remarks on the Translational Equivariance of Convolutional Layers
## 4 Experimental Results
### On the Translational Invariance of Fusion Strategy  
In this section, we compared the four networks by varying translation from 0 to 5 pixels. We selected three results; one for a shallow network with a small kernel size, another for a deep network with a  
Figure 5: The test accuracy for the Net-1 network of \(3\times 3\) kernels trained with spatial images only.  
Figure 6: The accuracy of the networks tested with N-pixel translated (depicted by Trans. N) images. (**a**) for the deep networks with varying convolutional layers of up to 17 with the same \(3\times 3\) kernel size, (**b**) for the Net-1 network with varying kernel sizes of up to 37.  
small kernel size, and the other for a shallow network with a large kernel size. The test error rates are shown in Tables 1-3, respectively.  
From Table 1, all the networks obtained over 98% in accuracy in the case of zero translation. This indicates that all the networks were successfully trained with the original centered training dataset. For all the networks, the error rates significantly increase as the pixel translation increases, especially after 2-pixel translation. However, focusing on relative performance improvement, we can note that all the fusion networks outperform the baseline network regardless of pixel translation. The late-fusion networks achieved at least 13.79% to 26.29% performance improvement in the absence of pixel translation. In addition, the concatenation network achieved a 66.28% performance improvement in the case of 2-pixel translation. We can remark that the fusion networks are far superior to the baseline network in terms of translational invariance.  
Experimental results for a typical deep CNNs with a \(3\times 3\) kernel size are shown in Table 2. The error rate was reduced compared to Table 1 for all cases. In the case of early fusion, there are cases where the performance improvement is lower than the baseline network. Overall, all the fusion networks are superior to the baseline network. For late fusion by concatenation, it is still far superior to  
\begin{table}
\begin{tabular}{c c c c c c c} \hline \hline
**Translation by Pixel** & **No Fusion** & \multirow{2}{*}{**Early Fusion**} & \multicolumn{2}{c}{**Late-Fusion**} & \multicolumn{2}{c}{**Late-Fusion**} \\  & **Baseline** & & & **Concatenation** & \multicolumn{2}{c}{**Addition**} \\ \hline
0 & 1.98 & 1.66 & (19.28) & 1.74 & (13.79) & 1.56 & (26.92) \\
1 & 8.47 & 6.04 & (40.23) & 5.08 & (66.73) & 5.03 & (68.39)  
the baseline network by 62.55% in the case of 2-pixel translation. We can remark that the concatenation strategy consistently achieves high performance improvements over the baseline network.  
Table 3 shows the experimental results for a very large kernel size, but a shallow network. Compared to Table 2, the error rates were reduced in all cases. This indicates that the kernel size is more effective for translational invariance than for network depth. Early fusion performs relatively well in the absence of pixel translation. Overall, the concatenation strategy is superior to all the other networks. The performance improvement of this strategy compared to the baseline network is 20.35% for 1-pixel translation and 73.88% for 2-pixel translation. Table 3 remarks that it is intrinsic to increase the kernel size to improve translational invariance when the network depth is shallow. We note that the concatenation strategy still achieves high performance improvements.