# Learning Class-Specific Features with Class Regularization for Videos
## 4 Experiments
### Direct Comparisons with and without Class Regularization  
In Table 3, we pairwise compare three architectures after training directly on Kinetics, and after adding _Class Regularization_ blocks and fine-tuning. For each architecture and dataset, networks with _Class Regularization_ outperform those without. The largest gain is observed in the 101-layer 3D Resnet (+2.45%), while we obtained improvements of +1.37% and +1.43% on the Wide-ResNet50 and I3D, respectively. Our approach appears to be especially useful for deeper networks. Since the effective description of classes is achieved through large feature spaces, _Class Regularization_ significantly benefits models that include complex and large class weight spaces. The set of influential class features can be better distinguished with minimal computational costs, as shown in Figure 4.  
When transferring weights, the retraining process does not change for the _Class Regularization_ networks as the additional training phase is only performed in order for the model to learn the feature correspondence. \begin{table}
\begin{tabular}{l c c c c c} \hline \hline
**Model** & **Backbone** & **Depth** & **\# Params (M)** & **GFLOPS** & **Top-1 (\%)** \\ \hline ResNet50-3D [16] & ResNet & 50 & 36.72 & 80.32 & 0.636 \\ \hline ResNet101-3D [16] & ResNet & 101 & 69.06 & 110.98 & 0.652 \\ \hline ResNeXt101-3D [16] & ResNet & 101 & 69.06 & 148.91 & 0.667 \\ \hline Wide ResNet50-3D [16] & ResNet & 50 & 140.94 & 72.32 & 0.640 \\ \hline I3D [15] & Inception & 48 & 12.07 & 55.79 & 0.664 \\ \hline R(2+1)D-ResNet50 [19] & Resnet & 50 & 34.86 & 89.14 & 0.645 \\ \hline R(2+1)D-ResNet101 [19] & ResNet & 101 & 67.22 & 159.21 & 0.668 \\ \hline MF-Net [36] & ResNet & 50 & 8.03 & 22.7 & 0.653 \\ \hline ResNet101-3D w/_Class Regularization_ & ResNet & 101 + 4 & 37.10 & 126.13 & 0.677 \\ \hline Wide ResNet50-3D w/_Class Regularization_ & ResNet & 50 + 4 & 141.32 & 82.67 & 0.653 \\ \hline I3D w/_Class Regularization_ & Inception & 48 + 3 & 69.44 & 62.96 & **0.678** \\ \hline \hline \end{tabular}
\end{table} 
Table 2: Accuracy rates of different spatiotemporal convolutional architectures on the Kinetics-400 dataset. \begin{table}
\begin{tabular}{l c c c c} \hline \hline
**Model** & **Added Latency** & **Kinetics** & **UCF101** & **HMDB51** \\ \hline ResNet101 & - & 65.29\% & 88.23\% & 62.47\% \\ \hline ResNet101 & + 98.786 & 67.74\% & 88.84\% & 63.31\% \\ \hline Wide ResNet50 & - & 63.96\% & 87.52\% & 61.62\% \\ \hline Wide ResNet50 & +102.995 & 65.33\% & 89.11\% & 63.24\% \\ \hline I3D & - & 66.42\% & 91.80\% & 64.27\% \\ \hline I3D & +68.34 & 67.85\% & 93.17\% & 65.83\% \\ \hline \hline \end{tabular}
\end{table} 
Table 3: Direct comparison of architectures with (in orange) and without (in black) _Class Regularization_ blocks. Latency in msec. Advancements can also be achieved in transfer learning, as shown by the rates for UCF-101 (split 1) and HMDB-51 (split 1) in Table 3. Accuracy for non-_Class-Regularized_ networks are recalculated in order to ensure the same training setting. The largest gain from the original implementation in UCF-101 is on the Wide ResNet50 with +1.59% followed by I3D with +1.26% and ResNet101 +0.61%. For the HMDB-51 dataset, the model pair that exhibits the largest gap in performance is Wide ResNet50 with a +1.62% improvement, I3D with +1.56%, and ResNet101 with +0.84%. Overall, the minor deterioration of the accuracy gains in transfer learning could be contributed to the fact that kernels have been already trained in conjunction with class information from a different dataset. As shown in Figure 5, _Class Regularization_ demonstrates improvements for most classes in Kinetics. The greatest improvements are observed for classes that can be better defined based on their execution instead of their appearance. Examples are \"bench pressing\" (9.1% improvement), \"high jump\" (8.9% improvement), and \"jogging\" (15.6% improvement). Amplifying features that are characteristic of those classes improves the model's recognition capabilities.