# Learning Class-Specific Features with Class Regularization for Videos
## 4 Experiments  
We demonstrate the merits of _Class Regularization_ on the action recognition classification performance on three benchmark datasets, and using a number of widely used CNN architectures. Results are summarized in Table 1. We further statistically compare the classification performance between predictions from different architectures and from different blocks within the same architecture.  
For our experiments, we consider the widely used Kinetics-400 [30] dataset as a baseline. Then, each of the selected models is further fine-tuned on both UCF-101 [31] and HMDB-51 [32] by training the 1D convolutions inside _Class Regularization_ to ensure a dimensionality correspondence between the new class weight vectors and the activation maps of each layer that the method is applied to. We further allow updates on the final two convolution blocks of the selected networks.  
**Training.** The models trained on Kinetics are initialized with a standard Kaiming initialization [33] without inflating the 3D weights. This allows for a direct comparison between architectures with and without _Class Regularization_ and to compare the respective accuracy rates. For all the experiments, we use an _SGD_ optimizer with 0.9 momentum. _Class Regularization_ is added at the end of each bottleneck block in the ResNets and at the end of each mixed block in I3D.  
Figure 3: Visualization of feature amplification. As class-specific activations are reused by the network, informative spatiotemporal features for specific classes during an iteration are amplified. The effect of this amplification is propagated to deeper layers in the network through the connections of the layers in which _Class Regularization_ is applied.  
Figure 2: Layerwise class feature correspondence. Each of the _Saliency Tubes_[29] represents the class activations of a _Class Regularized_ Wide-ResNet50 model in layers _res3_ and _predictions (fc)_. Both clips correspond to visualizations for class _bowling_ from Kinetics-400 [30] for two different examples with variations in their spatiotemporal regions.  
We use transformations for both spatial and temporal dimensions. In the temporal dimension, we use clips of 16 frames that are randomly extracted from different subsequences of the video. For the validation sets, we only use the center 16 frames. Spatially, we use a cropping size of \(112\times 112\) and \(256\times 256\) for fine-tuning. All models are trained for 170 epochs, as no further improvements were observed afterwards in our initial experiments with all three networks on Kinetics. We also used a step-based learning rate reduction of 10% every 50 epochs.  
**Datasets.** Kinetics-400 consists of roughly 240 K training videos and 20 k validation videos of 400 different human actions. We report the top-1 accuracy alongside the computational cost (FLOPs) for each of the networks using spatiotemporally cropped clips. UCF-101 and HMDB-51 have 13 k and 9 k videos, respectively. They are used to demonstrate the transfer abilities of the proposed _Class Regularization_ as well as the usability of our method in smaller datasets.