Repository: open-mmlab/mmaction2

Issue:
Configuration of  SlowFast,4x16,R50 and SlowFast,8x8,R50 do not match that of the paper

<img width="613" alt="Screenshot 2020-12-28 at 1 37 48 PM" src="https://user-images.githubusercontent.com/10518587/103191955-e6c6ec00-4911-11eb-862f-4f7011a3128d.png">

According to the paper 4x16 and 8x8 refers to T x \tau, which means \tau=16 for 4x16 and \tau=8 for 8x8. However, based on the setting below, \tau=4 for 8x8 and \tau=8 for 4x16. 

https://github.com/open-mmlab/mmaction2/blob/master/configs/recognition/slowfast/slowfast_r50_8x8x1_256e_kinetics400_rgb.py 

<img width="340" alt="Screenshot 2020-12-28 at 1 38 40 PM" src="https://user-images.githubusercontent.com/10518587/103192000-1249d680-4912-11eb-88bf-1fdc96da0c88.png"> 

https://github.com/open-mmlab/mmaction2/blob/master/configs/recognition/slowfast/slowfast_r50_4x16x1_256e_kinetics400_rgb.py 

<img width="384" alt="Screenshot 2020-12-28 at 1 38 48 PM" src="https://user-images.githubusercontent.com/10518587/103192003-14ac3080-4912-11eb-9d4e-ab84fac588db.png">

Is there a mistake here?


Discussion:
Hi @yxchng , thanks for your interest! There is no mistake here, please print out the tensor shape in the forward function.

The reason is that we merge the subsampling into the pipeline. In the paper, the origin input is 64 x 1, but we do subsampling in `SampleFrames`
```
    dict(type='SampleFrames', clip_len=32, frame_interval=2, num_clips=1),

```
Therefore, the \tau is not so big as the paper. We believe this can relieve the computation in io.

![image](https://user-images.githubusercontent.com/25632694/103194416-930cd080-491a-11eb-871e-29ddee5162a6.png)
The data sampling layer in the red box is moved to data preprocessing. Therefore, when we use \tau = 8, it actually means \tau = 16.
@SuX97 Does this have value to be added to its README?
> @SuX97 Does this have value to be added to its README?

Maybe add to the docstring of the arguments for \tau? This may cause similar confusion as it is not the same as the paper.
That's good

Resolving patch:
diff --git a/mmaction/models/backbones/resnet3d_slowfast.py b/mmaction/models/backbones/resnet3d_slowfast.py
--- a/mmaction/models/backbones/resnet3d_slowfast.py
+++ b/mmaction/models/backbones/resnet3d_slowfast.py
@@ -371,9 +371,11 @@ class ResNet3dSlowFast(nn.Module):
     Args:
         pretrained (str): The file path to a pretrained model.
         resample_rate (int): A large temporal stride ``resample_rate``
-            on input frames, corresponding to the :math:`\\tau` in the paper.
-            i.e., it processes only one out of ``resample_rate`` frames.
-            Default: 16.
+            on input frames. The actual resample rate is calculated by
+            multipling the ``interval`` in ``SampleFrames`` in the
+            pipeline with ``resample_rate``, equivalent to the :math:`\\tau`
+            in the paper, i.e. it processes only one out of
+            ``resample_rate * interval`` frames. Default: 8.
         speed_ratio (int): Speed ratio indicating the ratio between time
             dimension of the fast and slow pathway, corresponding to the
             :math:`\\alpha` in the paper. Default: 8.


Test patch: