# HalluciNet-_ing_ Spatiotemporal Representations Using a 2D-CNN
## 4 Experiments
### Dynamic Scene Recognition  
Dataset: [PERSON] et al. introduced the YUP++ dataset [58] for the task of dynamic scene recognition. It has a total of 20 scene classes. Samples from this dataset encompass a wide range of conditions, including those arising from natural within scene category \begin{table}
\begin{tabular}{l c c c c c c c} \hline \hline  & \multicolumn{6}{c}{**(a)**} \\ \hline
**Method** & **CNN** & **\#Frames** & **P** & **A** & **RT** & **SS** & **TW** \\ \hline C3D-AVG [6] & 3D & 96 & 96.32 & 99.72 & 97.45 & 96.88 & 93.20 \\ MSCADC [6] & 3D & 16 & 78.47 & 97.45 & 84.70 & 76.20 & 82.72 \\ [PERSON] et al. [56] & 3D & 16 & 74.79 & 98.30 & 78.75 & 77.34 & 79.89 \\ \hline Ours VGG11 & **2D** & **6** & **90.08** & 99.43 & 92.07 & 83.00 & 86.69 \\ HalluciNet (VGG11) & **2D** & **6** & 89.52 & 99.43 & **96.32** & **86.12** & **88.10** \\ HalluciNet (R18) & **2D** & **6** & **91.78** & 99.43 & 95.47 & **88.10** & **89.24** \\ \hline \multicolumn{8}{c}{**(b)**} \\ \hline
**Method** & **CNN** & **\#Frames** & **Corr.** & & & \\ \hline Pose+DCT [57] & - & 96 & 26.82 & & & \\ C3D-SVR [5] & 3D & 96 & 77.16 & & & \\ C3D-LSTM [5] & 3D & 96 & 84.89 & & & \\ C3D-AVG-STL [6] & 3D & 96 & 89.60 & & & \\ MSCADC-STL [6] & 3D & 16 & 84.72 & & & \\ \hline Ours VGG11 & **2D** & **6** & 80.39 & & & \\ Ours HalluciNet (VGG11) & **2D** & **6** & **82.70** & & & \\ Ours HalluciNet (R18) & **2D** & **6** & **83.51** & & & \\ \hline \hline \end{tabular}
\end{table} 
Table 3: (a) Performance (Accuracy in % reported) comparison on dive recognition task. #Frames represents the number of frames that the corresponding method sees. P, AS, RT, SS, TW stand for position, arm stand, rotation type, number of somersaults, and number of twists. (b) Performance ([PERSON]’s rank correlation in % reported) on AQA task. differences, seasonal and diurnal variations as well as viewing parameters. For each scene class in the dataset, there are 60 color videos, with no two samples for a given class taken from the same physical scene. Half of the videos within each class were acquired with a static camera and half were acquired with a moving camera, with camera motions encompassing pan, tilt, zoom and jitter. The use of this dataset to evaluate the utility of inferred motion was suggested in [19]. In the work by [PERSON], 10% of the samples were used for training, while the remaining 90% of the samples were used for testing purposes. [PERSON] et al. [19] formed their own split, called _static-YUP++_. Protocol: For training and testing purposes, we considered the central frame of each sample. The first experiment considered standard dynamic scene recognition using splits from the literature and compared them with a spatiotemporal energy based approach (BoSE), slow feature analysis (SFA) approach, and temporal CNN (T-CNN). Additionally, we also considered versions based on Resnet50 and predictions averaged over 25 frames. As shown in Table 4a, HalluciNet showed minor improvement over the baseline 2D-CNN and outperformed studies in the literature. T-CNN might be the closest for comparison because it uses a stack of 10 optical flow frames; however, our HalluciNet outperformed it by a large margin. Note that we did not train our 3D-CNN on the scene recognition dataset/task, and used a 3D-CNN trained on the action recognition dataset, but we still observed improvements. The second experiment compared our approach with [19] in which we used their split for static-YUP++ (Table 4b). In this case, our vanilla 2D-CNN did not outperform studies in the literature, but our HalluciNet did--even when groundtruth motion information was used by im2 flow [19].