# Multi-Modal Low-Data-Based Learning for Video Classification
## 5 Experiments
### Few-Shot Classification with HMDB-51  
Our newly proposed low-data-based multi-modal video classification dataset is the first dataset that contains pairs of videos and attributes that capture multiple facets of the video content. Thus, future studies in this field will benefit from this dataset and be able to provide more real-world-compatible algorithms.  
Therefore, we proposed a variational-autoencoder-based baseline study to evaluate this dataset and serve as a starting point for future studies. However, the generalization ability of the basic model is important and should be able to achieve a competitive result, at least when the uni-modal low-data-based video classification, which is the closest research subject to the proposed dataset, is compared to the most successful models in the research area.  
In this manner, we have evaluated the BL model that does not have our proposed regularization technique and is adapted to the video domain from [11], and the I-BL baseline, which has the regularization technique, with a very well-known dataset in that research area. In order to make this evaluation happen, we first obtain the raw videos of the HMDB-51 dataset. Then, according to [45], meta-train, meta-validation, and meta-validation sets are arranged. Then, as presented in our proposed dataset construction stages, ViT-based visual features for the meta-set videos are extracted. Finally, the classification results of both the BL and our I-BL models and other studies in uni-modal few-shot video classification studies are presented in Table 7.  
The results of the HMDB-51 dataset, which are presented in Table 7, have similar patterns in comparison between \(BL\) and \(I\)-\(BL\) in terms of classification performance with our newly proposed dataset, HVU-LD. More precisely, while the BL model under-performs for all \(k\)-_shot_ classification settings, with the help of the regularization technique, the I-BL model has better scores between nearly 10% and 17%. Besides the BL and I-BL comparison, our proposed method, I-BL, has 2.6% better classification performance in a 5-way 5-shot classification setting. Besides that, our model has gained 3.77% better classification performance among presented 5-way 3-shot classification results.  
\begin{table}
\begin{tabular}{l c c c c} \hline \hline \multirow{2}{*}{_n-Way_} & **BL** & **I-BL** & **BL** & **I-BL** \\  & **CNN** & **CNN** & **ViT** & **ViT** \\ \hline ELMO & _32.46_ & 41.06 & 37.03 & 43.42 \\ Bert & 39.72 & 42.35 & 48.91 & 52.19 \\ \hline \hline \end{tabular}
\end{table}
Table 6: Results of zero-shot video classification on the meta-test set for different visual and textual architectures (results are presented in percentage).  
\begin{table}
\begin{tabular}{l c c c c} \hline \hline  & **1** & **3** & **5** & **10** \\  & **Shot** & **Shot** & **Shot** & **Shot** \\ \hline OTAM [28] & 54.5 & 65.7 & 68.0 & - \\ HYKSM [45] & 60.3 & 71.7 & 76.0 & - \\ STRM [61] & - & - & 81.30 & - \\ FSVC-ATA [62] & 65.78 & - & 82.27 & - \\ BL 1 & 42.92 & 64.47 & 71.55 & 79.02 \\
**I-BL (ours)**2 & 60.01 & **75.47** & **84.87** & **89.14** \\ \hline \hline \end{tabular} \({}^{\dagger}\)Refers to our baseline study. 2 Refers to our improved study.  
\end{table}
Table 7: Results of _5-way_ few-shot video classification on the HMDB-51 meta-test set on ViT-based architecture with different k-shot values. (Results are presented in percentage). \({}^{a}\)-,\({}^{b}\) indicates that there is no value presented in the original paper for that experiment.