# Multi-Modal Low-Data-Based Learning for Video Classification
## 3 A New Multi-Modal Video Classification Dataset
### HVU-LD Building Steps  
This subsection outlines the methodology used to construct the HVU-LD dataset, focusing on two key aspects. Firstly, the importance of including a variety of classes in each meta-set is discussed. Secondly, the process of selecting the appropriate classes for each meta-set is described. The HVU-LD dataset is a crucial resource for the low-data-based video classification domain, and understanding its construction is essential for interpreting and utilizing its results. The first rule stipulates that the classes in the meta-training, meta-validation, and meta-testing sets have to be disjoint. To satisfy this requirement, one approach is to randomly select a set of unique classes from all possible classes, ensuring that the classes in the meta-training, meta-validation, and meta-testing sets are disjoint. As stated in [26], 100 classes for meta-training, meta-validation, and meta-testing were selected from the Kinetics [9] dataset randomly. However, this randomness can cause some classes in different meta-sets to be close to each other semantically. Although semantically similar classes do not violate the disjointness rule, they do not help to build challenging real-world situations and may compromise the goal of learning completely novel classes for low-data-based learning. For this reason, we have developed the meta-set class selection algorithm, presented in Algorithm 1. The algorithm is designed to prevent the intersection of classes across different meta-sets and maximize the semantic distance between classes in separate meta-sets. Our proposed meta-set class selection algorithm extracts semantic features of all possible classes over their videos' detailed attributes using Fasttext [37]. Then, according to the semantic features, the K-Means clustering algorithm is applied. Since the aim is building a semantically distant group of three meta-sets, we set the \(K\) number as three. This clustering operation helps to identify which classes are close to each other and which are not. Then, according to the clustering result, the classes to be selected for each meta-set are taken from different clusters. Thus, we first verify that the meta-training set has 64, the meta-validation set has 12, and the meta-test set has 24 video classes; here, there are 100 videos for each class, regardless of the set. Second, each meta-set has unique video classes, and the selected classes are as far away as possible in terms of semantic distance, which is highly desirable in a low-data-based learning problem. Ultimately, both the classes in each meta-set are chosen differently from each other, and the classes that are as semantically distant as possible are chosen among the meta-sets. ```
\(\text{CLASS\_SELECTOR}(classNames)\) \(classFeatures\leftarrow empty\) list Extract semantic feature from each className using Fasttext method for all\(className\in classNames\)do \(classFeature\gets Fasttext{text}(className)\) \(classFeatures.add(classFeature)\) endfor Apply K-means overclassFeatures when K is 3 then obtain 3 cluster centers \(C1,C2,C3\)\(\xleftarrow{K\text{-}\text{M}ams}\)\(classFeatures\) For each meta-set, random classes are sampled from different clusters \(\text{Meta-Train Classes}\xleftarrow{select randomly}\)\(C1\) Meta-Val Classes\(\xleftarrow{select randomly}\)\(C2\) Meta-Test Classes\(\xleftarrow{select randomly}\)\(C3\) return[Train, Val, and Test Classes]
```  
**Algorithm 1** Meta-set Class Selection Algorithm  
According to the clustering-based algorithm, we analyzed the semantic distances between classes in the meta-sets of both our proposed dataset and existing ones, and we compared the results obtained using our meta-set class selection algorithm (Algorithm 1) with those obtained by randomly shuffling classes and constructing meta-sets without using the algorithm.