# SimCDL: A Simple Framework for Contrastive Dictionary Learning
## 5 Experiments
### Classification
#### 5.1.1 Dictionary Learning Initialization  
For the general DL problem, we conducted experiments on a large amount of samples. To enhance the understanding of the outcomes, we made two experiments using the AK-SVD algorithm with two different initializations. The first is a random initialization, \(\mathbf{D}_{0}\), while the second starts from \(\mathbf{D}_{0}\) and is trained with the SimCDL framework provided in Algorithm 1. We denote with \(\mathbf{D}_{t}\) the dictionary obtained by SimCDL. We further include the experimental details for the _YaleB_ dataset. For this experiment, we only used the training dataset, which consisted of 1216 samples and 32 for each of the 38 classes. The dictionary \(\mathbf{D}_{0}\) was initialized with a total of 1520 atoms, 40 atoms for each class. Using this dictionary, we further obtained the dictionary \(\mathbf{D}_{t}\) by updating it for 10 iterations of SGD following the SimCDL framework. We applied AK-SVD starting with the two initializations, \(\mathbf{D}_{0}\) and \(\mathbf{D}_{t}\). For this stage, we set the sparsity constraint to 40, which is equal to the number of atoms from each class. This constraint will prove useful in future experiments. To balance the experimental conditions, we allowed additional AK-SVD iterations when initialized with \(\mathbf{D}_{0}\) to compensate for the time needed for the computation of \(\mathbf{D}_{t}\). This adjustment was essential to ensure that the comparison between the dictionaries is fair. In general, the computation of the dictionary \(\mathbf{D}_{t}\) is equivalent to several iterations of AK-SVD. This depends on the experimental configuration. More precisely, the batch size, the employed transformations, and the number of optimization iterations give the execution time for the SimCDL initialization. The compared results were obtained with a different number of iterations but the same execution time. For the _YaleB_ dataset, we used 10 additional iterations to compensate for the SimCDL initialization. Even if the AK-SVD algorithm, when initialized with \(\mathbf{D}_{0}\), benefited from more iterations, the representation error still tended to have poorer results compared to the AK-SVD algorithm initialized with SimCDL. A similar experiment was carried out for the _UCF50_ dataset. We provide the results obtained on these two datasets in Figures 3 and 4. Figure 3: YaleB—representation error evolution. We can observe that the error representation is better when the starting point is a dictionary trained with SimCDL. To better understand the nature of the SimCDL initialization, we plotted the mutual coherence of the two dictionaries, \(\mathbf{D}_{0}\) and \(\mathbf{D}_{t}\). The result is shown in Figure 5. We observed that even though SimCDL follows a Self-Supervised Representation Learning strategy, the algorithm is capable of obtaining more incoherent atoms. This shows that the augmentation transformation tends to build some incoherence barriers around the samples. This situation propagates further into the process of dictionary optimization and leads to more diverse and incoherent atoms. During our investigation, we also wanted to understand how the augmentation transformation affected the samples. Since the datasets consist of large vectors, a dimensional reduction task was employed. For this, we used the t-distributed Stochastic Neighbor Embedding (t-SNE) [24] algorithm to reduce the data dimensions to two. We first reduced the original data points to 2D (Figure 6), and then reduced the size of the same samples, but only after the augmentation task was applied (Figure 7). Figure 4: UCF50—representation error evolution. Figure 5: YaleB—mutual coherence. We noticed that the augmentation process tended to produce a more compact form of the classes compared to the original dataset. It is also possible that t-SNE was helped by the augmentation process to better group the samples from the same class; in this case, it is likely that the classification algorithms also directly benefited from augmentation.