## 6 Parameters Initialization  
The overcomplete dictionary is initialized with a sufficiently large number of training samples on the order of 1.25 times the data. The data samples are randomly selected from the training data with replacement. We use OMP to compute sparse codes for the initialization of \(\mathbf{s}_{i}^{c}\). We initialize \(\mathbf{z}_{i}^{c}\) with all its components equal to one, except those having zero values for their corresponding components of \(\mathbf{s}_{i}^{c}\), in which case these are set equal to zero. The ridge regression technique is used to initialize the classifier \(\mathbf{B}\), using \(\mathbf{s}_{i}^{c}\) and training labels \(\mathbf{h}_{i}^{c}\in R^{C}\)[8; 23; 25]. We set all \(\pi_{k}\) values equal to 0.5 to make the selection of dictionary and classifier atoms equally probable for the representation of data samples and the corresponding labels. We follow Algorithm 1 for Gibbs sampling. ```
0: We refer to Figure 1, Equation (6), and conditional probabilities derivations for this algorithm. Initialize the hyperparameters \(a_{0}\), \(b_{0}\) with \(0<a_{0},b_{0}<\underset{c}{min}|L_{c}|\), \(c_{0}\), \(d_{0}\), \(e_{0}\), \(f_{0}\) with \(10^{-6}\), \(\lambda_{\phi_{0}},\lambda_{b_{0}}\) with \(M\) and \(C\), \(\lambda_{s}^{c}\) with 1, and \(\lambda_{a}\) and \(\lambda_{h}\) with \(10^{9}\). Initialize \(\mathbf{\Phi}\), \(\mathbf{B}\), \(\boldsymbol{\pi}_{k}^{c}\), \(\mathbf{z}_{k}^{c}\), and \(\mathbf{s}_{k}^{c}\) as already described in Section 6. 1:for\(i\in\{1,2,3,\ldots,500\}\)do
2: We can reduce the number of iterations in the inner loop by processing atoms in groups along with the associated parameters. 3:for\(k\in\{1,2,3,\ldots,K\}\)do
4: Sample \(\boldsymbol{\phi}_{k}\), \(\mathbf{b}_{k}\), \(\mathbf{s}_{k}^{c}\), \(\mathbf{z}_{k}^{c}\), and \(\boldsymbol{\pi}_{k}^{c}\) (using expressions of conditional distributions) \(\forall c\in\{1,2,3,\ldots,C\}\)
5:\(k=k+1\)
6:endfor
7: Sample \(\lambda_{a}\), \(\lambda_{h}\), and \(\lambda_{s}^{c}\) (\(\forall c\in\{1,2,3,\ldots,C\}\))
8:\(i=i+1\)
9:endfor
10: Compute sparse weights \(\boldsymbol{\alpha}\) of test data over the learned dictionary \(\mathbf{\Phi}\), using orthogonal matching pursuit (OMP) available in the SPAMS package in Python. Compute predicted labels by selecting indices of the maximum value components of each column of \(\mathbf{B}\boldsymbol{\alpha}\). 11: Compute the classification accuracy

## 7 Datasets and Preprocessing

In this research, two face recognition, two categorization, and one action recognition dataset are employed as benchmarks for experiments. We give the details of these datasets along with the preprocessing operations as follows. ### Extended YaleB

This database was developed for 38 subjects, each with about 64 samples, with sufficient variations in illuminations and facial expressions containing 2414 face images [54]. These variations carried by each subject set a challenging stage for classification approaches. For one of the subjects, these variations are shown in Figure 2. We used 504-dimensional random face features [8] extracted by projecting \(192\times 168\) cropped face images [9] on a 504-dimensional vector. The projection matrix was generated from random samples of standard normal distributions for this transformation. ### Ar Face Dataset

This database was developed by capturing 26 photographs of each of 126 subjects during two different sessions with larger variations in facial disguise, illumination, and expressions compared with YaleB [38]. Consequently, this database consists of over 4000 face images. Samples from this dataset are shown in Figure 3. We used 540-dimensional random face features that were extracted by projecting \(165\times 120\) cropped face images onto a 540-dimensional vector using a random projection matrix, as in Section 7.1. ### Caltech-101 Dataset

The Caltech-101 database [55] consists of 101 categories of objects and comprises 9144 image samples along with a class of background images.