# Manifolds-Based Low-Rank Dictionary Pair Learning for Efficient Set-Based Video Recognition
## 3 Manifolds-Based Low-Rank Dictionary Pair Learning
### MbLRDPL-SPD  
MbLRDPL-SPD uses the following covariance matrix to model the image set \(X_{i}^{k}\):  
\[C_{i}^{k}=\tfrac{1}{m_{i}-1}\sum\limits_{j=1}^{m_{i}}(x_{i,j}^{k}-m_{i})(x_{i, j}^{k}-m_{i})^{T}, \tag{1}\]  
where \(m_{i}\) denotes the mean vector of image set \(X_{i}^{k}\). Thus, all our gallery image sets are modeled to \(\{C_{i}^{k}\}_{i=1}^{N}\), and these models are points on a Riemannian manifold; more specifically, they are points on a symmetric positive definite manifold. Thereby, we construct the following SPD-manifold-based distance learning problem:  
\[\text{min}\quad||C_{i}^{k}\ominus D_{k}\otimes A_{i}^{k}||_{s}^{2}+\lambda_{1 }||A_{i}^{k}||_{1} \tag{2}\]  
where \(D_{k}\) and \(A_{i}^{k}\) denote the \(kth\) dictionary and the coding matrix on the SPD manifold, respectively; \(||C_{i}^{k}\ominus D_{k}\otimes A_{i}^{k}||_{s}^{2}\) denotes the distance between image set \(C_{i}^{k}\) and the \(kth\) class image set. Here, operators \(\ominus\) and \(\otimes\) are the manifold replacements for subtraction and multiplication in Euclidean space, respectively. Furthermore, different from the \(L_{2}\) norm in Euclidean space, \(||\cdot||_{s}^{2}\) is the geodesic distance metric on SPD manifolds. The optimization problem (2) is difficult to solve because we have not explicitly defined the operators \(\ominus\) and \(\otimes\). However, [PERSON] et al. [1] showed that we can embed the SPD manifold into Euclidean tangent space via the logarithm function \(\log(\cdot)\). As a result, the optimization problem (2) can be rewritten as  
\[\text{min}\quad||\log(C_{i}^{k})-D_{k}A_{i}^{k}||_{F}^{2}+\lambda_{1}||A_{i}^{ k}||_{1} \tag{3}\]  
where \(D_{k}\) and \(A_{i}^{k}\) denote the dictionary and coding matrix in Euclidean spaces, respectively. However, for Equation (3), we need to solve the \(L_{1}\)-norm optimization problem, which is often computationally demanding. \begin{table}
\begin{tabular}{c c} \hline \hline
**Notations** & **Description** \\ \hline \(X,Y,C,\cdots\) & a matrix \\ \(x,d,\cdots\) & a vector \\ \(k,p,m,\ldots\) & scalar \\ \(X_{i}^{k}\) & the _ith_ gallery image set from _kth_ class \\ \(x_{i,j}^{k}\) & the _jth_ image coming from _ith_ gallery video \\ \(\ominus\) & the manifold replacements for subtraction \\ \(\otimes\) & the manifold replacements for multiplication \\ \(||\cdot||_{s}^{2}\) & the geodesic distance metric \\ \(||\cdot||_{F}\) & the Frobenius norm \\ \(||\cdot||_{1}\) & \(||X||_{1}=\sum_{j}|x_{ij}|\) \\ \(||\cdot||_{2}\) & the \(L_{2}\) norm \\ \(||\cdot||_{*}\) & the nuclear norm \\ \hline \hline \end{tabular}
\end{table} 
Table 1: Important notations used in the paper. To avoid the \(L_{1}\)-norm optimization problem, problem (3) is extended to the following problem by introducing the DPL algorithm:  
\[ \begin{array}{ll}\min&\sum\limits_{k=1}^{K}\sum\limits_{i=1}^{n_{k}}||\log(C_{ i}^{k})-D_{k}P_{k}\log(C_{i}^{k})||_{F}^{2}\\ &\quad+\lambda_{1}\sum\limits_{k=1}^{K}\sum\limits_{j=1}^{n_{k}}||P_{k}\log(C_ {j}^{k})||_{F}^{2}+\lambda_{2}\sum\limits_{k=1}^{K}||D_{k}||_{*}\\ \text{s.t.}&||d_{j}^{k}||_{2}^{2}\leq 1,\ k=1,\cdots,K;\ j=1,\cdots,m,\end{array}  \tag{4}\]  
where \(D_{k}=[d_{1}^{k},d_{2}^{k},\cdots,d_{m}^{k}]\in\mathcal{R}^{p\times m}\) is the synthesis dictionary; \(P_{k}\in\mathcal{R}^{m\times p}\) is the analysis dictionary; \(m\) is the number of atoms in the dictionary; \(\{C_{j}^{k}\}_{j=1}^{n_{k}}\) are the complementary data of \(C_{j}^{k}\) in the whole gallery sets, i.e., all the gallery sets that do not belong to the \(kth\) class; \(\bar{n}_{k}\) is the number of complementary sets. Note that the first term in Equation (4) is the fidelity term, which is used to ensure that the learned dictionaries effectively reconstruct the input data. The second term is the discriminative term, which ensures that the analysis subdictionary \(P_{k}\) can project the gallery sets of other classes into a null subspace.