# Manifolds-Based Low-Rank Dictionary Pair Learning for Efficient Set-Based Video Recognition
## 3 Manifolds-Based Low-Rank Dictionary Pair Learning
### Optimization  
Here, we use symbol \(M\) to represent the modeling of the image sets, i.e., \(M\) can be a linear subspace \(Y\) or a covariance matrix \(C\). Additionally, we use the symbol \(E\) to represent the Euclidean tangent space representation of \(M\) (i.e., \(E\) can be \(\log(M)\) or \(MM^{\text{T}}\)). Thus, the optimization problems (4) and (7) can be rewritten as  
\[ \begin{array}{ll}\min&\sum\limits_{k=1}^{K}\sum\limits_{i=1}^{n_{k}}||E_{i}^ {k}-D_{k}P_{k}E_{i}^{k}||_{F}^{2}+\lambda_{1}\sum\limits_{k=1}^{K}\sum\limits _{j=1}^{n_{k}}||P_{k}E_{i}^{k}||_{F}^{2}+\lambda_{2}\sum\limits_{k=1}^{K}||D_{ k}||_{*}\\ \text{s.t.}&||d_{j}^{k}||_{2}^{2}\leq 1,\;k=1,\cdots,K;\;j=1,\cdots,m,\end{array}  \tag{9}\]  
Since our objective function (9) is a nonconvex problem, it cannot be directly solved. Hence, the Alternating Direction Method of Multipliers (ADMM) method is used to iteratively optimize it. Specifically, to facilitate the solving of the objective function, some auxiliary variables \(A_{i}^{k}\) and \(J_{k}\), \(k=1,\cdots,K\), \(i=1,\cdots,n_{k}\) are introduced. Thus, problem (9) becomes  
\[ \begin{array}{ll}\min&\sum\limits_{k=1}^{K}\sum\limits_{i=1}^{n_{k}}||E_{i}^ {k}-D_{k}A_{i}^{k}||_{F}^{2}+\lambda_{1}\sum\limits_{k=1}^{K}\sum\limits_{j=1 }^{\bar{n}_{k}}||P_{k}E_{j}^{k}||_{F}^{2}\\ &+\lambda_{2}\sum\limits_{k=1}^{K}||J_{k}||_{*}+\tau\sum\limits_{k=1}^{K}\sum \limits_{i=1}^{n_{k}}||P_{k}E_{i}^{k}-A_{i}^{k}||_{F}^{2}\\ &+\frac{\mu}{2}\sum\limits_{k=1}^{K}||J_{k}-(D_{k}+\frac{\tau}{\mu})||_{F}^{ 2}\\ \text{s.t.}&||d_{j}^{k}||_{2}^{2}\leq 1,\;k=1,\cdots,K;\;j=1,\cdots,m,\end{array}  \tag{10}\]  
where \(\tau\) and \(\mu\) are penalty parameters, and \(Z\) is the Lagrange multiplier. The above problem can be solved with the alternating minimization method, i.e., fix other variables and update the remaining variables in turn. (1) Fix the other variables and update \(A_{i}^{k}\). \(A_{i}^{k}\) can be obtained by solving the following optimization problem:  
\[ \begin{array}{ll}A_{i}^{k*}&=\underset{A_{i}^{k}}{\text{arg min}}\;||E_{i}^ {k}-D_{k}A_{i}^{k}||_{F}^{2}+\tau||P_{k}E_{i}^{k}-A_{i}^{k}||_{F}^{2}\\ \end{array}  \tag{11}\]  
After solving Equation (11), we can obtain the following closed-form solution:  
\[ \begin{array}{ll}A_{i}^{k}=(D_{k}^{T}D_{k}+\tau I)^{-1}(D_{k}^{T}E_{i}^{k}+ \tau P_{k}E_{i}^{k})\\ \end{array}  \tag{12}\]  
(2) Fix the other variables and update \(P_{k}\) by solving the problem:  
\[ \begin{array}{ll}{P_{k}}^{*}&=\underset{P_{k}}{\text{arg min}}\;\lambda_{1} \sum\limits_{j=1}^{n_{k}}||P_{k}E_{j}^{k}||_{F}^{2}+\tau\sum\limits_{i=1}^{n_{ k}}||P_{k}E_{i}^{k}-A_{i}^{k}||_{F}^{2}\\ \end{array}  \tag{13}\]  
After solving Equation (13), we can obtain the following closed-form solution:  
\[ \begin{array}{ll}P_{k}&=\tau\sum\limits_{i=1}^{n_{k}}A_{i}^{k}(E_{i}^{k})^{ T}(\lambda_{1}\sum\limits_{j=1}^{n_{k}}\bar{E}_{j}^{k}(\bar{E}_{j}^{k})^{T}+\tau \sum\limits_{i=1}^{n_{k}}E_{i}^{k}(E_{i}^{k})^{T}+\gamma I)^{-1}\\ \end{array}  \tag{14}\]  
where \(\gamma=10^{-4}\). (3) Fix the other variables and update \(D_{k}\) by solving the problem:  
\[ \begin{array}{ll}{D_{k}}^{*}=\underset{D_{k}}{\text{arg min}}\;\sum\limits_{i =1}^{n_{k}}||E_{i}^{k}-D_{k}A_{i}^{k}||_{F}^{2}\\ \hskip 14.226378 pt+\frac{\mu}{2}||J_{k}-(D_{k}+\frac{Z}{\mu})||_{F}^{2}+\rho||D_ {k}-S_{k}+T_{k}||_{F}^{2}\\ \end{array}  \tag{15}\]where \(S_{k}\) is the auxiliary variables, \(T_{k}\) is the Lagrange multiplier, and \(\rho\) is the penalty parameter. This problem can be solved by iteratively computing the following equations:  
\[\begin{cases}D_{k}=(2\sum\limits_{i=1}^{n_{k}}E_{i}^{k}{A_{i}^{k}}^{T}+2\rho(S_{ k}-T_{k})+\mu I_{k}-Z)(2\sum\limits_{i=1}^{n_{k}}A_{i}^{k}{A_{i}^{k}}^{T}+2\rho I+ \mu I)^{-1},\\ s_{j}^{k}= \begin{cases}d_{j}^{k}+t_{j}^{k},&\text{if }\sqrt{||\rho d_{j}^{k}+\rho t _{j}^{k}||_{2}^{2}}=\rho\\ \frac{\rho t_{j}^{k}+\rho t_{j}^{k}}{\sqrt{||\rho d_{j}^{k}+\rho t_{j}^{k}||_{ 2}^{2}}},&\text{if }\sqrt{||\rho d_{j}^{k}+\rho t_{j}^{k}||_{2}^{2}}>\rho\\ T_{k}=T_{k}+(D_{k}-S_{k}),\end{cases}