# Manifolds-Based Low-Rank Dictionary Pair Learning for Efficient Set-Based Video Recognition
## 1 Introduction  
With the development of imaging technology, people can increasingly easily obtain multimedia data, such as images and videos. Additionally, much multimedia data are usually stored in the form of image sets, such as personal photo albums, multi-view photo albums, and video frame sets. Therefore, studies of set-based video recognition or image set classification methods are needed.  
In set-based video recognition or image set classification tasks, only the spatial information of the video is considered. Thus, the key problems of image set classification (ISC) task are how to model the image sets and how to measure the distance between model representations; additionally, the choice of modeling strategy is the basis of distance measurement. The existing ISC modeling methods can be grouped into two types: static and dynamic modeling methods. Classical static modeling methods include Covariance Discriminative Learning (CDL) [1], Multi-Model Fusion Metric Learning (MMFML) [2], etc. This type of method provides a unique modeling representation for each image set and then performs distance metric learning based on the representation. However, although the calculations of such methods are fast, their classification accuracy is usually uncompeptive.  
Classical dynamic modeling methods include Image-Set-based Collaborative Representation and Classification (ISCRC) [3], Regularized Nearest Points (RNP) [4], etc. With this type of method, sparse representation or collaborative representation is usually used to simultaneously learn the representation coefficients (which is also known as dynamic modeling) and the distance between different image sets. The classification performanceof dynamic modeling methods is usually more accurate than that of the static modeling methods; however, better performance is achieved at the expense of computational efficiency. For example, suppose that we have \(N\) gallery image sets. When classifying one probe image set, the dynamic modeling methods need to solve \(N\) sparse or collaborative optimization problems, which is time-consuming. To reduce the computing time required for solving \(N\) sparse or collaborative optimization problems, [PERSON] et al. proposed the Convolutional Encoder-based Block Sparse Representation (CEBSR) [5] model to learn \(K\) discriminative dictionaries for \(K\) categories. When classifying one probe set, CEBSR only needs to solve \(K\) sparse optimization problems, and \(K<N\) is satisfied. Moreover, CEBSR uses the image set matrix as the feature representation of the probe set, neglecting the importance of set modeling. Additionally, solving the sparse representation problem is time-consuming. Recently, [PERSON] et al. proposed the projective Dictionary Pair Learning (DPL) [6] algorithm, which decomposes the original sparse representation into a linear reconstruction problem by introducing the analysis dictionaries, thereby considerably speeding up the sparse representation computing. Therefore, introducing DPL into the ISC field is a compelling idea.  
To increase the efficiency and accuracy of image set classification, in this study, we developed a new Manifolds-based Low-Rank Dictionary Pair Learning (MbLRDPL) method, which combines the advantages of both static and dynamic modeling methods. Specifically, static modeling (i.e., covariance matrix or linear subspace) is first used to model each image set; second, the Manifolds-based Discriminative Dictionary Learning algorithm, i.e., MbLRDPL, is developed by introducing DPL into the ISC task. In addition, the nuclear norm is imposed on the synthesis dictionaries for further strengthening the discriminative ability. The flowchart of the proposed method is shown in Figure 1.  
The rest of this paper is organized as follows: Section 2 provides an overview of related studies. In Section 3, the MbLRDPL algorithm is proposed. The experiments and results are described in Section 4. A discussion and the conclusions are outlined in Sections 5 and 6, respectively.