Kernel-Based Sparse Representation Learning With Global and Local Low-Rank Label Constraint

Authors:
raw_author_name: Luyao Teng
raw_affiliation_string: School of Information Engineering, Guangzhou Panyu Polytechic, Guangzhou, China
raw_author_name: Feiyi Tang
raw_affiliation_string: School of Information Engineering, Guangzhou Panyu Polytechic, Guangzhou, China
raw_author_name: Zefeng Zheng
raw_affiliation_string: School of Computer Science and Technology, Guangdong University of Technology, Guangzhou, China
raw_author_name: Peipei Kang
raw_affiliation_string: School of Computer Science and Technology, Guangdong University of Technology, Guangzhou, China
raw_author_name: Shaohua Teng
raw_affiliation_string: School of Computer Science and Technology, Guangdong University of Technology, Guangzhou, China

Abstract: Due to the large-scale and multiscale natures of social media data, sparse representation (SR) learning methods are widely followed. However, there are three problems associated with the existing SR methods: 1) they neglect the fact that the semantic features of data may change during iterative learning, which leads to weak semantic learning; 2) they often assume that the data are linearly separable, while the data might be nonlinear in many real-world applications; and 3) they cannot ensure the low-rank and discriminative properties of the data at the same time and might neglect the global properties of the data, leading to suboptimal solutions. To solve these problems, we propose a novel method, named kernel-based SR learning with global and local low-rank label (KSR-GL <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$^3$</tex-math> </inline-formula> ) constraint, which strengthens the semantic information and ensures the semantic features invariant during learning. First, we map the data into a high-dimensional feature space to learn the linear representation of samples. Second, global and local low-rank label (GL <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$^3$</tex-math> </inline-formula> ) constraint is used to ensure the semantic invariance, low-rankness, and discrimination of features during learning. Third, an <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$\ell_{2,1}$</tex-math> </inline-formula> is imposed to explore the sparseness of the subspace. Mathematical analyses show that GL <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$^3$</tex-math> </inline-formula> can retain the intrinsic properties of data during learning. By combining the above three components, a generalized power iteration (GPI) approach is applied to build the model and deal with the tricky optimization problem. By KSR-GL <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$ ^3$</tex-math> </inline-formula> , a sparse, low-rank, and discriminative subspace is produced from the high-dimensional and orthogonal representation of the data under the guidance of semantics, while the intrinsic properties of data are preserved. Extensive experiments on six datasets compared with five advanced algorithms demonstrate its promising prospects.