# Recursive Sample Scaling Low-Rank Representation

Authors: Wenyun Gao; Xiaoyun Li; Sheng Dai; Xinghui Yin; Stanley Ebhohimhen Abhadiomhen
Journal: Journal of Mathematics
 (2021)

---

## Abstract

The low-rank representation (LRR) method has recently gained enormous popularity due to its robust approach in solving the subspace segmentation problem, particularly those concerning corrupted data. In this paper, the recursive sample scaling low-rank representation (RSS-LRR) method is proposed. The advantage of RSS-LRR over traditional LRR is that a cosine scaling factor is further introduced, which imposes a penalty on each sample to minimize noise and outlier influence better. Specifically, the cosine scaling factor is a similarity measure learned to extract each sample’s relationship with the low-rank representation’s principal components in the feature space. In order words, the smaller the angle between an individual data sample and the low-rank representation’s principal components, the more likely it is that the data sample is clean. Thus, the proposed method can then effectively obtain a good low-rank representation influenced mainly by clean data. Several experiments are performed with varying levels of corruption on ORL, CMU PIE, COIL20, COIL100, and LFW in order to evaluate RSS-LRR’s effectiveness over state-of-the-art low-rank methods. The experimental results show that RSS-LRR consistently performs better than the compared methods in image clustering and classification tasks.


---

## Body

## 1. Introduction

The limitations of classical feature learning techniques such as PCA [1] easily made the robust principal component analysis (RPCA) method an efficient choice for dealing with noise and outliers. Specifically, RPCA is focused on learning a low-rank subspace directly from the original high-dimensional data to preserve its geometric structure in a low-dimensional subspace. And this strategy has shown tremendous improvements in several applications [2–5]. However, as RPCA only seeks a single low-rank subspace, it may still be limited with noise damage since high-dimensional data are known to reside in multiple low-dimensional subspaces [6]. Thus, extending RPCA’s idea, Liu et al. [4, 7] proposed a method named “low-rank representation” (LRR). LRR’s main advantage over RPCA lies in its aim to learn data’s multiple low-dimensional subspaces and their membership. This approach makes LRR very robust to the negative effect of noise and outliers [8].Therefore, considering LRR’s robustness mentioned above, several attempts were made in the literature such as references [9–13] to improve its performance. For example, to deal with the data from nonlinear subspaces, Tang et al. [13] proposed robust kernel LRR (RKLRR). Liu et al. [11] then adopted a fixed-rank strategy to accelerate LRR’s computation process. However, their performances could be reduced with insufficient or heavily corrupted samples. That is why Xiao et al. [14] had previously proposed the latent LRR (LatLRR) for joint subspace segmentation and feature selection. The idea behind LatLRR is to include hidden data in constructing the dictionary to improve robustness further. On the contrary, how to handle gross data damage remains unsolved. It is observed that the more the data corrupted by noise, the larger the degradation in the classification and clustering performance.To address this issue, a recursive sample scaling low-rank representation (RSS-LRR) method is proposed in this paper. Since some data samples will be more damaged in gross data corruption, we estimate each data sample’s importance using a cosine scaling factor. This scaling factor measures the angle connecting each data sample and the low-rank representation’s principal components in feature space. In this way, we then iteratively subdue noisy data samples to overcome their effect. Thus, the proposed RSS-LRR can effectively obtain a good low-rank representation than existing methods. Our main contributions are summarized as follows:(1)
We propose a novel method named “RSS-LRR,” which measures each data sample’s importance using a cosine scaling factor. This scaling factor is used in our model to extract each data sample’s relationship with the low-rank matrix’s principal components.(2)
The proposed RSS-LRR method can effectively handle noisy data samples by iteratively restricting noisy data samples using the sample scaling factor to suppress noise so as to obtain a good low-rank representation.(3)
Several experiments are performed with varying levels of corruption to evaluate RSS-LRR’s effectiveness. The experimental results show that RSS-LRR consistently outperforms other state-of-the-art methods in image clustering and classification tasks.


## 2. Related Work

This section presents a brief review of the baseline methods RPCA and LRR. First, in this paper, matrices are written in uppercase, e.g.,X. Thus, XF and X∗ denote the Frobenius norm and nuclear norm, respectively. X2 and X2,1 denote the vector norm and L2,1-norm, which are defined by X1=∑i,jxij and X2,1=∑i∑ixij21/2.


### 2.1. Robust Principal Component Analysis

To recover a subspace structure from corrupted data, RPCA was proposed in [2]. Its strategy is to decompose a given data matrix into two components matrices by solving the following optimization problem:(1)minD,ED∗+λE1s.t.X=D+E,where the data matrix X=x1,x2,…,xn∈ℜm×n is a matrix of n samples in the m-dimensional space, D is the low-rank matrix, E is a sparse error matrix, and λ is the regularized parameter to balance the effects of the two terms. Thus, RPCA’s main objective from the above formula is to obtain low-rank and sparse elements by combining the nuclear norm and L1-norm. This approach is proven to be possible under some assumptions [16]. However, as RPCA assumes a single low-rank subspace, its performance can degrade easily.


### 2.2. Low-Rank Representation

Liu et al. [5] proposed LRR to tackle RPCA’s limitations. Specifically, LRR is focused on pursuing a data representation matrix with the lowest rank. It can achieve that by using data’s self-expressiveness property such that the given data itself are utilized as a self-dictionary. This way, each data sample is then represented as a linear combination of similar samples belonging to the same class. The optimal low-rank matrix obtained by LRR is defined as follows:(2)minZ,EZ∗+λEps.t.X=XZ+E,where the data matrix X=x1,x2,…,xn∈ℜm×n denotes the self-dictionary E is used to capture the error components where .p denotes a certain norm, which can be determined based on the type of noise corruption. For example, while .F is a suitable candidate for data damaged by Gaussian noise, .1 is good for random noise. Besides, .2,1 is an efficient choice when only a part of data are contaminated.Although LRR’s approach is shown to be very effective, particularly in noisy settings, its performance may degrade with insufficient samples. For this reason, Liu et al. [15] also proposed latent LRR that exploits both the observed and the hidden data to construct the self-dictionary. This strategy is most useful for image restoration [17]. Consequently, the work of [18] proposed a method for line pattern noise removal to address contaminated instances. It is realized in a transform domain by using a line pattern’s directional property. Besides, other efforts were made in references [11, 12, 19–31] to improve LRR’s discriminative capability.Notably, Bing-Kun Bao et al. [23] used a fixed-rank approach so as to reduce LRR’s singular value decomposition cost. Zhang et al. [24] proposed two instantaneous methods: the first can reasonably handle noise interference by decomposing given data into two parts, namely, the low-rank sparse principal feature part and a noise-fitting error part. In [25], Tang et al. introduced a diversity regularization and a rank constraint to suppress the redundancy in different data views. In [26], Zhang et al. presented a method that adaptively preserves local information of salient features, thus guaranteeing a block-diagonal coefficient structure. Meanwhile, a compressive robust subspace clustering method was proposed in [27] for dimensionality reduction. However, because the subspace techniques, including LRR, do not provide linear dimensionality reduction (LDR) functionality, the feature selective projection (FSP) [28] was proposed. FSP combines feature extraction, feature selection, and LRR into a unified model to promote robust LDR. Likewise, a method was introduced in [29], which exploits a robust dictionary learning strategy to discover hybrid salient low-rank and sparse representations in a factorized compressed space. Furthermore, in an attempt to keep both similarity and local structures, the hierarchical weighted low-rank representation (HWLRR) [30] was proposed. Similarly, the more recent study [31] was focused on capturing cross-view information through an approach that preserves both diversity and consensus information of each data view.


## 2.1. Robust Principal Component Analysis

To recover a subspace structure from corrupted data, RPCA was proposed in [2]. Its strategy is to decompose a given data matrix into two components matrices by solving the following optimization problem:(1)minD,ED∗+λE1s.t.X=D+E,where the data matrix X=x1,x2,…,xn∈ℜm×n is a matrix of n samples in the m-dimensional space, D is the low-rank matrix, E is a sparse error matrix, and λ is the regularized parameter to balance the effects of the two terms. Thus, RPCA’s main objective from the above formula is to obtain low-rank and sparse elements by combining the nuclear norm and L1-norm. This approach is proven to be possible under some assumptions [16]. However, as RPCA assumes a single low-rank subspace, its performance can degrade easily.


## 2.2. Low-Rank Representation

Liu et al. [5] proposed LRR to tackle RPCA’s limitations. Specifically, LRR is focused on pursuing a data representation matrix with the lowest rank. It can achieve that by using data’s self-expressiveness property such that the given data itself are utilized as a self-dictionary. This way, each data sample is then represented as a linear combination of similar samples belonging to the same class. The optimal low-rank matrix obtained by LRR is defined as follows:(2)minZ,EZ∗+λEps.t.X=XZ+E,where the data matrix X=x1,x2,…,xn∈ℜm×n denotes the self-dictionary E is used to capture the error components where .p denotes a certain norm, which can be determined based on the type of noise corruption. For example, while .F is a suitable candidate for data damaged by Gaussian noise, .1 is good for random noise. Besides, .2,1 is an efficient choice when only a part of data are contaminated.Although LRR’s approach is shown to be very effective, particularly in noisy settings, its performance may degrade with insufficient samples. For this reason, Liu et al. [15] also proposed latent LRR that exploits both the observed and the hidden data to construct the self-dictionary. This strategy is most useful for image restoration [17]. Consequently, the work of [18] proposed a method for line pattern noise removal to address contaminated instances. It is realized in a transform domain by using a line pattern’s directional property. Besides, other efforts were made in references [11, 12, 19–31] to improve LRR’s discriminative capability.Notably, Bing-Kun Bao et al. [23] used a fixed-rank approach so as to reduce LRR’s singular value decomposition cost. Zhang et al. [24] proposed two instantaneous methods: the first can reasonably handle noise interference by decomposing given data into two parts, namely, the low-rank sparse principal feature part and a noise-fitting error part. In [25], Tang et al. introduced a diversity regularization and a rank constraint to suppress the redundancy in different data views. In [26], Zhang et al. presented a method that adaptively preserves local information of salient features, thus guaranteeing a block-diagonal coefficient structure. Meanwhile, a compressive robust subspace clustering method was proposed in [27] for dimensionality reduction. However, because the subspace techniques, including LRR, do not provide linear dimensionality reduction (LDR) functionality, the feature selective projection (FSP) [28] was proposed. FSP combines feature extraction, feature selection, and LRR into a unified model to promote robust LDR. Likewise, a method was introduced in [29], which exploits a robust dictionary learning strategy to discover hybrid salient low-rank and sparse representations in a factorized compressed space. Furthermore, in an attempt to keep both similarity and local structures, the hierarchical weighted low-rank representation (HWLRR) [30] was proposed. Similarly, the more recent study [31] was focused on capturing cross-view information through an approach that preserves both diversity and consensus information of each data view.


## 3. The Proposed Method

### 3.1. RSS-LRR

Since real-world data are not always perfect and inevitably corrupted by noise in practice, most existing low-rank methods cannot guarantee robust performance. Therefore, noise interference must be carefully handled to resolve the present drawback. As a result, a rational solution is pursued in this paper, which uses a cosine scaling factor to estimate the importance of each data sample. Essentially, we suppose that the clean data samples will give high significant values, while the noisy ones would differ from the principal component of the data. Thus, the cosine scaling factorD is introduced into the LRR formulation in equation (2) using the constraint XD=XDZ+E. The motivation behind our approach is straightforward: according to reference [32], Z can be decomposed as UΣVT, where U and V are left and right low-rank singular vectors. In other words, U or V becomes the pursued projection vector such that XUϵRm is the data projection in feature space. Therefore, XU1 is chosen as our maximum projection direction where U1, the maximum eigenvalue of Σ, is U’s column vectors. So, for an outlier data sample xj, the angle between it and the principal component vector XU1 would differ more than that of clean data xi, as described in Figure 1(a). Hence, di expressed in the following is used to estimate the importance of each data sample:(3)di=cosθi+ε=xiTXU1xiXU1+ε,where ε is a constant that stops di from 0. Thus, using the significant factor d, a given data matrix can be scaled to minimize the effect of noisy data samples, allowing the low-rank structure to be realized with clean data as shown in Figure 1(b). As such, the proposed sample scaling low-rank model is obtained as follows:(4)minZ,EZ∗+λE2,1s.t.XD=XDZ+E,where Z denotes the low-rank matrix and E is used to capture the noise elements, similar to that in equation (2). Then, D=d1⋱di⋱dn denotes the scaling factor of n samples.Figure 1
Our proposed model’s framework.From the above formulation, one can easily check that the scaling factorπ/2 suits our goal, as we can then detect the noisy points from closer angles to π/2, with lower values being assigned to such points. Illustratively, let us assume θi=π/2, and di is almost 0, so xi is subdued with xi^=xidi. Thus, X^=XD is used to obtain new training data. Besides, using SVD, X^=XD=UxΣxVxTD=Ux^Σx^Vx^T, meaning that both new singular vectors Ux^ and new projection vectors of sample space are obtained by suppressing the noisy data samples. As a result, our proposed method can then learn an optimal low-rank structure using new data X^ where the points closer to the principal component vector are enhanced.We give the summary of our model’s main characteristics as follows:(i)
Unlike the existing low-rank methods, which use the input dataX itself as the dictionary, a new dictionary is presented with X^=XD by imposing a recursive scaling factor D on X to suppress the effect of noisy samples.(ii)
Specifically, our recursive modeling is very useful for learning a good low-rank representation, most especially when the data are heavily contaminated. As shown in equation (4), our focus is to find Z by minimizing equation (4) with the constraint of X^=X^Z^, thus allowing Z to preserve a better low-rank structure using only data samples with huge cosine similarity (referred to as clean data samples as they would have the smaller angle with the principal component). In other words, Z is obtained by equation (4) using clean data X^.


### 3.2. Optimization

In this section, we propose an optimization algorithm to solve equation (4). First, following standard practice, we introduce a variable J=Z to relax equation (4) further. Thus, equation (4) can be recast as(5)argminJ,Z,EJ∗+λE2,1s.t.XD=XDZ+E,Z=J.The augmented Lagrangian function equation (5) is given as(6)LJ,E,Z,μ=J∗+λE2,1+trM1TXD−XDZ−E+trM2TZ−J+μ2XD−XDZ−EF2+Z−JF2,where M1 and M2 are Lagrange multipliers, μ>0 is a penalty parameter, and ⋅F denotes the Frobenius norm of a matrix. Many concepts for convex optimization have been developed [33–35], which rely on nuclear-norm regularization. And the optimization problem can be solved via the method in [36].


#### 3.2.1. Computation ofJ

According to references [37, 38], nuclear-norm minimization methods have a stable performance. For computing J, we rewrite equation (6) as(7)J∗=argminJJ∗+12J−Z+M2μF2.


#### 3.2.2. Computation ofZ

By fixingJ and E and substituting X^=XD, Z can be updated using the following formula:(8)Z∗=I+X^TX^−1X^TX^−X^TE+J+X^TM1−M2μ.


#### 3.2.3. Computation ofE

WithX^, Z, and J fixed, E can be solved as follows:(9)E∗=argminλμE2,1+12E−X^−X^Z+M1μF2.Following reference [39], equation (9) can be solved by the following lemma.Lemma 1.
(see [40]). Let Q=q1,q2,…,qn be a given matrix. If W∗ is the optimal solution, then(10)W∗=minwλW2,1+12W−QF2.Thus, theith column of W∗ is(11)W∗:,i=qi−λqiqi,ifλ<qi,0,otherwise.Based on Lemma1, supposing we have a matrix Q, W∗ can be reached directly using the above formula, making the E computation process very efficient.The complete solution is given in Algorithm1.Algorithm 1: Algorithm of our proposed RSS-LRR.
(1)
Input: training dataset X, regulation parameter λ(2)
Initialize:t=0, k=0, Jk=0=0, Zk=0=0, Ek=0=0, ϵ1=10−6,ϵ2=10−6, ε=0.0001, D=Diagonesn,1.(3)
While not converged do(4)
UpdateX^ by X^=XD.(5)
Initialize:M1=0, M2=0, μ>0, ρ>0.(6)
Set:Jk=Jk−1, Zk=Zk−1, Ek=Ek−1.(7)
While not converged do(8)
UpdateJ while fixing others by equation (7).J∗=argminjJ∗+1/2J−Z+M2/μF2(9)
UpdateZ while fixing others by equation (8).Z∗=I+X^TX^−1X^TX^−X^TE+J+X^TM1−M2/μ(10)
UpdateE while fixing others by equation (9).E∗=argminλ/μE2,1+1/2E−X^−X^Z+M1/μF2(11)
Update the multipliers.M1=M1+μX^−X^Z−E,M2=M2+μZ−J(12)
Updateμ by μ=minρμ,maxμ(13)
Updatetk=tk+1(14)
Check convergence conditionsX^−X^Zk−Ek∞<ϵ1,Jk−Ek∞<ϵ1(15)
end while(16)
U,S,V=svdZ,U1=U:,1;(17)
UpdateD by di=cosθi+ε and cosθi=xiTXU1/xiXU1.(18)
Updatek=k+1(19)
Check convergence conditions(20)
X^−X^Z−E∞<ϵ2,J−E∞<ϵ2(21)
end while(22)
Output: Z, E


### 3.3. Complexity Analysis

This section gives an analysis of the computational cost of Algorithm1. In Step 6 of Algorithm 1, the values of Jk,Zk,Ek are the same as Jk−1,Zk−1,Ek−1 in the inner loop. Therefore, as the number of iterations k increases, the time complexity in pursuing the low-rank variable Zk decreases faster. From the above discussion, the computing time in tk is far shorter than that in t1. The cost of SVD is On3, where n denotes the number of data vectors. Besides, Z costs On2m+n3 for computation, where m is the data dimension. E in Step 10 then costs Odn. Furthermore, supposing that the D subblock requires k times until convergence, the J,Z,E subproblems will then be calculated for tk iterations. Therefore, based on the number of iterations, the combined cost is Otn3+n2m+dn, such that t is a representation of the number of iterations. Thus, when n≥m, the cost’s upper bound would be Otn3. Accordingly, the overall computational cost of the proposed method is Oktn3.


## 3.1. RSS-LRR

Since real-world data are not always perfect and inevitably corrupted by noise in practice, most existing low-rank methods cannot guarantee robust performance. Therefore, noise interference must be carefully handled to resolve the present drawback. As a result, a rational solution is pursued in this paper, which uses a cosine scaling factor to estimate the importance of each data sample. Essentially, we suppose that the clean data samples will give high significant values, while the noisy ones would differ from the principal component of the data. Thus, the cosine scaling factorD is introduced into the LRR formulation in equation (2) using the constraint XD=XDZ+E. The motivation behind our approach is straightforward: according to reference [32], Z can be decomposed as UΣVT, where U and V are left and right low-rank singular vectors. In other words, U or V becomes the pursued projection vector such that XUϵRm is the data projection in feature space. Therefore, XU1 is chosen as our maximum projection direction where U1, the maximum eigenvalue of Σ, is U’s column vectors. So, for an outlier data sample xj, the angle between it and the principal component vector XU1 would differ more than that of clean data xi, as described in Figure 1(a). Hence, di expressed in the following is used to estimate the importance of each data sample:(3)di=cosθi+ε=xiTXU1xiXU1+ε,where ε is a constant that stops di from 0. Thus, using the significant factor d, a given data matrix can be scaled to minimize the effect of noisy data samples, allowing the low-rank structure to be realized with clean data as shown in Figure 1(b). As such, the proposed sample scaling low-rank model is obtained as follows:(4)minZ,EZ∗+λE2,1s.t.XD=XDZ+E,where Z denotes the low-rank matrix and E is used to capture the noise elements, similar to that in equation (2). Then, D=d1⋱di⋱dn denotes the scaling factor of n samples.Figure 1
Our proposed model’s framework.From the above formulation, one can easily check that the scaling factorπ/2 suits our goal, as we can then detect the noisy points from closer angles to π/2, with lower values being assigned to such points. Illustratively, let us assume θi=π/2, and di is almost 0, so xi is subdued with xi^=xidi. Thus, X^=XD is used to obtain new training data. Besides, using SVD, X^=XD=UxΣxVxTD=Ux^Σx^Vx^T, meaning that both new singular vectors Ux^ and new projection vectors of sample space are obtained by suppressing the noisy data samples. As a result, our proposed method can then learn an optimal low-rank structure using new data X^ where the points closer to the principal component vector are enhanced.We give the summary of our model’s main characteristics as follows:(i)
Unlike the existing low-rank methods, which use the input dataX itself as the dictionary, a new dictionary is presented with X^=XD by imposing a recursive scaling factor D on X to suppress the effect of noisy samples.(ii)
Specifically, our recursive modeling is very useful for learning a good low-rank representation, most especially when the data are heavily contaminated. As shown in equation (4), our focus is to find Z by minimizing equation (4) with the constraint of X^=X^Z^, thus allowing Z to preserve a better low-rank structure using only data samples with huge cosine similarity (referred to as clean data samples as they would have the smaller angle with the principal component). In other words, Z is obtained by equation (4) using clean data X^.


## 3.2. Optimization

In this section, we propose an optimization algorithm to solve equation (4). First, following standard practice, we introduce a variable J=Z to relax equation (4) further. Thus, equation (4) can be recast as(5)argminJ,Z,EJ∗+λE2,1s.t.XD=XDZ+E,Z=J.The augmented Lagrangian function equation (5) is given as(6)LJ,E,Z,μ=J∗+λE2,1+trM1TXD−XDZ−E+trM2TZ−J+μ2XD−XDZ−EF2+Z−JF2,where M1 and M2 are Lagrange multipliers, μ>0 is a penalty parameter, and ⋅F denotes the Frobenius norm of a matrix. Many concepts for convex optimization have been developed [33–35], which rely on nuclear-norm regularization. And the optimization problem can be solved via the method in [36].


### 3.2.1. Computation ofJ

According to references [37, 38], nuclear-norm minimization methods have a stable performance. For computing J, we rewrite equation (6) as(7)J∗=argminJJ∗+12J−Z+M2μF2.


### 3.2.2. Computation ofZ

By fixingJ and E and substituting X^=XD, Z can be updated using the following formula:(8)Z∗=I+X^TX^−1X^TX^−X^TE+J+X^TM1−M2μ.


### 3.2.3. Computation ofE

WithX^, Z, and J fixed, E can be solved as follows:(9)E∗=argminλμE2,1+12E−X^−X^Z+M1μF2.Following reference [39], equation (9) can be solved by the following lemma.Lemma 1.
(see [40]). Let Q=q1,q2,…,qn be a given matrix. If W∗ is the optimal solution, then(10)W∗=minwλW2,1+12W−QF2.Thus, theith column of W∗ is(11)W∗:,i=qi−λqiqi,ifλ<qi,0,otherwise.Based on Lemma1, supposing we have a matrix Q, W∗ can be reached directly using the above formula, making the E computation process very efficient.The complete solution is given in Algorithm1.Algorithm 1: Algorithm of our proposed RSS-LRR.
(1)
Input: training dataset X, regulation parameter λ(2)
Initialize:t=0, k=0, Jk=0=0, Zk=0=0, Ek=0=0, ϵ1=10−6,ϵ2=10−6, ε=0.0001, D=Diagonesn,1.(3)
While not converged do(4)
UpdateX^ by X^=XD.(5)
Initialize:M1=0, M2=0, μ>0, ρ>0.(6)
Set:Jk=Jk−1, Zk=Zk−1, Ek=Ek−1.(7)
While not converged do(8)
UpdateJ while fixing others by equation (7).J∗=argminjJ∗+1/2J−Z+M2/μF2(9)
UpdateZ while fixing others by equation (8).Z∗=I+X^TX^−1X^TX^−X^TE+J+X^TM1−M2/μ(10)
UpdateE while fixing others by equation (9).E∗=argminλ/μE2,1+1/2E−X^−X^Z+M1/μF2(11)
Update the multipliers.M1=M1+μX^−X^Z−E,M2=M2+μZ−J(12)
Updateμ by μ=minρμ,maxμ(13)
Updatetk=tk+1(14)
Check convergence conditionsX^−X^Zk−Ek∞<ϵ1,Jk−Ek∞<ϵ1(15)
end while(16)
U,S,V=svdZ,U1=U:,1;(17)
UpdateD by di=cosθi+ε and cosθi=xiTXU1/xiXU1.(18)
Updatek=k+1(19)
Check convergence conditions(20)
X^−X^Z−E∞<ϵ2,J−E∞<ϵ2(21)
end while(22)
Output: Z, E


## 3.2.1. Computation ofJ

According to references [37, 38], nuclear-norm minimization methods have a stable performance. For computing J, we rewrite equation (6) as(7)J∗=argminJJ∗+12J−Z+M2μF2.


## 3.2.2. Computation ofZ

By fixingJ and E and substituting X^=XD, Z can be updated using the following formula:(8)Z∗=I+X^TX^−1X^TX^−X^TE+J+X^TM1−M2μ.


## 3.2.3. Computation ofE

WithX^, Z, and J fixed, E can be solved as follows:(9)E∗=argminλμE2,1+12E−X^−X^Z+M1μF2.Following reference [39], equation (9) can be solved by the following lemma.Lemma 1.
(see [40]). Let Q=q1,q2,…,qn be a given matrix. If W∗ is the optimal solution, then(10)W∗=minwλW2,1+12W−QF2.Thus, theith column of W∗ is(11)W∗:,i=qi−λqiqi,ifλ<qi,0,otherwise.Based on Lemma1, supposing we have a matrix Q, W∗ can be reached directly using the above formula, making the E computation process very efficient.The complete solution is given in Algorithm1.Algorithm 1: Algorithm of our proposed RSS-LRR.
(1)
Input: training dataset X, regulation parameter λ(2)
Initialize:t=0, k=0, Jk=0=0, Zk=0=0, Ek=0=0, ϵ1=10−6,ϵ2=10−6, ε=0.0001, D=Diagonesn,1.(3)
While not converged do(4)
UpdateX^ by X^=XD.(5)
Initialize:M1=0, M2=0, μ>0, ρ>0.(6)
Set:Jk=Jk−1, Zk=Zk−1, Ek=Ek−1.(7)
While not converged do(8)
UpdateJ while fixing others by equation (7).J∗=argminjJ∗+1/2J−Z+M2/μF2(9)
UpdateZ while fixing others by equation (8).Z∗=I+X^TX^−1X^TX^−X^TE+J+X^TM1−M2/μ(10)
UpdateE while fixing others by equation (9).E∗=argminλ/μE2,1+1/2E−X^−X^Z+M1/μF2(11)
Update the multipliers.M1=M1+μX^−X^Z−E,M2=M2+μZ−J(12)
Updateμ by μ=minρμ,maxμ(13)
Updatetk=tk+1(14)
Check convergence conditionsX^−X^Zk−Ek∞<ϵ1,Jk−Ek∞<ϵ1(15)
end while(16)
U,S,V=svdZ,U1=U:,1;(17)
UpdateD by di=cosθi+ε and cosθi=xiTXU1/xiXU1.(18)
Updatek=k+1(19)
Check convergence conditions(20)
X^−X^Z−E∞<ϵ2,J−E∞<ϵ2(21)
end while(22)
Output: Z, E


## 3.3. Complexity Analysis

This section gives an analysis of the computational cost of Algorithm1. In Step 6 of Algorithm 1, the values of Jk,Zk,Ek are the same as Jk−1,Zk−1,Ek−1 in the inner loop. Therefore, as the number of iterations k increases, the time complexity in pursuing the low-rank variable Zk decreases faster. From the above discussion, the computing time in tk is far shorter than that in t1. The cost of SVD is On3, where n denotes the number of data vectors. Besides, Z costs On2m+n3 for computation, where m is the data dimension. E in Step 10 then costs Odn. Furthermore, supposing that the D subblock requires k times until convergence, the J,Z,E subproblems will then be calculated for tk iterations. Therefore, based on the number of iterations, the combined cost is Otn3+n2m+dn, such that t is a representation of the number of iterations. Thus, when n≥m, the cost’s upper bound would be Otn3. Accordingly, the overall computational cost of the proposed method is Oktn3.


## 4. Experiments

In this section, our proposed RSS-LRR method’s effectiveness is evaluated by comparing it with similar methods such as LRR [5], latent LRR [15], LRRLC, GLRR [8], and GODEC+ [41] in background modeling from video [42] and image denoising [43].


### 4.1. Background Modeling from Surveillance Video

#### 4.1.1. Experimental Settings

This experiment is performed using surveillance video with various illumination settings. It is composed of a chain of 200 grayscale frames of 32 × 32 dimensions. Thus, each algorithm’s effectiveness is evaluated using precision, recall, and F-score metrics, and their parameters are tuned according to the corresponding literature. Precisely, background modeling [44] is measured by manually quoting out the activities. In this experiment, 50% of frames are randomly selected as the training set, while the remaining are treated as the testing set.


#### 4.1.2. Experimental Results

In Figure2, we show each algorithm’s background recovery and activity segmentation performance. Additionally, it can be observed from Table 1 that RSS-LRR outperforms other methods in activity segmentation as it can better generalize to the testing frames. This result further substantiates our sample scaling factor approach’s effectiveness such that a more reliable low-rank object is obtained than that from the compared methods.Figure 2
Results obtained by all algorithms on surveillance video.Table 1
Activity results of all algorithms on surveillance video.
LRRGLRRGODEC+RSS-LRRTrainTestTrainTestTrainTestTrainTestPrecision0.91530.48720.92040.88590.92080.88400.93880.8919Recall0.78970.73580.79370.74780.79400.74920.79990.7594F-score0.84780.58620.85230.81100.85270.81100.86380.8213


### 4.2. Image Clustering with Varying Levels of Noise

#### 4.2.1. Experimental Settings

Here, several experiments are performed on three well-known image datasets, namely, ORL, CMU PIE, and COIL20, to evaluate the effectiveness of the proposed algorithm on image clustering. Each algorithm’s parameters are tuned according to the corresponding literature using the grid search strategy. We give a brief description of each one of these datasets as follows:ORL: it contains face images of ten individuals, with each of them contributing forty distinct images under various conditions such as facial details and different facial expressions.CMU PIE: it is a face image repository with images of sixty-eight individuals with different settings. It includes thirteen different poses, four different expressions, and forty-two different illuminations.COIL20: it is an object image dataset consisting of 20 separate objects. Each object contributes 72 grayscale images, amounting to a total of 1440 images.For each dataset, the images are resized to 32∗32 dimensions in our experiments. As illustrated in Figure 3, these datasets are then corrupted with 5%, 10%, 15%, and 20% random pixel noise to demonstrate each algorithm’s robustness to noise. Thus, a spectral clustering algorithm is applied to the similarity matrix of each algorithm to obtain the clustering results with ten multiple tries to ensure fairness [45].Figure 3
Example of the original image and corrupted one with different degree of noise on the ORL dataset.


#### 4.2.2. Experimental Results

Tables2–4 display the clustering results of each algorithm concerning the accuracy evaluation metric on ORL, CMU PIE, and COIL20, respectively. Therefore, it is obvious to see that the accuracy of our proposed RSS-LRR method consistently beats those of the compared methods in all three datasets. For example, on the ORL dataset, the accuracy of our proposed method is about 1% higher than that of its closest competitor on clean data. Then, gradually increasing the noise level, one may notice that all the algorithms had reduced performance. However, our proposed method shows more robustness than the other methods, especially with 20% noise damage. Let us take, for instance, the clustering result of LRR, which moved from 0.7505 to 0.3497, while that of the proposed method had a lower drop from 0.7609 to 0.5650.Table 2
Clustering results concerning the accuracy of different algorithms on the ORL dataset.
0%5%10%15%20%LRR0.7505 ± 1.650.7335 ± 1.820.6955 ± 1.760.6247 ± 1.750.3497 ± 1.95LRRLC0.7512 ± 1.170.7248 ± 1.690.7053 ± 1.720.6260 ± 1.270.3604 ± 2.34LLRR0.7518 ± 1.700.7371 ± 1.310.7169 ± 1.180.6397 ± 1.670.3953 ± 1.98GLRR0.7460 ± 1.030.7402 ± 0.950.7098 ± 1.480.6650 ± 1.650.5638 ± 1.22GODEC+0.7469 ± 1.270.7344 ± 1.030.7156 ± 1.440.6726 ± 1.760.5413 ± 1.75RSS-LRR0.7609 ± 1.090.7487 ± 1.380.7182 ± 1.120.6825 ± 1.230.5650 ± 1.50Table 3
Clustering results concerning the accuracy of different algorithms on the CMU PIE dataset.
0%5%10%15%20%LRR0.6964 ± 1.030.6906 ± 1.540.6631 ± 1.800.5924 ± 1.810.4703 ± 1.75LRRLC0.7086 ± 1.410.7017 ± 1.840.6777 ± 1.700.6075 ± 1.950.4919 ± 2.07LLRR0.7188 ± 1.270.7083 ± 1.420.6926 ± 1.530.6149 ± 1.960.5118 ± 1.83GLRR0.7212 ± 1.570.7109 ± 1.670.6947 ± 1.030.6776 ± 1.710.5657 ± 1.92GODEC+0.7297 ± 0.880.7147 ± 1.210.7058 ± 1.750.6406 ± 1.970.5422 ± 1.86RSS-LRR0.7308 ± 1.270.7286 ± 1.390.7125 ± 1.480.6834 ± 1.650.5725 ± 1.61Table 4
Clustering results concerning the accuracy of different algorithms on the COIL20 dataset.
0%5%10%15%20%LRR0.7047 ± 1.250.6869 ± 1.550.6203 ± 1.470.4757 ± 1.970.3955 ± 1.91LRRLC0.7158 ± 1.480.6875 ± 1.520.6519 ± 1.600.5291 ± 2.040.4112 ± 2.09LLRR0.7170 ± 1.710.6949 ± 1.690.6618 ± 1.970.5340 ± 1.930.4260 ± 1.99GLRR0.7234 ± 2.020.6976 ± 1.980.6757 ± 1.510.5687 ± 2.380.4718 ± 2.12GODEC+0.7224 ± 1.440.7006 ± 1.560.6822 ± 1.720.5977 ± 2.130.5069 ± 1.86RSS-LRR0.7285 ± 1.660.7103 ± 1.420.7055 ± 1.550.6123 ± 1.890.5417 ± 2.35Similarly, in Tables3 and 4, we can also see that the proposed method maintained its performance over other methods on the CMU PIE and COIL20 datasets. Particularly, RSS-LRR results with a 20% corruption level show that its accuracy is about 4% better than that of GODEC+, which is 0.5069 on the COIL20 dataset. Accordingly, we present the clustering variation graph of each method in Figure 4 that further reveals the robustness of the proposed method to noise. Thus, it is safe to conclude that the proposed method’s performance is steadier than that of the other algorithms, especially at a high level of corruption. We attribute that to our scaling factor approach in iteratively overcoming the noise effect.Figure 4
Clustering variation of different algorithms with varying noise levels on (a) ORL, (b) PIE, and (c) COIL20 datasets.
(a)(b)(c)


### 4.3. Image Recognition with Contiguous Occlusion

#### 4.3.1. Experimental Settings

In order to evaluate the robustness of RSS-LRR on image recognition under different levels of contiguous occlusions in images [19], we randomly add 6×6 and 8×846 block occlusions to each dataset, as illustrated in Figure 5. Therefore, 50% of samples in each dataset are selected as the training set and the rest as the testing set. Besides, we compare our proposed method’s performance with that of similar ones: LRR, LRRLC, latent LRR (LLRR), GLRR, and GODEC+, by adopting relevant experimental settings in Section 4.2. Thus, each algorithm’s classification accuracy is evaluated using the K nearest neighbor (KNN) classifier.Figure 5
Example of the original image and corrupted one with different occlusion sizes on the COIL20 dataset.


#### 4.3.2. Experimental Results

In Table5, the average classification accuracies are obtained on the ORL dataset in two levels of contiguous occlusions. From Table 5, we can see that the accuracy of RSS-LRR is only slightly higher than that of GODEC+. On the contrary, RSS-LRR’s accuracy is significantly better than that of LRRLC, LLRR, and LRR by about 5%, 4%, and 12%, respectively, under 6×6 occlusion. For example, the classification accuracy of RSS-LRR is 0.6015 and 0.4891 in LRR. This indicates that our proposed RSS-LRR can effectively obtain a good low-rank representation than existing methods.Table 5
Classification accuracies on ORL under 6 × 6 and 8 × 8 occlusions.
LRRLRRLCLLRRGLRRGODEC+RSS-LRR6 × 60.48910.54880.55340.57210.59530.60158 × 80.38990.40460.43370.44730.46200.4762From Table6, we can see that the accuracy of RSS-LRR is better than that of LRR by about 6%. Under 6∗6 occlusion, the accuracy of RSS-LRR is better than that of GODEC+ and GLRR by about 2%. Thus, our proposed RSS-LRR demonstrates approximately outstanding effectiveness on classification accuracies among all algorithms. In Table 7, the accuracy of RSS-LRR is higher than that of LRRLC and LRR by about 3% and 6% under 6∗6 occlusion. For example, the classification accuracy of GLRR is 0.6985 and 0.7130 in RSS-LRR, while the result of LRR is 0.6522 under 6∗6 occlusion.Table 6
Classification accuracies on CMU PIE under 6 × 6 and 8 × 8 occlusions.
LRRLRRLCLLRRGLRRGODEC+RSS-LRR6 × 60.78350.79800.80540.82470.82490.84158 × 80.75890.77730.78210.80810.79190.8112Table 7
Classification accuracies on COIL20 under 6 × 6 and 8 × 8 occlusions.
LRRLRRLCLLRRGLRRGODEC+RSS-LRR6 × 60.65220.68430.69120.69850.70560.71308 × 80.62820.63050.65380.64900.64330.6638Figures6(a)–6(f) illustrate the variations of classification accuracies with increasing feature dimensionalities in different occlusions on ORL, CMU PIE, and COIL20 datasets. From Figures 6(a) and 6(b), we can see that RSS-LRR achieves the highest accuracies among all algorithms. In Figures 6(c) and 6(d), we can see that the performance of RSS-LRR becomes better than that of others on the CMU PIE dataset when the feature dimensionality is over 70. From Figures 6(e) and 6(f), on the COIL20 dataset, we can see that the accuracies of RSS-LRR gradually show its superiority when the feature dimensionalities are more than 70.Figure 6
Variations of classification accuracies with increasing feature dimensionalities of6×6 and 8×8 occlusions on (a, b) ORL, (c, d) CMU PIE, and (e, f) COIL20 datasets.
(a)(b)(c)(d)(e)(f)From the above discussion, it can be seen that our proposed RSS-LRR shows better performance in classification accuracy compared with the other algorithms.


### 4.4. Experiments on Large-Scale Dataset

#### 4.4.1. Experimental Settings

In this experiment, RSS-LRR effectiveness is further evaluated on two larger datasets, namely, COIL100 and LFW. Also, the relevant settings from previous sections are adopted to perform the large-scale experiment. We give a brief description of each dataset as follows:COIL100: it has 7200 images of 100 objects, which amount to 72 images for each object with each image taken at pose intervals of 5 degrees.LFW: the Labeled Faces in the Wild (LFW) dataset originally contains more than 13000 face images, mainly from Internet sources. However, 2484 face images were extracted from 38 classes in our experiments due to fewer samples in some categories. Each image was resized to 64×64 pixels, yielding 4096 features per image.


#### 4.4.2. Experimental Results

From the classification results in Tables8 and 9, it can be noticed that RSS-LRR performance is consistently better than that of the compared methods. For instance, while RSS-LRR’s performance of 0.6052 on the COIL100 dataset corrupted with 6 × 6 block occlusion (Table 8) is slightly better than that of the second-best GODEC+ by over 1%, it is far better by over 2% under 8 × 8 block occlusion. Similar results are obtained on the LFW dataset (Table 9), where the proposed method’s performance is also better than that of the GODEC+ method, which follows closely, except that more margin of over 4% is obtained under 6 × 6 block occlusion.Table 8
Classification accuracies on COIL100 under 6 × 6 and 8 × 8 occlusions.
LRRLRRLCLLRRGLRRGODEC+RSS-LRR6 × 60.47650.53690.55010.56890.59120.60528 × 80.38120.39910.42870.44540.45880.4803Table 9
Classification accuracies on LFW under 6 × 6 and 8 × 8 occlusions.
LRRLRRLCLLRRGLRRGODEC+RSS-LRR6 × 60.51060.55620.57320.58990.60210.64578 × 80.40310.43210.44160.45410.49530.5276From the clustering results displayed in Tables10 and 11, the following can be observed:(i)
Although all methods obtained comparative performances on both datasets, their accuracies degrade significantly by increasing noise. This, however, is not unexpected because more corruption levels would mean that more discriminative data features are destroyed, making it difficult to accurately group similar data samples in the same cluster.(ii)
Furthermore, while the relatively newer method GODEC+ shows more robustness to noise than the older methods, its clustering accuracy on clean data is slightly lower than that of the other methods on the LFW dataset, perhaps due to class imbalance in this dataset.(iii)
Overall, RSS-LRR’s robustness to noise grows stronger with the increase in noise level. For example, its performance on COIL100 under 0% noise is merely 1% better than that of its closest competitor GODEC+, but it is over 2% better under 20% noise. The same can be said on the LFW dataset, where RSS-LRR’s clustering accuracy is only about 2% better than that of LRR on clean data, whereas it is more than 4% better than that of GODEC+, which is the closest result under the 20% noise level.Table 10
Clustering results concerning the accuracy of different algorithms on the COIL100 dataset.
0%5%10%15%20%LRR0.6854 ± 1.530.6741 ± 1.280.6376 ± 1.890.4964 ± 2.610.4236 ± 1.42LRRLC0.6931 ± 1.370.6823 ± 1.430.6423 ± 1.590.5191 ± 1.870.4243 ± 1.87LLRR0.7043 ± 1.460.6812 ± 1.530.6442 ± 1.640.5230 ± 2.010.4195 ± 1.78GLRR0.7102 ± 1.670.6854 ± 1.850.6532 ± 1.490.5694 ± 2.380.4832 ± 2.05GODEC+0.7156 ± 1.870.7001 ± 1.470.6588 ± 1.840.5854 ± 2.130.5154 ± 1.21RSS-LRR0.7264 ± 1.310.7113 ± 1.440.7015 ± 1.180.6103 ± 1.420.5417 ± 1.61Table 11
Clustering results concerning the accuracy of different algorithms on the LFW dataset.
0%5%10%15%20%LRR0.7605 ± 1.530.7305 ± 1.840.7122 ± 1.760.6263 ± 1.750.3547 ± 1.65LRRLC0.7512 ± 1.530.7298 ± 1.260.7099 ± 1.720.6310 ± 1.270.3864 ± 2.14LLRR0.7518 ± 1.320.7431 ± 2.170.7169 ± 2.180.6397 ± 1.740.4311 ± 1.98GLRR0.7560 ± 1.030.7457 ± 1.970.7098 ± 1.530.6780 ± 1.850.5256 ± 1.87GODEC+0.7491 ± 1.270.7434 ± 1.920.7156 ± 1.440.6856 ± 1.790.5413 ± 1.34RSS-LRR0.7832 ± 1.360.7567 ± 1.720.7269 ± 1.840.6952 ± 1.680.5893 ± 1.76Additionally, in order to demonstrate more novelty, RSS-LRR’s clustering and classification performances on large-scale datasets are further compared with those of two more recent state-of-the-art (SOTA) methods, namely, nonnegative sparse discriminative low-rank representation (NSDLRR) [47] and low-rank and collaborative representations for hyperspectral anomaly detection (LRCRD) [48]. From the classification results displayed in Figure 7, it can be observed that all three methods obtain correlative results on both datasets. However, RSS-LRR shows more robustness, especially on the LFW dataset. Similarly, the clustering results are shown in Figure 8, and the results are also close with those of the proposed method displaying the best overall performance.Figure 7
Classification results concerning the accuracy of different SOTA methods on (a) COIL100 and (b) LFW datasets under various block occlusion levels.
(a)(b)Figure 8
Clustering results concerning the accuracy of different SOTA methods on (a) COIL100 and (b) LFW datasets under various noise levels.
(a)(b)


### 4.5. Convergence Study

Based on established standards of inexact augmented Lagrange multiplier (IALM) optimization strategy in several pieces of literature [5, 49–51], the objective function values of the J, Z, and E subproblems were expected to decrease monotonically in each iteration until convergence. Besides, the Z subproblem is known to have a closed-form solution [51] since it has a convex function. Also, the convergence of the J and E subproblems is confirmed by references [36, 52–54], respectively. Thus, a slow convergence is supposed when everything is put together. Luckily, as shown in Figure 9, the proposed algorithm has a strong convergence property, as it converges within 150 iterations on COIL100 and ORL datasets.Figure 9
Convergence curve of the proposed algorithm on (a) COIL100 and (b) ORL corrupted with 6 × 6 block occlusion.
(a)(b)


## 4.1. Background Modeling from Surveillance Video

### 4.1.1. Experimental Settings

This experiment is performed using surveillance video with various illumination settings. It is composed of a chain of 200 grayscale frames of 32 × 32 dimensions. Thus, each algorithm’s effectiveness is evaluated using precision, recall, and F-score metrics, and their parameters are tuned according to the corresponding literature. Precisely, background modeling [44] is measured by manually quoting out the activities. In this experiment, 50% of frames are randomly selected as the training set, while the remaining are treated as the testing set.


### 4.1.2. Experimental Results

In Figure2, we show each algorithm’s background recovery and activity segmentation performance. Additionally, it can be observed from Table 1 that RSS-LRR outperforms other methods in activity segmentation as it can better generalize to the testing frames. This result further substantiates our sample scaling factor approach’s effectiveness such that a more reliable low-rank object is obtained than that from the compared methods.Figure 2
Results obtained by all algorithms on surveillance video.Table 1
Activity results of all algorithms on surveillance video.
LRRGLRRGODEC+RSS-LRRTrainTestTrainTestTrainTestTrainTestPrecision0.91530.48720.92040.88590.92080.88400.93880.8919Recall0.78970.73580.79370.74780.79400.74920.79990.7594F-score0.84780.58620.85230.81100.85270.81100.86380.8213


## 4.1.1. Experimental Settings

This experiment is performed using surveillance video with various illumination settings. It is composed of a chain of 200 grayscale frames of 32 × 32 dimensions. Thus, each algorithm’s effectiveness is evaluated using precision, recall, and F-score metrics, and their parameters are tuned according to the corresponding literature. Precisely, background modeling [44] is measured by manually quoting out the activities. In this experiment, 50% of frames are randomly selected as the training set, while the remaining are treated as the testing set.


## 4.1.2. Experimental Results

In Figure2, we show each algorithm’s background recovery and activity segmentation performance. Additionally, it can be observed from Table 1 that RSS-LRR outperforms other methods in activity segmentation as it can better generalize to the testing frames. This result further substantiates our sample scaling factor approach’s effectiveness such that a more reliable low-rank object is obtained than that from the compared methods.Figure 2
Results obtained by all algorithms on surveillance video.Table 1
Activity results of all algorithms on surveillance video.
LRRGLRRGODEC+RSS-LRRTrainTestTrainTestTrainTestTrainTestPrecision0.91530.48720.92040.88590.92080.88400.93880.8919Recall0.78970.73580.79370.74780.79400.74920.79990.7594F-score0.84780.58620.85230.81100.85270.81100.86380.8213


## 4.2. Image Clustering with Varying Levels of Noise

### 4.2.1. Experimental Settings

Here, several experiments are performed on three well-known image datasets, namely, ORL, CMU PIE, and COIL20, to evaluate the effectiveness of the proposed algorithm on image clustering. Each algorithm’s parameters are tuned according to the corresponding literature using the grid search strategy. We give a brief description of each one of these datasets as follows:ORL: it contains face images of ten individuals, with each of them contributing forty distinct images under various conditions such as facial details and different facial expressions.CMU PIE: it is a face image repository with images of sixty-eight individuals with different settings. It includes thirteen different poses, four different expressions, and forty-two different illuminations.COIL20: it is an object image dataset consisting of 20 separate objects. Each object contributes 72 grayscale images, amounting to a total of 1440 images.For each dataset, the images are resized to 32∗32 dimensions in our experiments. As illustrated in Figure 3, these datasets are then corrupted with 5%, 10%, 15%, and 20% random pixel noise to demonstrate each algorithm’s robustness to noise. Thus, a spectral clustering algorithm is applied to the similarity matrix of each algorithm to obtain the clustering results with ten multiple tries to ensure fairness [45].Figure 3
Example of the original image and corrupted one with different degree of noise on the ORL dataset.


### 4.2.2. Experimental Results

Tables2–4 display the clustering results of each algorithm concerning the accuracy evaluation metric on ORL, CMU PIE, and COIL20, respectively. Therefore, it is obvious to see that the accuracy of our proposed RSS-LRR method consistently beats those of the compared methods in all three datasets. For example, on the ORL dataset, the accuracy of our proposed method is about 1% higher than that of its closest competitor on clean data. Then, gradually increasing the noise level, one may notice that all the algorithms had reduced performance. However, our proposed method shows more robustness than the other methods, especially with 20% noise damage. Let us take, for instance, the clustering result of LRR, which moved from 0.7505 to 0.3497, while that of the proposed method had a lower drop from 0.7609 to 0.5650.Table 2
Clustering results concerning the accuracy of different algorithms on the ORL dataset.
0%5%10%15%20%LRR0.7505 ± 1.650.7335 ± 1.820.6955 ± 1.760.6247 ± 1.750.3497 ± 1.95LRRLC0.7512 ± 1.170.7248 ± 1.690.7053 ± 1.720.6260 ± 1.270.3604 ± 2.34LLRR0.7518 ± 1.700.7371 ± 1.310.7169 ± 1.180.6397 ± 1.670.3953 ± 1.98GLRR0.7460 ± 1.030.7402 ± 0.950.7098 ± 1.480.6650 ± 1.650.5638 ± 1.22GODEC+0.7469 ± 1.270.7344 ± 1.030.7156 ± 1.440.6726 ± 1.760.5413 ± 1.75RSS-LRR0.7609 ± 1.090.7487 ± 1.380.7182 ± 1.120.6825 ± 1.230.5650 ± 1.50Table 3
Clustering results concerning the accuracy of different algorithms on the CMU PIE dataset.
0%5%10%15%20%LRR0.6964 ± 1.030.6906 ± 1.540.6631 ± 1.800.5924 ± 1.810.4703 ± 1.75LRRLC0.7086 ± 1.410.7017 ± 1.840.6777 ± 1.700.6075 ± 1.950.4919 ± 2.07LLRR0.7188 ± 1.270.7083 ± 1.420.6926 ± 1.530.6149 ± 1.960.5118 ± 1.83GLRR0.7212 ± 1.570.7109 ± 1.670.6947 ± 1.030.6776 ± 1.710.5657 ± 1.92GODEC+0.7297 ± 0.880.7147 ± 1.210.7058 ± 1.750.6406 ± 1.970.5422 ± 1.86RSS-LRR0.7308 ± 1.270.7286 ± 1.390.7125 ± 1.480.6834 ± 1.650.5725 ± 1.61Table 4
Clustering results concerning the accuracy of different algorithms on the COIL20 dataset.
0%5%10%15%20%LRR0.7047 ± 1.250.6869 ± 1.550.6203 ± 1.470.4757 ± 1.970.3955 ± 1.91LRRLC0.7158 ± 1.480.6875 ± 1.520.6519 ± 1.600.5291 ± 2.040.4112 ± 2.09LLRR0.7170 ± 1.710.6949 ± 1.690.6618 ± 1.970.5340 ± 1.930.4260 ± 1.99GLRR0.7234 ± 2.020.6976 ± 1.980.6757 ± 1.510.5687 ± 2.380.4718 ± 2.12GODEC+0.7224 ± 1.440.7006 ± 1.560.6822 ± 1.720.5977 ± 2.130.5069 ± 1.86RSS-LRR0.7285 ± 1.660.7103 ± 1.420.7055 ± 1.550.6123 ± 1.890.5417 ± 2.35Similarly, in Tables3 and 4, we can also see that the proposed method maintained its performance over other methods on the CMU PIE and COIL20 datasets. Particularly, RSS-LRR results with a 20% corruption level show that its accuracy is about 4% better than that of GODEC+, which is 0.5069 on the COIL20 dataset. Accordingly, we present the clustering variation graph of each method in Figure 4 that further reveals the robustness of the proposed method to noise. Thus, it is safe to conclude that the proposed method’s performance is steadier than that of the other algorithms, especially at a high level of corruption. We attribute that to our scaling factor approach in iteratively overcoming the noise effect.Figure 4
Clustering variation of different algorithms with varying noise levels on (a) ORL, (b) PIE, and (c) COIL20 datasets.
(a)(b)(c)


## 4.2.1. Experimental Settings

Here, several experiments are performed on three well-known image datasets, namely, ORL, CMU PIE, and COIL20, to evaluate the effectiveness of the proposed algorithm on image clustering. Each algorithm’s parameters are tuned according to the corresponding literature using the grid search strategy. We give a brief description of each one of these datasets as follows:ORL: it contains face images of ten individuals, with each of them contributing forty distinct images under various conditions such as facial details and different facial expressions.CMU PIE: it is a face image repository with images of sixty-eight individuals with different settings. It includes thirteen different poses, four different expressions, and forty-two different illuminations.COIL20: it is an object image dataset consisting of 20 separate objects. Each object contributes 72 grayscale images, amounting to a total of 1440 images.For each dataset, the images are resized to 32∗32 dimensions in our experiments. As illustrated in Figure 3, these datasets are then corrupted with 5%, 10%, 15%, and 20% random pixel noise to demonstrate each algorithm’s robustness to noise. Thus, a spectral clustering algorithm is applied to the similarity matrix of each algorithm to obtain the clustering results with ten multiple tries to ensure fairness [45].Figure 3
Example of the original image and corrupted one with different degree of noise on the ORL dataset.


## 4.2.2. Experimental Results

Tables2–4 display the clustering results of each algorithm concerning the accuracy evaluation metric on ORL, CMU PIE, and COIL20, respectively. Therefore, it is obvious to see that the accuracy of our proposed RSS-LRR method consistently beats those of the compared methods in all three datasets. For example, on the ORL dataset, the accuracy of our proposed method is about 1% higher than that of its closest competitor on clean data. Then, gradually increasing the noise level, one may notice that all the algorithms had reduced performance. However, our proposed method shows more robustness than the other methods, especially with 20% noise damage. Let us take, for instance, the clustering result of LRR, which moved from 0.7505 to 0.3497, while that of the proposed method had a lower drop from 0.7609 to 0.5650.Table 2
Clustering results concerning the accuracy of different algorithms on the ORL dataset.
0%5%10%15%20%LRR0.7505 ± 1.650.7335 ± 1.820.6955 ± 1.760.6247 ± 1.750.3497 ± 1.95LRRLC0.7512 ± 1.170.7248 ± 1.690.7053 ± 1.720.6260 ± 1.270.3604 ± 2.34LLRR0.7518 ± 1.700.7371 ± 1.310.7169 ± 1.180.6397 ± 1.670.3953 ± 1.98GLRR0.7460 ± 1.030.7402 ± 0.950.7098 ± 1.480.6650 ± 1.650.5638 ± 1.22GODEC+0.7469 ± 1.270.7344 ± 1.030.7156 ± 1.440.6726 ± 1.760.5413 ± 1.75RSS-LRR0.7609 ± 1.090.7487 ± 1.380.7182 ± 1.120.6825 ± 1.230.5650 ± 1.50Table 3
Clustering results concerning the accuracy of different algorithms on the CMU PIE dataset.
0%5%10%15%20%LRR0.6964 ± 1.030.6906 ± 1.540.6631 ± 1.800.5924 ± 1.810.4703 ± 1.75LRRLC0.7086 ± 1.410.7017 ± 1.840.6777 ± 1.700.6075 ± 1.950.4919 ± 2.07LLRR0.7188 ± 1.270.7083 ± 1.420.6926 ± 1.530.6149 ± 1.960.5118 ± 1.83GLRR0.7212 ± 1.570.7109 ± 1.670.6947 ± 1.030.6776 ± 1.710.5657 ± 1.92GODEC+0.7297 ± 0.880.7147 ± 1.210.7058 ± 1.750.6406 ± 1.970.5422 ± 1.86RSS-LRR0.7308 ± 1.270.7286 ± 1.390.7125 ± 1.480.6834 ± 1.650.5725 ± 1.61Table 4
Clustering results concerning the accuracy of different algorithms on the COIL20 dataset.
0%5%10%15%20%LRR0.7047 ± 1.250.6869 ± 1.550.6203 ± 1.470.4757 ± 1.970.3955 ± 1.91LRRLC0.7158 ± 1.480.6875 ± 1.520.6519 ± 1.600.5291 ± 2.040.4112 ± 2.09LLRR0.7170 ± 1.710.6949 ± 1.690.6618 ± 1.970.5340 ± 1.930.4260 ± 1.99GLRR0.7234 ± 2.020.6976 ± 1.980.6757 ± 1.510.5687 ± 2.380.4718 ± 2.12GODEC+0.7224 ± 1.440.7006 ± 1.560.6822 ± 1.720.5977 ± 2.130.5069 ± 1.86RSS-LRR0.7285 ± 1.660.7103 ± 1.420.7055 ± 1.550.6123 ± 1.890.5417 ± 2.35Similarly, in Tables3 and 4, we can also see that the proposed method maintained its performance over other methods on the CMU PIE and COIL20 datasets. Particularly, RSS-LRR results with a 20% corruption level show that its accuracy is about 4% better than that of GODEC+, which is 0.5069 on the COIL20 dataset. Accordingly, we present the clustering variation graph of each method in Figure 4 that further reveals the robustness of the proposed method to noise. Thus, it is safe to conclude that the proposed method’s performance is steadier than that of the other algorithms, especially at a high level of corruption. We attribute that to our scaling factor approach in iteratively overcoming the noise effect.Figure 4
Clustering variation of different algorithms with varying noise levels on (a) ORL, (b) PIE, and (c) COIL20 datasets.
(a)(b)(c)


## 4.3. Image Recognition with Contiguous Occlusion

### 4.3.1. Experimental Settings

In order to evaluate the robustness of RSS-LRR on image recognition under different levels of contiguous occlusions in images [19], we randomly add 6×6 and 8×846 block occlusions to each dataset, as illustrated in Figure 5. Therefore, 50% of samples in each dataset are selected as the training set and the rest as the testing set. Besides, we compare our proposed method’s performance with that of similar ones: LRR, LRRLC, latent LRR (LLRR), GLRR, and GODEC+, by adopting relevant experimental settings in Section 4.2. Thus, each algorithm’s classification accuracy is evaluated using the K nearest neighbor (KNN) classifier.Figure 5
Example of the original image and corrupted one with different occlusion sizes on the COIL20 dataset.


### 4.3.2. Experimental Results

In Table5, the average classification accuracies are obtained on the ORL dataset in two levels of contiguous occlusions. From Table 5, we can see that the accuracy of RSS-LRR is only slightly higher than that of GODEC+. On the contrary, RSS-LRR’s accuracy is significantly better than that of LRRLC, LLRR, and LRR by about 5%, 4%, and 12%, respectively, under 6×6 occlusion. For example, the classification accuracy of RSS-LRR is 0.6015 and 0.4891 in LRR. This indicates that our proposed RSS-LRR can effectively obtain a good low-rank representation than existing methods.Table 5
Classification accuracies on ORL under 6 × 6 and 8 × 8 occlusions.
LRRLRRLCLLRRGLRRGODEC+RSS-LRR6 × 60.48910.54880.55340.57210.59530.60158 × 80.38990.40460.43370.44730.46200.4762From Table6, we can see that the accuracy of RSS-LRR is better than that of LRR by about 6%. Under 6∗6 occlusion, the accuracy of RSS-LRR is better than that of GODEC+ and GLRR by about 2%. Thus, our proposed RSS-LRR demonstrates approximately outstanding effectiveness on classification accuracies among all algorithms. In Table 7, the accuracy of RSS-LRR is higher than that of LRRLC and LRR by about 3% and 6% under 6∗6 occlusion. For example, the classification accuracy of GLRR is 0.6985 and 0.7130 in RSS-LRR, while the result of LRR is 0.6522 under 6∗6 occlusion.Table 6
Classification accuracies on CMU PIE under 6 × 6 and 8 × 8 occlusions.
LRRLRRLCLLRRGLRRGODEC+RSS-LRR6 × 60.78350.79800.80540.82470.82490.84158 × 80.75890.77730.78210.80810.79190.8112Table 7
Classification accuracies on COIL20 under 6 × 6 and 8 × 8 occlusions.
LRRLRRLCLLRRGLRRGODEC+RSS-LRR6 × 60.65220.68430.69120.69850.70560.71308 × 80.62820.63050.65380.64900.64330.6638Figures6(a)–6(f) illustrate the variations of classification accuracies with increasing feature dimensionalities in different occlusions on ORL, CMU PIE, and COIL20 datasets. From Figures 6(a) and 6(b), we can see that RSS-LRR achieves the highest accuracies among all algorithms. In Figures 6(c) and 6(d), we can see that the performance of RSS-LRR becomes better than that of others on the CMU PIE dataset when the feature dimensionality is over 70. From Figures 6(e) and 6(f), on the COIL20 dataset, we can see that the accuracies of RSS-LRR gradually show its superiority when the feature dimensionalities are more than 70.Figure 6
Variations of classification accuracies with increasing feature dimensionalities of6×6 and 8×8 occlusions on (a, b) ORL, (c, d) CMU PIE, and (e, f) COIL20 datasets.
(a)(b)(c)(d)(e)(f)From the above discussion, it can be seen that our proposed RSS-LRR shows better performance in classification accuracy compared with the other algorithms.


## 4.3.1. Experimental Settings

In order to evaluate the robustness of RSS-LRR on image recognition under different levels of contiguous occlusions in images [19], we randomly add 6×6 and 8×846 block occlusions to each dataset, as illustrated in Figure 5. Therefore, 50% of samples in each dataset are selected as the training set and the rest as the testing set. Besides, we compare our proposed method’s performance with that of similar ones: LRR, LRRLC, latent LRR (LLRR), GLRR, and GODEC+, by adopting relevant experimental settings in Section 4.2. Thus, each algorithm’s classification accuracy is evaluated using the K nearest neighbor (KNN) classifier.Figure 5
Example of the original image and corrupted one with different occlusion sizes on the COIL20 dataset.


## 4.3.2. Experimental Results

In Table5, the average classification accuracies are obtained on the ORL dataset in two levels of contiguous occlusions. From Table 5, we can see that the accuracy of RSS-LRR is only slightly higher than that of GODEC+. On the contrary, RSS-LRR’s accuracy is significantly better than that of LRRLC, LLRR, and LRR by about 5%, 4%, and 12%, respectively, under 6×6 occlusion. For example, the classification accuracy of RSS-LRR is 0.6015 and 0.4891 in LRR. This indicates that our proposed RSS-LRR can effectively obtain a good low-rank representation than existing methods.Table 5
Classification accuracies on ORL under 6 × 6 and 8 × 8 occlusions.
LRRLRRLCLLRRGLRRGODEC+RSS-LRR6 × 60.48910.54880.55340.57210.59530.60158 × 80.38990.40460.43370.44730.46200.4762From Table6, we can see that the accuracy of RSS-LRR is better than that of LRR by about 6%. Under 6∗6 occlusion, the accuracy of RSS-LRR is better than that of GODEC+ and GLRR by about 2%. Thus, our proposed RSS-LRR demonstrates approximately outstanding effectiveness on classification accuracies among all algorithms. In Table 7, the accuracy of RSS-LRR is higher than that of LRRLC and LRR by about 3% and 6% under 6∗6 occlusion. For example, the classification accuracy of GLRR is 0.6985 and 0.7130 in RSS-LRR, while the result of LRR is 0.6522 under 6∗6 occlusion.Table 6
Classification accuracies on CMU PIE under 6 × 6 and 8 × 8 occlusions.
LRRLRRLCLLRRGLRRGODEC+RSS-LRR6 × 60.78350.79800.80540.82470.82490.84158 × 80.75890.77730.78210.80810.79190.8112Table 7
Classification accuracies on COIL20 under 6 × 6 and 8 × 8 occlusions.
LRRLRRLCLLRRGLRRGODEC+RSS-LRR6 × 60.65220.68430.69120.69850.70560.71308 × 80.62820.63050.65380.64900.64330.6638Figures6(a)–6(f) illustrate the variations of classification accuracies with increasing feature dimensionalities in different occlusions on ORL, CMU PIE, and COIL20 datasets. From Figures 6(a) and 6(b), we can see that RSS-LRR achieves the highest accuracies among all algorithms. In Figures 6(c) and 6(d), we can see that the performance of RSS-LRR becomes better than that of others on the CMU PIE dataset when the feature dimensionality is over 70. From Figures 6(e) and 6(f), on the COIL20 dataset, we can see that the accuracies of RSS-LRR gradually show its superiority when the feature dimensionalities are more than 70.Figure 6
Variations of classification accuracies with increasing feature dimensionalities of6×6 and 8×8 occlusions on (a, b) ORL, (c, d) CMU PIE, and (e, f) COIL20 datasets.
(a)(b)(c)(d)(e)(f)From the above discussion, it can be seen that our proposed RSS-LRR shows better performance in classification accuracy compared with the other algorithms.


## 4.4. Experiments on Large-Scale Dataset

### 4.4.1. Experimental Settings

In this experiment, RSS-LRR effectiveness is further evaluated on two larger datasets, namely, COIL100 and LFW. Also, the relevant settings from previous sections are adopted to perform the large-scale experiment. We give a brief description of each dataset as follows:COIL100: it has 7200 images of 100 objects, which amount to 72 images for each object with each image taken at pose intervals of 5 degrees.LFW: the Labeled Faces in the Wild (LFW) dataset originally contains more than 13000 face images, mainly from Internet sources. However, 2484 face images were extracted from 38 classes in our experiments due to fewer samples in some categories. Each image was resized to 64×64 pixels, yielding 4096 features per image.


### 4.4.2. Experimental Results

From the classification results in Tables8 and 9, it can be noticed that RSS-LRR performance is consistently better than that of the compared methods. For instance, while RSS-LRR’s performance of 0.6052 on the COIL100 dataset corrupted with 6 × 6 block occlusion (Table 8) is slightly better than that of the second-best GODEC+ by over 1%, it is far better by over 2% under 8 × 8 block occlusion. Similar results are obtained on the LFW dataset (Table 9), where the proposed method’s performance is also better than that of the GODEC+ method, which follows closely, except that more margin of over 4% is obtained under 6 × 6 block occlusion.Table 8
Classification accuracies on COIL100 under 6 × 6 and 8 × 8 occlusions.
LRRLRRLCLLRRGLRRGODEC+RSS-LRR6 × 60.47650.53690.55010.56890.59120.60528 × 80.38120.39910.42870.44540.45880.4803Table 9
Classification accuracies on LFW under 6 × 6 and 8 × 8 occlusions.
LRRLRRLCLLRRGLRRGODEC+RSS-LRR6 × 60.51060.55620.57320.58990.60210.64578 × 80.40310.43210.44160.45410.49530.5276From the clustering results displayed in Tables10 and 11, the following can be observed:(i)
Although all methods obtained comparative performances on both datasets, their accuracies degrade significantly by increasing noise. This, however, is not unexpected because more corruption levels would mean that more discriminative data features are destroyed, making it difficult to accurately group similar data samples in the same cluster.(ii)
Furthermore, while the relatively newer method GODEC+ shows more robustness to noise than the older methods, its clustering accuracy on clean data is slightly lower than that of the other methods on the LFW dataset, perhaps due to class imbalance in this dataset.(iii)
Overall, RSS-LRR’s robustness to noise grows stronger with the increase in noise level. For example, its performance on COIL100 under 0% noise is merely 1% better than that of its closest competitor GODEC+, but it is over 2% better under 20% noise. The same can be said on the LFW dataset, where RSS-LRR’s clustering accuracy is only about 2% better than that of LRR on clean data, whereas it is more than 4% better than that of GODEC+, which is the closest result under the 20% noise level.Table 10
Clustering results concerning the accuracy of different algorithms on the COIL100 dataset.
0%5%10%15%20%LRR0.6854 ± 1.530.6741 ± 1.280.6376 ± 1.890.4964 ± 2.610.4236 ± 1.42LRRLC0.6931 ± 1.370.6823 ± 1.430.6423 ± 1.590.5191 ± 1.870.4243 ± 1.87LLRR0.7043 ± 1.460.6812 ± 1.530.6442 ± 1.640.5230 ± 2.010.4195 ± 1.78GLRR0.7102 ± 1.670.6854 ± 1.850.6532 ± 1.490.5694 ± 2.380.4832 ± 2.05GODEC+0.7156 ± 1.870.7001 ± 1.470.6588 ± 1.840.5854 ± 2.130.5154 ± 1.21RSS-LRR0.7264 ± 1.310.7113 ± 1.440.7015 ± 1.180.6103 ± 1.420.5417 ± 1.61Table 11
Clustering results concerning the accuracy of different algorithms on the LFW dataset.
0%5%10%15%20%LRR0.7605 ± 1.530.7305 ± 1.840.7122 ± 1.760.6263 ± 1.750.3547 ± 1.65LRRLC0.7512 ± 1.530.7298 ± 1.260.7099 ± 1.720.6310 ± 1.270.3864 ± 2.14LLRR0.7518 ± 1.320.7431 ± 2.170.7169 ± 2.180.6397 ± 1.740.4311 ± 1.98GLRR0.7560 ± 1.030.7457 ± 1.970.7098 ± 1.530.6780 ± 1.850.5256 ± 1.87GODEC+0.7491 ± 1.270.7434 ± 1.920.7156 ± 1.440.6856 ± 1.790.5413 ± 1.34RSS-LRR0.7832 ± 1.360.7567 ± 1.720.7269 ± 1.840.6952 ± 1.680.5893 ± 1.76Additionally, in order to demonstrate more novelty, RSS-LRR’s clustering and classification performances on large-scale datasets are further compared with those of two more recent state-of-the-art (SOTA) methods, namely, nonnegative sparse discriminative low-rank representation (NSDLRR) [47] and low-rank and collaborative representations for hyperspectral anomaly detection (LRCRD) [48]. From the classification results displayed in Figure 7, it can be observed that all three methods obtain correlative results on both datasets. However, RSS-LRR shows more robustness, especially on the LFW dataset. Similarly, the clustering results are shown in Figure 8, and the results are also close with those of the proposed method displaying the best overall performance.Figure 7
Classification results concerning the accuracy of different SOTA methods on (a) COIL100 and (b) LFW datasets under various block occlusion levels.
(a)(b)Figure 8
Clustering results concerning the accuracy of different SOTA methods on (a) COIL100 and (b) LFW datasets under various noise levels.
(a)(b)


## 4.4.1. Experimental Settings

In this experiment, RSS-LRR effectiveness is further evaluated on two larger datasets, namely, COIL100 and LFW. Also, the relevant settings from previous sections are adopted to perform the large-scale experiment. We give a brief description of each dataset as follows:COIL100: it has 7200 images of 100 objects, which amount to 72 images for each object with each image taken at pose intervals of 5 degrees.LFW: the Labeled Faces in the Wild (LFW) dataset originally contains more than 13000 face images, mainly from Internet sources. However, 2484 face images were extracted from 38 classes in our experiments due to fewer samples in some categories. Each image was resized to 64×64 pixels, yielding 4096 features per image.


## 4.4.2. Experimental Results

From the classification results in Tables8 and 9, it can be noticed that RSS-LRR performance is consistently better than that of the compared methods. For instance, while RSS-LRR’s performance of 0.6052 on the COIL100 dataset corrupted with 6 × 6 block occlusion (Table 8) is slightly better than that of the second-best GODEC+ by over 1%, it is far better by over 2% under 8 × 8 block occlusion. Similar results are obtained on the LFW dataset (Table 9), where the proposed method’s performance is also better than that of the GODEC+ method, which follows closely, except that more margin of over 4% is obtained under 6 × 6 block occlusion.Table 8
Classification accuracies on COIL100 under 6 × 6 and 8 × 8 occlusions.
LRRLRRLCLLRRGLRRGODEC+RSS-LRR6 × 60.47650.53690.55010.56890.59120.60528 × 80.38120.39910.42870.44540.45880.4803Table 9
Classification accuracies on LFW under 6 × 6 and 8 × 8 occlusions.
LRRLRRLCLLRRGLRRGODEC+RSS-LRR6 × 60.51060.55620.57320.58990.60210.64578 × 80.40310.43210.44160.45410.49530.5276From the clustering results displayed in Tables10 and 11, the following can be observed:(i)
Although all methods obtained comparative performances on both datasets, their accuracies degrade significantly by increasing noise. This, however, is not unexpected because more corruption levels would mean that more discriminative data features are destroyed, making it difficult to accurately group similar data samples in the same cluster.(ii)
Furthermore, while the relatively newer method GODEC+ shows more robustness to noise than the older methods, its clustering accuracy on clean data is slightly lower than that of the other methods on the LFW dataset, perhaps due to class imbalance in this dataset.(iii)
Overall, RSS-LRR’s robustness to noise grows stronger with the increase in noise level. For example, its performance on COIL100 under 0% noise is merely 1% better than that of its closest competitor GODEC+, but it is over 2% better under 20% noise. The same can be said on the LFW dataset, where RSS-LRR’s clustering accuracy is only about 2% better than that of LRR on clean data, whereas it is more than 4% better than that of GODEC+, which is the closest result under the 20% noise level.Table 10
Clustering results concerning the accuracy of different algorithms on the COIL100 dataset.
0%5%10%15%20%LRR0.6854 ± 1.530.6741 ± 1.280.6376 ± 1.890.4964 ± 2.610.4236 ± 1.42LRRLC0.6931 ± 1.370.6823 ± 1.430.6423 ± 1.590.5191 ± 1.870.4243 ± 1.87LLRR0.7043 ± 1.460.6812 ± 1.530.6442 ± 1.640.5230 ± 2.010.4195 ± 1.78GLRR0.7102 ± 1.670.6854 ± 1.850.6532 ± 1.490.5694 ± 2.380.4832 ± 2.05GODEC+0.7156 ± 1.870.7001 ± 1.470.6588 ± 1.840.5854 ± 2.130.5154 ± 1.21RSS-LRR0.7264 ± 1.310.7113 ± 1.440.7015 ± 1.180.6103 ± 1.420.5417 ± 1.61Table 11
Clustering results concerning the accuracy of different algorithms on the LFW dataset.
0%5%10%15%20%LRR0.7605 ± 1.530.7305 ± 1.840.7122 ± 1.760.6263 ± 1.750.3547 ± 1.65LRRLC0.7512 ± 1.530.7298 ± 1.260.7099 ± 1.720.6310 ± 1.270.3864 ± 2.14LLRR0.7518 ± 1.320.7431 ± 2.170.7169 ± 2.180.6397 ± 1.740.4311 ± 1.98GLRR0.7560 ± 1.030.7457 ± 1.970.7098 ± 1.530.6780 ± 1.850.5256 ± 1.87GODEC+0.7491 ± 1.270.7434 ± 1.920.7156 ± 1.440.6856 ± 1.790.5413 ± 1.34RSS-LRR0.7832 ± 1.360.7567 ± 1.720.7269 ± 1.840.6952 ± 1.680.5893 ± 1.76Additionally, in order to demonstrate more novelty, RSS-LRR’s clustering and classification performances on large-scale datasets are further compared with those of two more recent state-of-the-art (SOTA) methods, namely, nonnegative sparse discriminative low-rank representation (NSDLRR) [47] and low-rank and collaborative representations for hyperspectral anomaly detection (LRCRD) [48]. From the classification results displayed in Figure 7, it can be observed that all three methods obtain correlative results on both datasets. However, RSS-LRR shows more robustness, especially on the LFW dataset. Similarly, the clustering results are shown in Figure 8, and the results are also close with those of the proposed method displaying the best overall performance.Figure 7
Classification results concerning the accuracy of different SOTA methods on (a) COIL100 and (b) LFW datasets under various block occlusion levels.
(a)(b)Figure 8
Clustering results concerning the accuracy of different SOTA methods on (a) COIL100 and (b) LFW datasets under various noise levels.
(a)(b)


## 4.5. Convergence Study

Based on established standards of inexact augmented Lagrange multiplier (IALM) optimization strategy in several pieces of literature [5, 49–51], the objective function values of the J, Z, and E subproblems were expected to decrease monotonically in each iteration until convergence. Besides, the Z subproblem is known to have a closed-form solution [51] since it has a convex function. Also, the convergence of the J and E subproblems is confirmed by references [36, 52–54], respectively. Thus, a slow convergence is supposed when everything is put together. Luckily, as shown in Figure 9, the proposed algorithm has a strong convergence property, as it converges within 150 iterations on COIL100 and ORL datasets.Figure 9
Convergence curve of the proposed algorithm on (a) COIL100 and (b) ORL corrupted with 6 × 6 block occlusion.
(a)(b)


## 5. Conclusion

In this paper, we propose a recursive sample scaling low-rank representation method named “RSS-LRR.” Different from the existing methods, each data sample’s importance is estimated by introducing a cosine scaling factor. This scaling factor is used to extract each sample’s relationship with the low-rank representation’s principal components in the feature space. Thus, our proposed model can effectively reduce the noise effect by iteratively reducing the importance of noisy samples in learning the robust low-rank matrix. Several experimental results on well-known benchmark datasets demonstrate that RSS-LRR performs better in clustering and classification tasks than the state-of-the-art methods. It includes various experiments conducted on gross corrupted data. Therefore, we will extend RSS-LRR’s idea to multiview data in future work.



---

*Source: 2999001-2021-12-27.xml*