# New Robust Tensor PCA via Affine Transformations andL2,1 Norms for Exact Tubal Low-Rank Recovery from Highly Corrupted and Correlated Images in Signal Processing

Authors: Peidong Liang; Chentao Zhang; Habte Tadesse Likassa; Jielong Guo
Journal: Mathematical Problems in Engineering
 (2022)

---

## Abstract

In this latest work, the Newly Modified Robust Tensor Principal Component Analysis (New RTPCA) using affine transformation andL2,1 norms is proposed to remove the outliers and heavy sparse noises in signal processing. This process is done by decomposing the original data matrix as the low-rank heavy sparse noises. The determination of the potential variables is casted as constrained convex optimization problem, and the Alternating Direction Method of Multipliers (ADMM) method is considered to reduce the computational loads in an iterative manner. The simulation results validate the effectiveness of the new method as compared with that of the state-of-the-art methods.


---

## Body

## 1. Introduction

The current growth of big data has attracted the researchers to conduct the research in high dimensional increase in images and videos data across the globe. One of the major dilemmas therein is to scrutinize image and video visualization data to prune out the impacts of outliers and heavy sparse noises [1–4] due to spatial, the temporal, or other kinds of variations. These potential challenges are encountered in different scenarios such as pattern recognition, images and video reconstruction, and surveillance and medical images in healthcare [5]. It is thus of significance to build a new method for image alignment and reconstruction for the high-dimensional tensor which can further decompose the original high-dimensional images from the contaminated ones and, more significantly, are robust through pruning out the potential impacts of different noises illumination.A variety of robust tensor methods have been suggested for high-dimensional images reconstructions. For instance, Peng et al. [6] suggested a Robust method for Sparse and Low-rank decomposition (RASL), the authors in [7] suggested a modified robust principal component analysis, and authors in [1–3] suggested different robust PCA approaches for image recovery to remove errors in images incurred by corruption and occlusion, but the issue of the tensor is not taken into consideration. To tackle this dilemma, the authors in [8] addressed a fast method through the L1-based regularization by decomposing several images.Additionally, the authors in [9–11] suggested a new RPCA relying on the constrained convex optimization problem, in which the reconstruction of the images as the low rank is promising. Wang et al. [12] addressed a new model for Frequency-Weighted Robust Tensor PCA to decompose images as the reconstructed low rank and sparse errors and noises; however, it still lacks robustness when the number of parameters is increasing related to tensor. Following this, Chen et al. [13] addressed Nonconvex optimization techniques to recover highly corrupted, and correlated images reduce the impact of errors on image alignment. Moreover, Song et al. [14] proposed online robust methods for image alignment using the linearization of the objective function via incorporating geometric transformations. Liu et al. [15] addressed a modified method to tackle the issue of the subspace clustering problems. Oh et al. [16] proposed the Partial Sum of Singular Values (PSSVs) instead of using the usual the nuclear norms to separate the original corrupted images. He et al. [17] developed a modified method to tackle the issue of computational loads in addition to image reconstruction. Qu et al. [18] mentioned a new model that incorporated the rank as a prior information in the neural networks to improve the performance of the regression modified model.To enhance the robustness of RPCA [9] and RASL [1–3, 6], this work proposes a new robust tensor PCA to reconstruct the low-rank components from the highly correlated images using the tensor train low-rank along with the affine transformation and L2,1 norms. As RPCA [9] and RASL [6] consider handling two-way data matrix, but, the proposed method tries to recover highly correlated datasets mainly real face images which are also in the tensors. However, inspired by [4, 6] and [4, 19, 20], the tensor train low-rank along with affine transformation and the L2,1 norms is incorporated to remove the outliers and heavy sparse errors in big data processing particularly in signal processing. As the data under consideration is occluded and abnormal, the incorporation of the tensor low-rank along with the two novel ideas suggested is indispensable to mitigate the potential impact of annoying effects. As such, the new method is relatively better in capturing the potential impacts of all the aforementioned effects, where the robustness of the proposed method is confirmed.Moreover, to alleviate the complexity, the new parameters and optimal equations are taken into considerations to derive the convex constrained optimization involved in iterative manners. Motivated by [21], the tensor nuclear norm is modified and the robustness of the method is enhanced using the affine transformation and the L2,1 norms and considering the relationship as the matrix cases. These novel ideas have enhanced the performance of the approach in reconstruction of high-dimensional images particularly in signal processing.Along with the tensor modified nuclear norm assisted through an affine transformation and theL2,1 norms, the convex TRPCA model is relatively better in reconstruction of images in signal processing over the RPCA [22]. The major contributions of this work include the following:(1)
To make the new method robust, affine transformations andL2,1 norm are incorporated to a new tensor model to remove all the potential impacts of annoying effects.(2)
The ADMM method is taken into account to optimize the updating parameters through solving the new convex constrained optimization problem and a set of updating parameters and equations are developed to iteratively solve this problem in an iterative process.(3)
The new model considers both a set of an affine transformation andL2,1 norms which are used to minimize the gross errors but cannot guarantee the convergence of the whole method; thus, the new proposed approach integrates error minimization and error support into one regression model, and its ADMM method theoretically converges well.(4)
The experimental results justify our theoretical applications for face images and handwritten digits image reconstruction demonstrating the effectiveness of our method.The main process of this work is summarized as shown in Figure1.Figure 1
The flow chart of the proposed method. First, an affine transformation and theL2,1 norms are applied to the corrupted high-dimensional images in signal processing, and then low-rank tensor is separated from the corrupted high-dimensional images. Ultimately, we attain the low-rank recovered tensor high-dimensional images or the true underlying objects.This work is illustrated as follows. Section2 indicates the details of the problem formulations of the latest paper. Section 3 develops and describes the updating equations to solve the proposed problem. In Section 4, experimental results are conducted to justify the proposed method. Section 5 provides some concluding remarks to summarize the paper.


## 2. Related Works

There are a number of works reported for image recovery [15, 23–28]. For example, the authors in [26, 28, 29] considered the least trimmed squares to mitigate errors in the regression to explicitly search a data subset that reduces the squared of the errors. The discriminatory least square regression [30] and the worst-case linear discriminant analysis [31] were addressed to solve the least square loss function that influences the correlation between the explanatory and response variables. However, the robust regression methods still lack dealing with high-dimensional tensor data. Wang et al. [32] developed a robust regression approach via self-scaled regularization to boost the performance in the presence of gross outliers. These methods, however, are not that robust for high-dimensional tensor images. Bunea et al. [33] presented a rank selection criterion to select the best rank estimator of the coefficient regression matrix in the multivariate regression approach. To deal with linearly structured matrices, Zachariah et al. [34] considered an iterative method via least square estimation for low-rank matrix reconstruction, but it required the prior knowledge of the matrix structure. Chen et al. [35] addressed an iterative reweighted least squared method for sparsity recovery, which incorporated the structure of sparsity along with an orthogonal basis and the total variation. However, it requires to conduct matrix inversion at each iteration, leading to high computational complexity. Instead of using the handcrafted least square regressions as in [36–38], some more recent methods [39–44] have improved the visual quality in face reconstruction via the low-rank approximation. Recently, several methods, which combined regression with RPCA, were proposed to further boost the effectiveness of the methods. For instance, Ji et al. [45] addressed a regularized sparse regression via combining RPCA [9] with lasso regression [46] to mitigate the influence of outliers in the head pose estimation. However, the performance of this relaxation method degrades when the fitting error is large. Yin et al. [47] considered a robust multinomial logistic and binary regression to remove the sparse noise and outliers from the contaminated data. Yang et al. [48] proposed a matrix regression scheme for face image representation based on the nuclear norm. To tackle the problem of the existing methods, the authors in [49, 50] addressed a new tensor method via robustifying tensor decomposition.High-dimensional tensor image recovery has also received a considerable amount of research attention. For instance, the authors in [51] addressed the robust tensor to overcome the drawbacks of the linear regression methods which were considered in [52–54] for head pose estimation and high-dimensional image alignment and recovery. These methods, however, still lack in pruning out the potential impacts of occlusions, outliers, and uncontrolled illuminations. Additionally, the authors in [55] tackled the drawbacks of the robust regression methods such as [56–58] to caste the outliers and others annoying effects in high-dimensional images. Additionally, several novel methods were used to solve dynamic problems for real-time applications, for instance, [59–63]; yet the issue of pruning out the potential annoying effect needs more robust tensor method via affine transformations. To tackle the setback, different new models were developed by [1–5, 53], extending the existing work into the high tensor approaches via incorporating the affine transformations, while the authors in [64, 65] proposed a probabilistic method for head pose estimation by directly mapping the feature vectors onto the yaw angles. Diaz-Chito et al. [66] addressed a method to narrow down the gap between the head yaw angles and the regression by combining manifold embedding methods with linear regression. Meyer et al. [67] considered a three-dimensional head poses estimation method to handle large pose angles and partial occlusions. However, it cannot prune out outliers from disjoint subspaces in head pose estimation. To tackle the drawbacks of robust regressions, the author in [55] addressed the robust spectral method for over-complete tensor decomposition. Additionally, there are several methods proposed for image tensor recovery using the new tensor methods [68, 69]. Additionally, Zhang and Lerman [69] developed a new tensor nuclear norm to prune out the potential impact of outliers and heavy sparse. For further summary of the new proposed approach as compared with the other state-of-the-art works, see Table 1.Table 1
Comparison of the proposed approach with other related methods in terms of the objective function and constraints (TRPCA is with high-dimensional three tensors while the others methods are with two dimension).
MethodsObjective functionConstraints[1, 3, 22, 70, 71]minA,EA∗+λE1M=A+E[1, 6]minA,EA∗+λE1Moτ+∑i=1nJiΔτviviT=A+E[15, 72, 73]minC,E,Q,AC∗+λE2,1M=A×C+E,C=Q[3, 74, 75]minA,E,C,Q,ΔτA∗+C∗+λ1Q2,1+λ2E2,1Moτ+∑i=1nJiΔτviviT=A×C+E,C=Q,Q±0[21, 76, 77]TRPCAminL,EL∗+λE1M=A+EProposed methodminL,EL∗+λE2,1Xoτ+∑i=1nTiΔτuiuiT=A+E,Ai=Ji∀i∈1,2,..,N


## 3. Problem Formulation

Given three-way tensor datumX∈n1×n2×n3, then X=A+E [78], where A=vecI10∣…∣vecIn0∈Rn1×n2×n3 denotes an approximated low-rank matrix, E∈Rn1×n2×n3 represents the sparse errors and noises, vec⋅ is the vector stacking operator [79], and n3 denotes input images of the same objects which are impacted by different variations. However, in reality, the objects in images Ii0 are often occluded by some errors ϵi, so it may be appropriate to assume that Ii=Ii0+εi,i=1,…,n. Now, the original tensor X can be represented as X=A+E, remove consequently. Invoked by [4, 80], for real tensor data, in order to compensate for the misalignment’s dilemma, certain transformations like τ1,τ2,…,τn are applied to the original data. To mitigate this setback, we need to apply the affine transformation τi to each misaligned tensor image Ii and then the original corrupted high-dimensional images can be changed to Xoτ=A+E, where Xoτ indicates the transformed data after applying τi to the distorted and misaligned image Ii [6, 13]. Since the solution of Xoτ=A+E is not good because of the nature of nonlinearity, we need to further linearize Xoτ. As such the difference of τ is small, then τ=τ1∣…∣τn∈Rp×n1×n2 can be approximated by Δτ1….Δτn∈Rp×∈Rn1×n2, in which p is used to denote the number of updating parameters. When the initial value of τ is known, then we can rewrite Xoτ+Δτ≈Xoτ+∑i=1nJiΔτuiuiT, where Ji=∂/∂τivecIioτi∈Rn3×p represents the Jacobian of the ith images with respect to τi and ui denotes the standard basis for Rn1×n2. Ultimately, we can get Xoτ+∑i=1nJiΔτviviT=A+E.The new incorporation of the two novel ideas is an interesting conceptual framework, including the new tensor concepts and their relationships, and is consistent with the one for the matrix cases. As such, we need to develop a new robust tensor PCA to reconstruct the low tubal rank componentA0 from the original and highly correlated images by removing different variation impacts of E0 from noisy observations X=A+E∈Rn1×n2×n3 (this work focuses on the 3-way tensor) by convex constrained optimization. To make the work easily understandable, the linearization procedure is given in Algorithm 1 [6]. Thereby, invoked by the idea of tensor RPCA [21, 76], our new method incorporated an affine transformation and the L2,1 norms to prune the potential impact of annoying effects. Therefore, using different relaxation, together with the novel ideas of affine transformation and the L2,1 norms, the new robust tensor PCA can be developed which leads the following constrained optimization problem:(1)minA,E,Ji,ΔτaiJi∗+λ1E2,1s.tXoτ+∑i=1nTiΔτviviT=A+E,Ai=Ji∀i∈1,2,..,N,where Xoτ∈Rn1×n2×n3, A∈R∈Rn1×n2×n3, E∈Rn1×n2×n3, and λ=1/maxn1,n2n3. The nuclear norm is given by Ji∗=∑i=1mσiJi, and E2,1=∑i=1n∑j=1mEji21/2 denotes the L2,1 norm of E.Algorithm 1: Iterative linearization.
Input: Given the corrupted images I1, I2,…, initial transformations τ1,τ2,…,τn and λ Not Converged(1)
Step 1: Thus, we need to normalize the corrupted data Xoτ=vecI1oτ1/vecI1oτ1vecI2oτ2/vecI2oτ2…….vecInoτn/vecInoτn(2)
Step 2: Solve the linearized convex optimization minimizeA,E,Ji,Δτ∑i=1NaiJi˜∗+λE2,1s.tXoτ+∑i=1NTiΔτviviT=A+E,Ai=Ji∀i∈1,2,..,N(3)
Step 3: Similarly, we need to find the Jacobian updating matrices with respect to τTi=∂/∂τivecIioτi/vecIioτi2,i=1,2,…,n


## 4. New Proposed Method

To find the constrained convex optimizing problems in (1), we used the augmented Lagrangian multipliers [6, 11, 81], in which we are going to estimate the best approximates iteratively both of the Lagrange multipliers and the optimal updating parameters by minimizing the augmented Lagrangian function given by(2)LJi˜,A,E,Z,Δτ,J2i=argminA,E,JI,Δτ,Z∑i=1NliJi˜∗+λE2,1+Z,Xoτ+∑i=1nTiΔτviviT−A−E+∑i=1NAi−Ji,Y2i+μ2Xoτ+∑i=1nTiΔτviviT−A−EF2+∑i=1Nμ2Ai−Ji2,where μ indicates positive penalty parameters, Z and Y2i are the Lagrangian multipliers. Notice that Z is tensor Lagrangian multipliers whereas Y2i is a set of N matrices of multipliers. The minimization of Lμ with respect to Ji for i∈1,2,…,N can be done as follows .,. and is inner product and Frobenious norm, respectively. We cannot solve (2) directly, thus, as [6, 81] updating parameters can be done iteratively in an alternating way. First, we fix E, Δτ, and Z as constants and determine A by(3)Jik+1=argminJiLJi˜,A,E,Z,Δτ,J2i,Jik+1=argminJiaiJi∗+Ai−Ji,Y2i+μ2Ai−JiF2,Jik+1=argminJiaiμ−1‖Ji‖∗+12‖Ji−Ai+μ−1Y2i‖F2.(4)Jik+1==ℚaiμ−1Ai+μ−1Y2i.Thus, (4) is separable and involves solving a convex constrained program. Following the procedures in [81], we need to update Ak+1 by using the Augmented Lagrangian Multiplier given by(5)Unew,Σnew,Vnew=SVDXoτ+∑i=1NTiΔτkuiuiT−E+μk−1Zk,Ak+1=UnewS1/μkΣnewVnewT,where S1/μkΣnew represents the soft-threshold operator and Σnew is the covariance, in which the soft-threshold operator is defined as S1/μkΣnew=sgnΣnew.maxΣnew−1/μk,0 with sgnΣnew denoting the signs functions.Following this, to updateA, we need to fix all E, J2i, Δτ, and Z as constants and determine Ak+1 by(6)Ak+1=argminALJi˜,A,E,Z,Δτ,J2i,Ak+1=argminAX−A−E,Z+∑i=1NAi−Ji,Y2i+μ2X−A−EF2+∑i=1Nμ2Ai−JiF2,Ak+1=argminAX−A−E,Z+∑i=1NA−Ji,Y2i+μ2X−A−EF2+∑i=1Nμ2A−JiF2.Here, we have considered the tensorised forms of eachJi and Y2i, represented by Ji and Y2i, respectively, in order to void the repeated matrix of A. By tensor form, we simply mean the rearrangement of the matrix entries such that they form a tensor off the appropriate dimensions, which can be thought of as the reverse process of the matrix. The above can be easily minimized by forcing its derivative with respect to A equal to zero. This entails and produces the following results: ∂/∂A=X−A−E,Z=−Z∂/∂A=∑i=1NA−JJi,y2I=∑i=1Ny2i∂/∂A=∑i=1NX−A−EF2=μA+E−X∂/∂A∑i=11Nμ/2A−JJiF2=μNA−μ∑i=1NJJi. Thus, by minimization of the above equation as ∂Lμ/A=0, finally we can get(7)N+1A−X+E−μ−1Z−∑i=1NJJi−μ−1y2i=0.(8)A=N+1−1X−E+μ−1Z+∑i=1NJJi−μ−1Y2i.By using all these approaches, we are also interested in updatingE by(9)Ek+1=argminEλE2,1+Z,Xoτ+Z,Xoτ+∑i=1nTiΔτuiuiT−A−E,+μ2Xoτ+∑i=1nTiΔτuiuiT−A−EF2.Again by using the lemma [82], the update of the ith column of Ek+1, Qik+1 is given by(10)=Ek+1=argminEλE2,1+μk2Xoτ+∑I=1nTiΔτuiuuiT−A−E−ZμF2.Similarly by applying lemmas shown in [82], the update of the ith column of Qk+1, Qik+1, is given by(11)Qik+1=Pik2−λ1/μ2kPik2Pik,ifPik2≥λ1μ2k0,otherwise,where Pk=Ek+1+Z2k/μ2k.Fourthly, by keepingA, E, Ji, and Z as constants, Δτ can be updated by(12)Δτk+1=argminΔτLAk+1,Ek+1,Δτk,Zk.Moreover, further applying the Augmented Lagrangian Function and Singular Value Threshold, the update ofΔτ is given by(13)Δτk+1=∑i=1NTi+×Ak+1−Xoτ−μk−1ZkuiuiT,where Ti+ denotes Moore–Penrose pseudoinverse of Ti. Ultimately, the new updates of Lagrangian Multiplier Z and μ are given, respectively, by(14)Zk+1=Zk+μkhAk+1,Ek+1,Δτk+1.(15)μk+1=ρμk.To make the entire works easy, the updating equations of the proposed method are given in Algorithm2.Algorithm 2: New Tensor Robust PCA by ADMM.
Input: Tensor DataX, parameterλ, with initializationLk=Sk=0,ρ=1.1,μ=1×e−3While not converged Do(1)
Update:Jjk+1 by (4)(2)
Update:Ak+1 by (8)(3)
Update:Ek+1 by (12)(4)
Update:Δτk+1 by (14)(5)
Update:Z1k+1 by (15)(6)
Update:μ1k+1 by (16)(7)
Check the convergence conditions:Ak+1−Ak∞≤εEk+1−Ek∞≤εAk+1+Ek+1−X∞≤εEnd whileΔτOutputs: Low-rank component Ak, sparse component Ek


### 4.1. Convergence of the Proposed Method

This subsection considers the convergence characteristics of the updating equations and parameters suggested in Algorithm2. Thus, relying on these concepts, we also additionally proposed two theorems for further analysis.Theorem 1.
Suppose thatPk=Ak,Ek,Zk is formulated from the proposed Algorithm 2, then Pk is bounded. Thus, the new points P∗=A∗,E∗,Z∗ go to Pkk=1+∞ which is KKT stationary point, where we are going to satisfy the KKT conditions Z∗ε∂Al−γ,A∗+E∗,Zε∂E2,1. Proof of both theorems is extended from [83].


## 4.1. Convergence of the Proposed Method

This subsection considers the convergence characteristics of the updating equations and parameters suggested in Algorithm2. Thus, relying on these concepts, we also additionally proposed two theorems for further analysis.Theorem 1.
Suppose thatPk=Ak,Ek,Zk is formulated from the proposed Algorithm 2, then Pk is bounded. Thus, the new points P∗=A∗,E∗,Z∗ go to Pkk=1+∞ which is KKT stationary point, where we are going to satisfy the KKT conditions Z∗ε∂Al−γ,A∗+E∗,Zε∂E2,1. Proof of both theorems is extended from [83].


## 5. Simulations and Discussion

First, we need to assess the effectiveness and efficiency of the updated proposed method through extensive computer simulations to justify the advantage of the proposed method in recovering and aligning big data processing. Three state-of-the-art methods are conducted for comparison, including TRPCA-A [21], Low-Rank Tensor Completion (LRTC) [84], and WSTNN based on TRPCA [85]. The overall effectiveness of the new mew method compared with the baselines is scrutinized in two ways. In the first case, we examine the effectiveness of these methods using the visualization by tackling the potential impacts in highly correlated signal processing. By following this, next we apply the widespread Peak Signal Noise Ratio PSNR, which is defined as(16)PSNRf,g=10log102552/1/n1×n2×n3∑i=1n1∑i=1nfij−gij2,where n1×n2×n3 indicates the sizes of both of the original image f and the recovered g. Thus, the Peak Signal to Noise rate (PSNR) is to measure the quality of the aligned and recovered high-dimensional images, in which the PSNR tests are implemented on the platform of Windows 7 and MATLAB (R2017b) with an Intel Core i5-4590 3.30 GHz and 16 GB RAM.


### 5.1. Low-Rank Tensor Completion

Following this, in this subsection, we test the effectiveness of the proposed method using the real public image data. First, we employed the dummy face and handwritten images [6, 86]; following this, we also considered the complicated images taken from [87]. Our main goal is to remove the potential impacts of outlier and heavy sparse noises removal problem on high-dimensional images. A total of a number of different contaminated images were considered to assess the effectiveness of the proposed method based on the protocols of [76]. Each image from the high-dimensional data is with different sizes. The image backgrounds change slightly, and the variability is mainly caused by the outliers, occlusions, illumination, and heavy sparse noises. Thereby, the effectiveness of the proposed method is compared with that of the state-of-the-art works such as TRPCA-A [21], Low-Rank Tensor Completion (LRTC) [84], and WSTNN based on TRPCA [85].


### 5.2. Experimental Convergence Performance

In this subsection, we provide more simulations to see the performance of the new method based on four different sets of images (handwritten images, dummy face images, complicated windows, and HSI images). In the first place, we consider a set of image which is the randomly generated dummy face images, the second one is handwritten digits from the MINST database, the third one is complicated window images, and the fourth one is HSI images. As [88], the convergence issue is analyzed by using Relative Square Error (RSE) vs. the iteration number, where RSE is defined as(17)g^−gFgF,in which g^ and g are denoting the recovered and original images, respectively. Also, we set λ1=1/maxn1,n2,n3 and λ1=1/maxn1,n2,n3 as suggested in [88]. The result corresponding to convergence analysis is as shown in Figure 2, from which we can understand that the RSEs of the proposed robust tensor method for all considered images with the iteration number decrease and then reach a constant after a few iterations. Finally, we also verified experimentally the convergence issue of the proposed method as shown in Figure 2, from which we note that the proposed method is fast converging. This fact further validates the convergence of the new method (see Figure 2).Figure 2
Convergence curve of the proposed method based on four different datasets: (a) dummy face images; (b) handwritten images; (c) complicated windows; (d) HSI images.
(a)(b)(c)(d)


### 5.3. Ablation Studies

In this section, first, we are going to examine the performance of the new method using eitherL1 or L2,1 norms of E and with and without further considering an affine transformations on three data bases, as given in Table 2, from which we recognize that the PSNR performance can be increased with the addition of the affine transformations in the low-rank plus sparse decomposition. This is because the L2,1 norm is considered, which is better than the L1 norm in dealing with outliers. Also, the performance can be further improved if we consider the L2,1 norm instead of the L1 norm of E in the objective function in (1). The performance improvement on the handwritten digits, dummy face images, complicated windows, and the HSI face images is more promising. This is because the L2,1 norm can remove the corrupted correlated samples across the images, so it is better in pruning out adverse effects.Table 2
Comparison of the proposed approach with theL1 or L2,1 norm of Sparse Error and with and without the affine transformations on handwritten images, dummy face images, complicated windows, and HSI images.
MethodDummyHandwrittenComplicatedHSI imagesL1 norm15.0319.1818.0918.09L1 norm with affine transformations19.1821.9318.8419.89L2,1 norm20.1623.1221.5023.54L2,1 norm with affine transformations25.9828.0031.1351.09


### 5.4. Image Reconstruction and Alignment

To check the performance, we consider face images of sizes49×49 taken from 100 images of a dummy which are corrupted in nature [76, 89, 90]. The result of the study shows that the proposed method is outperformed as compared with the baselines. This also verifies the performance of the new method in removing the perturbed and occluded effect from the contaminated data. To ensure the performance, the result of the PSNRs of the aforementioned method is as shown in Table 3, from which we can see that the new approach can indeed yield a larger PSNR as compared with the other four state-of-the-art works. This result is also in line with the visual quality of images which more resembled with the visualization results. The increment of the performance of the new method is due to the incorporation of the prior information of the partial column rank, which makes it to be more robust to outliers, occlusions, and abnormal data. As shown in Figure 3(e), the recovered handwritten images provide clearer visual quality by properly removing the adverse effects such as outliers and heavy sparse noises. This is in agreement with the results in Table 3 and further justifies that the proposed approach is more resilient to outliers and heavy sparse noises.Table 3
Mean peak signal noise ratio.
ApproachesFace images (time)Handwritten digits (time)LRTC [84]24.923.4523.50.462TRPCA-A [21]24.500.46323.400.763WSTNN-based TRPCA [85]24.501.03423.400.464Proposed method282.03425.982.012Figure 3
Removing random noises from face images. (a) Original image; (b) recovered images by LRTC; (c) recovered images by TRPCA-A; (d) recovered images by WSTNN based on TRPCA; (e) recovered images by the proposed method.


### 5.5. Handwritten Digits Image Recovery Alignment

Following this, we also carried out other experiment simulations on the 30 handwritten digits of sizes29×29 of “3” which are taken from MINST database [86] in which the performance of the proposed method is good as compared with that of the proposed method. The recovered images based on the performance of our approach as compared with the other three baselines are shown in Table 3. This also entails that the proposed method has better visual quality as compared with the other three baselines. To further assess the new approach, a comparison of the PSNRs of the aforementioned methods is shown in Table 3, from which we can find that the new method again yields the largest PSNR. This validates that inclusion of the partial column rank matrix in the decomposition process can enhance the quality of recovered handwritten digits as compared with previous works.As an illustration, some visual images of the recovered handwritten digits based on the aforementioned methods are displayed and shown in Figure3, from which we can see that the proposed method recovers the handwritten images better as compared with the other four baselines. As shown in Figure 4(e), the recovered handwritten images provide clearer visual quality by properly removing the adverse effects such as outliers and heavy sparse noises. This is in agreement with the results in Table 3 and further justifies that the proposed approach is more resilient to outliers and heavy sparse noises.Figure 4
Removing random noises from handwritten images. (a) Original image; (b) recovered images by LRTC; (c) recovered images by TRPCA-A; (d) recovered images by WSTNN based on TRPCA; (e) recovered images by the proposed method.


### 5.6. Recovery of Complicated Images for Performance Evaluation

Figure5 gives the comparison of the PSNR values along with the running time on a database of human segmented natural images [87]. Some examples with the recovered images are shown in Figure 5. From these results, we have the following observations. First, the proposed method has better performance as compared with the baselines. The reason is that RPCA performs on each channel independently and thus is notable to use the information across channels. The tensor methods instead take advantage of the multidimensional structure of data. Generally, the new method outperforms as compared with all the state-of-the-art works. This also justifies the new tensor method assisted through affine transformation, and the L2,1 norms are more resilient with the outliers, heavy sparse noises, occlusions and illuminations, and errors. This result is also more consistent with Table 4.Figure 5
Recovery performance comparison on three example images. (a) Original image; (b) observed image; (c) recovered images by LRTC, TRPCA-A, WSTNN-based on TRPCA, and the proposed method, respectively. Table3 shows the comparison of PSNR values and running time (second) on the above three images.Table 4
Mean peak signal noise ratio and time complexity.
ApproachesComplicated window imagesElapsed timeLRTC [84]24.5623.45TRPCA-A [21]23.600 10.463 928WSTNN-based TRPCA [85]24.500.567 89Proposed method31.126 62.034 387Again, as an illustration, some recovered complicated images based on the proposed method and the aforementioned baselines are given in Figure5, where the complicated images with different corruptions are depicted in Figure 5(a). The recovered images by the aforementioned methods are shown in Figures 5(b)–5(e), from which we can see that the visual quality of the proposed method is better than all of the baselines. This is in line with the numerical results in Table 4.


## 5.1. Low-Rank Tensor Completion

Following this, in this subsection, we test the effectiveness of the proposed method using the real public image data. First, we employed the dummy face and handwritten images [6, 86]; following this, we also considered the complicated images taken from [87]. Our main goal is to remove the potential impacts of outlier and heavy sparse noises removal problem on high-dimensional images. A total of a number of different contaminated images were considered to assess the effectiveness of the proposed method based on the protocols of [76]. Each image from the high-dimensional data is with different sizes. The image backgrounds change slightly, and the variability is mainly caused by the outliers, occlusions, illumination, and heavy sparse noises. Thereby, the effectiveness of the proposed method is compared with that of the state-of-the-art works such as TRPCA-A [21], Low-Rank Tensor Completion (LRTC) [84], and WSTNN based on TRPCA [85].


## 5.2. Experimental Convergence Performance

In this subsection, we provide more simulations to see the performance of the new method based on four different sets of images (handwritten images, dummy face images, complicated windows, and HSI images). In the first place, we consider a set of image which is the randomly generated dummy face images, the second one is handwritten digits from the MINST database, the third one is complicated window images, and the fourth one is HSI images. As [88], the convergence issue is analyzed by using Relative Square Error (RSE) vs. the iteration number, where RSE is defined as(17)g^−gFgF,in which g^ and g are denoting the recovered and original images, respectively. Also, we set λ1=1/maxn1,n2,n3 and λ1=1/maxn1,n2,n3 as suggested in [88]. The result corresponding to convergence analysis is as shown in Figure 2, from which we can understand that the RSEs of the proposed robust tensor method for all considered images with the iteration number decrease and then reach a constant after a few iterations. Finally, we also verified experimentally the convergence issue of the proposed method as shown in Figure 2, from which we note that the proposed method is fast converging. This fact further validates the convergence of the new method (see Figure 2).Figure 2
Convergence curve of the proposed method based on four different datasets: (a) dummy face images; (b) handwritten images; (c) complicated windows; (d) HSI images.
(a)(b)(c)(d)


## 5.3. Ablation Studies

In this section, first, we are going to examine the performance of the new method using eitherL1 or L2,1 norms of E and with and without further considering an affine transformations on three data bases, as given in Table 2, from which we recognize that the PSNR performance can be increased with the addition of the affine transformations in the low-rank plus sparse decomposition. This is because the L2,1 norm is considered, which is better than the L1 norm in dealing with outliers. Also, the performance can be further improved if we consider the L2,1 norm instead of the L1 norm of E in the objective function in (1). The performance improvement on the handwritten digits, dummy face images, complicated windows, and the HSI face images is more promising. This is because the L2,1 norm can remove the corrupted correlated samples across the images, so it is better in pruning out adverse effects.Table 2
Comparison of the proposed approach with theL1 or L2,1 norm of Sparse Error and with and without the affine transformations on handwritten images, dummy face images, complicated windows, and HSI images.
MethodDummyHandwrittenComplicatedHSI imagesL1 norm15.0319.1818.0918.09L1 norm with affine transformations19.1821.9318.8419.89L2,1 norm20.1623.1221.5023.54L2,1 norm with affine transformations25.9828.0031.1351.09


## 5.4. Image Reconstruction and Alignment

To check the performance, we consider face images of sizes49×49 taken from 100 images of a dummy which are corrupted in nature [76, 89, 90]. The result of the study shows that the proposed method is outperformed as compared with the baselines. This also verifies the performance of the new method in removing the perturbed and occluded effect from the contaminated data. To ensure the performance, the result of the PSNRs of the aforementioned method is as shown in Table 3, from which we can see that the new approach can indeed yield a larger PSNR as compared with the other four state-of-the-art works. This result is also in line with the visual quality of images which more resembled with the visualization results. The increment of the performance of the new method is due to the incorporation of the prior information of the partial column rank, which makes it to be more robust to outliers, occlusions, and abnormal data. As shown in Figure 3(e), the recovered handwritten images provide clearer visual quality by properly removing the adverse effects such as outliers and heavy sparse noises. This is in agreement with the results in Table 3 and further justifies that the proposed approach is more resilient to outliers and heavy sparse noises.Table 3
Mean peak signal noise ratio.
ApproachesFace images (time)Handwritten digits (time)LRTC [84]24.923.4523.50.462TRPCA-A [21]24.500.46323.400.763WSTNN-based TRPCA [85]24.501.03423.400.464Proposed method282.03425.982.012Figure 3
Removing random noises from face images. (a) Original image; (b) recovered images by LRTC; (c) recovered images by TRPCA-A; (d) recovered images by WSTNN based on TRPCA; (e) recovered images by the proposed method.


## 5.5. Handwritten Digits Image Recovery Alignment

Following this, we also carried out other experiment simulations on the 30 handwritten digits of sizes29×29 of “3” which are taken from MINST database [86] in which the performance of the proposed method is good as compared with that of the proposed method. The recovered images based on the performance of our approach as compared with the other three baselines are shown in Table 3. This also entails that the proposed method has better visual quality as compared with the other three baselines. To further assess the new approach, a comparison of the PSNRs of the aforementioned methods is shown in Table 3, from which we can find that the new method again yields the largest PSNR. This validates that inclusion of the partial column rank matrix in the decomposition process can enhance the quality of recovered handwritten digits as compared with previous works.As an illustration, some visual images of the recovered handwritten digits based on the aforementioned methods are displayed and shown in Figure3, from which we can see that the proposed method recovers the handwritten images better as compared with the other four baselines. As shown in Figure 4(e), the recovered handwritten images provide clearer visual quality by properly removing the adverse effects such as outliers and heavy sparse noises. This is in agreement with the results in Table 3 and further justifies that the proposed approach is more resilient to outliers and heavy sparse noises.Figure 4
Removing random noises from handwritten images. (a) Original image; (b) recovered images by LRTC; (c) recovered images by TRPCA-A; (d) recovered images by WSTNN based on TRPCA; (e) recovered images by the proposed method.


## 5.6. Recovery of Complicated Images for Performance Evaluation

Figure5 gives the comparison of the PSNR values along with the running time on a database of human segmented natural images [87]. Some examples with the recovered images are shown in Figure 5. From these results, we have the following observations. First, the proposed method has better performance as compared with the baselines. The reason is that RPCA performs on each channel independently and thus is notable to use the information across channels. The tensor methods instead take advantage of the multidimensional structure of data. Generally, the new method outperforms as compared with all the state-of-the-art works. This also justifies the new tensor method assisted through affine transformation, and the L2,1 norms are more resilient with the outliers, heavy sparse noises, occlusions and illuminations, and errors. This result is also more consistent with Table 4.Figure 5
Recovery performance comparison on three example images. (a) Original image; (b) observed image; (c) recovered images by LRTC, TRPCA-A, WSTNN-based on TRPCA, and the proposed method, respectively. Table3 shows the comparison of PSNR values and running time (second) on the above three images.Table 4
Mean peak signal noise ratio and time complexity.
ApproachesComplicated window imagesElapsed timeLRTC [84]24.5623.45TRPCA-A [21]23.600 10.463 928WSTNN-based TRPCA [85]24.500.567 89Proposed method31.126 62.034 387Again, as an illustration, some recovered complicated images based on the proposed method and the aforementioned baselines are given in Figure5, where the complicated images with different corruptions are depicted in Figure 5(a). The recovered images by the aforementioned methods are shown in Figures 5(b)–5(e), from which we can see that the visual quality of the proposed method is better than all of the baselines. This is in line with the numerical results in Table 4.


## 6. Time Complexity

In this section, we support the simulation results of the time complexity of the proposed method as compared with the baselines. On a very standard desktop computer [21], TRPCA-A is more efficient as compared with the state-of-the-art works mainly including TRPCA [9], LRTC, TRPCA-A, and WSTNN-based TRPCA which has relatively a less time computation as compared with TRPCA [76]. The new method guaranteed faster convergence than the state-of-the-art methods as shown in Tables 3 and 4.


### 6.1. HSI Completion Complicated Images Recovery

To further validate the effectiveness of the proposed approach as compared with the state-of-the-art works, we employ the newly developed approach to test and justify the effectiveness to recover the complicated HSIs Washington DC Mall images. Table5 shows the results of the values of PSNR in testing HSIs recovered by the aforementioned different methods along with the proposed method. We observe that compared with other methods, the proposed method consistently achieves the highest values in terms of the PSNs evaluation indexes.Table 5
HSI mean peak signal noise ratio.
ApproachesHSI images with 0.10 sample ratioLRTC [84]48.98TRPCA-A [21]26.96WSTNN-based TRPCA [85]49.594Proposed method51.09Again, as an illustration, some recovered Natural Face images based on the proposed method and the aforementioned baselines are given in Figure6, where the complicated HSI images with different corruptions are depicted in Figure 6 with the first column. The recovered images by the aforementioned method are shown in Figure 6 on column three, from which we can see that the visual quality of the proposed method is better than all of the baselines. This is in line with the numerical results in Table 4.Figure 6
Recovery of HSI images: (a) First Column Original image; (b) Second Column observed image; (c) recovered images by TRPCA [21]; (d) the proposed method.


## 6.1. HSI Completion Complicated Images Recovery

To further validate the effectiveness of the proposed approach as compared with the state-of-the-art works, we employ the newly developed approach to test and justify the effectiveness to recover the complicated HSIs Washington DC Mall images. Table5 shows the results of the values of PSNR in testing HSIs recovered by the aforementioned different methods along with the proposed method. We observe that compared with other methods, the proposed method consistently achieves the highest values in terms of the PSNs evaluation indexes.Table 5
HSI mean peak signal noise ratio.
ApproachesHSI images with 0.10 sample ratioLRTC [84]48.98TRPCA-A [21]26.96WSTNN-based TRPCA [85]49.594Proposed method51.09Again, as an illustration, some recovered Natural Face images based on the proposed method and the aforementioned baselines are given in Figure6, where the complicated HSI images with different corruptions are depicted in Figure 6 with the first column. The recovered images by the aforementioned method are shown in Figure 6 on column three, from which we can see that the visual quality of the proposed method is better than all of the baselines. This is in line with the numerical results in Table 4.Figure 6
Recovery of HSI images: (a) First Column Original image; (b) Second Column observed image; (c) recovered images by TRPCA [21]; (d) the proposed method.


## 7. Conclusions

In this article, we developed a new robust tensor method for exact image alignment and reconstruction via affine transformation and theL2,1 norms to efficaciously handle sparse errors, occlusions, and illumination. To handle the decomposition process, we additionally considered the affine transformation and the L2,1 norms. A new set of updating equations is also derived to alternatively solve the convex constrained optimization involved, using the ADMM approach iteratively in a round Robin manner. As such, we attained a robust method which is simultaneously effective and efficient in big signal processing. We also experimentally verified that our robust tensor method outperforms the state-of-the-art methods during image alignment and reconstruction based on the several datasets.



---

*Source: 3002348-2022-03-31.xml*