# Low Tensor Rank Constrained Image Inpainting Using a Novel Arrangement Scheme
## 3 Methods
### Method 1: The Low Unfolding Matrix Rank-Based Method  
The unfolding method is widely used to permute order-3 video or dynamic magnetic resonance images into an unfolding matrix and then exploit the low rankness of this matrix for data reconstruction [23; 42]. The unfolding matrix has a low-rank structure because of the similarity of every slightly changed slice along the time dimension. We try to dig out the potential low unfolding-matrix rankness of a color image by a flexible QA scheme and call this scheme the first flexible QA scheme. Take a 256 \(\times\) 256 \(\times\) 3 Lena image as an example, as shown in Figure 2a. First, the image is permuted into the 3-order tensor size of 32 \(\times\) 32 \(\times\) 192 using the basic QA scheme. Then, similar slices of this 3-order are unfolded. Lastly, the balanced (the context of 'balanced' is that the size changes from the unbalanced 256 \(\times\) 256 \(\times\) 3 to the more balanced size of 1024 \(\times\) 192) unfolding matrix size of 1024 \(\times\) 192 is obtained. As the slices (32 \(\times\) 32) in the 3-order tensor are similar, the unfolding matrix is low rank, as shown in Figure 2b. In practice, the size of the designed unfolding matrix should be balanced so that the minimization of the unfolding matrix rank is efficient. Figure 1: (**a**) Examples of the basic QA scheme. Using the basic QA scheme, the matrix M size of 8 \(\times\) 8 can be turned into an order-3 tensor with a size of 4 \(\times\) 4 \(\times\) 4. (**b**) Using the basic QA scheme, the Lena image can be divided into 4 small Lena images. (**c**) The four-pixel values curves of the four smaller Lena images have overlapped into one curve. We exploit the low unfolding matrix rank in image inpainting and give the low unfolding matrix rank-based model as follows:  
\[\min_{X}\|M(\Phi_{1}X)\|_{*}\text{ subject to }X(i,\,j)=Y(i,\,j),\,\forall(i,\,j)\in\Omega \tag{1}\]  
where \(X\) denotes the image to be recovered; \(\Phi_{1}\) denotes the operator of permuting the image into a suitable 3-order tensor by multiple basic QA schemes; \(M\) denotes the operator of the unfolding process, which unfolds every slice along the third dimension of the 3-order tensor \(\Phi_{1}X\); \(\Omega\) is the position without painting; and \(Y\) is the painted image with damaged entries at the positions \(\Omega^{C}\). To reduce the computational complexity, in model (1), the following SVD-free approach [43, 44] is exploited to constrain the low rankness of the unfolding matrix \(M(\Phi_{1}X)\) instead of the nuclear norm:  
\[\min_{U\,\,V^{H}=M(\Phi_{1}X)}\frac{1}{2}\Big{(}\|U\|_{F}^{2}+\|V\|_{F}^{2} \Big{)}=\|M(\Phi_{1}X)\|_{*} \tag{2}\]  
As total variation (TV) has been proved as an effective constraint of smooth prior [45, 46], model (1) is incorporated with 2D TV to exploit the local smooth priors of visual image data. Then, the image inpainting model (1) turns into the following:  
\[ \begin{array}{l}\min_{U,\,V,\,X^{2}}\frac{1}{2}\Big{(}\|U\|_{F}^{2}+\|V\|_{F} ^{2}\Big{)}+\frac{\beta}{2}\|X\|_{TV}\\ \text{subject to }\,\,X(i,\,j)=Y(i,\,j),\,\,\forall(i,\,j)\in\Omega,\,M\Phi_{1}X= UV^{H}\end{array}  \tag{3}\]  
where \(\beta\) is the regularization parameter. The algorithm of alternating direction method of multipliers (ADMM) is used to solve the low unfolding matrix rank and TV-based image inpainting model (3). Firstly, an auxiliary variable \(Z=DX\) is introduced, where \(D\) is the finite difference operator, and then (3) is rewritten as the unconstrained convex optimization problem (4):  
\[ \begin{array}{l}\min_{U,\,V,\,\Lambda,\,L,\,Z,\,X}\tau_{\Omega}(X)+\frac{1} {2}\Big{(}\|U\|_{F}^{2}+\|V\|_{F}^{2}\Big{)}+\frac{\rho_{1}}{2}\big{\|}M\Phi_{ 1}X-UV^{H}+\Lambda\big{\|}_{F}^{2}\\ +\frac{\rho_{1}}{2}\big{\|}Z\big{\|}_{1}+\frac{\beta\rho_{2}}{2}\|DX-Z+L\|_{F} ^{2}\end{array}  \tag{4}\]  
Figure 2: (**a**) The first flexible QA scheme to obtain the unfolding matrix. Take the Lena RGB image as an example.