A unified approach for minimizing composite norms
We propose a first-order augmented Lagrangian algorithm (FALC) to solve the composite norm minimization problem $$\begin{aligned} \begin{array}{ll} \min \limits _{X\in \mathbb{R }^{m\times n}}&\mu _1\Vert \sigma (\mathcal{F }(X)-G)\Vert _\alpha +\mu _2\Vert \mathcal{C }(X)-d\Vert _\beta ,\\ \text{ subject} \text{ to}&\mathcal{A }(X)-b\in \mathcal{Q }, \end{array} \end{aligned}$$where \(\sigma (X)\) denotes the vector of singular values of \(X \in \mathbb{R }^{m\times n}\), the matrix norm \(\Vert \sigma (X)\Vert _{\alpha }\) denotes either the Frobenius, the nuclear, or the \(\ell _2\)-operator norm of \(X\), the vector norm \(\Vert .\Vert _{\beta }\) denotes either the \(\ell _1\)-norm, \(\ell _2\)-norm or the \(\ell _{\infty }\)-norm; \(\mathcal{Q }\) is a closed convex set and \(\mathcal{A }(.)\), \(\mathcal{C }(.)\), \(\mathcal{F }(.)\) are linear operators from \(\mathbb{R }^{m\times n}\) to vector spaces of appropriate dimensions. Basis pursuit, matrix completion, robust principal component pursuit(PCP), and stable PCP problems are all special cases of the composite norm minimization problem. Thus, FALC is able to solve all these problems in a unified manner. We show that any limit point of FALC iterate sequence is an optimal solution of the composite norm minimization problem. We also show that for all \(\epsilon >0\), the FALC iterates are \(\epsilon \)-feasible and \(\epsilon \)-optimal after \(\mathcal{O }(\log (\epsilon ^{-1}))\) iterations, which require \(\mathcal{O }(\epsilon ^{-1})\) constrained shrinkage operations and Euclidean projection onto the set \(\mathcal{Q }\). Surprisingly, on the problem sets we tested, FALC required only \(\mathcal{O }(\log (\epsilon ^{-1}))\) constrained shrinkage, instead of the \(\mathcal{O }(\epsilon ^{-1})\) worst case bound, to compute an \(\epsilon \)-feasible and \(\epsilon \)-optimal solution. To best of our knowledge, FALC is the first algorithm with a known complexity bound that solves the stable PCP problem.
norm minimization basis pursuit convex optimization conic constraints augmented lagrangian method first order method iteration complexity \(\ell _1\)-minimization nuclear norm principal component pursuit sparse optimization 90c25 90c06 90c22 49m29 90c90 65k05