RPCANet++: Deep Interpretable Robust PCA for Sparse Object Segmentation
Authors: name: Fengyi Wu; name: Yimian Dai; name: Tianfang Zhang; name: Yixuan Ding; name: Jian Yang; name: Ming-Ming Cheng; name: Zhenming Peng
Published: 2025-08-06 08:19:37

Abstract: Robust principal component analysis (RPCA) decomposes an observation matrix
into low-rank background and sparse object components. This capability has
enabled its application in tasks ranging from image restoration to
segmentation. However, traditional RPCA models suffer from computational
burdens caused by matrix operations, reliance on finely tuned hyperparameters,
and rigid priors that limit adaptability in dynamic scenarios. To solve these
limitations, we propose RPCANet++, a sparse object segmentation framework that
fuses the interpretability of RPCA with efficient deep architectures. Our
approach unfolds a relaxed RPCA model into a structured network comprising a
Background Approximation Module (BAM), an Object Extraction Module (OEM), and
an Image Restoration Module (IRM). To mitigate inter-stage transmission loss in
the BAM, we introduce a Memory-Augmented Module (MAM) to enhance background
feature preservation, while a Deep Contrast Prior Module (DCPM) leverages
saliency cues to expedite object extraction. Extensive experiments on diverse
datasets demonstrate that RPCANet++ achieves state-of-the-art performance under
various imaging scenarios. We further improve interpretability via visual and
numerical low-rankness and sparsity measurements. By combining the theoretical
strengths of RPCA with the efficiency of deep networks, our approach sets a new
baseline for reliable and interpretable sparse object segmentation. Codes are
available at our Project Webpage https://fengyiwu98.github.io/rpcanetx.