Towards theoretically-founded learning-based denoising

Denoising a stationary process (X <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">i</sub> ) <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">i∈Z</sub> corrupted by additive white Gaussian noise (Z <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">i</sub> ) <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">i∈Z</sub> , i.e., recovering Xn from Y <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">n</sup> = X <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">n</sup> + Z <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">n</sup> , is a classic and fundamental problem in information theory and statistical signal processing. Theoretically-founded and computationally-efficient denoising algorithms which are applicable to general sources are yet to be found. In a Bayesian setup, given the distribution of X <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">n</sup> , a minimum mean square error (MMSE) denoiser computes E[X <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">n</sup> |Y <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">n</sup> ]. However, for general sources, computing E[X <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">n</sup> |Y <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">n</sup> ] is computationally very challenging, if not infeasible. In this paper, starting from a Bayesian setup, a novel denoiser, namely, quantized maximum a posteriori (Q-MAP) denoiser, is proposed and its asymptotic performance is analyzed. Both for memoryless sources, and for structured first-order Markov sources, it is shown that, asymptotically, as σ <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sup> (noise variance) converges to zero, 1/σ <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sup> E[(X <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">i</sub> -X <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Q-MAP</sup> ) <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sub> ] converges to the information dimension of the source. For the studied memoryless sources, this limit is known to be optimal. A key advantage of the QMAP denoiser is that, unlike a MMSE denoiser, it highlights the key properties of the source distribution that are to be used in its denoising. This naturally leads to a learning-based denoising algorithm. Using ImageNet database for training, initial simulation results exploring the performance of such a learning-based denoiser in image denoising are presented.