# Is Texture Denoising Efficiency Predictable?

O. Rubel, V. Lukin, S. Abramov, B. Vozel, O. Pogrebnyak, K. Egiazarian

► To cite this version:

O. Rubel, V. Lukin, S. Abramov, B. Vozel, O. Pogrebnyak, et al., Is Texture Denoising Efficiency Predictable?. International Journal of Pattern Recognition and Artificial Intelligence, 2018, 32 (01), pp.1860005. 10.1142/S0218001418600054. hal-01622383v2

HAL Id: hal-01622383

https://univ-rennes.hal.science/hal-01622383v2

Submitted on 22 Jun 2023

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>
# Is Texture Denoising Efficiency Predictable?

Oleksii Rubel*,¶, Vladimir Lukin*,||, Sergey Abramov*,**, Benoit Vozel†,††,
Oleksiy Pogrebnyak‡,‡‡ and Karen Egiazarian§,§§

*National Aerospace University

Department of Receivers

Transmitters and Signal Processing

Kharkov, Ukraine, 61070

†University of Rennes 1 – Enssat

IETR UMR CNRS 6164, Lannion, France, 22300

‡Instituto Politecnico Nacional

Centro de Investigación en Computación

Ave. Juan de Dios Batiz S/N, Mexico, D.F., 07738

§Tampere University of Technology

Department of Signal Processing

Tampere, Finland, 33720

¶edu.rubel@gmail.com

||vladimlukin@yahoo.com

**s.abramov@rtsla.khai.edu

††benoit.vozel@univ-rennes1.fr

‡‡olek@cic.ipn.mx

§§karen.egiazarian@tut.fi

Received 29 October 2016

Accepted 24 May 2017

Published 24 July 2017

Images of different origin contain textures, and textural features in such regions are frequently employed in pattern recognition, image classification, information extraction, etc. Noise often present in analyzed images might prevent a proper solution of basic tasks in the aforementioned applications and is worth suppressing. This is not an easy task since even the most advanced denoising methods destroy texture in a more or less degree while removing noise. Thus, it is desirable to predict the filtering behavior before any denoising is applied. This paper studies the efficiency of texture image denoising for different noise intensities and several filter types under different visual quality criteria (quality metrics). It is demonstrated that the most efficient existing filters provide very similar results. From the obtained results, it is possible to generalize and employ the prediction strategy earlier proposed for denoising techniques based on the discrete cosine transform. Accuracy of such a prediction is studied and the ways to improve it

<footnote>|| Corresponding author.</footnote>

<footnote>This is an Open Access article published by World Scientific Publishing Company. It is distributed under the terms of the Creative Commons Attribution 4.0 (CC-BY) License. Further distribution of this work is permitted, provided the original work is properly cited.</footnote>
O. Rubel et al.

are considered. Some practical recommendations concerning a decision to undertake whether it
is worth applying a filter are given.

Keywords: Texture denoising; noise suppression; image processing; visual quality.

# 1. Introduction

Texture regions found in almost all natural scene images can occupy a different percentage of image area.<sup>18</sup> Textures play an important role in geomorphometry<sup>21,45</sup> content-based image retrieval,<sup>9</sup> remote sensing,<sup>19,40,46</sup> pattern recognition and classification,<sup>25,47</sup> etc. Meanwhile, texture features are often masked or distorted due to noise present in the acquired images. This noise can be of a different type (additive, multiplicative, signal-dependent<sup>13,46,48</sup>) and origin being inherent for different types of images (optical, radar, medical, hyperspectral). Therefore, a task is to remove this noise preserving the texture features in a maximally careful manner.<sup>17,32,35,36,46,54</sup>

One might expect that this task of efficient texture denoising which was already relevant a decade or two ago,<sup>17,32,46</sup> and now with all recent advancements (nonlocal filtering methods) in image denoising<sup>11,12,14,26,46</sup> has been successfully solved. However, this is not true. As it was shown by Milanfar and Chatterjee,<sup>7</sup> the potential of nonlocal filtering approach is limited for textural images. This has been empirically confirmed in Ref. 16 for highly textural images from TID2013 database.<sup>27</sup> It has been shown in Refs. 35 and 36 that the problems in noise removal arise for filters based on discrete cosine transform (DCT)<sup>16,22,26</sup> and one of the most advanced nonlocal filtering methods, BM3D (block matching three dimensional) filter.<sup>11</sup> Then, one might think that denoising techniques based on other principles are able to cope with a noise in texture images in a better way. Analysis carried out in the recent papers (see Refs. 34 and 36) shows that this is not true also for many advanced and efficient modern filters such as translation invariant wavelet shrinkage (TI-WS),<sup>10</sup> Bayesian least squares of Gaussian scale mixtures (BLS-GSM),<sup>31</sup> nonlocal mean (NLM),<sup>4</sup> a filter based on principal components analysis with local pixel grouping (LPG-PCA) in the spatial domain,<sup>6</sup> spatially adaptive iterative filtering (SAIF).<sup>50</sup> Recently, the powerful clustering-based denoising schemes have been proposed: KSVD<sup>43</sup> and KLLD<sup>3</sup> that use learned dictionaries in different ways, and a filter based on gradient histogram preservation (GHP).<sup>54</sup> Their analysis has shown that most of the aforementioned denoising techniques perform similarly (approximately at the same level as standard DCT-based filter<sup>22,26</sup> and BM3D filter<sup>11</sup>) whilst NLM and LPG-PCA filters<sup>4,6</sup> perform sufficiently worse.

Moreover, it is worth performing a more careful analysis. There are several reasons behind this. The paper in Ref. 36 does not present data for the filters GHP, LPG-PCA, SAIF, KSVD, and KLLD.<sup>3,6,43,50,54</sup> The paper in Ref. 34 gives data for these filters in the form of scatter-plots used to predict filtering efficiency which are difficult to analyze and compare. Besides, data is given as scatter-plots only for one denoising efficiency metric, improvement of peak signal-to-noise ratio (IPSNR),
whilst for visual quality metrics such as Peak-Signal-to-Noise Ratio with accounting Human Visual System and masking (PSNR-HVS-M),³⁰ Multiscale Structural Similarity Index Measure (MSSIM),⁵² and Feature Similarity Index Measure (FSIM)⁵³ only efficiency approximation curves are presented.³⁴

Here we would like to mention two important aspects in texture filtering that explain our attention to the listed metrics. Firstly, visual quality of denoised images is important for many applications.²³,³⁶,⁴⁹ Thus, it is expedient to employ adequate visual quality metrics²³,²⁸,⁴¹ in the analysis of filtering efficiency. Secondly, a positive effect from denoising (noise suppression) is often comparable to a negative effect of texture smearing or distorting.³⁶ Then, the following question arises — is it worth applying denoising at all? Accompanying questions are: can we predict expediency of filtering for each particular case, and is it possible to reliably undertake a decision to carry out or to skip denoising?

What can be said about visual quality and metrics that can be employed for its characterization: there are no commonly accepted and fully reliable metrics. Studies in this direction continue.²³,²⁸ If one wants to have a reliable assessment, it is reasonable to employ several adequate visual quality metrics, and to check the consistency of conclusions based on the analysis of these metrics. Below, we followed this approach and considered the aforementioned metrics PSNR-HVS-M, MSSIM, and FSIM, which are among the best for the case of grayscale image denoising.²⁸

Concerning the prediction of denoising efficiency: the efficiency as we mean is(are) the value(s) of some parameter(s) (indicator(s)) that can quantitatively characterize changes in image quality due to filtering. These can be the improvement of PSNR (IPSNR), reduction of the output Mean Square Error (MSE) compared to a noise variance in the original image, or other improvements of visual quality metrics. An idea that such indicators of denoising efficiency can be predicted (estimated before image denoising is applied) has been put forward in papers.¹,⁸ The way proposed in Ref. 8 requires considerable computations and, thus, the time needed to derive a prediction indicator is comparable to a filtering procedure itself that restricts the practical application of this approach.

In fact, one needs a simple, fast and accurate enough way to predict the efficiency quantitatively. The approach in Ref. 1 is just fast and simple. It implies calculation of one input parameter over a limited (small) number of 8 × 8 pixel blocks for which 2D DCT is performed. This input parameter is then used for calculating the output parameter. Input and output parameters are linked by a function that can be also called approximation (prediction) curve. This curve could be obtained by regression in offline mode (in advance, before applying it for prediction). Due to a small number of blocks and simple calculations in them to determine the input parameter, the prediction can be carried out much faster than filtering. An important question is then: how accurately is the efficiency indicator estimated?

More or less extensive analysis of this accuracy is performed in papers Refs. 1, 35, 36, 34 showing that the accuracy depends upon many factors including what are the
input and output parameters, how regression is calculated, what filter is analyzed, etc. Good prediction characteristics are already obtained for many modern filters if IPSNR is predicted. However, the prediction is considerably less accurate for visual quality metrics. There exist methods to improve this accuracy³³,³⁸,⁵¹ that, in particular, deal with a joint use of two or more input parameters. But, these options have not been tested for texture image denoising yet.

Therefore, the main contributions of this paper are the following. Firstly, a thorough analysis of the denoising efficiency for texture images corrupted by noise of different intensity using several metrics is performed. It allows carrying out comparisons of the filtering efficiency for the aforementioned set of filters and give well-motivated practical recommendations on their use. In particular, it is shown that NLM and LPG-PCA filters²⁹,³⁰ do not perform well for texture images. Secondly, the prediction accuracy is analyzed and ways to improve it are proposed. Also, it is shown that the use of input PSNR as the second input parameter allows improving prediction accuracy.

The paper is structured as follows. Section 2 briefly describes the image/noise model, considered test images, analyzed efficiency criteria (metrics) and filtering techniques. Section 3 deals with the analysis of denoising efficiency. Approaches to the prediction of denoising efficiency indicators are considered in Sec. 4. New solutions to the prediction are presented in Sec. 5. Finally, the conclusions follow.

## 2. Image-Noise Model, Test Images, Metrics and Filters

### 2.1. Image-noise model and used test images

In our study, we use a typical simple observation model for noisy grayscale (or components of multichannel) images

$$I_{ij}^{n} = I_{ij}^{tr} + n_{ij}. \qquad (1)$$

Here i, j denote pixel indices, $I_{ij}^{tr}$ and $n_{ij}$ are the true image value and noise, respectively, i = 1, ..., $I_{lm}$ and j = 1, ..., $J_{lm}$, $I_{lm}$ and $J_{lm}$ define the image size. It is well understood that (1) is the idealized noise model.

Recall that texture is a prime interest. So, the images to be tested have to be either fully textural or to contain large areas that belong to textures. Besides, we analyze the case of grayscale images here and are interested in the result generality. Taking these aspects into consideration, 12 textural images presented in Fig. 1 were used in experiments same as in our previous paper.³⁶ Nine images that have indices 1–8, and 11, have been taken from USC-SIPI Image Database⁴⁴ and they can be treated as fully textural images where the texture is the same for the entire image. Two other test images #9 and #10 (Baboon and Grass) have been widely used in optical image analysis. The last test image (#12) is a good example of aerial remote sensing images of terrain having a complex (textural) structure; such textures are often used in geomorphometric analysis.⁴⁵
Is Texture Denoising Efficiency Predictable?

<figure><img src="image_2.png" /><figcaption>Fig. 1. Test texture images and their indices.</figcaption></figure>
O. Rubel et al.

Noise $n_{ij}$ in the observation model (1) is supposed to be zero mean, additive, white and Gaussian (AWGN), which is the most widely used model in image processing literature$^{36,48}$ and it is quite adequate for many practical situations. Moreover, if a noise is signal-dependent or multiplicative, processing is often carried out using a homomorphic or a variance stabilizing transform$^{24,38,40}$ that makes noise to be additive and close to Gaussian in the transformed images subject to filtering. Note, that the case of spatially correlated noise is more complicated, and some additional pre-whitening may be required. A more complex model has been briefly considered in Ref. 36. Consideration of spatially correlated noise models falls out of the scope of this paper; this could be a subject for future work.

A parameter that characterizes AWGN intensity is its variance $\sigma_0^2$ or a standard deviation (STD), $\sigma_0$. In general, noise variance values used in the filter performance analysis vary in a very wide range where one might often meet noise standard deviation (STD) values up to 100 in some recent studies. In our opinion, it is enough to use three practical values where STD = 5 relates to the case of hardly noticeable noise$^{16}$ for 8-bit represented images; the noise with such STD will be further treated as a low intensity noise. The STD = 10 corresponds to a middle intensity noise and the case of STD = 15 relates to an intensive noise that can be regarded as annoying.

## 2.2. Used metrics

Denoising efficiency can be analyzed and quantitatively characterized in many different ways. Probably, the most common is to present and analyze the output MSE (for a given noise variance)

$$ \text{MSE}_{\text{out}} = \frac{\sum_{i=1}^{I_{\text{Im}}} \sum_{j=1}^{J_{\text{Im}}} (I_{ij}^{f} - I_{ij}^{tr})^2}{(I_{\text{Im}} \cdot J_{\text{Im}})} \quad (2) $$

where $I_{ij}^f$ is a denoised image value for an $ij$th pixel. It is also possible to employ the ratio $\text{MSE}_{\text{out}}/\sigma_0^2$ Ref. 1. One more standard parameter is the output PSNR or the efficiency indicator, called improvement of PSNR, determined as $\text{IPSNR} = 10 \log_{10}(\sigma_0^2/\text{MSE}_{\text{out}})$ and expressed in dB.

Peculiarities of the Human Vision System (HVS) are in one way or another taken into account in the visual quality metrics (also called HVS-metrics). For example, the metric PSNR-HVS-M$^{30}$ is determined as

$$ \text{PSNR-HVS-M} = \frac{10 \log(255^2/\text{MSE}_{\text{HVS-M}})}{10} \quad (3) $$

where MSE$_{HVS-M}$ is the mean square error calculated in the DCT domain for 8 × 8 pixel blocks with weighting that takes into account two peculiarities of HVS: less sensitivity to distortions for higher spatial frequencies and masking effects, typical for texture. PSNR-HVS-M is also expressed in dB and, similarly to IPSNR, it is reasonable to calculate and analyse an improvement of PSNR-HVS-M (IPSNR-HVS-M) subtracting the input PSNR-HVS-M value from the output value.$^{36}$
Besides, we consider two other metrics called MSSIM<sup>52</sup> and FSIM.<sup>53</sup> These metrics are among the best in literature, especially for conventional types of distortions such as different types of noise, blur, distortions caused by lossy compression or filtering of noisy images.<sup>27</sup> In contrast to PSNR-HVS-M, MSSIM and FSIM metrics values vary from 0 to 1, where unity corresponds to the perfect visual quality. Similar to IPSNR-HVS-M, one can analyze an improvement of MSSIM (IMSSIM) and an improvement of FSIM (IFSIM) that are determined as differences in their values before and after denoising. A question is: how informative are IMSSIM and IFSIM without knowing MSSIM or FSIM for the original image. Note, that MSSIM or FSIM for original (noisy) images cannot be accurately determined in practice since the true image is absent.

## 2.3. Considered filters

We have already mentioned above which filters will be used. Our particular goal is to consider state-of-the-art filtering methods that belong to different classes. Let us briefly describe them. The DCT-based filter performs data processing in 8 × 8 pixel fully overlapping blocks using a hard thresholding of the DCT coefficients with the threshold set to 2.7σ₀. The BM3D filter<sup>11</sup> employs a search for similar blocks and two stages of 3D DCT-based processing to find similar patches with thresholding and weighted aggregation. Principles of operation for the wavelet-based filter TI-WS<sup>10</sup> and the nonlocal mean (NLM) filter<sup>4</sup> are well known. The BLS-GSM filter<sup>31</sup> exploits a complex model of wavelet coefficient statistics for their thresholding. Another considered filter, LPG-PCA, uses principal components analysis with a local pixel grouping to suppress noise in the spatial domain.<sup>50</sup> This filter employs vector variables to represent neighbor pixels as training patches for block matching. LPG-PCA can be performed iteratively to improve the denoising efficiency starting with adjusting the noise level from the second iteration.

The denoising performed iteratively to improve filtering efficiency has also been applied for spatial domain filters. Spatially adaptive iterative filter<sup>43</sup> (SAIF, available at https://users.soe.ucsc.edu/~htalebi/SAIF.php) processes iteratively the image local content using some base filter (we have used a NLM filter for this purpose) and automatically optimizes the iteration number with respect to the mean-squared error estimated by the SURE risk estimator.<sup>42</sup> We have also studied nonlocal denoising techniques in the transform domain. Two efficient clustering-based denoising schemes have been proposed recently: KSVD<sup>3</sup> and KLLD.<sup>6</sup> These filters learn dictionaries in different ways. KSVD (available at http://www.cs.technion.ac.il/~elad/software/) represents the signal in a sparse and redundant form and learns flexible and sparse dictionaries that are compact and provide efficient representation of sample data. This scheme has demonstrated its advantages performing 3D denoising. Alternatively, KLLD (available at https://users.soe.ucsc.edu/~priyam/K-LLD/) employs as features the local weight functions based on a steering kernel regression and uses the SURE risk estimator for denoising efficiency
O. Rubel et al.

improvement. A special texture-oriented denoising method based on gradient histogram preservation⁵⁴ (GHP, available at http://www4.comp.polyu.edu.hk/~cslzhang/code) enforces the gradient histogram to be close to the reference gradient histogram of the original image. The method estimates the reference histogram from noisy observations of unknown images. For NLM, LPG-PCA, KSVD, and KLLD filters, we have used parameter values recommended by authors in their scripts at the above mentioned sites.

### 3. Analysis of Texture Denoising Efficiency

Recall that our simulations have been carried out for 12 test images depicted in Fig. 1. Ten realizations of AWGN with aforementioned values of STDs (5, 10, and 15) have been added to each test image. For each test image and a given noise variance, denoising has been performed by each of the filters described above, namely, DCTF, BM3D, BLS-GSM, NLM, LPG-PCA, SAIF, KSVD, KLLD, and GHP. For each denoised image, the following parameters have been calculated:

* MSE<sub>out</sub>, then PSNR<sub>out</sub> and IPSNR;

* MSE<sub>HVS-M</sub>, then PSNR-HVS-M<sub>out</sub> and IPSNR-HVS-M;

* output value MSSIM<sub>out</sub> and, then, IMSSIM;

* output value FSIM<sub>out</sub> and, then, IFSIM.

It is supposed that the input values of controlled metrics (obtained for input, i.e. noisy images) are known or pre-estimated. The considered output metric values obtained for a given test image and a given noise variance have been later averaged for the analyzed noise realizations. Note that the values of the metrics IPSNR and IPSNR-HVS-M change from one realization to another by no more than 0.2 dB, whereas the values of other metrics did not change much.

The results for the noise STD equal to 5 are presented in Fig. 2. The first and the most obvious observation is that the data for the seven best filters (BM3D, BLS-GSM, DCT, SAIF, KSVD, KLLD, GHP) are very similar — plots for them practically coincide (to show difference in performance of these filters more clearly, Figures 2(c) and 2(d) present the plots for IPSNR and IPSNR-HVS-M using another scale). Meanwhile, the data for two filters, namely LPG-PCA and NLM, are sufficiently worse than for other filters according to all quality metrics.

The second observation is that, according to all visual quality metrics (in terms of their improvements), even the best filters do not, in fact, noticeably improve the quality of the processed texture images. Meanwhile, the use of LPG-PCA or NLM filters can lead to considerable degradations of the denoised image compared to the original. According to the metrics IPSNR and IPSNR-HVS-M, the test image #10 seems to be the most “unfavorable” for NLM. This filter smears the texture (see an example in Fig. 3(c)) thus degrading a visual quality. According to the metrics IMSSIM and IFSIM, the test images #5 and #12 are the most unfavorable for LPG-PCA.
Is Texture Denoising Efficiency Predictable?

<figure><img src="image_3.png" /><figcaption>Fig. 2. Determined metrics for the considered filters, AWGN with STD = 5.</figcaption></figure>

The third observation is that, according to IPSNR, the image quality can be slightly improved for some test images such as the test images #3 and #8. However, the improvement is so small (0.5–1 dB in the former case and 1.5–2.0 in the latter case) that it seems not worth applying filtering for these test images as well if noise is not intensive (for STD about 5 and less). Comparing image fragments in Figs. 3(a)
O. Rubel et al.

<figure><img src="image_4.png" /><figcaption>Fig. 3. The same fragment of noise-free (a), noisy (b), and denoised by NLM (c) test image # 2.</figcaption></figure>

and 3(b), one may conclude that they appear practically identical (noise is almost invisible and/or texture masks it).

Let us now consider the case of the moderate noise (AWGN with STD = 10). The obtained results are presented in Fig. 4. The results for seven filters (BM3D, BLS-GSM, DCT, SAIF, KSVD, KLLD, GHP) are again similar to each other according to all four metrics analyzed (to show a difference in performance of these filters more clearly, Figures 4(e) and 4(f) represent the plots for δMSSIM and δFSIM in a different scale) whilst performance of the NLM and LPG-PCA filters is still considerably worse for most test images.

Here, the difference in the filtering efficiency for the considered test images becomes essential. For example, IPSNR reaches 4 dB for the test image #8 whilst IPSNR for the test image #10 is less than 1 dB for the best filters. Visual quality does not improve sufficiently for all, even the best, filters. The IPSNR-HVS-M exceeds 1 dB only in a few cases; the metrics IMSSIM and IFSIM are also close to zero. Despite IPSNR is about 4 dB for the test image #8, the visual quality metrics do not indicate any improvement by filtering. One fragment of the noise-free, noisy and filtered test image #10 is presented in Fig. 5. Although noise is quite intensive, it is seen only in quasi-homogeneous regions (as leaves or grass) that occupy a small percentage of the total image area. Meanwhile, noise is masked by texture in other regions. Denoising removes noise well in the aforementioned quasi-homogeneous regions. In the textural fragments, noise is partly removed whilst texture is partly smeared. In aggregate, there is an impression that the visual quality has been slightly improved by filtering, and it is possible to “agree” with the visual quality metrics which indicate a small improvement.

Analysis of data in Fig. 4 shows that there is a certain agreement between results for the metrics IPSNR and IPSNR-HVS-M. At least, the largest improvements are observed for the test images #3, #8, #9, and #12. The metrics IMSSIM and IFSIM are, in general, in agreement between each other indicating the largest improvements for the test images #9 and #12. On average, we can state that the best performance
Is Texture Denoising Efficiency Predictable?

<figure><img src="image_5.png" /><figcaption>Fig. 4. Determined metrics for the considered filters, AWGN with STD = 10.</figcaption></figure>

is provided by GHP and BM3D filters although five others perform only a little worse.

Finally, let us consider the data obtained for intensive noise. The results are presented in Fig. 6. Compared to the earlier considered cases of less intensive noise, the improvements due to denoising are larger. The values of IPSNR vary from 1 to 5 dB for most filters. Moreover, even for the filters LPG-PCA and NLM the IPSNR
<figure><img src="image_6.png" /><figcaption>Fig. 5. The same fragment of noise-free (a), noisy (b), and denoised by GHP (c) test image #10.</figcaption></figure>

values are positive for all test images except one (#10). Concerning the visual quality metrics, LPG-PCA and NLM do not improve image visual quality. Other filters, in most cases, provide some improvement of the visual quality although this improvement is not large. According to the presented results, the filter GHP performs

<figure><img src="image_7.png" /><figcaption>Fig. 6. Determined metrics for the considered filters, AWGN with STD = 15.</figcaption></figure>
Is Texture Denoising Efficiency Predictable?

slightly better than its counterparts including BM3D. But, GHP filter performs slower and requires more memory than BM3D. Note that most filters improve the quality of the test image #8 in the largest degree.

The presented examples demonstrate that even if IPSNR-HVS-M exceeds 1 dB or IMSSIM exceeds 0.01, the improvement of visual quality after denoising is not obvious. There are even more problems with IPSNR, at least, in the case of texture image denoising. The presented example (Fig. 7) demonstrates that even the values

<figure><img src="image_8.png" /><figcaption>Fig. 7. The same fragment of noise free image #1 (a), noisy image with STD = 15 (b) and output images for LPG-PCA (c), DCTF (d), SAIF (e), KSVD (f), KLLD (g), BM3D (h), and GHP (i).</figcaption></figure>
O. Rubel et al.

of IPSNR about 3..4 dB do not guarantee that visual quality of the texture images
has been improved by filtering. An empirical rule can be the following: if a filter
provides IPSNR over 3 dB and IPSNR-HVS-M over 1 dB, then one can expect that
the filtering is expedient. Certainly, other empirical rules to perform denoising or to
skip it can be applied.

The difference between the input images and the denoised images for DCTF, NLM, BM3D, and K-SVD filters are presented in Ref. 34. It is shown there that the differences in texture regions are larger than the ones in homogeneous regions. The largest distortions are introduced by the NLM filter.

4. Filtering Efficiency Prediction

By denoising efficiency prediction, we mean that some indicator (metric) able to
quantitatively and adequately characterize filter performance can be estimated
without (before) denoising itself. Then, this indicator is somehow used (analyzed) in
order to undertake a decision: whether it is worth denoising this image, what filter to
use, what parameter of a chosen filter to employ, etc.

The used prediction procedure proposed in Ref. 42 and further advanced in
Refs. 33, 37, 38 and 51 is based on the following assumptions:

* There are input statistical parameters that can jointly or separately describe image complexity and noise intensity for an image to be denoised (we assume here that a noise type and parameters are either known or pre-estimated with an appropriate accuracy in advance. Currently, there are methods that provide an accurate estimation in a blind manner, see Ref. 11).

* There are indicators that are able to adequately characterize filtering performance.

* There is a strict correlation (available in advance, before filtering) that allows estimating this indicator (indicators) with certain accuracy.

Then, the prediction itself for a given noisy image presumes the following steps: calculation of input parameters and their use as arguments for estimating output parameters (denoising efficiency indicators).

Keeping this in mind, it becomes clear that there are certain requirements to such a prediction. The main points of these requirements are the following: (1) input parameters have to be calculated considerably faster than denoising; (2) output parameters have to be estimated (predicted) accurately enough for further processing (decision undertaking, filter parameter settings, etc.). Next, a set of subtasks needs to be solved: (1) What input parameters to apply? (2) What output parameters to use? (3) How to establish dependence between input and output parameters? (4) What accuracy of output parameter estimation (prediction) is appropriate for solving further tasks and how to provide such an accuracy? It is difficult to thoroughly study all these subtasks within the scope of a paper. Thus, let us discuss what is already known and point out what subtasks will be considered below.
Earlier studies<sup>33,38,51</sup> have shown the following.

Firstly, there exist quite simple statistics of DCT coefficients which can serve as good input parameters. They are, e.g. mean probabilities $P_{\alpha\sigma}$ that absolute values of AC DCT coefficients in a limited number of 8 × 8 pixel blocks do not exceed $\alpha\sigma_0$, where $\alpha$ is a non-negative value of the order 0.5–2.0 and STD of AWGN $\sigma_0$ is assumed a priori known or accurately pre-estimated. By saying a limited number of blocks, we meant that the local estimates $\hat{P}_{\alpha\sigma q}, q = 1, ..., Q$ are obtained in $Q$ (at least, 300–500) randomly selected blocks or more that allow estimating the aforementioned probabilities quite accurately.33 In general, the estimated $P_{\alpha\sigma}$ depends upon image properties and on the block positions, but the error in determination of $P_{\alpha\sigma}$ is not the dominant factor in prediction accuracy.

Secondly, it has been already demonstrated that IPSNR for many filters can be predicted quite accurately for DCT-based filters<sup>1,33,38,51</sup> and some other denoising techniques.<sup>34</sup> Accuracy can be characterized differently where the root mean square error (RMSE) of estimates is one of the most adequate and widely used quantitative criteria.<sup>5</sup> The results presented in Ref. 34 show that RMSE for IPSNR is smaller than 1 dB for most considered filters if $P_{0.5\sigma}$ is used as only one input parameter and simple dependences of exponential or polynomial types are employed in the scatter-plot data regression.

Now, we come to the methodology of obtaining the approximating (prediction, regression) curves that can be calculated in advance. Figure 8 presents examples of two scatter-plots obtained for metrics IPSNR (a) and IPSNR-HVS-M (b) on the input parameter $P_{2\sigma}$. Horizontal coordinate of a scatter-plot point corresponds to the metric value and the argument relates to the input parameter value for a given test image corrupted by AWGN with a certain variance, and then denoised by a considered filter. The data in Fig. 8 is presented for a DCT based filter where 40 test images have been used and STD values equal to 3, 5, 10, 15, 20, 25, and 30 cover a wide range of possible values for input and output parameters.

<figure><img src="image_9.png" /><figcaption>Fig. 8. Scatter-plots of IPSNR versus $P_{2\sigma}$ (a) and IPSNR-HVS-M versus $P_{2\sigma}$ (b).</figcaption></figure>
O. Rubel et al.

Examples in Fig. 8 additionally show regression curves where the following simple fitted exponential functions are used: IPSNR = 0.012 · exp(6.7 · P₂σ) and IPSNR-HVS-M = 0.002 · exp(8.3 · P₂σ). Visual analysis of the data in Fig. 8 shows the following. General tendencies in dependences of IPSNR and IPSNR-HVS-M on P₂σ are clear: if P₂σ is larger, the metric is larger too. Meanwhile, a comparison of the scatter-plots also reveals that it is much harder to predict IPSNR-HVS-M than IPSNR since the former scatter-plot exhibits a larger dispersion of points. This is confirmed by data in Ref. 34 where the RMSE values for IPSNR-HVS-M are mostly larger than 1 dB.

One more criterion that directly characterizes fitting (regression) and sufficiently influences the prediction accuracy is a goodness of fit parameter $R^2$ (see Ref. 5) that should approach unity if the output parameter dependence on the input parameter is essential and fitting is performed well. As is demonstrated,³⁴ $R^2$ for most filters exceeds 0.9 for IPSNR and is smaller than 0.9 for IPSNR-HVS-M. Thus, an accurate prediction of IPSNR-HVS-M is more problematic. A similar situation holds for IMSSIM and IFSIM where $R^2$ for them is about 0.85 for most of filters.³⁴ This means that an improvement of prediction is more important and problematic just for the visual quality metrics.

**5. New Solutions for Prediction of Denoising Efficiency Indicators**

As has been already demonstrated above and in Refs. 34, 33 and 51 it is more problematic to predict visual quality metrics than IPSNR. Several ways to improve the prediction (to increase $R^2$ and to reduce fitting RMSE) have been already proposed. They presume finding a better input parameter,³³ search and employment of a better approximation function,⁵¹ and usage of two or more input parameters³⁸ aggregated in one or another way. In the latter case, different statistics of local (calculated in analyzed blocks) probabilities have been employed: mean, median, variance, skewness, kurtosis. Below, we propose and study another approach where two input parameters are used in which the first is statistical and the second characterizes the quality of the original image subject to denoising.

Let us explain why we expect this approach to be reasonable. Firstly, looking at the scatter-plots in Figs. 8(b) and 9, these scatter-plots can be divided into three regions:

* $P_{2\sigma} < 0.5$ or $P_{0.5\sigma} < 0.25$ that mainly correspond to highly textural images corrupted by non-intensive noise for which denoising is practically useless since improvement of quality according to all considered metrics is negligible;

* $0.5 \le P_{2\sigma} < 0.9$ or $0.25 \le P_{0.5\sigma} < 0.35$ (that relate to either middle complexity images or to textural images corrupted by quite intensive noise) for which there is an essential diversity of metric values and denoising seems to be expedient for many, but not for all images;
Is Texture Denoising Efficiency Predictable?

<figure><img src="image_10.png" /><figcaption>Fig. 9. Scatter-plot of IMSSIM versus $P_{0.5\sigma}$ and the fitted curve for DCTF.</figcaption></figure>

* $P_{2\sigma} \ge 0.9$ or $P_{0.5\sigma} \ge 0.35$ for which it is worth employing denoising with a high probability of a positive result.

So, the main task is to improve prediction just for the second region ($0.5 \le P_{2\sigma} < 0.9$ or $0.25 \le P_{0.5\sigma} < 0.35$). Note, that IMSSIM less than 0.005 or IPSNR-HVS-M smaller than 0.5 dB practically cannot be considered as a visual quality improvement.³⁷

Secondly, it has been shown in Refs. 26 and 51 that IPSNR about 3 dB is not recognized as efficient denoising if noise in original images is intensive. Therefore, we can expect that the values of a metric that characterizes quality of the original (noisy) image can be helpful for better prediction of the metric that describes the image quality improvement due to denoising.

Let us now check our assumption for IPSNR and input PSNR used together. Recall that input PSNR can be easily determined for a known noise type and parameters and is able to characterize noise intensity irrespectively of image complexity.

There are numerous methods to aggregate two or more input parameters. To have an easy option of 2D curve fitting into scatter-plot of two arguments (see example in Fig. 10), let us use the fitting (regression) defined as $M_{\text{out}} = a \exp(bP_{0.5\sigma} + cM_{\text{inp}})$, where $M_{\text{out}}$ and $M_{\text{inp}}$ are used as output and input metrics, respectively; a, b, and c are the determined parameters of the fitted function.

Analysis of the data in Fig. 10 shows the following. The tendency of IPSNR to increase if $P_{0.5\sigma}$ increases remains. There is also a tendency for IPSNR increasing if
<figure><img src="image_11.png" /><figcaption>Fig. 10. Scatter-plot of two arguments for IPSNR and the fitted 2D function for DCTF.</figcaption></figure>

the input PSNR becomes smaller (the scatter-plot in Fig. 10 was obtained for the conventional DCT-based filter varying input PSNR from about 20 dB using noise STD = 30 to about 40 dB for noise STD = 3).

As a result of 2D fitting for the case in Fig. 10, we have $a = 0.12, b = 11.46$, and $c = 0.01$. The parameters that characterize fitting accuracy are the following: $R^2 = 0.956$ and RMSE = 0.671. These parameters both confirm that fitting is quite good and RMSE of IPSNR prediction is less than 1 dB. Meanwhile, the influence analysis of both input parameters shows that the role of $P_{0.5\sigma}$ is dominating: whilst the factor $\exp(bP_{0.5\sigma})$ varies by about 100 times in the limits of $P_{0.5\sigma}$ variation, the factor $\exp(cPSNR_{inp})$ varies only by about from 10 to 30%. In the case of fitting for the data obtained for only one input parameter for predicting IPSNR as IPSNR = $a \exp(bP_{0.5\sigma})$, the parameters are the following: $a = 0.18, b = 10.79, R^2 = 0.953$ and RMSE = 0.695, i.e. fitting results are only slightly worse than in the case of two input parameters.

The case considered above is good from different viewpoints. Firstly, fitting is fine for both cases of one and two input parameters. Secondly, in practice, it is possible to use one input parameter since this procedure is easier, but almost is of the same accuracy when two input parameters are used. Thirdly, the second input parameter can be calculated as well as PSNR = $10 \log_{10}(255^2/\sigma_0^2)$.

Unfortunately, such favorable conditions do not always take place in practice. For example, consider the case of IMSSIM and input MSSIM which is hypothetical (cannot be estimated) when the scatter-plot can be obtained by simulations. The
Is Texture Denoising Efficiency Predictable?

<figure><img src="image_12.png" /><figcaption>Fig. 11. Scatter-plot of two arguments for IMSSIM and the fitted 2D function (surface) for DCTF.</figcaption></figure>

scatter-plot and the fitted curve IMSSIM = *a* exp(*b*P<sub>0.5σ</sub> + *c*MSSIM<sub>inp</sub>) is represented in Fig. 11 for the conventional DCTF. First of all, the input MSSIM variation range is rather narrow (from about 0.7 to almost unity) although the same wide range of noise standard deviation variation has been used. This means that though the range from 0 to 1 is declared for the metric MSSIM, only a part of it is of value. Moreover, input values are concentrated in the neighborhood quite close to unity. It makes the use of the metric MSSIM quite complicated (the same relates to FSIM). Secondly, IMSSIM value depends on both input parameters sufficiently. In the case of two input parameters, *a* = 0.02, *b* = 13.68, *c* = -3.96, *R*² = 0.935, and RMSE = 0.02. Negative values of the parameter *c* mean that the IMSSIM value increases if the MSSIM value becomes smaller (this can be seen in Fig. 11). Depending upon the input MSSIM value, the IMSSIM value varies by several times in the limits of the input MSSIM variation. Therefore, it is desirable to take the input MSSIM value into account for the prediction (for the case in Fig. 9, fitting is characterized by *R*² = 0.856 and RMSE = 0.029, i.e. it is sufficiently less accurate). Since the input MSSIM value is not available (to our best knowledge, there are no methods to estimate it), the considered option to improve prediction cannot be realized in practice.

Let us give two more examples, both for the BM3D filter. Figure 12 represents the scatter-plot for IPSNR-HVS-M versus two input parameters. Fitting leads to $R^2 = 0.852$ and RMSE = 0.954. If IPSNR-HVS-M is predicted using only $P_{0.5\sigma}$, then $R^2 = 0.78$ and RMSE = 1.16. Obviously, a prediction is possible, but its accuracy is worse than in the case of two input parameters. Consider that PSNR-HVS-M for a noisy image again cannot be estimated.
<figure><img src="image_13.png" /><figcaption>Fig. 12. Scatter-plot of two arguments for IPSNR-HVS-M and the fitted 2D function (surface) for BM3D filter.</figcaption></figure>

Figure 13 presents the scatter-plot for IFSIM versus two input parameters. Similar to MSSIM, FSIM values vary in the limited range (0.7–1.0) and IFSIM considerably depends on both input parameters. Fitting is rather good: $R^2 = 0.906$ and RMSE = 0.017, but it is impossible to determine the input FSIM value in practice. In the case of using only one input parameter ($P_{0.5\sigma}$), fitting is sufficiently worse: $R^2 = 0.836$ and RMSE = 0.022.

Summarizing the obtained results, it is possible to conclude the following. Firstly, a prediction is, in general, possible not only for IPSNR and IPSNR-HVS-M (shown in our previous publications³³,³⁸) but for some other HVS-metrics as well, e.g. IMSSIM and IFSIM. Secondly, one potential way to improve the prediction accuracy is to use more than one input parameter.

Some input parameters such as input PSNR-HVS-M, MSSIM or FSIM cannot be determined. Therefore, we propose to use the input PSNR value as the second input parameter keeping in mind that it can be determined for a noisy image. The results obtained in this case for DCTF and BM3D filter are given in Table 1 in columns defined as $P_{0.5\sigma}$ + PSNR under the assumption that the input PSNR value is estimated without error.

As one can see, there is a sufficient accuracy improvement for predicting IPSNR-HVS-M compared to the case of using one input parameter (see the data in Table 1, columns marked as $P_{0.5\sigma}$). There is practically no improvement in the accuracy of predicting IMSSIM and IFSIM, and the results are worse than for a hypothetic case
Is Texture Denoising Efficiency Predictable?

<figure><img src="image_14.png" /><figcaption>Fig. 13. Scatter-plot of two arguments for IFSIM and the fitted 2D function (surface) for BM3D filter.</figcaption></figure>

(see the data in Fig. 13 and in Table 1 columns marked as $P_{0.5\sigma}$ + RealPar). We have also tried the noise standard deviation as the second input parameter employed alongside $P_{0.5\sigma}$. The obtained data (not presented in Table 1) is very similar to the earlier case using input PSNR. Hence, we prefer applying input PSNR as a more general characteristic of noise. The best results for a practically realizable combination of two input parameters are marked in Table 1 in bold.

Table 2 presents results for accuracy of fitting characterized by RMSE. The same notations are used. These results coincide well with the data in Table 1. If $R^2$ in Table 1 for a given filter and metric is larger, RMSE in Table 2 is smaller. The smallest RMSE for each filter and practically realizable combination of two input parameters is marked in bold. Although the RMSE values for IPSNR and IPSNR-HVS-M are considerably larger than those for IMSSIM and IFSIM, the former two metrics are expressed in dB and vary in infinite limits. Also, note that the accuracy of predicting IPSNR-HVS-M is always worse than the accuracy of predicting IPSNR. Similarly, the accuracy of predicting IFSIM is better than the accuracy of predicting IMSSIM.

We have also analyzed another approach to improve the accuracy that was proposed in Ref. 38. The input parameters are some statistics of the local estimates $\hat{P}_{\alpha\sigma q}, q = 1, \dots, Q$ of the probabilities in blocks. The best results have been obtained in Ref. 44 for the mean and variance (VarP) of the local estimates. Hence, let us consider this combination in our experiments. The obtained results are presented in columns marked as $P_{0.5\sigma}$ + VarP.
<table><caption>Table 1. $R^2$ for fitted exponential curves.</caption><thead><tr><th rowspan="2">Pred. Param.</th><th colspan="4">DCTF</th><th colspan="4">BM3D</th></tr><tr><th>P<sub>0.5σ</sub></th><th>P<sub>0.5σ</sub> + RealPar</th><th>P<sub>0.5σ</sub> + PSNR</th><th>P<sub>0.5σ</sub> + VarP</th><th>P<sub>0.5σ</sub></th><th>P<sub>0.5σ</sub> + RealPar</th><th>P<sub>0.5σ</sub> + PSNR</th><th>P<sub>0.5σ</sub> + VarP</th></tr></thead><tbody><tr><td>IPSNR</td><td>0.953</td><td><strong>0.956</strong></td><td><strong>0.956</strong></td><td>0.953</td><td><strong>0.949</strong></td><td><strong>0.953</strong></td><td>0.953</td><td>0.949</td></tr><tr><td>IPSNR-HVS-M</td><td>0.772</td><td>0.848</td><td><strong>0.854</strong></td><td>0.812</td><td>0.780</td><td>0.852</td><td><strong>0.856</strong></td><td>0.818</td></tr><tr><td>IMSSIM</td><td>0.856</td><td>0.935</td><td>0.865</td><td><strong>0.915</strong></td><td>0.856</td><td>0.933</td><td>0.864</td><td><strong>0.914</strong></td></tr><tr><td>IFSIM</td><td>0.843</td><td>0.917</td><td>0.847</td><td><strong>0.892</strong></td><td>0.837</td><td>0.908</td><td>0.840</td><td><strong>0.882</strong></td></tr></tbody></table>

<table><caption>Table 2. RMSE for fitted exponential curves.</caption><thead><tr><th rowspan="2">Predicted Parameter</th><th colspan="4">DCTF</th><th colspan="4">BM3D</th></tr><tr><th>P<sub>0.5σ</sub></th><th>P<sub>0.5σ</sub> + RealPar</th><th>P<sub>0.5σ</sub> + PSNR</th><th>P<sub>0.5σ</sub> + VarP</th><th>P<sub>0.5σ</sub></th><th>P<sub>0.5σ</sub> + RealPar</th><th>P<sub>0.5σ</sub> + PSNR</th><th>P<sub>0.5σ</sub> + VarP</th></tr></thead><tbody><tr><td>IPSNR</td><td>0.695</td><td><strong>0.671</strong></td><td><strong>0.671</strong></td><td>0.693</td><td>0.756</td><td><strong>0.727</strong></td><td><strong>0.727</strong></td><td>0.757</td></tr><tr><td>IPSNR-HVS-M</td><td>1.103</td><td>0.902</td><td><strong>0.885</strong></td><td>1.006</td><td>1.164</td><td>0.954</td><td><strong>0.943</strong></td><td>1.061</td></tr><tr><td>IMSSIM</td><td>0.029</td><td>0.020</td><td>0.028</td><td><strong>0.022</strong></td><td>0.030</td><td>0.020</td><td>0.029</td><td><strong>0.023</strong></td></tr><tr><td>IFSIM</td><td>0.021</td><td>0.016</td><td>0.021</td><td><strong>0.018</strong></td><td>0.022</td><td>0.017</td><td>0.022</td><td><strong>0.019</strong></td></tr></tbody></table>
Is Texture Denoising Efficiency Predictable?

<table><caption>Table 3. Approximation functions for prediction of efficiency indicators.</caption><thead><tr><th>Filter</th><th>Metric</th><th>Formula</th><th>Formula Parameters</th><th>RMSE</th><th>R<sup>2</sup></th></tr></thead><tbody><tr><td>DCTF</td><td>IPSNR</td><td>a * exp(b * P<sub>0.5σ</sub> + c * PSNR)</td><td>a = 0.11608;<br>b = 11.4586;<br>c = 0.0088937</td><td>0.671</td><td>0.956</td></tr><tr><td>DCTF</td><td>IPSNR-HVS-M</td><td>a * exp(b * P<sub>0.5σ</sub> + c * PSNR)</td><td>a = 0.001865;<br>b = 18.4505;<br>c = 0.057281</td><td>0.885</td><td>0.854</td></tr><tr><td>DCTF</td><td>IMSSIM</td><td>a * exp(b * P<sub>0.5σ</sub> + c * VarP)</td><td>a = 8.7575e-06;<br>b = 30.0195;<br>c = -260.1751</td><td>0.022</td><td>0.915</td></tr><tr><td>DCTF</td><td>IFSIM</td><td>a * exp(b * P<sub>0.5σ</sub> + c * VarP)</td><td>a = 8.3438e-07;<br>b = 35.2152;<br>c = -256.571</td><td>0.018</td><td>0.892</td></tr><tr><td>BM3D</td><td>IPSNR</td><td>a * exp(b * P<sub>0.5σ</sub> + c * PSNR)</td><td>a = 0.11074;<br>b = 11.6527;<br>c = 0.0097466</td><td>0.727</td><td>0.953</td></tr><tr><td>BM3D</td><td>IPSNR-HVS-M</td><td>a * exp(b * P<sub>0.5σ</sub> + c * PSNR)</td><td>a = 0.0015764;<br>b = 19.0617;<br>c = 0.057539</td><td>0.943</td><td>0.856</td></tr><tr><td>BM3D</td><td>MSSIM</td><td>a * exp(b * P<sub>0.5σ</sub> + c * VarP)</td><td>a = 7.6169e-06;<br>b = 30.4887;<br>c = -264.8369</td><td>0.022</td><td>0.914</td></tr><tr><td>BM3D</td><td>FSIM</td><td>a * exp(b * P<sub>0.5σ</sub> + c * VarP)</td><td>a = 6.1862e-07;<br>b = 36.0104;<br>c = -254.8657</td><td>0.019</td><td>0.882</td></tr></tbody></table>

Analysis shows that there is a considerable improvement of the prediction accuracy for all visual quality metrics compared to the case of using only $P_{0.5σ}$. Obviously, there is a difference between the considered metrics. For IPSNR-HVS-M, it is better to use the input PSNR value as the second parameter. On the contrary, it is better to apply the combination $P_{0.5σ} + VarP$ for IMSSIM and IFSIM.

The fitting curve parameters for the best combinations of two input parameters (those marked in bold in Tables 1 and 2) are presented in Table 3. As is seen, IPSNR can be predicted well even if only one input parameter is used (see data in Tables 1 and 2). The potential accuracy of the IPSNR-HVS-M prediction is worth improving although the use of the second input parameter (input PSNR) helps to provide a sufficiently better accuracy. For IMSSIM and IFSIM, it is worth using VarP as the second input parameter and the $R^2$ values are already about 0.9. This shows that a rather good prediction is possible but improving its efficiency is still worth trying. We should stress besides that the fitting function parameters for a given metric (e.g. IPSNR) are very close for the DCT and BM3D filters.

One might think that the obtained prediction results relate only to DCT-based filters since one input parameter is $P_{0.5σ}$. This is not so. The prediction approaches are considerably more general. To demonstrate this, we have collected data for six
<figure><img src="image_15.png" /><figcaption>Fig. 14. Scatter-plot of IMSSIM versus $P_{0.5\sigma}$ and the fitted curve for all six filters.</figcaption></figure>

filters (namely, DCTF, BM3D, BLSGSM, SAIF, KSVD, and KLLD) into joint scatter-plots — with one input parameter if only $P_{0.5\sigma}$ is employed, and two input parameters (pairs $P_{0.5\sigma}$ and input PSNR, $P_{0.5\sigma}$ and VarP). Fitting functions for one and two arguments have been obtained. Two examples are presented in Figs. 14 and 15. The scatter-plot in Fig. 14 can be compared to the scatter-plot in Fig. 9. It is seen that the main properties of these scatter-plots are very similar. Moreover, the $R^2$ and RMSE values are very similar, too. The only difference in these scatter-plots is in the number of points (six times more points for the scatter-plot in Fig. 14).

If two parameters are used, the prediction is more accurate (see data in Fig. 15). The value $R^2$ increases and RMSE reduces sufficiently.

It is possible to present all obtained scatter-plots. Instead, to save space, only the main conclusions and data are given. For IPSNR and IPSNR-HVS-M, it is worth using input PSNR as the second parameter, whilst it is better to employ VarP as the second parameter for IMSSIM and IFSIM. The obtained $R^2$ and RMSE values are very close to those presented for DCTF and BM3D in Tables 1 and 2. Here, we present only the best results and parameters of the fitting functions in Table 4. As is seen, the values of analyzed parameters are very close to the corresponding values in Table 3. We can state that the provided approximations can be used for all six filters. In other words, for each particular image to be denoised, it is possible to predict what the efficiency indicators for the best existing filters are.

Let us come back to the prediction accuracy. Clearly, it sufficiently depends upon quality of fitting, but there are also other factors mentioned earlier. Obtaining the
Is Texture Denoising Efficiency Predictable?

<figure><img src="image_16.png" /><figcaption>Fig. 15. Scatter-plot of IMSSIM versus $P_{0.5\sigma}$ and VarP and the fitted function for all six filters.</figcaption></figure>

scatter-plot and approximating functions can be considered as a special learning task. If so, a question is to verify the prediction. For this purpose, we have taken an extra 36 images from the database ESPL-LIVE HDR Image Quality database¹⁵ not used for learning. Scatter-plots have been obtained for the DCTF filter and then the $R^2$ and RMSE parameters have been calculated with respect to the earlier obtained approximations (Table 4). The new data is collected in Table 5. Its analysis shows

<table><caption>Table 4. Approximation functions for prediction of efficiency indicators for six filters.</caption><thead><tr><th>Metric</th><th>Formula</th><th>Formula Parameters</th><th>RMSE</th><th>$R^2$</th></tr></thead><tbody><tr><td>IPSNR</td><td>a * exp(b * $P_{0.5\sigma}$ + c * PSNR)</td><td>a = 0.129; b = 11.404; c = 0.006</td><td>0.739</td><td>0.950</td></tr><tr><td>IPSNR-HVS-M</td><td>a * exp(b * $P_{0.5\sigma}$ + c * PSNR)</td><td>a = 0.002; b = 18.328; c = 0.052</td><td>0.885</td><td>0.851</td></tr><tr><td>IMSSIM</td><td>a * exp(b * $P_{0.5\sigma}$ + c * VarP)</td><td>a = 9.801e-6; b = 29.654; c = -256.345</td><td>0.023</td><td>0.912</td></tr><tr><td>IFSIM</td><td>a * exp(b * $P_{0.5\sigma}$ + c * VarP)</td><td>a = 8.754e-7; b = 34.892; c = -246.875</td><td>0.018</td><td>0.879</td></tr></tbody></table>

<table><caption>Table 5. Approximation function verification for the DCTF.</caption><thead><tr><th>Metric</th><th>Formula</th><th>RMSE</th><th>$R^2$</th></tr></thead><tbody><tr><td>IPSNR</td><td>a * exp(b * $P_{0.5\sigma}$ + c * PSNR)</td><td>1.441</td><td>0.821</td></tr><tr><td>IPSNR-HVS-M</td><td>a * exp(b * $P_{0.5\sigma}$ + c * PSNR)</td><td>1.852</td><td>0.451</td></tr><tr><td>IMSSIM</td><td>a * exp(b * $P_{0.5\sigma}$ + c * VarP)</td><td>0.043</td><td>0.810</td></tr><tr><td>IFSIM</td><td>a * exp(b * $P_{0.5\sigma}$ + c * VarP)</td><td>0.044</td><td>0.732</td></tr></tbody></table>
O. Rubel et al.

the following. As can be seen, RMSE values have increased almost twice and $R^2$ have reduced. The IPSNR-HVS-M prediction using a generalized approximation for all filters is unsatisfactory and still requires some improving. For the other three metric indicators, we can state that the designed prediction is quite general, stable, and accurate.

From this analysis, it is possible to conclude the following. Firstly, we can recommend using individual approximating functions instead of generalized approximations given in Table 4. Secondly, it is worth using more than 40 test images to obtain scatter-plots for further curve fitting. Thirdly, some examples of scatter-plots show that more complex functions can be used in fitting to obtain smaller RMSE and larger $R^2$. In spite of all these ideas and recommendations for further improvements of the prediction accuracy, we can state that a prediction of denoising efficiency indicators is possible.

If an answer to the question in the paper title is positive, then the next question is whether to apply denoising or not. The following procedure has been proposed in Ref. 26. The initial step 1 is to skip filtering if $P_{0.5\sigma} < 0.25$. For larger $P_{0.5\sigma}$, the rule could be: apply filtering if the predicted IPSNR value exceeds 3.5 dB and the predicted IPSNR-HVS-M value exceeds 1 dB.

Step 1 is motivated by an analysis of many scatter-plots, e.g. those given in Figs. 9, 10, and 12. Improvements of the metrics in this case are negligible. The situation changes if $P_{0.5\sigma}$ exceeds 0.25. Then, a rather large IPSNR ($>3.5$ dB) is needed to guarantee an essential improvement of the image quality. Concerning the visual quality metrics, the following study has been carried out for images in the database TID2013. A reliable denoising efficiency measure is the opinion of observers that have assessed a quality of noisy and filtered images. Note, such images and assessments exist for the databases TID2008 and TID2013.²⁷,²⁹ There are images distorted by the AWGN (distortion type # 1) and distortions due to denoising (distortion type # 9) in these databases. They contain 25 test images and four and five levels of distortions for TID2008 and TID2013, respectively. In our further analysis, we have used data for the database TID2013 since it is more advanced. Each database image is characterized by the corresponding mean opinion score (MOS) values that can be treated as a reliable assessment of the image visual quality (higher MOS corresponds to a better visual quality).

The scatter-plot of MOS vs PSNR-HVS-M values is presented in Fig. 16 where the fitted straight lines are given (points of red color relate to images corrupted by AWGN, blue color points correspond to images with residual distortions after denoising). These lines are in a good agreement (the angle between them is small). This shows that the metric PSNR-HVS-M correlates with MOS well enough for the analyzed types of distortions. If PSNR-HVS-M is about 35–40 dB (almost invisible distortions), MOS values for the filtered images are higher than for the images corrupted by AWGN. The situation is slightly different for PSNR-HVS-M smaller than 30 dB. To be sure that the denoising is worth applying, one needs the predicted
Is Texture Denoising Efficiency Predictable?

<figure><img src="image_17.png" /><figcaption>Fig. 16. Scatter-plots of MOS versus PSNR-HVS-M for noisy and filtered images with fitted lines.</figcaption></figure>

value of IPSNR-HVS-M to be positive. This explains why we have proposed using
the threshold equal to 1 dB at the second stage of our procedure.

The metrics IMSSIM and/or IFSIM can be potentially used in the decision un-
dertaking as well. However, their peculiarities described above prevent giving simple
and direct rules. More studies are necessary to provide such rules.

**6. Conclusions**

Analysis of denoising efficiency has been carried out for several modern filters with the application to texture images corrupted by AWGN. Different visual quality metrics are employed in the analysis and comparisons. It is demonstrated that noise removal from texture images is complicated, and even the most sophisticated existing filters often have low filtering efficiency in terms of the used metrics. Visual examples confirm this observation. In such situations, it is reasonable to skip denoising in order to save resources.

The corresponding decision can be undertaken in an automatic manner based on
the prediction of the parameters characterizing the filtering efficiency. Such a pre-
diction can be fast and accurate enough. Several ways to improve the accuracy are
put forward. The use of input PSNR as the second input parameter provides a
considerable improvement of a prediction accuracy. It is shown that a general pre-
diction approach is possible for the set of the best existing filters despite the fact that
O. Rubel et al.

they belong to different groups according to their operation mechanisms and the exploited image and noise properties.

Analysis has demonstrated that the prediction accuracy is worse for visual quality metrics than for the conventional PSNR. Although the prediction accuracy has been improved for visual quality metrics, it is worth continuing research in this direction.

## Acknowledgment

This work was partially supported by Instituto Politecnico Nacional as a part of research project No. 20171559.

## References

1. S. Abramov, S. Krivenko, A. Roenko, V. Lukin, I. Djurovic and M. Chobanu, Prediction of filtering efficiency for DCT-based image denoising, in *Proc. 2-nd Mediterranean Conf. Embedded Computing (MECO)*, (Budva, Montenegro, 2013), pp. 97–100.

2. V. V. Abramova, S. K. Abramov, V. V. Lukin, K. O. Egiazarian and J. T. Astola, On required accuracy of mixed noise parameter estimation for image enhancement via denoising, *EURASIP J. Image Video Process.* **2014** (2014) 3.

3. M. Aharon, M. Elad and A. M. Bruckstein, The K-SVD: An algorithm for designing of overcomplete dictionaries for sparse representations, *IEEE Trans. Signal Process.* **54**(11) (2006) 4311–4322.

4. A. Buades, A. Coll and J. M. Morel, A non-local algorithm for image denoising, in *Proc. Computer Vision and Pattern Recognition (CVPR)* (San Diego, USA, 2005), pp. 60–65.

5. C. Cameron, A. Windmeijer, A. G. Frank, H. Gramajo, D. E. Cane and C. Khosla, An R-squared measure of goodness of fit for some common nonlinear regression models, *J. Econom.* **77**(2) (1997) 329–342.

6. P. Chatterjee and P. Milanfar, Clustering-based denoising with locally learned dictionaries, *IEEE Trans. Image Process.* **18**(7) (2009) 1438–1451.

7. P. Chatterjee and P. Milanfar, Is denoising dead? *IEEE Trans. Image Process.* **19**(4) (2010) 895–911.

8. P. Chatterjee and P. Milanfar, Practical bounds on image denoising: From estimation to information, *IEEE Trans. Image Process.* **20**(5) (2011) 1221–1233.

9. F. Cheikh, B. Cramariuc and M. Gabbouj, MUVIS: A system for content-based indexing and retrieval in large image databases, in *Proc. Workshop Very Low Bit Rate Coding, VLBV* (Urbana, USA, October 1998), pp. 41–44.

10. R. R. Coifman and D. Donoho, Translation-invariant denoising, in *Wavelets and Statistics* (Springer Verlag, 1995), pp. 125–150.

11. K. Dabov, A. Foi, V. Katkovnik and K Egiazarian, Image denoising by sparse 3D transform-domain collaborative filtering, *IEEE Trans. Image Process.* **16**(8) (2007) 2080–2095.

12. C. A. Deledalle, L. Denis and F. Tupin, How to compare noisy patches? Patch similarity beyond Gaussian noise, *Int. J. Comput. Vis.* **99**(1) (2012) 86–102.

13. C. O. Dumitru, J. Singh and M. Datcu, Selection of relevant features and TerraSAR-X products for classification of high resolution SAR images, *EUSAR 2012*, 24–26 April 2012, Nuremberg, Germany, pp. 243–246.
Is Texture Denoising Efficiency Predictable?

14. M. Elad and M. Aharon, Image denoising via sparse and redundant representations over learned dictionaries, *IEEE Trans. Image Process.* **15**(6) (2006) 3736–3745.

15. ESPL-LIVE HDR Subjective Image Quality Database http://live.ece.utexas.edu/research/HDRDB/hdr_index.html.

16. D. Fevralev, V. Lukin, N. Ponomarenko, S. Abramov, K. Egiazarian and J. Astola, Efficiency analysis of color image filtering, *EURASIP J. Adv. Signal Process.* **2011** (2011) 41.

17. G. Gilboa, N. Sochen and Y. Y. Zeevi, Variational denoising of partly-textured images by spatially varying constraints, *IEEE Trans. Image Process.* **15**(8) (2006) 2281–2289.

18. R. Haralick and D. Dori, A pattern recognition approach to detection of complex edges, *Pattern Recognit. Lett.* **16**(5) (1995) 517–529.

19. S. Lazebnik, C. Schmid and J. Ponce, A sparse texture representation using local affine regions, *IEEE Trans. Pattern Anal. Mach. Intell.* **27**(8) (2005) 1265–1278.

20. M. Lebrun, M. Colom, A. Buades and J. M. Morel, Secrets of image denoising cuisine, *Acta Numer.* **21** (2012) 475–576.

21. A. Lucieer and A. Stein, Texture-based landform segmentation of LiDAR imagery, *Int. J. Appl. Earth Obs. Geoinf.* **6** (2005) 261–270.

22. V. Lukin, R. Oktem, N. Ponomarenko and K. Egiazarian, Image filtering based on discrete cosine transform, *Telecommun. Radio Eng.* **66**(18) (2007) 1685–1701.

23. V. Lukin, N. Ponomarenko and K. Egiazarian, HVS-metric-based performance analysis of image denoising algorithms, in *Proc. European Workshop Visual Information Processing (EUVIP)* (Paris, France, 2011), pp. 156–161.

24. F. Murtagh and J. L. Starck, Astronomical image and signal processing, *IEEE Signal Processing Magazine* **18**(2) (2001) 30–40.

25. M. Petrou and P. Gacia Sevilla, *Image Processing: Dealing with Texture* (Wiley, 2006).

26. O. Pogrebnyak and V. Lukin, Wiener discrete cosine transform based image filtering, *SPIE J. Electron. Imaging* **21**(4) (2012) 1–15.

27. N. Ponomarenko, O. Ieremeiev, V. Lukin, K. Egiazarian, L. Jin, J. Astola, B. Vozel, K. Chehdi, M. Carli, F. Battisti and C. C. Jay Kuo, Color image database TID2013: Peculiarities and preliminary results, in *Proc. EUVIP* (Paris, France, June 2013), pp. 106–111.

28. N. Ponomarenko, O. Ieremeiev, V. Lukin, K. Egiazarian, L. Jin, J. Astola, B. Vozel and K. Chehdi, Image visual quality metrics verification by TID2013: Exploring of mean square error drawbacks, in *Proc. European Workshop Visual Information Processing (EUVIP)* (Paris, France, 2014), pp. 1–6.

29. N. Ponomarenko, V. Lukin, A. Zelensky, K. Egiazarian, M. Carli and F. Battisti, TID2008 - A database for evaluation of full-reference visual quality assessment metrics, *Adv. Modern Radioelectron.* **10** (2009) 30–45.

30. N. Ponomarenko, F. Silvestri, K. Egiazarian, M. Carli, J. Astola and V. Lukin, On between-coefficient contrast masking of DCT basis functions, in *Proc. Int. Workshop Video Process. Quality Metrics (VPQM-07)* (Scottsdale, USA, January 2007).

31. J. Portilla, V. Strela, M. J. Wainwright and E. Simoncelli, Image denoising using scale mixtures of gaussians in the wavelet domain, *IEEE Trans. Image Process.* **12**(11) (2003) 1338–1351.

32. T. Randen and J. H. Husoy, Filtering for texture classification: A comparative study, *IEEE Trans. Pattern Anal. Mach. Intell.* **21**(4) (1990) 291–310.

33. O. Rubel and V. Lukin, An improved prediction of DCT-based filters efficiency using regression analysis, *Inf. Telecommun. Sci.* **5**(1) (2014) 30–41.
O. Rubel et al.

34. O. Rubel, V. Lukin, S. Abramov, B. Vozel, K. Egiazarian and O. Pogrebnyak, Efficiency of texture image filtering and its prediction, *Signal, Image Video Process.* **10** (2016) 1543–1550. doi: 10.1007/s11760-016-0969-3.

35. A. Rubel, V. Lukin and O. Pogrebniak, Efficiency of DCT-based denoising techniques applied to texture images, in *Proc. MCPR 2014* (Cancun, Mexico, 2014), pp. 261–270.

36. A. Rubel, V. Lukin, M. Uss, B. Vozel, O. Pogrebnyak and K. Egiazarian, Efficiency of texture image enhancement by DCT-based filtering, *Neurocomput.* **155** (2016) 948–965, doi: 10.1016/j.neucom.2015.04.119.

37. O. S. Rubel, V. V. Lukin and F. S. de Medeiros, Prediction of despeckling efficiency of DCT-based filters applied to SAR images, in *Proc. Int. Conf. Distributed Comput. Sensor Syst. (DCOSS)* (Fortaleza, Brazil, June 2015), pp. 159–168.

38. A. Rubel, A. Naumenko and V. Lukin, A neural network based predictor of filtering, in *Proc. Microwaves, Radar and Remote Sensing Symposium (MRRS)* (Kiev, Ukraine, 2014), pp. 14–17.

39. O. Rubel, N. Ponomarenko, V. Lukin, K. Egiazarian and J. Astola, HVS-based local analysis of denoising efficiency for DCT-based filters, in *Proc. Conf. Problems Information, Science and Technology* (Kharkov, Ukraine, 2015), pp. 189–192.

40. R. Schowengerdt, *Remote Sensing: Models and Methods for Image Processing* (Academic Press, 2006).

41. H. R. Sheikh, M. F. Sabir and A. C. Bovik, A statistical evaluation of recent full reference image quality assessment algorithms, *IEEE Trans. Image Process.* **15**(11) (2006) 3441–3452.

42. C. M. Stein, Estimation of the mean of a multivariate normal distribution, *Ann. Stat.* **9**(6) (1981) 1135–1151. doi: 10.1214/aos/1176345632.JSTOR2240405.

43. H. Talebi, X. Zhu and P. Milanfar, How to SAIF-ly boost denoising performance, *IEEE Trans. Image Process.* **22**(4) (2013) 1470–1485.

44. The USC-SIPI Image Database, http://sipi.usc.edu/database/.

45. S. Trevisani and M. Rocca, MAD: Robust image texture analysis for applications in high resolution geomorphometry, *Comput. Geosci.* **81** (2015) 78–92.

46. O. Tsymbal, V. Lukin, N. Ponomarenko, A. Zelensky, K. Egiazarian and J. Astola, Three-state locally adaptive texture preserving filter for radar and optical image processing, *EURASIP J. Appl. Signal Process.* **8** (2005) 1185–1204.

47. M. Tuceryan and A. Jain, *The Handbook of Pattern Recognition and Computer Vision*, 2 edn. (World Scientific Publishing Co., 1998).

48. M. L. Uss, B. Vozel, V. V. Lukin and K. Chehdi, Image informative maps for component-wise estimating parameters of signal-dependent noise, *J. Electron. Imaging.* **22** (1) (2013) 013019. doi: 10.1117/1.JEI.22.1.013019.

49. E. Vansteenkiste, D. Van der Weken, W. Philips and F. Kerre, Perceived image quality measurement of state-of-the-art noise reduction schemes, *Lect. Notes Comput. Sci.* **4179** (2006) 114–124.

50. M. Vijay and S. V. Subha, Spatially adaptive image restoration using LPG-PCA and JBF, in *Proc. Int. Conf. Machine Vision and Image Processing* (Tamil Nadu, India, 2012), pp. 53–56.

51. B. Vozel, O. Rubel, A. Zemliachenko, S. Abramov, S. Krivenko, R. Kozhemiakin, V. Lukin and K. Chehdi, Processing of multichannel remote-sensing images with prediction of performance parameters, environmental applications of remote sensing, Ed. Prof. Maged Marghany, InTech, 2016. doi:10.5772/61853.

52. Z. Wang, A. C. Bovik, H. R. Sheikh and E. P. Simoncelli, Image quality assessment: From error visibility to structural similarity, *IEEE Trans. Image Process.* **13**(4) (2004) 600–612.
53. L. Zhang, X. Mou and D. Zhang, FSIM: A feature similarity index for image quality assessment, *IEEE Trans. Image Process.* **20**(5) (2011) 2378–2386.

54. W. Zuo, L. Zhang, C. Song, D. Zhang and H. Gao, Gradient histogram estimation and preservation for texture enhanced image denoising, *IEEE Trans. Image Process.* **23**(6) (2014) 2459–2472.

<figure><img src="image_18.png" /></figure>

Oleksii Rubel received his Ph.D. in Digital Signal Processing for Remote Sensing from the National Aerospace University, Department of Receivers, Transmitters and Signal Processing, Kharkiv, Ukraine, in February 2016. His interests include image enhancement and

design of image processing algorithms. His current work focuses on image processing algorithms adaptation to different noise models and its effectiveness prediction.

<figure><img src="image_19.png" /></figure>

Sergey Abramov graduated from the National Aerospace University, Kharkov, Ukraine in 2000 and received his Diploma with Honors in Radio Engineering. Since then, he has been with the Department of Transmitters, Receivers and Signal Processing of National

Aerospace University. He received the Candidate of Technical Science degree in DSP for remote sensing from the National Aerospace University in 2004. His research interests include digital signal/image processing automation, blind noise characteristics evaluation, image filtering and compression.

<figure><img src="image_20.png" /></figure>

Vladimir V. Lukin graduated from Kharkov Aviation Institute (now National Aerospace University) in 1983 and received his Diploma with Honors in Radio Engineering. Since then, he has been with the Department of Transmitters, Receivers and Signal

Processing of National Aerospace University. He presented his thesis of Candidate of Technical Science in 1988 and Doctor of Technical Science in 2002 in DSP for Remote Sensing. Since 1995, he has been in cooperation with Tampere University of Technology. Currently he is the department vice-chairman and a professor. His research interests include digital signal/image processing, remote sensing data processing, image filtering and compression.

<figure><img src="image_21.png" /></figure>

Benoit Vozel obtained his State Engineering degree, his M.Sc degree in Control and Computer Science in 1991 and his Ph.D. in 1994 from École Centrale de Nantes (France). As a Ph.D. student, he was with the Signal Processing Group at the Institut de Recherche

en Communication et Cybernétique de Nantes, where he worked on the detection of abrupt changes in signals. Since 1995, he has been with the ENSSAT, where he is currently with the signal and multicomponent/multimodal image processing research team (TSI2M) within the Institute of Electronics and Telecommunications of Rennes (IETR). His research interests generally concern blind estimation of noise characteristics, image filtering and restoration, and adaptive image and remote sensing data processing.
O. Rubel et al.

<figure><img src="image_22.png" /></figure>

**Oleksiy Pogrebnyak**
received his Ph.D. from Kharkov Aviation Institute (now National Aerospace University), Ukraine, in 1991. Currently he is with The Center for Computing Research of National Polytechnic Institute, Mexico. His research interests include digital signal/image filtering and compression, remote sensing.

<figure><img src="image_23.png" /></figure>

**Karen O. Egiazarian**
received his Ph.D. from Moscow M. V. Lomonosov State University, Russia, in 1986, and his Doctor of Technology degree from Tampere University of Technology, Finland, in 1994. He is a leading scientist in signal, image, and video processing with about 300 refereed journal and conference articles, three book chapters, and a book published by Marcel Dekker, Inc. His main interests are in the field of multirate signal processing, image and video denoising and compression, and digital logic. He is a member of the DSP Technical Committee of the IEEE Circuits and Systems Society.