Edge-aware Image Smoothing with Relative Wavelet Domain Representation

Authors: Huiqing QI, Xiaoliu Luo, Tingting Li, Fang Li
Primary area: optimization
Keywords: Image smoothing, Wavelet transformation, Relative wavelet domain representation, Edge-preserving, Non-convex optimization

Abstract:
Image smoothing is a fundamental technique in image processing, designed to eliminate perturbations and textures while preserving dominant structures. It plays a pivotal role in numerous high-level computer vision tasks. More recently, both traditional and deep learning-based smoothing methods have been developed. However, existing algorithms frequently encounter issues such as gradient reversals and halo artifacts. Furthermore, the smoothing strength of deep learning-based models, once trained, cannot be adjusted for adapting different complexity levels of textures.  These limitations stem from the inability of previous approaches to achieve an optimal balance between smoothing intensity and edge preservation. Consequently, image smoothing while maintaining edge integrity remains a significant challenge. To address these challenges, we propose a novel edge-aware smoothing model that leverages a relative wavelet domain representation. Specifically, by employing wavelet transformation, we introduce a new measure, termed Relative Wavelet Domain Representation (RWDR), which effectively distinguishes between textures and structures. Additionally, we present an innovative edge-aware scale map that is incorporated into the adaptive bilateral filter, facilitating mutual guidance in the smoothing process. This paper provides complete theoretical derivations for solving the proposed non-convex optimization model. Extensive experiments substantiate that our method has a competitive superiority with previous algorithms in edge-preserving and artifact removal. Visual and numerical comparisons further validate the effectiveness and efficiency of our approach in several applications of image smoothing.

Decision: accept (poster)
Reviewer scores: 6, 6, 6

Reviews and discussion:
<thread>
creation date:
1730511723728

thread_type:
official_review

<thread_object>
creator: reviewer_m1zn

<content>
summary:
In this article, the author reviews image smoothing methods based on local information, global information, and deep learning, and discusses the limitations of current image smoothing techniques when dealing with image textures and image structural edges. To address this issue, first, the author proposes a novel edge-aware smoothing model that more effectively distinguishes between image textures and image structures through relative wavelet domain representation (RWDR). Second, the author reintroduces edge-aware scale maps into bilateral filters to improve image edges during the smoothing process. Finally, the author demonstrates the superiority of this method in texture preservation and artifact removal after image smoothing through comprehensive theoretical derivations and experimental results compared to other algorithms.

strengths:
1.This paper proposes a relative wavelet domain representation and an edge-aware smoothing model, achieving certain progress in image smoothing technology; 2.The paper utilizes extensive theoretical proofs to establish a mathematical model for the relative wavelet domain. The experimental validation is well-supported by theory; 3.The writing of this paper is relatively fluent and conforms to the standards of English academic writing.

weaknesses:
1.In the experimental part of this paper, there is a predominance of qualitative analysis of images. However, due to the significant subjective factors inherent in qualitative experiments, supplementing with more quantitative experiments would enhance the persuasiveness of the results; 2.Image smoothing operations, as one of the fundamental image processing tasks, play a crucial role in various visual tasks. However, the paper seems to lack exploration of specific visual tasks (for example, in super-resolution tasks, the textures and structures preserved after image smoothing are vital for image reconstruction).

questions:
1.In the experimental part of this paper, there is a predominance of qualitative analysis of images. However, due to the significant subjective factors inherent in qualitative experiments, supplementing with more quantitative experiments would enhance the persuasiveness of the results; 2.Image smoothing operations, as one of the fundamental image processing tasks, play a crucial role in various visual tasks. However, the paper seems to lack exploration of specific visual tasks (for example, in super-resolution tasks, the textures and structures preserved after image smoothing are vital for image reconstruction).
</content>
</thread_object>

</thread>
<thread>
creation date:
1730620833331

thread_type:
official_review

<thread_object>
creator: reviewer_kxjp

<content>
summary:
The author introduces a mutually guided edge-aware smoothing model based on relative wavelet domain representation. Their proposed RWDR serves as a novel measure for effectively differentiating between textures and structures.

strengths:
1. The solution of the proposed model is supported by a complete theoretical guarantee, which is a strong point.
2. Extensive experiments prove that the proposed method outperforms existing algorithms in mitigating gradient reversals,
 staircase artifacts, and halos and achieves a superior performance in balancing smoothing strength and edge preservation.

weaknesses:
1. Though the authors support their claims by extensive qualitative results, but they should also provide the quantitative results to validate their points in the main paper or at least in the supplementary. For instance, the authors can include PSNR (Peak Signal-to-Noise Ratio), SSIM (Structural Similarity Index), or MSE (Mean Squared Error) on standard synthetic benchmark datasets and LPIPS, MUSIQ, NIQE, MANIQA for real-world tasks. This would allow for a more objective comparison with existing method.
2. The method section needs to be refined, as mentioned in Fig1 (that the detail enhancement image is boosted by four detail layers), this statement is not explained in the method section, how the four detail layers are being generated , is it from the wavelet decomposition?
3. The paper has lacks ablation study. The authors have given mathematical proofs of choosing the particular operations like RWDR and the edge-aware scale map, they should also try to prove the effectiveness of each proposed component  on the overall model.
4. The authors should also try to report the results on some real-world applications in Super-Resolution (RealSR, DrealSR,RealLR200), denoising (SIDD, DND) that would further prove the use of the proposed model, if the time permits and should also check on synthetic SR datasets like Manga109, Urban100, and BSD68 (for denoising).

questions:
Please check the weakness section.
</content>
</thread_object>

</thread>
<thread>
creation date:
1730650128221

thread_type:
official_review

<thread_object>
creator: reviewer_bdth

<content>
summary:
The main contribution of this work is the introduction of RWDR that effectively distinguishes textures from primary structures and preserves weaker edges. Additionally, the paper proposes an innovative edge-aware scale map method that dynamically adjusts scale based on the image structure, resulting in clearer distinctions between structure and texture. Experimental results demonstrate that the proposed approach provides superior edge-preserving smoothing compared to existing methods.

strengths:
1. This paper introduces relative wavelet domain representation into bilateral filtering, which is reasonable and novel.

2. The method achieves superior visual results compared to previous studies. 

3. The paper includes comprehensive theoretical derivations, technical descriptions, and runtime analysis of the algorithm.

weaknesses:
1. The paper provides extensive visual results, but I’m curious how different algorithms are objectively evaluated based on visual quality. The authors should consider comparing performance on downstream tasks with objective metrics. A user study could also statistically confirm the advantages of the proposed method.

2. As a new method, it likely performs well in certain scenarios. However, I am more interested in its robustness and stability. In other words, can the authors provide a lower bound for the algorithm's performance? In which scenarios might it fail? Additionally, how sensitive is the algorithm to parameter changes?

questions:
Could the authors provide an online demo to allow users to test the method easily? While it’s not essential for acceptance, it would add value for potential users.
</content>
</thread_object>

</thread>
<thread>
creation date:
1732331230283

thread_type:
official_comment

<thread_object>
creator: authors

<content>
title:
Response Summary

comment:
We sincerely appreciate all reviewers' thorough and constructive comments. We are pleased that the reviewers recognized our method as **reasonable and novel (Reviewer bdtH)**, **complete theoretical guarantee and superior performance (Reviewer kXjP)**, and our **fluent and academic writing (Reviewer M1ZN)**.

Our main responses are as following five folds:

1. Provided comprehensive **objective qualitative results** in the tables below;

2. Clarified the **generation mechanism of the detail layer** in Figure 1;

3. Conducted **detailed ablation studies** in Figure 8 and Figure 9;

4. Clarified our method's **parameters sensitive analysis and recommended settings**;

5. Provided the **specific visual task analysis** and the importance of our approach in real-world applications.

Notably, all these improvements are incorporated into our revised manuscript while maintaining clarity and technical depth.
***
We are grateful for helping us improve our manuscript’s quality and completeness.
</content>
</thread_object>

</thread>


Metareview:
metareview:
This paper proposes an effective Relative Wavelet Domain Representation (RWDR) for edge-preserving image smoothing. Experimental results show that the proposed method preserve main edges well. 

The major concerns of reviewers including adding more experimental results (e.g., subjective evaluations) and limitation analysis. 

In the rebuttal, the authors solve the concerns of reviewers. Based on the recommendations of reviewers, the paper can be accepted.