Venue: NeurIPS 2021 Conference

Reviews:

final_score: rating: 6: Marginally above the acceptance threshold
review_content: summary: This paper proposes a dual-stream network design equipped with an interactive strategy (coined YTMT) to tackle the blind source separation tasks, especially the single image reflection separation. The underlying design motivation comes from a very simple mathematical derivation, i.e., any information discarded from one layer might be beneficial to reconstruct another layer. They demonstrate performance improvement on the task of single image reflection removal. The possible extensions to other source separation tasks, e.g., the blind image denoising and intrinsic image decomposition, are also investigated. 
main_review: ## Pros
* The introduction part is well written; the design motivation is clear and reasonable. 
* Good performance on SIRS against state-of-the-art methods. 
* The potential applications to other blind source separation tasks. 

## Cons
Though the underlying principle looks sound, it largely relies on the assumption that blind source separation could be better handled by  dual-stream networks. Such an assumption is, however, not established in majority of source separation tasks. To separate two layers, one may argue only a single network with two output branches at tail is sufficient to interact information within shared backbone in an implicit manner. The authors should consider adding such an ablation study (with comparable computation overhead) to demonstrate the benefit of dual-stream network design.

Besides, in Table 3, the performance of two-UNet should be identical to a single UNet, as there is no interaction from another stream, thus the comparison there is not so meaningful. The descriptions of Split UNet and Concat. UNet are also unclear, the authors should give some explanations on why they design such architectures for ablation. The ablation study in Table 4 looks redundant, since neither the two stage design nor the attention module is the authors' contribution. The authors should reorganize the paper by moving other content, e.g., the intrinsic image decomposition, from the suppl. material into the main paper. 

## Summary
Overall, the proposed idea is simple yet reasonable, but more experimental verifications are needed to justify its superiority. 
I tentatively give a borderline score and would change it according to the authors' response. 

## Post rebuttal
After reading the rebuttal and other reviewers' comments, I think most issues raised by the reviewers are repairable, and many of them have been already addressed in the rebuttal. As a result, I lean to borderline accept and it's ok for me if the rejection is made on this paper. 
Also, I highly recommend the authors refine the method and experiment parts, particularly, describing the abbreviations (e.g., Concat. UNet, YTMT-UCS) carefully to avoid potential confusion and distraction. 
limitations_and_societal_impact: N/A
review_title: null

final_score: rating: 6: Marginally above the acceptance threshold
review_content: summary: In this work, the authors address the problem of how to get a good approximation of a scene affected by reflections produced by a pane of glass. To do this, the authors proposed a dual-stream decomposition strategy called YTMT, where the main idea is how to effectively exchange information to enforce a better approximation of the target scene (and also of the reflection layer). At the optimisation level, the authors use a composite loss function which involves the reconstruction term along with three additional terms to enforce better output quality based for instance in constraining the gradients of the image.  Experimental setting shows readily competing results with respect to the existing techniques for reflection removal. Moreover, authors show the potential of the technique for other tasks such as blind denoising.


main_review: [Strengths] 
--   The main strength of the paper is the idea of how to efficiently exchange/use information through a dual-stream decomposition strategy.  Moreover, the experimental results are competitive wrt the body of literature. 

[Weaknesses]
------ A major point that turns down somehow the paper is the lack of formalism and intuition of the technique. More precisely:

-- The authors consider the image reflection model as a linear superposition between two layers I=T+R. However, different works, even early ones in the area, for  example [*] [**] showed that the reflection is less in focus and often blurred then considering in the model a weighting factor w regarding the strength of the reflections and a blurring kernel k. Why do the authors not consider this more realistic scenario as a model?

[*] Y.  Schechner,  N.  Kiryati,  and  R.  Basri,  “Separation  of  transparent layers  using  focus,”International  Journal  of  Computer  Vision  (IJCV),vol. 39, no. 1, pp. 25–39, 2000.
[**]Li  and  M.  S.  Brown,  “Single  image  layer  separation  using  relative smoothness,”  inIEEE  Conference  on  Computer  Vision  and  PatternRecognition (CVPR), 2014, pp. 2752–2759.

-- The main driving optimisation (7) is a combination of existing works and ideas that has been already applied to the task at hand including the gradients constraint and the exclusion term. Firstly, can the authors provide further thoughts on the level of novelty on the loss? Secondly, authors argue that they use an adversarial loss to ‘regulate the solution on the image manifold’  Can the authors argue where is coming that regularisation and where the space of images is modelling as a manifold?

-- The authors in the introduction motivate the approach and discuss an inherent ‘error/residual Q’. How does it fit into the big picture of the main driving optimisation and how Q is obtained? 

-- From (3) authors set ‘a’ ‘b’ ‘c’ and ‘d’ empirically but what  was the range of values searched to set such values? Similarly for the set of  \lambda in (7). What is the intuition behind those parameters wrt the performance (e.g. what is the effect of increasing or decreasing them)? 


------------ [Experimental Results] Although the experimental setting has merit, there are some points to clarify. 
-- The authors are using SSIM and PSNR metrics for the quantitative results. However, several works including [a] and [b] have discussed that need for more perceptual meaningful metrics such as the sLMSE [b]. Why not using more perceptural metrics such as the sLMSE? That avoid for example  that a single edge error to dominate the overall performance

-- What is the motivation of Table 2? One is interested in recovering the target scene rather than the reflection itself. Whilst the reflection layer indeed helps in the model formulation the relevant output is the target scene (transmission layer).

-- From Table I, It is unclear the level of performance gain from the proposed technique. The numerics seem not to have a statistical significant difference wrt ERRNet (and Lei et al). Authors should argue the main benefit of the proposed technique. 

-- The authors in the opening of the paper (footnote 1 in page 1) mentioned that the proposed technique can be applied to other tasks but they are not being explored in the paper as the main topic is reflection removal. However, authors actually include experiments regarding blind denoising -- Therefore, the title is not matching the content of the paper. Authors need to further motivate the inclusion of section 5


[a]Wan, R., Shi, B., Duan, L. Y., Tan, A. H., & Kot, A. C. (2017). Benchmarking single-image reflection removal algorithms. In Proceedings of the IEEE International Conference on Computer Vision (pp. 3922-3930).
[b] Grosse, M. K. Johnson, E. H. Adelson, and W. T. Free-man. Ground truth dataset and baseline evaluations for in-trinsic image algorithms. InProc. ICCV, 2009.



------------------------------------------------------------
POST-REBUTTAL

After reading the rebuttal carefully, the authors addressed my concerns and even provided further experiments to further support their statements. * And additional minor comment is regarding Q9, what I wanted to covey was that, from a reader point of view, it would be better to have only results on the task of reflection removal rather than adding another task (blind denoising) with a short description. Overall, I’m more positive about this paper than before and this is therefore reflected in the final suggested score -- expecting the authors to implement the points from the rebuttal in a the revised version for completeness. 




limitations_and_societal_impact: There is not a clear discussion on the limitations of their work.
review_title: null

final_score: rating: 7: Good paper, accept
review_content: summary: This paper presents a nonlinear module that leverages both positive and negative ReLUs for feature separation. This enables applications such as reflection removal, blind denoising and other layer separation tasks. The paper shows several variants of adding the proposed module into existing CNN frameworks. Results show superior performance on several benchmark datasets.
main_review: The idea of this paper is simple and intuitive. It's executed well with promising results and good ablation study. The novelty of this paper is the way of exploiting all features learned in a CNN and identifying its effectiveness in layer separation tasks where pixels are interfered but useful. The training scheme, data generation, objective functions are all from prior works, but the simple design of keeping all features in the network by adding dual-ReLU branch is interesting and novel, and can potentially inspire further research works.

Nevertheless, I have the following concerns:

**Results**

Although the model is tested on several datasets, not all results are shown in the supplement and the paper. Quality degrades (compared to results in the paper) for some images in fig 4 and 5 in the supplement. The limitation of the relatively worse performance isn't clear.

**Ablation**

While the paper presents several ablation studies on how features are fused and the backbone architectures, some supporting experiments are missing for the key claim of keeping the 'trash' features. In other words, is the benefit from having the negative branch?

What happens if the model only keeps the negative ReLU or only the positive ReLU (which is similar to the vanilla backbone)? For example, to keep the number of parameters the same, how does the model perform if replacing all negative ReLU with positive ones and vice versa? What features are lost and how does it show up in the image space? (e.g. is it like residual artifacts? etc.)

The paper also mentions the design alleviates dead ReLU. Can the two branch use a Leaky ReLU? What would be the difference? There are many interesting questions raised from the proposed framework, but the paper seems to only touch on a few. I'd hope to get more insights from the authors and how they made their design choice of only experimenting with ReLU and how they are thinking about this problem.

**Others**

Line 102 mentions the iterative approach suffers from accumulated errors. However, how is the proposed two-stage method handling this problem, given the second stage also depends on the output from the first stage?


## **Post-rebuttal**

The authors have addressed my concerns with more quantitative results and intuitive explanations. I'd encourage the authors to add some of these explanations into the main paper for clarity. I support acceptance of this paper. The simple design of positive-negative ReLU to preserve all features in the network is interesting and novel, and can potentially inspire further research works.

limitations_and_societal_impact: **Missing references**

The paper is missing a set of works on internal learning for layer separation: 
- Ulyanov, Dmitry, Andrea Vedaldi, and Victor Lempitsky. "Deep image prior." CVPR 2018.
- Gandelsman, Yosef, Assaf Shocher, and Michal Irani. "" Double-DIP": Unsupervised Image Decomposition via Coupled Deep-Image-Priors." CVPR 2019.

**Writing**

Writing is mostly clear.
- line 164 & 238: the choose of -> the choice of
review_title: null

final_score: rating: 6: Marginally above the acceptance threshold
review_content: summary: 
This paper presents a method that recovers reflection and transmission components from a given image degraded with reflections. The proposed network employs a dual-stream interactive strategy where the discarded information (by ReLU) from one branch helps improving the feature representation of the other branch and vice versa. Experiments are performed for several image processing tasks, including image reflection removal, denoising, demoireing, and intrinsic image decomposition.  

main_review: The idea is simple and provides favorable results for image decomposition tasks, especially for image reflection removal.  However, I have the following concerns.


* Image reflection removal is a position-sensitive task where pixel-to-pixel correspondence from the input to output images is required. For such restoration tasks, those methods that operate at high-resolution (having no downsampling operations) usually perform better than the UNet-like architectures. Why did you use YTMT-UCT as a default choice over the YTMT-PCT version? 
    
    

* Among the UNet versions of YTMT, Why do you opt for YTMT-UCT as a default architecture to perform main experiments, when in the ablation experiments (Table 4) YTMT-UAT shows overall better performance than the YTMT-UCT version?
    


* The evaluation for the image denoising task is not convincing. The proposed method performs poorly when compared to DnCNN (which is five years old method) for Gaussian noise levels up to sigma=75. The difference in performance might increase even further when compared to recent image denoising approaches such as RIDNet, RDN, PAN, etc. It is only when the noise levels are sigma=80 and above, the proposed method shows performance improvement over DnCNN. Don't you think that the experimental setting with such high noise levels is unrealistic and impractical?  



* Since the evaluation for the image reflection removal task is performed on real-world datasets, it would have been interesting to see the performance of YTMT on \textit{blind} image denoising task on real datasets such as SIDD and DND.

    

* In Table 1 you are comparing the results produced by the proposed YTMT method with those of the IBCLN algorithm (CVPR20). The authors of IBCLN also provided a new real-world dataset called \textit{Nature}. Did you compute the image quality scores on the Nature dataset?
    


* The main architecture diagram (Figure 2a) is complicated and the text explaining it isn't well written either (Sec 3.2). While some parts are fine, the overall paper is not easy to follow.



 * * * *  *  *  *  *

### **Post Rebuttal Comments**
* The authors have provided a convincing response to my concerns. Overall, the idea of this paper is simple and intuitive and provides good results. I recommend this paper for acceptance. However, I would suggest the authors to revise the paper to improve the text and simplify the figures.   
limitations_and_societal_impact: Adequately addressed.
review_title: null

Meta-review: The reviewers in general liked the idea and results of the paper: simple designs of keeping all features in the network by adding dual-ReLU branch; good results on SIRS against SOTA. Many issues were raised, e.g. lack of comparison, formalism and intuition of the technique. The reviewers were satisfied with the rebuttal and bumped up the score. Please incorporate the post-rebuttal comments from the reviewers, e.g. having only results on the task of reflection removal rather than adding another task with a short description (can use the space to make the figures larger), and improving the text and simplifying the figures.
Decision: Accept (Poster)