Venue: ICLR 2021 Conference

Reviews:

final_score: rating: 5: Marginally below acceptance threshold
review_content: This paper proposes a setting called "recognition-aware image processing." The key idea is to make the images output by image processing methods still be readily recognized by image recognition methods. Realizing this will help to better meet the requirement from both human observers and machines. Formally, this is formulated as a combined optimization in which the losses from image processing and recognition tasks are jointly considered. This framework is further extended to unsupervised case and the case of intermediate transformer to make it more flexible. The transferability issue is discussed and it is observed that the model trained by the proposed method can generally help even when other recognition models or tasks are used. Experimental study is conducted to demonstrate the performance of the proposed method. 

The issue considered in this paper is interesting, and the proposed method is a good attempt to address this issue. The result shows that it is possible to balance between the performance of image processing and that of image recognition. This paper is well written and easy to follow. 

Meanwhile, 

1. This paper may need more solid cases to justify the necessity of such a setting in practice. The benefit of combining image processing and image recognition from this perspective needs to be better explained. As shown by the experiment, image processing quality is more or less affected, and a careful tuning of the tradeoff value may be needed. In this case, separating these two tasks may lead to an overall easier solution.  

2. The technical contribution of this work is a bit limited. The essence of the proposed work is just a weighted combination of the image processing loss and image recognition loss, with some extension to the unsupervised case and the intermediate transformer case. However, the overall novelty does not seem to be sufficient. 

3. The transferability highlighted in this work needs to be more rigorously analyzed and justified. 

--- Thank the authors for the detailed response. This paper investigates an interesting issue. However, the overall technical contribution and novelty of this work remains to be a concern. After reading the response and the comments of peer reviewers, the rating is maintained as follows. 
review_title: An interesting paper but not strong enough

final_score: rating: 5: Marginally below acceptance threshold
review_content: This paper proposes an interesting image processing approach to deal with the issue that the processed images cannot be recognized by machines. To address this issue,  the author proposes the so-called recognition-aware image processing strategy in which an extra discriminator is employed to classify the semantic annotation of given samples.

Strength:
-  The idea is technically sound. Such a simple design can theoretically improve the quality of processed images in either supervised or unsupervised manner.
-  The paper is easy to understand and follow. 
-  The experiments are solid to demonstrate the usefulness of the proposed method. The authors evaluate their design on three different image processing scenarios, i. e., super-resolution, denoising, deblocking, and the improvements in all scenarios are impressive. 

Weakness:
- The novelty is somewhat limited. The idea proposed in this paper is very similar to the motivation of multi-task learning and also the ideas in several few-shot learning works [1,2], which employ extra discriminators to improve the feature quality. To this end, the contribution of this paper maybe not enough.
- Is there any clear illustration of why an image processing network has negative impacts on recognition accuracy? 

[1] Boosting Few-Shot Visual Learning with Self-Supervision 

[2] Rethinking Class Relations: Absolute-relative Few-shot Learning
review_title: An interesting paper but the novelty is somewhat not enough

final_score: rating: 6: Marginally above acceptance threshold
review_content: The paper proposed a learnable image processing methods that improve machine interpretability of processed image. The paper mainly claimed that improvement of machine recognition is transferrable when evaluated on models of different architectures, recognized categories, tasks and training datasets. Additionally, the paper also try to explain this transferability phenomenon by demonstrating the similarities of different models’ decision boundaries.

Pros:
(1) The writing is clear and easy to follow. The problem is well formulated and experiments is comparably extensive.
(2) The paper proposed a unsupervised RA scheme without the Recognition label, which maximize the agreement of recognition output between original and processed image, this approach sounds novel to me.
(3) The Decision Boundaries Analysis and Transferability part is interesting, to my knowledge is the first work that to investigate the transferability of RA.

Cons:
(1) The problem that high-level task can benefits low-level task has already been well-studied in multiple literature [1][2][3][4], the first half of the paper is just a revisit of those methods while swaping differen backbones and tasks. I acknowledge that 3 representative low-level tasks has been studied, however for high-level part the paper only studied recognition and detection task, more semantically intensive task such as segmentation is missing.
(2) The paper simply adding a transformer in between Process module and Recogition module, to guarantee the image processing quality not affected. This seem to be a trivial approach and need more experiment to justify.


[1] Sreya Banerjee, Rosaura G VidalMata, Zhangyang Wang, and Walter J Scheirer. Report on ugˆ 2+ challenge track 1: Assessing algorithms to improve video object detection and classiﬁcation from unconstrained mobility platforms. arXiv preprint arXiv:1907.11529, 2019. 3
[2] Sicheng Wang, Bihan Wen, Junru Wu, Dacheng Tao, and Zhangyang Wang. Segmentation-aware image denoising without knowing true segmentation. arXiv preprint arXiv:1905.08965, 2019.
[3] Boyi Li, Xiulian Peng, Zhangyang Wang, Jizheng Xu, and Dan Feng. End-to-end united video dehazing and detection. In Thirty-Second AAAI Conference on Artiﬁcial Intelligence, 2018.
[4] Ding Liu, Bihan Wen, Xianming Liu, Zhangyang Wang, and Thomas S Huang. When image denoising meets high-level vision tasks: A deep learning approach. arXiv preprint arXiv:1706.04284, 2017.
review_title: Interesting Paper

Meta-review: Three reviewers have reviewed this paper and they maintain their findings after the rebuttal. The reviewers are mainly concerned about the novelty (several highly-related papers exist) and well as the technical contribution (more theoretical developments are needed). Therefore, this paper in its current form cannot be accepted.
Decision: Reject