Venue: ICLR 2020 Conference

Reviews:

final_score: rating: 8: Accept
review_content: This paper is about a novel method to add watermarks to images and audio that is highly robust to several transformations that is closely related to gan methods. The idea is that the watermark signal is learned concurrently to the detector network, which share similarities to a generator and detector networks. Five standard attack transformations are considered and a specific optimization to reduce the transferability of the watermark is considered. The method is compared against Broken arrows on Cifar10 and Imagenet. It shows similar or better performance for Gaussian noise attack given the same amount of perturbance allowed in a signal while very much better performance for the other attacks. Going beyond the five attacks, the paper also includes an estimation of the probability of confidence of finding watermarked images given a fixed l2 norm radius. Finally the method is also tested on audio on a proprietary dataset with a deepspeaker architecture which still shows very good performance and it is confirmed by a human evaluation where participants found the watermarked audio not significantly worse or degraded.

The paper is well written and of excellent presentation. The proposed method is novel and goes in the interesting direction of learning watermarks using adversarial training techniques. Experiments shows that the method has good performance but it is only compared to Broken Arrows (i.e. a zero bit watermarking) which is the state of the art of this type of watermarking. It would have been interesting a comparison to other key based watermarking methods to also deeply evaluate the signal transformation attacks. All in all, I found the paper to be significant and I would like to see it accepted.
review_title: Official Blind Review #3

final_score: rating: 6: Weak Accept
review_content: The authors introduce ReSWAT, a method for transformation-resilient watermarking of images via adversarial training. The high level idea is to learn a watermark/detector pair (W,D). W can be any transformation (in this paper, an l-infty bounded perturbation) that imputes an imperceptible distortion to a given input, while D is a detector that distinguishes watermarked from non-watermarked images. There is an additional requirement that the detector should be robust to simple transformations such as rotations, cropping, flipping, and contrast enhancement. 

The authors pose a min-max style learning problem for learning (W,D) that leads to a natural adversarial training scheme. In particular, it can be viewed as creating an adversarial defense to the Expectations-over-transformations attack of Athalye et al. Experimental results on a bunch of different datasets confirm that the method works,

The paper is well-written and the contributions are clear. In terms of conceptual or theoretical novelty, the paper is limited (the method essentially boils down to regular adversarial training with a slightly non-standard loss function) but the connection to watermarking seems novel and is nicely executed.

My only concern is the lack of comparisons with baselines. I am not a digital forensics expert -- so I don't know what the state of the art is -- but the only comparisons made are with the Broken Arrows (BA) watermarking , which seems to be over 10 years old, so I am not sure how to evaluate the results.
review_title: Official Blind Review #1

final_score: rating: 1: Reject
review_content: The paper proposes a watermark design algorithm that is based on an adversarial training paradigm where a watermark detector and a watermark generator are jointly trained to maximize the likelihood of watermark detection while bounding the manipulation of individual pixels. 

I have several concerns about the clarity of the paper, its applicability to the motivating problem, and the completeness of the results that motivate my "reject" recommendation.

The description does not make it clear how the detector and watermark mechanism are jointly obtained in (1) - intuitively, a detector should know the type of watermark to be observed in order to pose a feasible detection model. The discussion focuses on "fixed detector" and "fixed watermark" cases only, but it is not clear how an iterative optimization would be initialized, and no discussion of how to set up the problem appears in the manuscript. 

The motivation for the paper is also unclear. The authors discuss the need to be able to detect deep fakes, which is a relevant and meritorious problem. However, it seems that the application of the proposed approach in this setting would require deep fake creators to add watermarks to their creations, which would be a naive assumption. Watermarks are used to demonstrate provenance, but deep fake creators would not want to have such provenance verified.

The watermark transferability premise seems to rely on the random initialization of the underlying deep learning watermark detector. One could posit that if a third party is interested in creating false positives, they could train the deep learning network using a database of watermarked and non-watermarked images. Another concern here is that the formulation provided in (3) would potentially lead to low transferability for the specific classifiers provided there, but one does not know which classifier an adversary would construct - or why an adversary would construct an adversary that does not have knowledge of the specific watermark being used, or of samples of the watermark. It would have been interesting to see if it is possible to construct a good watermark detector this way (given enough training data being available). It would also be good to have some intuition as to why this robustness formulation also improves "certified robustness".

Finally, the performance comparison only considers a 10-year old watermarking algorithm; some discussion as to why this is sufficient should have been included.

Minor comments
The norm infinity subindex is missing from the minimization conditions in (1).
Figure 3 caption: watermark's -> watermarks
It is not clear why no numerical results were given for Section 4.2.4.
review_title: Official Blind Review #2

Meta-review: This paper offers an interesting and potentially useful approach to robust watermarking.  The reviewers are divided on the significance of the method.  The most senior and experienced reviewer was the most negative.  On balance, my assessment of this paper is borderline; given the number of more highly ranked papers in my pile, that means I have to assign "reject".
Decision: Reject