Provenance detection through learning transformation-resilient watermarking

1 INTRODUCTION . Generative models have contributed to impressive advancements in content generation and representation learning in both digital image and audio domains ( Brock et al . ( 2018 ) ; Kalchbrenner et al . ( 2018 ) ; Mehri et al . ( 2016 ) ; Zhu et al . ( 2017 ) ; Prenger et al . ( 2019 ) ; Donahue & Simonyan ( 2019 ) ; Oord et al . ( 2016 ) ; Goodfellow et al . ( 2014 ) ; Kingma & Welling ( 2013 ) ) . However , as generative models learn to better match a target distribution , the distinction between natural signals and synthetic signals generated by a model has blurred , leading to a raft of concerns over the potential misuse of these models ( Chesney & Citron ( 2018 ) ) . For example , synthetic videos that are indistinguishable from natural videos , sometimes referred to as deep fakes , have the potential to cause widespread distrust in traditional media . While the problem of detecting synthetic signals is interesting in its own right , it is challenging to do in a manner that is independent of the model used to generate the signal . Instead , we consider the problem of provenance detection via watermarking , a technique that involves injection of a carefully chosen ( but imperceptible ) perturbation ( watermark ) into the signal at the time of creation . The presence of the perturbation in the signal can be later used to detect the provenance ( or ultimate source ) of this signal . While the simplicity of the technique makes watermarking an appealing technique for detecting synthetic ( as well as naturally generated ) signals , it is susceptible to adversarial actors that can systematically try to remove the watermark by transforming the signal ( so as to obfuscate the provenance ) . In the case of images , this may take the form of cropping , addition of Gaussian noise or blurring , rotating images etc . Many watermarking schemes break down under these types of transformations . In this paper , we propose a novel transformation-resilient watermarking scheme- Resilient Signal Watermarking via Adversarial Training ( ReSWAT ) that is able to detect the presence of a watermark even after the signal has been through systematic attempts to remove the watermark . We use ideas from the literature on adversarial training ( Madry et al. , 2017 ) to learn to synthesize watermarks ( encoding the provenance of a signal ) that can be detected even after the watermarked signal has been through transformations that seek to remove the watermark . Our detection method can operate in any domain with continuous data representations . Experimental results demonstrate that our method is effective and substantially improves over existing methods - it can reliably detect the provenance of a synthetic signal even if the signal has been deformed or manipulated . 1.1 CONTRIBUTIONS . 1 We formulate the transformation resilient watermarking problem and show how it can be reduced to an empirical risk minimization problem with a minimax loss function . 2 We develop a transformation resilient watermarking scheme , named ReSWAT ( Resilient Signal Watermarking via Adversarial Training ) and show that the learning procedure is closely related to adversarial training techniques ( Madry et al. , 2017 ; Athalye et al. , 2017 ) . 3 Empirically , we show across a set of image and audio datasets that our scheme can produce imperceptible watermarks that can be detected even after the signal has been through a series of adversarial transformations that preserve fidelity to the original signal . We do this using both standard metrics for signal watermarking and via human evaluation on audio signals . Furthermore , we formal provable guarantees of detection if an adversarial transformation has a bounded ` 2 norm . 1.2 RELATED WORK . The problem of detecting synthetic signals has recently received a significant amount of attention . Marra et al . ( 2018 ) investigate how a generative adversarial network ( GAN ) ( Goodfellow et al. , 2014 ) leaves an identifiable fingerprint in the image it generates , while Nataraj et al . ( 2019 ) detect if an image was generated by a GAN by training a detection model on co-occurence matrices of the RGB channels of images . Similarly , Yu et al . ( 2018 ) also train a model that learns to detect if an image was generated by a GAN , and attribute a generated image to its source model . Unfortunately , these detection methods suffer from some limitations : ( 1 ) they are specifically designed to operate only in the image domain and ( 2 ) they are not designed to be resilient to transformations that an attacker could apply . Digital watermarking consists of a two stage process : An embedding stage , where the original signal is combined with a hidden message ( also referred to as the watermark ) , producing a watermarked version of the signal . Secondly , a decoding stage , where the hidden message is retrieved from the watermarked signal . Watermarking schemes are typically evaluated on fidelity of the watermarked signal to the original signal , the resilience of the watermark to various post-processing transformations and the false positive rates ( i.e . whether the decoder detects watermarks in non-watermarked signals ) . Robustness of watermarking techniques is measured against a number of practical attacks that aim to remove or degrade the watermark signal . For example , adding Gaussian noise , image cropping and image compression represent watermark degrading , watermark removal and watermark geometric ( repositioning ) attacks , respectively . In this paper , we are primarily interested in zero-bit watermarking , where one is simply interested in detecting the presence of a watermark ( rather than decoding a hidden message from the watermark ) . The state-of-the-art zero-bit watermarking scheme in the image domain is referred to as Broken Arrows ( BA ) ( Furon & Bas , 2008 ) and won the “ Break Our Watermarking System ” ( BOWS ) competition ( Bennour et al. , 2007 ) . BA has a provable minimum false positive rate under Gaussian noise , however the scheme does not provide robustness against geometric attacks ( like rotations and cropping ) . 2 FORMULATION OF THE ROBUST WATERMARKING PROBLEM . We study signals that live in a space X and are generated by a probabilistic source Ps - our framework applies regardless of whether Ps is a natural source ( images of natural scenes ) or an artificial source ( samples from a VAE or a GAN ) . We are interested in developing a scheme to watermark signals generated by this source , with the requirement that : 1 The detector should detect a watermark in any watermarked signal from source Ps and not detect a watermark in any other signal . 2 Even if post-processing transformations ( for example , compression/cropping/frequency shift etc ) are applied to the signal , the detector detects the watermark , or the absence thereof . Formally , we define the robust watermarking problem as follows : Definition 2.1 ( Transformation resilient watermarking scheme ) . Consider a watermarking scheme ( W , D ) where W : X 7→ X is a watermarking routine and D : X 7→ { 0 , 1 } is a watermark detector . Let T be a space of transformations with T : X 7→ X for each T ∈ T . We consider the watermarking scheme resilient with respect to a set of transformations T for a source Ps if D ( T ( W ( s ) ) ) = 1 , D ( T ( s ) ) = 0 , ∀T ∈ T with high probability for s ∼ Ps . This formulation suggests an empirical risk minimization approach for training D. Given a fixed W , we can train D to minimize the empirical risk E s∼Ps [ max T∈T ` ( D ( T ( W ( s ) ) ) , 1 ) + ` ( D ( T ( s ) ) , 0 ) ] where ` is a loss function measuring the discrepancy between the prediction of D and the desired label ( 0 for non-watermarked signals and 1 for watermarked signals ) . The inner maximization transformations resembles the objective used in adversarial training ( Madry et al. , 2017 ) , motivating our approach to learning a transformation resilient watermarking . We develop this idea in the following section . 3 RESWAT : RESILIENT WATERMARKING VIA ADVERSARIAL TRAINING . We parameterize the detector D as a neural network fθ , parameterized by θ . If we fix the detector , the watermark embedding process W should embed a watermark that is “ strongly detected ” by the detector , i.e. , the watermark embedding mechanism pushes the watermark deep inside the decision boundary of D. However , we also want the watermark to be imperceptible so we require that the watermark does not significantly change the original signal according to some distance measure . While it is challenging to define the space of imperceptible distortions of the input , a convenient proxy is to limit the change in terms of the ` ∞ norm between the watermarked and the original signal . Thus , we construct the watermark , δ , by solving the following optimization problem : W ( s ) = min ‖δ‖∞≤ ` ( fθ ( s+ δ ) , 1 ) This optimization problem can be solved efficiently using a projected gradient descent ( PGD ) method , and is mathematically very similar to computing an adversarial example for a neural network ( however , in this case , the adversary is friendly and actually pushes the example to minimize the loss of the detector ) . The parameter controls the perceptibility of the watermark . If is too large , the watermark would be clearly perceptible while if it is too small , the watermark may easily be washed out by post processing steps . Thus , the right choice of achieves the optimal trade-off between perceptibility and transformation resilient detection . Note that we use the ` ∞ norm of the perturbation instead of another norm since this distributes the watermark over the entire signal , and so will be resilient to geometric attacks ( that black out patches of the input , or obfuscate inputs ) . With this watermark embedding method , we can now train the detector to perform transformation resilient watermark detection as follows : minimize θ , ‖δ‖∞≤ E s∼Ps [ max T∈T ` ( fθ ( T ( s+ δ ) ) , 1 ) + ` ( fθ ( T ( s ) ) , 0 ) ] ( 1 ) This is similar to the expectation over transformation attack ( Athalye et al . ( 2017 ) ) , that aims to construct adversarial examples that persist through a set of transformations . During training , we only sample from differentiable transformations , this is so we do not have to approximate the gradient , which would be prohibitively slow to training . In practice , at each iteration in training we approximate finding maxT∈T with max { T1 , T2 , ... , Tn } , where Ti ∈ T , i ∈ { 1 , ... , n } . Theoretically , if we sample more transformations we achieve a better approximation for the true gradient over the full transformation distribution , however , empirically , we found it too expensive to trade more parallel transformations for longer training time . We explore how resilience to transformations is affected by the number of sampled transformations at each step in training in appendix C. All following experimental results are taken from a model trained with a sample size of one at each iteration of training .

Summary: This paper is about a novel method to add watermarks to images and audio that is highly robust to several transformations that is closely related to gan methods. The idea is that the watermark signal is learned concurrently to the detector network, which share similarities to a generator and detector networks. Five standard attack transformations are considered and a specific optimization to reduce the transferability of the watermark is considered. The method is compared against Broken arrows on Cifar10 and Imagenet. It shows similar or better performance for Gaussian noise attack given the same amount of perturbance allowed in a signal while very much better performance for the other attacks. Going beyond the five attacks, the paper also includes an estimation of the probability of confidence of finding watermarked images given a fixed l2 norm radius. Finally the method is also tested on audio on a proprietary dataset with a deepspeaker architecture which still shows very good performance and it is confirmed by a human evaluation where participants found the watermarked audio not significantly worse or degraded.