Provenance detection through learning transformation-resilient watermarking

1 INTRODUCTION . Generative models have contributed to impressive advancements in content generation and representation learning in both digital image and audio domains ( Brock et al . ( 2018 ) ; Kalchbrenner et al . ( 2018 ) ; Mehri et al . ( 2016 ) ; Zhu et al . ( 2017 ) ; Prenger et al . ( 2019 ) ; Donahue & Simonyan ( 2019 ) ; Oord et al . ( 2016 ) ; Goodfellow et al . ( 2014 ) ; Kingma & Welling ( 2013 ) ) . However , as generative models learn to better match a target distribution , the distinction between natural signals and synthetic signals generated by a model has blurred , leading to a raft of concerns over the potential misuse of these models ( Chesney & Citron ( 2018 ) ) . For example , synthetic videos that are indistinguishable from natural videos , sometimes referred to as deep fakes , have the potential to cause widespread distrust in traditional media . While the problem of detecting synthetic signals is interesting in its own right , it is challenging to do in a manner that is independent of the model used to generate the signal . Instead , we consider the problem of provenance detection via watermarking , a technique that involves injection of a carefully chosen ( but imperceptible ) perturbation ( watermark ) into the signal at the time of creation . The presence of the perturbation in the signal can be later used to detect the provenance ( or ultimate source ) of this signal . While the simplicity of the technique makes watermarking an appealing technique for detecting synthetic ( as well as naturally generated ) signals , it is susceptible to adversarial actors that can systematically try to remove the watermark by transforming the signal ( so as to obfuscate the provenance ) . In the case of images , this may take the form of cropping , addition of Gaussian noise or blurring , rotating images etc . Many watermarking schemes break down under these types of transformations . In this paper , we propose a novel transformation-resilient watermarking scheme- Resilient Signal Watermarking via Adversarial Training ( ReSWAT ) that is able to detect the presence of a watermark even after the signal has been through systematic attempts to remove the watermark . We use ideas from the literature on adversarial training ( Madry et al. , 2017 ) to learn to synthesize watermarks ( encoding the provenance of a signal ) that can be detected even after the watermarked signal has been through transformations that seek to remove the watermark . Our detection method can operate in any domain with continuous data representations . Experimental results demonstrate that our method is effective and substantially improves over existing methods - it can reliably detect the provenance of a synthetic signal even if the signal has been deformed or manipulated . 1.1 CONTRIBUTIONS . 1 We formulate the transformation resilient watermarking problem and show how it can be reduced to an empirical risk minimization problem with a minimax loss function . 2 We develop a transformation resilient watermarking scheme , named ReSWAT ( Resilient Signal Watermarking via Adversarial Training ) and show that the learning procedure is closely related to adversarial training techniques ( Madry et al. , 2017 ; Athalye et al. , 2017 ) . 3 Empirically , we show across a set of image and audio datasets that our scheme can produce imperceptible watermarks that can be detected even after the signal has been through a series of adversarial transformations that preserve fidelity to the original signal . We do this using both standard metrics for signal watermarking and via human evaluation on audio signals . Furthermore , we formal provable guarantees of detection if an adversarial transformation has a bounded ` 2 norm . 1.2 RELATED WORK . The problem of detecting synthetic signals has recently received a significant amount of attention . Marra et al . ( 2018 ) investigate how a generative adversarial network ( GAN ) ( Goodfellow et al. , 2014 ) leaves an identifiable fingerprint in the image it generates , while Nataraj et al . ( 2019 ) detect if an image was generated by a GAN by training a detection model on co-occurence matrices of the RGB channels of images . Similarly , Yu et al . ( 2018 ) also train a model that learns to detect if an image was generated by a GAN , and attribute a generated image to its source model . Unfortunately , these detection methods suffer from some limitations : ( 1 ) they are specifically designed to operate only in the image domain and ( 2 ) they are not designed to be resilient to transformations that an attacker could apply . Digital watermarking consists of a two stage process : An embedding stage , where the original signal is combined with a hidden message ( also referred to as the watermark ) , producing a watermarked version of the signal . Secondly , a decoding stage , where the hidden message is retrieved from the watermarked signal . Watermarking schemes are typically evaluated on fidelity of the watermarked signal to the original signal , the resilience of the watermark to various post-processing transformations and the false positive rates ( i.e . whether the decoder detects watermarks in non-watermarked signals ) . Robustness of watermarking techniques is measured against a number of practical attacks that aim to remove or degrade the watermark signal . For example , adding Gaussian noise , image cropping and image compression represent watermark degrading , watermark removal and watermark geometric ( repositioning ) attacks , respectively . In this paper , we are primarily interested in zero-bit watermarking , where one is simply interested in detecting the presence of a watermark ( rather than decoding a hidden message from the watermark ) . The state-of-the-art zero-bit watermarking scheme in the image domain is referred to as Broken Arrows ( BA ) ( Furon & Bas , 2008 ) and won the “ Break Our Watermarking System ” ( BOWS ) competition ( Bennour et al. , 2007 ) . BA has a provable minimum false positive rate under Gaussian noise , however the scheme does not provide robustness against geometric attacks ( like rotations and cropping ) . 2 FORMULATION OF THE ROBUST WATERMARKING PROBLEM . We study signals that live in a space X and are generated by a probabilistic source Ps - our framework applies regardless of whether Ps is a natural source ( images of natural scenes ) or an artificial source ( samples from a VAE or a GAN ) . We are interested in developing a scheme to watermark signals generated by this source , with the requirement that : 1 The detector should detect a watermark in any watermarked signal from source Ps and not detect a watermark in any other signal . 2 Even if post-processing transformations ( for example , compression/cropping/frequency shift etc ) are applied to the signal , the detector detects the watermark , or the absence thereof . Formally , we define the robust watermarking problem as follows : Definition 2.1 ( Transformation resilient watermarking scheme ) . Consider a watermarking scheme ( W , D ) where W : X 7→ X is a watermarking routine and D : X 7→ { 0 , 1 } is a watermark detector . Let T be a space of transformations with T : X 7→ X for each T ∈ T . We consider the watermarking scheme resilient with respect to a set of transformations T for a source Ps if D ( T ( W ( s ) ) ) = 1 , D ( T ( s ) ) = 0 , ∀T ∈ T with high probability for s ∼ Ps . This formulation suggests an empirical risk minimization approach for training D. Given a fixed W , we can train D to minimize the empirical risk E s∼Ps [ max T∈T ` ( D ( T ( W ( s ) ) ) , 1 ) + ` ( D ( T ( s ) ) , 0 ) ] where ` is a loss function measuring the discrepancy between the prediction of D and the desired label ( 0 for non-watermarked signals and 1 for watermarked signals ) . The inner maximization transformations resembles the objective used in adversarial training ( Madry et al. , 2017 ) , motivating our approach to learning a transformation resilient watermarking . We develop this idea in the following section . 3 RESWAT : RESILIENT WATERMARKING VIA ADVERSARIAL TRAINING . We parameterize the detector D as a neural network fθ , parameterized by θ . If we fix the detector , the watermark embedding process W should embed a watermark that is “ strongly detected ” by the detector , i.e. , the watermark embedding mechanism pushes the watermark deep inside the decision boundary of D. However , we also want the watermark to be imperceptible so we require that the watermark does not significantly change the original signal according to some distance measure . While it is challenging to define the space of imperceptible distortions of the input , a convenient proxy is to limit the change in terms of the ` ∞ norm between the watermarked and the original signal . Thus , we construct the watermark , δ , by solving the following optimization problem : W ( s ) = min ‖δ‖∞≤ ` ( fθ ( s+ δ ) , 1 ) This optimization problem can be solved efficiently using a projected gradient descent ( PGD ) method , and is mathematically very similar to computing an adversarial example for a neural network ( however , in this case , the adversary is friendly and actually pushes the example to minimize the loss of the detector ) . The parameter controls the perceptibility of the watermark . If is too large , the watermark would be clearly perceptible while if it is too small , the watermark may easily be washed out by post processing steps . Thus , the right choice of achieves the optimal trade-off between perceptibility and transformation resilient detection . Note that we use the ` ∞ norm of the perturbation instead of another norm since this distributes the watermark over the entire signal , and so will be resilient to geometric attacks ( that black out patches of the input , or obfuscate inputs ) . With this watermark embedding method , we can now train the detector to perform transformation resilient watermark detection as follows : minimize θ , ‖δ‖∞≤ E s∼Ps [ max T∈T ` ( fθ ( T ( s+ δ ) ) , 1 ) + ` ( fθ ( T ( s ) ) , 0 ) ] ( 1 ) This is similar to the expectation over transformation attack ( Athalye et al . ( 2017 ) ) , that aims to construct adversarial examples that persist through a set of transformations . During training , we only sample from differentiable transformations , this is so we do not have to approximate the gradient , which would be prohibitively slow to training . In practice , at each iteration in training we approximate finding maxT∈T with max { T1 , T2 , ... , Tn } , where Ti ∈ T , i ∈ { 1 , ... , n } . Theoretically , if we sample more transformations we achieve a better approximation for the true gradient over the full transformation distribution , however , empirically , we found it too expensive to trade more parallel transformations for longer training time . We explore how resilience to transformations is affected by the number of sampled transformations at each step in training in appendix C. All following experimental results are taken from a model trained with a sample size of one at each iteration of training .

Summary: The authors introduce ReSWAT, a method for transformation-resilient watermarking of images via adversarial training. The high level idea is to learn a watermark/detector pair (W,D). W can be any transformation (in this paper, an l-infty bounded perturbation) that imputes an imperceptible distortion to a given input, while D is a detector that distinguishes watermarked from non-watermarked images. There is an additional requirement that the detector should be robust to simple transformations such as rotations, cropping, flipping, and contrast enhancement.