Title: Generative Adversarial Networks for Image Steganography
Authors: Denis Volkhonskiy
Boris Borisenko
Evgeny Burnaev
Venue: ICLR 2017
Keywords: Computer vision
Deep learning
Unsupervised Learning
Applications
Supervised Learning

Abstract: Steganography is collection of methods to hide secret information ("payload") within non-secret information ("container"). Its counterpart, Steganalysis, is the practice of determining if a message contains a hidden payload, and recovering it if possible. Presence of hidden payloads is typically detected by a binary classifier. In the present study, we propose a new model for generating image-like containers based on Deep Convolutional Generative Adversarial Networks (DCGAN). This approach allows to generate more setganalysis-secure message embedding using standard steganography algorithms. Experiment results demonstrate that the new model successfully deceives the steganography analyzer, and for this reason, can be used in  steganographic applications.

TL;DR: We consider a new type of GAN model and apply it to secure image steganography

Review discussion:
Have you explored how many bits may be stored in an adversarial perturbation that might not be detected? I would imagine that if the perturbation were larger, more bits might be communicated but at a risk of being detected easier.
In steganographic articles the payload bit rate is usually set to 0.4-0.6 bits per pixel (bpp). In this paper we used the default 0.4 bpp, but will analyze the sensitivity in the further experiments.
How do the samples from the proposed model perform compared to using real images as containers? What is the main incentive to use generated containers as opposed to randomly picking real data?
In section 5.3 we showed, that if we have trained on faces Steganalyzer 'S*', we can deceive it with generated faces. Our goal was to create images from the same distribution, that 'S*' has seen.
Due to adversarial nature in which the container image generator was trained, it is better suited for being undetectable by a jointly trained stegoanalyzer 'S*', which itself trains to better detect concealed payloads.
I reviewed the manuscript as of December 6th.
Summary:
The authors build upon generative adversarial networks for the purpose of steganalysis -- i.e. detecting hidden messages in a payload. The authors describe a new model architecture in which a new element, a 'steganalyser' is added a training objective to the GAN model.
Major Comments:
The authors introduce an interesting new direction for applying generative networks.
That said, I think the premise of the paper could stand some additional exposition. How exactly would a SGAN method be employed? This is not clear from the paper. Why does the model require a generative model? Steganalysis by itself seems like a classification problem (i.e. a binary decision if there a hidden message?) Would you envision that a user has a message to send and does not care about the image (container) that it is being sent with? Or does the user have an image and the network generates a synthetic version of the image as a container and then hide the message in the container? Or is the SGAN somehow trained as a method for detecting hidden codes performed by any algorithm in an image? Explicitly describing the use-case would help with interpreting the results in the paper.
Additionally, the experiments and analysis in this paper is quite light as the authors only report a few steganalysis performance numbers in the tables (Table 1,2,3). A more extensive analysis seems warranted to explore the parameter space and provide a quantitative comparison with other methods discussed (e.g. HUGO, WOW, LSB, etc.) When is it appropriate to use this method over the others? Why does the seed effect the quality of results? Does a fixed seed correspond realistic scenario for employing this method?
Minor comments:
- Is Figure 1 necessary?
- Why does the seed value effect the quality of the predictive performance of the model?
Thank you for your review.
"Alice" wants to send some private notification in cleartext to "Bob" inside a unique photograph of a human face. An eavesdropping adversary "Eve" knows that Alice would only use human faces as stegocontainers and wants to detect if an intercepted image contains some concealed payload information, but doesn't know the particular stego algorithm used for embedding and the source seed used for generation (may be allow Eve to know the Stego(...) function, but then allow Alice to send ciphered messages). Eve also knows that Alice is aware of the existence of an adversary with Eve's capabilities.
In this setting it is best for Alice to explicitly model Eve detection capabilities inside a SGAN as the network S. This allows to Alice to train a generator that is secure against the best Eve's detector, since during training the model of Eve has access to more information (the faces dataset, source and “loaded" image, gradients and stego/empty labelling, and synthetic/realistic) than the real Eve, who has access to the same dataset, the labelling and source vs. “loaded" image.
We further allow Eve to have a labeled train dataset with synthetic images and real images to train a classifier S^*.
I found this paper very original and thought-provoking, but also a bit difficult to understand. It is very exciting to see a practical use case for image-generating GANs, with potentially meaningful benchmarks aside from subjective realism.
I found eq. 4 interesting because it introduces a potentially non-differentiable black-box function Stego(...) into the training of (S, G). Do you in fact backprop through the Stego function?
- For the train/test split, why is the SGAN trained on all 200k images? Would it not be cleaner to use the same splits for training SGAN as for "steganalysis purposes"? Could this account for the sensitivity to random seed shown in table 2?
- Sec. 5.3: "Steganographic Generative Adversarial Networks can potentially be used as a universal tool for generating Steganography containers tuned to deceive any specific steganalysis algorithm.". This experiment showed that SGAN can fool HUGO, but I do not see how it was "tuned" to deceive HUGO, or how it could be tuned in general for a particular steganalyzer.
Although S* seems to be fooled by the proposed method, in general for image generation the discriminator D is almost never fooled. I.e. contemporary GANs never converge to actually fooling the discriminator, even if they produce samples that sometimes fool humans. What if I created an additional steganalyzer S**(x) = S*(x) * D(x)? This I think would be extremely difficult to fool reliably because it requires realistic image generation.
After reading the paper several times, it is still a bit unclear to me how or why precisely one would use a trained SGAN. I think the paper could be greatly improved by detailing, step by step, the workflow of how a hypothetical user would use a trained SGAN. This description should be aimed at a reader who knows nothing or very little about steganography (e.g. most of ICLR attendees).
Thank you for your review.
Yes, potentially this function is non-differentiable, but given fixed payload the final result of its acton onto an image is a fixed additive distortion. This makes backprop through it quite straightforward. Note, that during training the action matrix of the Stego(...) is effectively random: we use the same pool of payload information for each batch, but within a batch random payload information is embedded in each image.
We apply GAN approach in the paper purely as a training technique to get an image generator (G) that achieves both realism with respect to a discriminator (D) and security against stegodetector (S), both jointly adversarially trained (eq. 4). This way we get a generator, the images of which have the pixel distribution, specifically tailored to securely concealing images with a particular Stego(...) function. Since we are not using the GAN trained D and S any further and want to use G solely as an image generator, we are therefore free to train it on the whole dataset. This permits us to verify how well the GAN approach works in stego.
The 90-10 train-test split is related to training/validating the independent stego-detector S^*, used to assess the security of the generated images.
In this paper we have tested our model for the +-1 embedding algorithm only, but in the future we plan to test it for other steganographic algorithms.The key idea of the paper is that SGAN produces a generator the images of which have pixel distributions better "aligned" with the distortions introduced by a particular Stego-algorithm, and are sufficiently realistic-looking.
In fact, this is slightly more powerful than simply trying to deceive a particular family of stegoanalysers.
This paper proposes an interesting application of the GAN framework in steganography domain.
In addition to the normal GAN discriminator, there is a steganalyser discriminator that receives the negative examples from the generator and positive examples from the generator images that contain a hidden payload.
As a result, the generator, not only learn to generate realistic images by fooling the discriminator of the GAN, but also learn to be a secure container by fooling steganalyser discriminator. The method is tested by training an independent steganalyser S* on real images and generated images.
Given that in the ICLR community, not many people are familiar with the literature of steganography, I think this paper should have provided more context about how exactly this method can be used in practice, what are the related works on setganalysis-secure message embedding and probably a more thorough sets of experiments on more than one dataset.
The proposed SGAN framework (Figure 2) does make sense to me, and I think it is very general and can have more applications other than the steganography domain.
But it is not clear to me why fooling the steganalyser discriminator S, necessarily mean that we can fool an independent discriminator S*?
Also I find it surprising that a different seed value, can make such a huge difference in the accuracy.
In short, the ideas of this paper are interesting and potentially useful, but I think the presentation of this paper should be improved so that it becomes more suitable for the ICLR and machine learning community.
Thank you for your review.
Indeed, there is no guarantee, that the trained generator is secure against S^*, since G is trained to deceive the jointly trained steganalyzer S, which is suboptimal with respect to S^*. The steganalyzer S^*, as a binary classifier, is trained using the pairs of raw and "loaded" images.
However, training the generator with an adversarial technique, we in effect get a generator, trained against a more powerful opponent than S^*: GAN  provides S with the feedback gradients of the generator in the backprop, which is obviously unavailable while training S^*, since the generator is assumed unavailable to the eavesdropping party.

Meta-review:
role: PC
data: comment: This paper examines an application of that deviates from the usual applications presented at ICLR. The idea seems very interesting to the reviewers, but a number of reviewers had trouble really understanding why the proposed SGAN would be attractive for this problem, and this problem setup with the SGAN in general. Clearer concrete 'use case scenarios' and experimentation that helps clarify the precise application setting and the advantages of the SGAN formulation would help make this work more impactful on the community. Given the quality of other paper submitted to ICLR this year the reviewer scores are just short of the threshold for acceptance

Decision: Reject