F IXED N EURAL N ETWORK S TEGANOGRAPHY : T RAIN THE IMAGES , NOT THE NETWORK
A BSTRACT
Recent attempts at image steganography make use of advances in deep learning to train an encoder-decoder network pair to hide and retrieve secret messages in images.
These methods are able to hide large amounts of data, but they also incur high decoding error rates (around 20%).
In this paper, we propose a novel algorithm for steganography that takes advantage of the fact that neural networks are sensitive to tiny perturbations.
Our method, Fixed Neural Network
Steganography (FNNS), yields signiﬁcantly lower error rates when compared to prior state-of-the-art methods and achieves 0% error reliably for hiding up tobits per pixel (bpp) of secret information in images.
FNNS also successfully evades existing statistical steganalysis systems and can be modiﬁed to evade neural steganalysis systems as well.
Recovering every bit correctly, up to 3 bpp, enables novel applications that requires encryption.
We introduce one speciﬁc use case for facilitating anonymized and safe image sharing.
Our code is available at https://github.com/varshakishore/FNNS .
I NTRODUCTION
Image steganography aims to hide a secret digital message within a cover image (Morkel et al., 2005) — ideally, through minimal alterations, such that only intended recipients are aware of the hidden secret.
Steganography has been widely used in applications such as watermarking (Wolfgang & Delp, 1996), copyright certiﬁcation (Lu, 2004) and private information storage (Srinivasan et al., 2004).
Classic steganography tools use pixel statistics to hide information in images (Pevn ` y et al., 2010).
Secret messages hidden with these methods can be recovered with 0% error, but to evade detection by steganalysis tools, they can only hide up to 0.5 bits per pixel (bpp) of information (Pevn ` y et al., 2010;
Holub & Fridrich, 2012; Holub et al., 2014).
Encouraged by data-driven deep learning techniques, recent methods propose training deep encoder-decoder networks to hide and recover up to 6 bpp of information in images (Zhang et al., 2019a; Zhu et al., 2018; Baluja, 2017; Wu et al., 2018; Hayes & Danezis, 2017).
These methods do achieve higher bpp rates, but they also result in higher error rates for the retrieved messages (Zhang et al., 2019a).
Many applications have low error rate requirements for the steganography algorithm.
In some scenarios, the hidden message has no redundancy for error correction, and there is zero tolerance for even a single incorrectly recovered bit.
For example, if the secret message is encrypted, it will be a random bit string that must be recovered with zero error for successful decryption.
In this paper, we propose a novel approach for image steganography that aims to simultaneously and reliably achieve high steganographic capacity and low error rates.
Notably, we achieve 0.0% error when encoding up to 3 bpp of information.
We make no assumptions on the secret message and allow it to be any arbitrary bit string.
We show that our method can be used with randomly initialized neural networks or in conjunction with pre-trained networks.
Unlike previous steganography methods that train deep networks to hide and recover messages in a speciﬁc dataset (Zhang et al., 2019a; Zhu et al., 2018; Baluja, 2017), our method is based upon a very different approach, which originated in the context of adversarial attacks on neural networks (Szegedy et al., 2013).
Adversarial attacks are based on the key insight that deep neural networks are highly sensitive to small changes to the input.
An adversary can therefore manipulate an image with imperceptible perturbations to inﬂuence the prediction of a neural network that uses this image as input.
The last eight years have witnessed an outpouring of analysis (Kurakin et al., 2016b; Carlini & Wagner, 2016;
Xu et al., 2017; Shafahi et al., 2018; Meng & Chen, 2017; Li & Li, 2017; Lu et al., 2017a; Graese et al., 2016; Dziugaite et al., 2016; Lu et al., 2017b; Gu & Rigazio, 2015; Kurakin et al., 2016a;
Miyato et al., 2015; Nokland, 2015; Cisse et al., 2017; Hu et al., 2019;
Guo et al., 2020; 2017) and methods (Liu et al., 2016; Moosavi-Dezfooli et al., 2016; Carlini &
Wagner, 2017; Tram`er et al., 2017; Papernot et al., 2017; 2016; Biggio et al., 2013) to understand and create adversarial perturbations.
Notably, it is fair to say that vulnerability to adversarial attacks is generally considered inevitable in most settings (Shafahi et al., 2018) and frustratingly hard to defend against (Carlini & Wagner, 2017; Dziugaite et al., 2016), especially when the target network architecture is known to the adversary (which is generally referred to as white-box setting).
Adversarial attacks are typically considered a nuisance, or a limitation of machine learning.
However, in this paper we utilize their persistence and reliability as a desired feature for what we refer to as Fixed Neural Network Steganography (FNNS) .
In a nutshell, FNNS is based on the following procedure (see Figure 1):
We initialize a neural network (decoder) that takes as input an image and produces sufﬁcient binary outputs.
Given a secret message and a cover image, the sender (Alice) perturbs the original image in a fashion similar to adversarial perturbations (Madry et al., 2017).
However, instead of targeting a single prediction bit (e.g. the classiﬁcation of an image), the sender manipulates thousands or even millions of output bits simultaneously.
The intended recipient (Bob) can use the same decoder network and recover the hidden message.
We show that FNNS reliably yields 0% error rate for hiding up to 3 bpp of information and lower error rates than current state-of-the-art methods for higher bit rates on multiple datasets.
FNNS can also be used in conjunction with existing trained encoder-decoder methods (like
SteganoGAN (Zhang et al., 2019a)) to further reduce the error rates obtained by the trained methods.
Additionally, we show that FNNS evades existing statistical steganalysis methods (Boehm, 2014; Dumitrescu et al., 2002a;b; B ¨ ohme & Westfeld, 2004; Zhang & Ping, 2003) and can be made resistant to JPEG compression (Wallace, 1992) for low bpp.
Finally, we introduce an example application of error-free steganography for anonymized image sharing: We replace faces in images with
GAN generated substitutes that contain the original faces encrypted and hidden through FNNS — ensuring that only intended recipients (with the secret key) can recover the original images.
R ELATED W ORK
Statistical image steganography methods (Pevn ` y et al., 2010) typically pre-date the use of neural networks.
Least-Signiﬁcant Bit (LSB) methods modify lower-order bits of each pixel to encode a secret message (Van Schyndel et al., 1994; Wolfgang & Delp, 1996; Katzenbeisser & Petitcolas, 2000).
Although compellingly simple and lossless, these methods are easily detectable (Dumitrescu et al., 2002a;b; B¨ohme & Westfeld, 2004; Zhang & Ping, 2003) and often lack robustness (Qin et al., 2010).
Many statistical image steganography methods were proposed to evade detection by such steganalysis algorithms.
Highly Undetectable Steganography (HUGO)
(Pevn`y et al., 2010), is one such method, that uses hand crafted features to measure distortion caused by modifying pixels and modiﬁes pixels that cause the least amount of distortion.
Wavelet Obtained Weights (WOW)
(Holub & Fridrich, 2012) uses directional high-pass ﬁlters to ﬁnd regions of the cover image with high texture and penalizes changes in low-textured regions.
S-UNIWARD (Holub et al., 2014) is similar to WOW but is designed to work with non-spatial domains (e.g. the frequency domain).
The main limitation of statistical methods is that the number of bits they encode is relatively low ( ≤ 0 .
5 bpp).
Deep learning image steganography methods have recently achieved impressive results in terms of bpp rates (Zhang et al., 2020b; Baluja, 2017; Rahim et al., 2018; Zhang et al., 2019b).
In general, they mostly share a similar pipeline that can be trained end-to-end:
An encoder network takes as input a cover image and a message that should be concealed within the image.
From these inputs it generates a steganographic image that has hidden information but is visually similar to the cover image.
A subsequent decoder network recovers the hidden message from the steganographic image.
Multiple loss functions ensure that 1) the generated image is close to the original one; 2) the decoder’s output matches the secret message.
Zhu et al.
(2018)’s
HiddenNet pioneered such an encoder-decoder pipeline, and their HiddenNet could hide up to 0.2 bpp with an error rate of 10 − 5 .
SteganoGAN (Zhang et al., 2019a) uses a slightly different encoder-decoder architecture and introduces an additional critic network that ensures the produced images look realistic, i.e. like a natural image.
The authors show experiments of hiding up to 6 bpp of information with an error rate of 530% (depending on how many bits are hidden).
In a similar vein, AdvSGAN
(Li et al., 2021) hides up to 1 bpp by learning an image steganography scheme that plays an adversarial game between a restricted neural coder and a critic.
Deep Steganography (Baluja, 2017), ISGAN (Dong et al., 2018),
Attention Based Data Hiding (Yu, 2020), Universal Deep Hiding Zhang et al.
(2020a)
and End-toend CNN for Image Steganography (Rahim et al., 2018) use similar encoder-decoder architectures to hide and recover structured images instead of random bits.
These methods assume the secret message is an image, which allows them to learn image priors that aid in hiding the secret image.
Invertible networks have also been explored to hide images within images (Jing et al., 2021; Lu et al., 2021).
Despite handling a large number of bits, these end-to-end neural approaches also share a set of disadvantages: 1) the error rate for the recovered messages is very high, 2) they assume access to hundreds or even thousands of training images from the target domain to train encoder and decoder pairs, and 3) there is little recourse if the model produces an image with high error rate or distortions.
There are also methods that have explored hiding messages in physical photographs (Wengrowski & Dana, 2019; Tancik et al., 2020), but our work focuses on digital images.
Imperceptible image perturbations.
Adversarial examples (Goodfellow et al., 2014) are inputs to machine learning models that an attacker has intentionally designed to cause the model to make a mistake.
Many approaches (Szegedy et al., 2013; Goodfellow et al., 2014; Guo et al., 2019; Xu et al., 2020; De Palma et al., 2021; Yuan et al., 2019) try to construct adversarial examples by perturbing image pixels.
Moosavi-Dezfooli et al.
(2017) propose a systematic algorithm for computing universal perturbations that many deep neural networks are highly vulnerable to.
Su et al.
(2019) propose a differential evolution method to generate low-dimensional one-pixel adversarial perturbations that change the output of a classiﬁcation network.
Athalye et al.
(2018b) create adversarial examples that are robust to afﬁne image transformations, noises, and other distortions.
Projected gradient descent (PGD) (Madry et al., 2017) is one of the most widely used algorithms to generate adversarial examples by adding small perturbations to the input.
This method iteratively updates the input with gradient descent until a desired output is obtained.
The input is projected, or more precisely clipped to be within [ − (cid:15), (cid:15) ] at the end of every step.
This precaution ensures that the perturbations stay reasonably small for all pixels and remain imperceptible.
Ghamizi et al.
(2019) propose performing stenography by ﬁnding perturbations using PGD with a classiﬁcation network.
However, the amount of information they are able to hide is low and they need multiple images to hide long messages.
F IXED N EURAL N ETWORK S TEGANOGRAPHY
Setting.
Let X ∈ [0 , 1] H × W × 3 be an RGB color image with height H and width W .
Further, let
M ∈ { 0 , 1 } H ×
W
×
D be a message that we are trying to conceal in X , where D speciﬁes the number of bits we need to hide per pixel.
1 We assume there are two parties involved – the sender , Alice, who hides M in X , creating ˜ X ; and the receiver , Bob, who extracts M out of ˜ X .
Given a decoder network F : [0 , 1] H × W × 3 → [0 , 1] H × W × D , our goal is to generate a perturbed image ˜ X , which is close to X (according to metrics speciﬁed in the subsequent section) such that F ( ˜ X )
=
M .
Encoding and Decoding.
In order to generate image ˜ X , we use an approach similar to adversarial perturbations (Madry et al., 2017).
Given the cover image X and the ground truth message M , the sender (Alice) solves the following optimization problem over the perturbed image ˜
X :
[Equation]
where (cid:104) , (cid:105) denotes inner-product across all H × W × D dimensions and the objective function is the binary cross entropy (BCE) loss.
The ﬁrst of the two linear constraints enforces that the maximum
[Table]
[Equation] [Equation] [Equation] [Equation]
[Table]
[Equation]
[Table]
[Equation]
[Table]
[Equation]
[Table]
[Equation]
[Table]
[Equation]
[Equation]
[Equation]
[Equation]
[Equation]
[Equation]
[Equation] [Equation] [Equation] [Equation]
[Equation] [Equation] [Equation] [Equation]
[Equation]
[Equation]
[Equation]
[Equation]
[Equation]
[Equation]
[Equation]
[Equation]
[Equation] [Equation] [Equation]
[Equation]
[Equation] [Equation]
[Equation]
[Equation] [Equation]
[Equation] [Equation]
[Equation] [Equation]
[Equation] [Equation]
[Equation] [Equation]
[Equation] [Equation]
[Equation] [Equation]
[Equation] [Equation]
[Equation] [Equation]
[Equation] [Equation]
[Equation] [Equation]
[Equation] [Equation]
[Equation] [Equation]
[Equation] [Equation]
[Equation] [Equation]
[Equation] [Equation]
[Equation] [Equation]
[Equation] [Equation]
[Equation] [Equation]
[Equation] [Equation]
[Equation] [Equation]
[Equation] [Equation]
[Equation] [Equation]
[Equation] [Equation]
[Equation] [Equation]
[Equation] [Equation]
[Equation] [Equation]
[Equation] [Equation]
[Equation] [Equation]
[Equation] [Equation]
[Equation] [Equation]
[Equation] [Equation]
[Equation] [Equation]
[Equation] [Equation]
[Equation] [Equation]
[Equation] [Equation]
[Equation] [Equation]
[Equation] [Equation]
[Equation] [Equation]
[Equation] [Equation]
[Equation] [Equation]
[Equation] [Equation]
[Equation] [Equation]
[Equation] [Equation]
[Equation] [Equation]
[Equation] [Equation]
[Equation] [Equation]
[Equation] [Equation]
[Equation] [Equation]
[Equation] [Equation]
[Equation] [Equation]
[Equation] [Equation]
[Equation] [Equation]
[Equation] [Equation]
[Equation] [Equation]
[Equation] [Equation]
[Equation] [Equation]
[Equation] [Equation]
[Equation] [Equation]
[Equation] [Equation]
[Equation] [Equation]
[Equation] [Equation]
[Equation] [Equation]
[Equation] [Equation]
[Equation] [Equation]
[Equation] [Equation]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure] [Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Figure]
[Equation]
[Figure]
[Equation]
[Figure]
[Equation]
[Figure]
[Equation]
[Figure]
[Equation]
[Figure]
[Equation]
[Figure] [Equation] [Figure]
[Equation]
[Figure]
[Equation]
[Figure]
[Equation]
[Figure]
[Equation]
[Figure] [Equation] [Figure] [Equation]
[Figure] [Equation] [Figure] [Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Equation]
[Figure]
[Equation]
[Table]
[Equation] [Equation]
[Equation] [Equation]
Decoding
BCE Loss L BCE L BCE
Fixed Decoder
Network
Encoding
Input
Decoding
Gradient
[Figure]
Secret Message MM Perturbations δδ
FF
FF
˜ X
LBFGS ( δ ← clip (cid:15) − (cid:15) { ˜ X − X }
8: return ˜ X perturbation does not exceed a value of (cid:15) and stays imperceptible.
The second constraint enforces that the perturbed image, ˜ X , is a well-deﬁned image, with pixel values ranging within [0 , 1] .
Our optimization algorithm is outlined in Algorithm 1, where clip 10 ( x ) = max(min
( x , 1) , 0) .
We use the unconstrained L-BFGS (Fletcher, 2013) algorithm to optimize the objective with respect to ˜
X .
We used L-BFGS because it keeps track of second order gradient statistics and results in faster optimization.
To ensure that the constraints are not violated, we project the solution back into the feasible region after k steps.
We assume that the recipient (Bob) has access to the same network F as the sender (Alice).
He can then recover the concealed message M by computing F ( ˜ X ) .
Decoder weights and initialization.
Our encoding procedure optimizes the image ˜
X
directly and considers the weights of F ﬁxed throughout.
This gives us the freedom to explore multiple ways to initialize ˜ X and to obtain weights for the decoder F , yielding three distinct variants of our method: 1) FNNS-R: F is a random network and ˜ X is initialized to be the cover image X .
When a random network is used, the sender and the receiver only need to share the architecture of the decoder network and the random seed used to initialize its weights; the actual weights of the network do not have to be shared.
Additionally, if the image quality of the perturbed image is low, a different random decoder can easily be initialized with a new random seed and the optimization can be repeated.
2) FNNS-DE:
Given a trained encoder-decoder pair (from any of the prior neural work mentioned in section 2), we can deﬁne F to be the pre-trained decoder and initialize ˜ X as Enc ( X , M ) , where
Enc is the trained encoder that is paired with the decoder F .
With this initialization, a part of the message M is already encoded into ˜ X such that F ( ˜ X ) ≈ M .
As a result, the optimization is much faster.
However
, the
encoding step sometimes deteriorates
image quality, and
it
is hard for the optimization
algorithm
to “recover”
in terms of quality
in such cases. 3) FNNS-D: F is a pre-trained decoder and ˜ X is initialized to be the cover image X .
With a trained decoder, messages can be hidden in images using both perturbations and by training weights
conducive to hiding information in the images.
As a result, the output image quality is better and the optimization does not suffer from getting stuck in bad local optima.
Decoder architecture.
Prior work on adversarial attacks has explored network designs that are especially robust to small perturbations (Xu et al., 2020).
In contrast, for our decoder we want network architectures that are particularly susceptible in that regard.
We develop our method based on the basic decoder of SteganoGAN (Zhang et al., 2019a), a 4-layer convolutional neural network that takes an H × W × 3 RGB image as input and outputs a bit string { 0 , 1 } H × W × D (after rounding).
We choose SteganoGAN because it has been shown to achieve state-of-the-art performance for hiding arbitrary messages in images.
For our experiments, we set the pre-trained encoder and decoder for FNNS-D and FNNS-DE, to be the trained SteganoGAN encoder and decoder.
For the FNNS-R random network we empirically explore many variations with different depths, widths, normalization layers, and activation functions, and evaluate them w.r.t.
bit error rates, PSNR, and SSIM.
Due to space constraints, we summarize our results in Appendix B. Throughout, we use 128 hidden channels in FNNS-R and 32 in FNNS-D and FNNS-DE.
This is because for trained SteganoGAN models, increasing the number of hidden units leads to no signiﬁcant improvements in accuracy or image quality (see Appendix E), but slows down the optimization process.
3. E VALUATION M ETRICS
[Equation]
We use three popular metrics to evaluate our results.
The bit error rate,
[Equation]
, where (cid:98)·(cid:101) denotes rounding function, measures how many bits are incorrectly recovered.
Peak signal-to-noise ratio (PSNR) is a common metric used to measure image distortions between X and ˜ X and has been shown to be correlated with human evaluation scores (Isola et al., 2017).
3. Structural Similarity Index
(SSIM) is another metric used to measure the similarity between two images.
SSIM differs from PSNR in that it tries to capture the change in structural information as opposed to considering pixel-wise changes.
PSNR and SSIM between two images X and ˜ X with maximum possible pixel value max X , averages µ, ˜ µ , standard deviation σ, ˜ σ and co-variance σ X ˜ X are deﬁned in Table 1.
S TEGANOGRAPHY R ESULTS AND D ISCUSSION
Experimental Setup We evaluate FNNS on three diverse datasets – a scenic image dataset
Div2k (Agustsson & Timofte, 2017), a 2D object detection dataset MS-COCO
(Lin et al., 2014) and a human face dataset CelebA (Liu et al., 2015).
For each dataset, we use the provided test/validation images (if unavailable, we use the ﬁrst 100 images in the dataset for validation).
We randomly
[Table]
initialized the message bit strings; each bit in the string is independently sampled from a Bernoullidistribution with probability 12 .
The hyper-parameters used for Algorithm 1 are as follows: perturbation bound (cid:15) = 0 .
3 , optimization steps n = 100 , and L-BFGS iterations k = 10 with early stopping if the output has zero error.
In cases where the image quality of ˜ X is poor, we restart optimization with a different learning rate α .
Concretely, we set the learning rate to 0.1 and change it to 0.05 or 0.5 if the output image gets a PSNR lower than 20.
We train SteganoGAN models for only one epoch for FNNS-D and FNNS-DE, as we observe that a fully-trained (32 epochs) SteganoGAN decoder over-ﬁts to its training objective such that it’s hard to use it for FNNS.
Appendix G shows the result of using SteganoGAN models trained for 32 epochs.
Quantitative Comparison.
We compare FNNS with SteganoGAN, the current state-of-the-art method, in Table 2.
In addition (not shown in the table), we also compare with steganography methods that hide lower payload messages (that is < 0 .
5 bpp messages): HUGO (Pevn ` y et al., 2010),
UNIWARD Holub et al.
WOW (Holub & Fridrich, 2012) and HiDDeN (Zhu et al., 2018).
These methods can achieve an error rate of 0% but only for < 0 .
5 bpp messages.
In contrast, FNNS also achieves 0% error for signiﬁcantly higher bit rates as shown in Table 2.
FNNS-D achieves the best performance in terms of both low error rates and good image quality.
FNNS-DE is also able to achieve error rates of 0%, but the corresponding image quality is worse when compared to FNNSD.
From Table 2, we also see that for Div2k it is hard to achieve 0% error with 3 bpp.
Div2k is a small dataset with only 800 images and as a result there is not enough diversity to train a ﬂexible
SteganoGAN model; this is evidenced by the fact that 0% error for Div2k with 3 bpp can be achieved by using a model trained on MS-COCO.
Qualitative Comparison.
Several qualitative examples are presented in Figure 2 (more images from all the method variants can be found in Appendix A).
Even with 4 bpp information hidden, the cover and stenographic images look identical.
There are a few examples where the stenographic images look pixelated, especially when a random network is used.
This is either because the image is over-optimized to get a low bit error rate or because of a bad random seed.
To produce a better stenographic image, the random network can be re-initialized (if using a random network), the optimization can be stopped earlier or the hyper-parameters of FNNS can be changed slightly (in order to change the optimization problem).
Domain independence.
The performance of trained encoder-decoder models like SteganoGAN degrade for out-of-domain cover images.
For example, a SteganoGAN (Zhang et al., 2019a) model trained on Div2k images (which contains primarily landscapes) produces noticeable artefacts when continue for to ensure that even
˜ into values (integers between 0 to 255), F ( ˜ X ) =
M are shown
Appendix F tested on facial images from CelebA (Figure 3).
The only recourse to correcting artifacts or quality would be to train another model with in-domain data.
In contrast, FNNS-R is completely domain independent.
Although FNNS-D uses a trained decoder, we ﬁnd it to be just as robust against domain shift.
We explain these ﬁndings by the fact that the secret message is hidden through an optimization procedure that is very robust to the exact ﬁlter values of the decoder network.
Optimization time.
Most encoder-decoder deep stenography methods require many hours of training on hundreds of images.
However, once trained, it takes less than a second for inference with the model given any cover image and message.
FNNS-D and FNNS-DE perform a per-image optimization process for encoding a message in an image and this takes on average 10 seconds for 1-2bpp and 20 seconds for 3-4bpp.
For FNNS-R encoding a message takes about 3 minutes because optimizing the random network is a harder task.
Appendix C has a table showing the amount of time required to use each method variant with different bpp rates.
[Table]
low error rates when the bpp rate is low.
JPEG Compression.
A desirable property for any steganography system is robustness against lossy image compression.
Unfortunately the goal of imperceptibility is inherently at odds with most image compression methods, which by design aim to remove imperceptible, and therefore unnecessary, information.
JPEG (Wallace, 1992) is a lossy compression method for digital images that transforms an image into frequency space and removes high frequency components via quantization and rounding.
Past work has shown that adversarial perturbations can be removed via JPEG compression (Dziugaite et al., 2016) and trained encoder-decoder steganography networks also fail to be resistant to JPEG compression.
Modiﬁcations to the loss functions can improve resistance to compression, but signiﬁcantly increase error rates.
We can improve the robustness of FNNS to JPEG compression by adding a JPEG layer (with quality factor 80) in our optimization pipeline, in which the back-propagated gradients are approximated with identity transformation (Athalye et al., 2018a).
We evaluate on MS-COCO and show results in Table 3.
Although we can successfully encode 0.
bpp (enforcing BCE loss on 10% of pixels) at only 0.06% error, it is fair to say that bit-rates beyond 0.5 bpp are currently still out of reach.
Alternative approaches that we may explore in the future include adopting the Shadow Attack approach from (Ghiasi et al., 2020) to ﬁnd larger semantically meaningful perturbations and incorporating global smoothness constraints.
S TEGANALYSIS
[Table]
Steganalysis tools are used to identify whether an image has a hidden message.
Broadly, these tools can be divided into two categories – statistical and neural steganalysis.
The former uses statistical methods to detect steganography, while the latter trains a neural network to distinguish between natural images and steganographic images.
Statistical steganalysis methods were developed to detect
LSB (least signiﬁcant bit) steganography (Gupta et al., 2012) in lossless images.
Boehm (2014) presents StegExpose, a library combining many existing steganalysis techniques including Sample Pairs (Dumitrescu et al., 2002b), RS Analysis (Fridrich et al., 2001), Chi Squared Attack (Westfeld & Pﬁtzmann, 1999), and Primary Sets (Dumitrescu et al., 2002a).
We show the detection rate of StegExpose for the Div2K dataset in in Table 4.
As seen in the table, some images are detected, but the detection rates are low in comparison with traditional methods that are detected with an accuracy of over 75% (Zhu et al., 2018).
It is worth noting that even if the image is detected as stegnographic, messages cannot be accessed if they are encrypted.
Neural steganalysis makes use of recent advances in deep learning to detect images with hidden information (Ye et al., 2017; You et al., 2020; Lerch-Hostalot, 2021).
Neural networks are powerful and are capable of accurately detecting steganographic images from most existing methods, even when they hide less than 0.5 bpp of information.
Recent work has explored using signal from steganalysis systems in the encoding process to evade detection (Tang et al., 2019; Bernard et al.,
; Zhang et al., 2019b).
Along similar lines, we use the gradient signal from a differentiable steganalysis network, SiaStegNet(You et al., 2020), and include its classiﬁcation score as an auxiliary detection loss in FNNS optimization, and ﬁnd perturbations such that SiaStegNet does not classify the image as steganographic.
We evaluate this method’s feasibility and compare results in
Table 5.
The detection rates are shown in the right most section of the table.
Since SiaStegNet performs random cropping when loading the image, it is hard to guarantee 100% detection avoidance.
Still, with < 2 bpp, we see that FNNS achieves 0.0% error rate, while 80% of generated images can successfully evade detection.
Furthermore, the weight for this detection loss can be adjusted to obtain higher quality images with a slightly higher error rate (see Appendix H).
We also present the detection results for hiding <
1 bpp of information in Appendix H.
In future, we would like to investigate to what extent we can evade detection by black box steganalysis networks and ﬁnd robust defenses against them.
[Table]
can be detected as steganographic by SiaStegNet.
A PPLICATION - F ACE A NONYMIZATION
As shown in table 2, we can reliably obtain
% error rates for hiding 3 bpp messages.
The ability to recover the bit string with zero error allows us to encrypt the message, which is currently the only provable way to safeguard private information.
This property enables us to create a protocol where an encrypted message or image is sent from one party to another, and the ﬁle is itself an indistinct looking image — allowing users to use photo sharing webpages or social media as a medium of transmission.
Below, we describe how FNNS can be used to create a protocol to share images that are anonymized to the public but not to trusted recipients.
Motivation.
Social media platforms have become a central part of our communication, with photo sharing as one of the center activities (e.g. Instagram, Facebook).
There are however well documented dangers with such practices.
If photos become publicly available, the owner loses control.
Snapshots intended to amuse friends or grandparents can “go viral”, with traumatic consequences for individuals portrait in the photos.
Further, public photos are quickly indexed by image search engines (e.g. https://clearview.ai/ ) and can haunt individuals decades later.
We use FNNS to facilitate a mechanism that allows users to share pictures with their friends on social media, but if these are leaked (e.g. through wrong privacy settings on social media pages, or careless behavior), the identity of the people in the images is preserved.
FNNS face anonymization.
Current methods for face anonymization either signiﬁcantly alter the image by using modiﬁcations (like blurring or masking)
(Neustaedter et al., 2006; Boyle et al., 2000), or remove the face completely and replace it with a “fake” face (Hukkel ˚ as et al., 2019).
The former approach can still leak information and cause the quality of the image to degrade, and the latter approach non-reversibly changes the image.
We propose a novel approach for face anonymization using FNNS (see Figure 4).
We make use of DeepPrivacy (Hukkel ˚ as et al., 2019) to detect faces in the original image (left) and replace them with plausible looking GAN-generated faces to obtain an anonymized public image (middle image).
Because the original face is cut before a fake face is generated, the public image leaks no information about the sensitive face and can safely be shared.
Since FNNS can reconstruct the message perfectly, each original private face can be hidden inside its corresponding fake face in the public image.
To ensure the private face is secure, we ﬁrst compress it (e.g. using JPEG (Wallace, 1992)) and then encrypt it into a cryptographically secure bit string (e.g. using AES (Standard, 2001)).
The intended recipient can decode this bit-string, using the decoder network and decrypt the image with a private key to obtain the recovered image (right image).
Any third party will only observe the public image without any access to private information.
See Figure 6 for a layout of the application pipeline.
Because all information is hidden inside the faces in the public image, it is naturally compatible with existing image sharing pages, and the encryption / decryption portion can be realized easily in practice e.g. through a simple browser plugin.
As each fake face contains all the information required to reconstruct the original private face, this approach can be applied to an arbitrary number of (non-overlapping) faces in an image.
An example is presented in Figure 5.
C ONCLUSION
In this paper, we propose a novel and effective algorithm for image steganography based on techniques from the adversarial attacks literature.
We show that our method can achieve very low error rates while making visually unnoticeable changes to the input image.
In contrast to prior work, we are able to achieve error rates of exactly 0% for up to 3 bpp, which in turn enables new applications.
In the future we plan to explore other variants and extension of FNNS.
One useful variant would be to explore new spaces in which to conduct the optimization.
Currently we use RGB space, to ﬁnd perturbations robust to JPEG compression, FNNS only has effects in low-frequency regions. However, other color spaces like YCbCr or non-spatial spaces like the frequency space can be used as well with FNNS.
Using different spaces could result in better image quality and could allow for easier encoding of constraints.
In the future, we would also like to explore meta-learning algorithms for ﬁnding networks that are conducive to FNNS optimization.
A CKNOWLEDGEMENTS
This research is supported by grants from the National Science Foundation NSF (IIS-2107161, III1526012, IIS-1149882, and IIS-1724282), DARPA, the Cornell Center for Materials Research with funding from the NSF MRSEC program (DMR-1719875), and SAP America.
We would like to thank Oliver Richardson, Katie Luo, Kate Donahue and our reviewers for their valuable feedback and insightful comments.
E THICS S TATEMENT
Steganography is a tool and it’s usage depends on the user.
Steganography can potentially be used for activities with negative societal impact, such as hidden communication for criminal activities, or espionage.
But it can also be used for beneﬁcial applications like preventing copyright infringements through watermarking, tracing illegal images on social media or anonymizing publicly shared images (as shown in the paper).
R EPRODUCIBILITY S TATEMENT
For our experiments, we use three publicly available datasets as mentioned in section 4.
We use common metrics to evaluate our results and these metrics are explained in subsection 3.1.
Our method is outlined in Algorithm 1 and we have also clearly described all the hyper-parameter values we used to obtain our results in section 4.
Our code is available at Our code is available at https: //github.com/varshakishore/FNNS .
A A DDITIONAL QUALITATIVE EXAMPLES A. E XAMPLES FROM FNNS-R
[Figure]
Error: 0.14%, PSNR: 36.
Steganographic Image (1 bpp)
Error: 1.29%, PSNR: 31.
Steganographic Image (2 bpp)
Error: 0.79%, PSNR: 21.
Steganographic Image (3 bpp)
Error: 24.93%, PSNR: 35.
Steganographic Image (4 bpp)
Error: 0.0%, PSNR: 39.
Error: 0.06%, PSNR: 34.
Error: 1.41%, PSNR: 31.
Error: 7.56%, PSNR: 31.
Error: 0.01%, PSNR: 38.
Error: 0.1%, PSNR: 33.
Error: 3.18%, PSNR: 32.
Error: 7.9%, PSNR: 30.
Error: 0.0%, PSNR: 36.
Error: 0.04%, PSNR: 32.
Error: 1.96%, PSNR: 31.
Error: 1.46%, PSNR: 22.
Error: 0.0%, PSNR: 20.
Error: 0.11%, PSNR: 32.
Error: 2.04%, PSNR: 31.
Error: 11.77%, PSNR: 30.
Error: 0.0%, PSNR: 36.
Error: 0.0%, PSNR: 21. Error: 0.13%, PSNR: 21.
Error: 3.97%, PSNR: 22.
A. E XAMPLES FROM FNNS-D
[Figure]
Error: 0.0%, PSNR: 35.
Steganographic Image (1 bpp)
Error: 0.0%, PSNR: 32.
Steganographic Image (2 bpp)
Error: 0.0%, PSNR: 27.
Steganographic Image (3 bpp)
Error: 3.11%, PSNR: 29.
Steganographic Image (4 bpp)
Error: 0.0%, PSNR: 35.
Error: 0.0%, PSNR: 29.
Error: 0.0%, PSNR: 28.
Error: 3.08%, PSNR: 31.
Error: 0.0%, PSNR: 31.
Error: 0.0%, PSNR: 28.
Error: 0.0%, PSNR: 24.
Error: 1.56%, PSNR: 22.
Error: 0.0%, PSNR: 31.
Error: 0.0%, PSNR: 28.
Error: 0.0%, PSNR: 24.
Error: 1.58%, PSNR: 22.
Error: 0.0%, PSNR: 38.
Error: 0.0%, PSNR: 36.
Error: 0.0%, PSNR: 29.
Error: 13.57%, PSNR: 34.
Error: 0.0%, PSNR: 40.
Error: 0.0%, PSNR: 37.
Error: 0.0%, PSNR: 28.
Error: 16.5%, PSNR: 37.
A. E XAMPLES FROM FNNS-DE
[Figure]
Error: 0.0%, PSNR: 22.
Steganographic Image (1 bpp)
Error: 0.0%, PSNR: 22.
Steganographic Image (2 bpp)
Error: 0.0%, PSNR: 21.
Steganographic Image (3 bpp)
Error: 2.13%, PSNR: 22.
Steganographic Image (4 bpp)
Error: 0.0%, PSNR: 23. Error: 0.0%, PSNR: 23.
Error: 0.0%, PSNR: 22.
Error: 3.44%, PSNR: 22.
Error: 0.0%, PSNR: 22.
Error: 0.0%, PSNR: 21.
Error: 0.0%, PSNR: 20.
Error: 2.14%, PSNR: 20.
Error: 0.0%, PSNR: 21.
Error: 0.0%, PSNR: 22.
Error: 0.0%, PSNR: 21.
Error: 2.01%, PSNR: 20.
Error: 0.0%, PSNR: 24. Error: 0.0%, PSNR: 24.
Error: 0.0%, PSNR: 22.
Error: 0.53%, PSNR: 24.
Error: 0.0%, PSNR: 27.
Error: 0.0%, PSNR: 26.
Error: 0.0%, PSNR: 25.
Error: 0.43%, PSNR: 25.
B R ANDOM D ECODER A RCHITECTURE
The decoder introduced in section 3 can be any network that takes an image as input and produces sufﬁciently many binary outputs, but some architectures are better suited for the task at hand.
Unlike most prior work, that explores networks that are robust to adversarial attack (Xu et al., 2020), we ﬁnd ourselves looking for architectures that are susceptible to it.
Consequently, in our search for a suitable decoder, we empirically explore many network architectures with different network depth, width, normalization and activation functions and evaluate them w.r.t.
pixel error rates, PSNR, and SSIM.
For the experiments in this sections, we test different architectures using 100 images from the training set of the Div2K (Agustsson &
Timofte, 2017) dataset, with dimensions
× 512 ×unless otherwise noted.
Since, our input consists of images, we focus on convolutional networks (CNN) (Goodfellow et al., 2016).
Network depth and width.
Figure 10 presents results of convolutional networks with varying height and width, and their respective error rates.
Similar to the ﬁnding in Wu et al.
(2020), we observe that wider networks are less robust and that increasing the width can lead to networks that are more sensitive to perturbations.
However, we notice that the opposite is true for depth; the deeper the network, the less sensitive it is to perturbations.
Although a width of 256 seems clearly superior, it is fair to say that the optimization is also slower.
Finding a perturbed version of a 512 × 512 ×image with 3 bpp takes 1 minute for network with a width of 32, but 6 minutes with width 256 (on a NVIDIA GTX 1080 GPU).
Normalization
Error Rate(%) ↓
PSNR(dB) ↑
SSIM ↑
None 36.1 ±
[Table]
Normalization.
As the weights of the decoder network are set randomly, the gradients can have high variance.
In
Table 6 we experiment with two types of normalization layers that do not require mini-batch statistics.
We include Positional
Normalization (PONO) (Li et al., 2019) and Instance normalization (IN) (Ulyanov et al., 2016) in our model.
The results show that both normalization methods improve the error rates and both visual metrics, but IN performs the best in this setting.
We hypothesize this is because normalization stabilizes the gradients and brings activations near the activation tipping points, which eases the optimization process.
Unlike IN, PONO removes the structural statistics (Li et al., 2019), which might be informative to create small perturbations.
[Table]
Activation.
One important design choice for neural networks are the activation functions.
In Table 7 shows results for various popular choices.
We observe that LeakyReLu results in better performance when compared to other activations like sigmoid or tanh.
The PSNR and SSIM values for sigmoid or tanh are (trivially) high because the optimization fails and very few changes are made have explored of fully them to far rates to the input image.
Omitting the activation layer, despite having low error rates, produces images of inferior quality.
Adding the ReLU non-linearity makes the optimization easier and more stable.
The different architectures have different trade-offs in terms of memory, time, and performance.
Wider and deeper networks require more time and memory.
For our random network experiments, we settled on four convolutional layers (three intermediate layers and one output layer) withhidden units each, along with instance normalization and ReLU activation.
C T IMING
For all our reported results, we used images with have a resolution of 512 × 512 (this which results in 1,048,576 hidden bits at 4-BPP).
Table 8 shows the amount of time required to encode a message with different FNNS variants and different bit rates (with standard deviations in parentheses) on a NVIDIA GTX 1080 GPU.
We also explored using images of different sizes and found that the encoding time scales approximately linearly with the number of pixels; for every factor of 4 increase in the number of pixels, the encoding time increases by a little less than a factor of 4.
Note that the encoding optimization is signiﬁcantly slower if run on a CPU.
We have currently not optimized the code explicitly for CPU or mobile GPU cards, and this is an interesting avenue for future work.
[Table]
D U NCONSTRAINED O PTIMIZATION
As we saw in Equation 1, we have two constraints- 1) to ensure that the pixels are between 0 and 1 and 2) to ensure that no pixel changes by more than (cid:15) .
We tried translating this optimization problem deﬁned in Equation 1 to an unconstrained optimization problem by reparameterizing to check if unconstrained optimization yielded better results.
Constraint 1 can easily be relaxed by reparameterizing X ∈ [0 , 1] H × W × 3 to Z ∈ R H
× W × 3 by applying an inverse sigmoid transform
Z = σ − 1 ( X ) and optimizing Z instead of Z .
We can also relax the second constraint with a slightly more involved process by computing the softmax over the set of admissible pixel values.
However, we see no improvements by using these relaxations.
We believe that the main source of error is not the projection step (that is clipping), but rather the restrictions (subpixels remain in [0,1] and change by at most δ ) on the perturbation.
Regardless of the parameterization, these restrictions remain and in some cases will not be satisﬁable.
E S TEGANO GAN H IDDEN L AYER S IZE
9 shows difference in performance when using 32 hidden channels vs 128 hidden channels.
As seen in the table the difference is quite small.
However, using networks with a smaller hidden size speeds up both the training of the network and using it for FNNS.
Hence, for FNNS we use networks with 32 hidden channels.
[Table]
F H IGHER B ITS PER P IXEL
Table 10 shows the performance of FNNS with large message payloads (5-6 bpp).
SteganoGAN results in a very high error rate.
FNNS-based methods can achieve lower error rates with its optimization.
However, in some cases the image quality is not great.
Since the SteganoGAN model cannot encode and decode 5-6 bpp well, FNNS works better with randomly initialized weights than with the pre-trained decoder.
[Table]
G O PTIMIZE F ULLY T RAINED S TEGANO GAN
For FNNS-D and FNNS-DE we used SteganoGAN models trained for 1 epoch as mentioned in section 4.
This ensured that the model was ﬂexible enough to achieve 0% error.
Table 11 shows the result of using SteganoGAN models trained to completion for 32 epochs.
As seen in the table, the error rate decreases signiﬁcantly for FNNS-D when compared with the numbers in Table
[Table]
The learning rate was set to 0.5 and the number of optimization steps was set to 200 when using a
SteganoGAN model trained for 32 epochs.
All other hyper-parameters were the same as listed in section
[Table]
H S TEGANALYSIS E VASION
In section 5, we see that we can add signal from SiaStegNet into the FNNS optimization process by adding an auxiliary SiaStegNet loss term, to evade detection from SiaStegNet.
In Table 12 we follow the same setting and show results for hiding <
1 bpp of information.
To hide less than 1 bpp of information we only compute L BCE with a subset of pixels in the image.
We see that FNNS is able to achieve 0 .
0% error rate and low detection rates for hiding <
1 bpp of information.
Table 5 shows that the image quality is a little low when the additional SiaStegNet loss term is added.
To increase the image quality, we can decrease the weight on the SiaStegNet loss term (from 1000 to 100) and the results of doing so are shown in Table 13.
This results in a slight increase in the error rate but the error rate is still under 1% for < 3 bpp.
[Table] [Table]