Year: 2021

Abstract:
Steganography is the task of hiding and recovering secret data inside a non-secret container data while making imperceptible changes to the container. When using steganography to hide audio inside an image, current approaches neither allow the encoding of a signal with variable length nor allow making a trade-off between secret data reconstruction quality and imperceptibility in the changes made to the container image. To address this problem, we propose VLVQ (Variable Length Variable Quality Audio Steganography), a deep learning based steganographic framework capable of hiding variable-length audio inside an image by training the network to iteratively encode and decode the audio data from the container image. Complementary to the standard reconstruction loss, we propose an optional conditional loss term that allows the users to make quality trade-offs between audio and image reconstruction on inference time, without needing to train a separate model for each trade-off setups. Our experiments on ImageNet and AudioSet demonstrate VLVQ’s ability to retain reasonable image quality (28.99 psnr) and audio reconstruction quality (23.79 snrseg) while encoding 19 seconds of audio. We also show VLVQ’s capability to generalize to signals longer than what is seen during training.

heading: 1 INTRODUCTION
text: Natural images are redundant (Ruderman, 1994). They contain predictable structures, repeating textures, and non-uniform color distribution that contribute to their low overall entropy. Meanwhile, the human visual system is insensitive to minor changes to its visual field. These features make steganography, the science of hiding a secret data within a non-secret container data, possible. Several researches have hidden various data inside images such as binary messages (Zhu et al., 2018),
separate images (Baluja, 2017; Tamimi et al., 2013; Wu et al., 2018; Zhang et al., 2019a;b; Duan et al., 2019), and audio signals (Kaul & Bajaj, 2013; Santosa & Bao, 2005; Huu et al., 2019). Here, we focus on the task of hiding audio inside images.
Many algorithms have successfully hidden audio data inside images, especially with the recent advances on deep learning based steganography. However, the use of deep learning restricts these algorithms to only receiving input dimensions that are allowed by the design of the network architecture, which usually have fixed dimensions. This condition makes these methods difficult to operate in the real world, where the signals may easily vary in length. Naively applying these traditional models iteratively on different chunks of the audio fails to work, as the input distribution starts to diverge from that of the training set.
Furthermore, as the bottleneck of most steganographic systems comes from the container data’s capacity, hiding a varying length signal inside a fixed-sized container requires the capability to trade-off audio quality and image fidelity. One approach to do so is by changing the coefficients of the different loss terms that each control the reconstruction quality of the container and the secret data. However, this requires the training of several systems with different coefficients, which is not only time and memory consuming, but also fails to cover the full continuous range of possible coefficient combinations.
We propose a new steganographic framework VLVQ (Variable Length Variable Quality Audio Steganography) to encode variable length audio signals in an image and make audio quality and image fidelity trade-off with a single model on inference time. Figure 1 demonstrates an example of VLVQ encoding 19 seconds of audio inside an 224x224 image then decoding it back.
Given the initial container image, VLVQ iteratively embeds an audio chunk inside the image until all of the audio has been encoded. To recover the audio back from this final container image, VLVQ isolates the latest audio chunk embedded inside the container, and update the container image to feed itself back into the decoder recursively. This is repeated until all audio is recovered. An important design choice is to use the trade-off coefficient that linearly weighs the image and audio reconstruction quality as an input to the network. By randomly sampling this coefficient at each training step and feeding it as input to the model, a single model can learn a distribution of solutions.
Section 2 reviews previous works related to VLVQ. Section 3 describes the problem setup and VLVQ’s methodology in depth. Then, Section 4 evaluates our approach on ImageNet (Russakovsky et al., 2015) and AudioSet (Gemmeke et al., 2017), demonstrating reasonable reconstruction quality even on signals longer than what is seen during training. Finally, Section 5 discusses various observations and possible directions for future work.

heading: 2 RELATED WORKS
text: Image Steganography. Steganography is the task of encoding a secret data inside a non-secret container data without making noticeable changes to the container. Image steganography describes a scenario that uses an image as the container that carries the secret data. The limitations of the human visual system in noticing slight changes in brightness or color among many pixels make these image steganography methods effective. Traditional image steganography methods exploit the image spatial domain by modifying the least significant bits (LSB) of the image, or the frequency domain with Discrete Fourier Transform (DFT), Discrete Cosine Transform (DCT), and Discrete Wavelet Transform (DWT). With the help of recent advancements in learning-based algorithms, steganographic systems have seen tremendous advancements. Many of the learning-based steganographic systems employs deep neural networks that are trained end-to-end with reconstruction losses that minimize the change in the container while maximizing the similarity between the recovered secret data and the ground truth. Baluja (2017) uses convolutional architecture to hide images inside images, while Zhu et al. (2018) adopts differentiable augmentations to hide and recover binary data inside images with robustness to image corruptions. Different approaches make improvements in architectural designs (Duan et al., 2019; Cui et al., 2020; Wu et al., 2018) while some adopt Generative Adversarial Networks (GANs) (Goodfellow et al., 2014) to make the changes further imperceptible.
Hiding Audio in Image. Hiding audio signal inside images is a specific instance of image steganography. Similar to image steganography in general, methods such as modifying the least significant bits (LSB) (Kaul & Bajaj, 2013) as well as wavelet transformations (Kaul & Bajaj, 2013; Santosa &
Bao, 2005) have been applied to the task. Deep learning based approaches such as Huu et al. (2019) encodes STFT features of audio into the container image with a convolutional architecture. Ye et al. (2019) incorporates GANs. Still, these methods are limited to fixed dimensional inputs, prohibiting its applications in the real world with audio signals that vary in length.
Loss-Conditional Training. Generally, deep learning handles multi-objective training by linearly combining the different sub-objective terms with their respective coefficients. If the coefficient values are closely tied to the model’s use cases, one must train multiple models with different coefficient combinations to cover all such cases, which is computationally expensive and impractical. Lossconditional training resolves this by conditioning the model on the randomly sampled coefficients during training. Babaeizadeh & Ghiasi (2018) first adopted loss-conditional training in style transfer to make the style and content coefficients in style transfer adjustable at inference time. Dosovitskiy & Djolonga (2019) then proposed a general framework for loss-conditional training, demonstrating its applications in autoencoder, image compression, and style transfer. Lee et al. (2020) adopts loss-conditional training in creating a dynamic residual network with adaptable sparsity. Since the importance of audio data reconstruction quality varies among the different use cases of steganography (e.g., in speaker identification, high audio quality may not be necessary as long as the voice is identifiable), we adopt an optional loss-conditional training process to allow the trade-off between these two objectives at inference time.

heading: 3 METHOD
text: In this section, we describe the VLVQ (Variable Length and Variable Quality Audio Steganography) framework in detail. We first describe the problem setting, then elaborate on the specific methods. Please refer to Algorithm 1 for an overview of the methods.

heading: 3.1 PROBLEM SETUP
text: In steganography, a system H takes a container data c with a secret data s to produce a modified container ĉ = H(c, s) that encodes s while remaining visually similar to c. Then, a system F takes ĉ to recover the secret data s ∼ ŝ = F(ĉ). In Audio-in-Image Steganography, c is an image, and s is an audio of variable length. Since the container’s capacity is the limiting factor in what is capable of being hidden, we adopt γ to represent the trade-off parameter between the reconstruction quality of the container and the secret data. Here, a larger value of γ increases the quality of the audio reconstruction at the sacrifice of the container image quality, while a smaller value makes the change on the container image less noticeable at the sacrifice of the audio reconstruction quality.
The audio data goes through STFT (Short-Time Fourier Transform) during the preprocessing step and converts into the frequency domain. This makes s, ŝ ∈ RN×T×2, with N as the number of frequencies, T as the varying length of the audio, and 2 as the real and complex channel. c and ĉ is
Algorithm 1: VLVQ Framework conditioned on γ Result: Fully trainedH and F Algorithm parameters: Container image c, Secret audio s, γ range γmin, γmax, total iteration Ttotal, audio chunk number range Cmin, Cmax, frequency number and image resolution h,w; InitializeH,F with random weights; repeat Ttotal times
Sample random integer k in range [Cmin, Cmax]; Let T = w × k, c1 = c; Sample γ from log(γ) ∼ U(log(γmin), log(γmax)); Sample s ∈ Rh×T×2 and c ∈ Rh×w×3 from the dataset; Crop s into si ∈ Rh×w×2, i = 1 . . . k; for i = 1 to k do
ci+1 = H(ci, si, γ); end ĉk+1 = ck+1; for i = k + 1 to 2 do
ĉi−1, ŝi−1 = F(ĉi, γ); end Limg = 1k ∑k+1 i=2 ||ci − c1||2;
Ldec = 1k ∑k
i=1 ||ĉi − ĉi+1||2; Lstft = 1k ∑k i=1 ||si − ŝi||2; Ltotal = Limg + Ldec + γLstft; Backpropagate and optimizeH and F with respect to Ltotal;
end
an RGB image with c, ĉ ∈ Rh×w×3. We set N = h = w for simplicity and only use h and w to denote spatial dimensions, thus s, ŝ ∈ Rh×T×2.

heading: 3.2 RECURSIVE PATCH INFERENCE
text: Algorithm. T is a value that changes across different inputs. However, having T = h = w would keep the spatial resolution of both s and c identical and constant, which is an ideal condition for inference on a deep neural network. We sample s to have T as a multiple of w during training, so we can crop s into k = Tw chunks, with each chunk si ∈ R
h×w×2, i = 1 . . . k. To guarantee this during inference, we pad s to s ∈ Rh×wd Tw e×2 and crop s into k = dTw e chunks. Encoding Process. Let ci denote the container image with i − 1 audio chunks encoded inside. c1 = c, and ci+1 = H(ci, si), which denotes that the current container and a chunk of the secret data is fed into H to update the container, and it is recursively fed back into H with the next chunk of the secret data until i = k + 1. Finally, ĉk+1 = ck+1, which denotes the final container that conceals all the audio chunks.
Decoding Process. Then, ĉi−1, ŝi−1 = F(ĉi), which recursively feeds the container image to F to decode each audio chunks, while updating the container image to recursively feed itself back into F . With this process, we obtain ŝ1 . . . ŝk that corresponds to s1 . . . sk. Figure 3 visualizes both the encoding and the decoding process.

heading: 3.3 OPTIMIZATION OBJECTIVE
text: We optimize both H and F with respect to Ltotal, an objective function that linearly weighs three sub-objective functions.
Audio Container Distance. To ensure that the decoded audio is similar to the ground truth audio at the same chunk index, we minimize the average of their mean-squared distances Lstft. Also, to minimize the change H makes on the container image, we minimize Limg which is the average of
the mean-squared distances between c1 and ci for i = 2 . . . k + 1.
Lstft = 1
k k∑ i=1 ||si − ŝi||2 Limg = 1 k k+1∑ i=2 ||ci − c1||2 (1)
Decoding Consistency. F continuously updates the container image from the previous iteration. To prevent this process from destroying the information about the remaining audio chunks, it is essential to prevent F from making large modifications to the container. We achieve this by minimizing the mean-squared distance between the container images of two consecutive decoding iterations, Ldec.
Ldec = 1
k k∑ i=1 ||ĉi − ĉi+1||2 (2)
Full Objective. The final loss term Ltotal is a linear combination of the above three loss terms, and the hyperparameter γ determines the trade-off between audio and image reconstruction quality. Figure 2 demonstrates the effects of this parameter by visualizing the changes of the residuals. We simply keep the coefficient of Ldec as 1.0, as it didn’t have a significant effect on the outcome.
Ltotal = Limg + Ldec + γLstft (3)

heading: 3.4 TRADE-OFF COEFFICIENT CONDITIONING
text: Modifying γ in equation 3 gives control over the trade-off between audio reconstruction quality and container image quality. However, to allow such trade-offs at inference time with varying γ terms, one must train several VLVQ frameworks, which would be computationally expensive. To address this, the VLVQ framework has the option to adopt a strategy of directly conditioning the models with γ, such thatH(ci, si) and F(ĉi) may becomeH(ci, si, γ) and F(ĉi, γ). γ Conditioning. Both H and F are composed of several convolutional blocks, with each block following the order of convolution (Conv), normalization (Norm), and activation function (Act). Given the input and output of a block X,Y ∈ RH×W×C , with γ ∈ R1, γ > 0.
Y = Act(Norm(Conv(X))) (4)
To condition γ, we adopt the FiLM conditioning layer of Perez et al. (2018), which does a linear transformation on the convolutional feature with scale and shift parameters generated from the conditional variable.
α = f(γ) β = h(γ)
H = Norm(Conv(X))
Y = Act(αH+ β)
(5)
Here, α ∈ RC , β ∈ RC , H,Y ∈ RH×W×C . f and h are two-layer fully-connected networks with ReLU activation. Figure 4 illustrates this conditioning module.
Sampling Distribution of Gamma Since γ is no longer a hyperparameter, we sample γ from a fixed distribution each training iteration. We apply a commonly used hyperparameter distribution of log-uniform distribution (Bergstra & Bengio, 2012) and find γmin = 0.01 and γmax = 100.0 to be a reasonable range.
log(γ) ∼ U(log(γmin), log(γmax)) (6)

heading: 4 EXPERIMENTS
text: Section 4.1 describe the implementation details of the VLVQ framework, and Section 4.2 describes the datasets and metrics used in our experiments. Section 4.3 demonstrates the effects of modifying the γ parameter to both conditional and non-conditional versions of the framework, and Section 4.4 tests VLVQ’s capability to extrapolate and encode variable length signals.

heading: 4.1 IMPLEMENTATION DETAILS
text: Architecture. We adopt a U-Net (Ronneberger et al., 2015) - like architecture for both H and F . When conditioning the network on γ, we insert a FiLM layer (Perez et al., 2018) in all convolutional and transposed convolutional blocks. H receives a 5-channel input by concatenating ci and si, and F produces a 5-channel output which is split in the channel dimension into ĉi and ŝi. Please refer to the supplementary materials for more details.
Hyperparameters. The network is trained end-to-end via Adam (Kingma & Ba, 2014) with the initial learning rate of 0.001, batch size of 4, and betas of 0.9 and 0.999 for two epochs. The learning rate decays by a factor of 10 halfway through the training. The framework is implemented on the PyTorch (Paszke et al., 2019) framework, and all models are trained on a single Nvidia Tesla V100 GPU.

heading: 4.2 EXPERIMENT SETUP
text: Dataset. For the images, we use the ILSVRC 2012 subset of the ImageNet (Russakovsky et al., 2015) dataset, containing 1.28 million natural images for training and 50k for validation. Images are preprocessed with a resize and a center crop of 224x224. For the audio, we use the AudioSet (Gemmeke et al., 2017) dataset, containing 1.7M audio clips that are each 10 seconds long. We convert all the samples into the frequency domain image with a height of 224 via STFT. After concatenating all the samples, one to three 224x224 random crops are taken from them every training iteration, with each crop representing an audio chunk that accounts for approximately 6 seconds of audio.
Evaluation metrics. For the container image quality, we measure PSNR (Peak Signal-to-Noise Ratio) between the original container image and the current container image (with the audio data embedded inside). For the audio reconstruction, we measure SegSNR (Segmental Signal-to-Noise Ratio) between the original audio and the reconstructed audio.

heading: 4.3 EFFECTS OF MODIFYING GAMMA
text: Verifying Trade-offs. Increasing the γ parameter on VLVQ should increase the audio quality at the sacrifice of the container image quality and vice versa. Such trend is shown for both conditional and non-conditional models in Figure 6, demonstrating that modifications in γ yield meaningful trade-offs between different objectives. An ideal conditional VLVQ framework should have near-identical results compared with the non-conditional VLVQ framework individually trained on their respective γ parameters. However, a slight degradation is expected as the consequence of having a difficult optimization space that covers many values of γ, forcing the model to learn a distribution of solutions. Again, such trend is observed between the conditional and non-conditional model.

heading: 4.4 NUMBER OF ENCODED AUDIO CHUNKS
text: We test whether the VLVQ framework stands to the claim of being capable of encoding audio signals with varying length. To evaluate this, we test beyond the expected number of chunks during training (Cmin = 1, Cmax = 3) by evaluating up to 7 audio chunks. For both non-conditional and conditional framework, Figure 5a, 5b and Table 1 each shows that while both image and audio reconstruction quality drops with longer encoded sequences, the drop continues to maintain a gradual decline from 1 to 7 instead of a rapid drop at 4. This trend demonstrates VLVQ’s reasonable extrapolation capability.

heading: 5 DISCUSSION
text: Defenses Against Steganalysis. While we make no claims regarding VLVQ’s defensibility against steganalysis, future work may evaluate and equip VLVQ with defenses (Lyu & Farid, 2002; Fridrich, 2004; Fridrich & Kodovsky, 2012; Kodovsky et al., 2011; Qian et al., 2015) to prevent algorithmic detections.
Failure Modes. When dealing with multiple audio chunks, F sometimes fails to separate the signals between different chunks, resulting in the leakage of audio chunks into different timesteps. This usually happens when different audio chunks greatly vary in volume, as the louder chunks dominate the signals in the container image. Future work may employ an adversarial framework by introducing an adversarial framework to discourage F from producing signals that can be classified as originating from a different chunk.
Conditioning Methods While we used FiLM on all convolutional blocks due to its simple architecture choice, one may search for a better method of conditioning γ, such as searching for the optimal layers to place the conditioning module.
Limitations. Although a simple U-Net (Ronneberger et al., 2015) was sufficient to yield satisfactory results, there are clear limitations to this architecture. For example, a convolutional layer’s inductive bias forces the network to rely mostly on local information. Figure 2 demonstrates that most residuals are visible at the top of the image, where the most prominent STFT features lie. To exploit the full capacity of the container image, architectures with minimal inductive biases such as transformers (Vaswani et al., 2017) may be employed.

heading: 6 CONCLUSION
text: In this work we present VLVQ, a steganographic framework that encodes variable length audio data into images with varying quality trade-offs. Compared to other frameworks of this kind, VLVQ enables a greater degree of freedom in terms of secret audio length and infernce-time quality tradeoffs. We achieve this through recursive inference of multiple audio chunks and conditioning the model with the trade-off parameter via FiLM layers. Experiments on ImageNet and Audioset verify these claims and demonstrate extrapolation capabilities beyond audio length seen during training. We hope these efforts make audio-in-image steganography feasible in diverse real-world scenarios.

heading: A ETHICS STATEMENT
text: With growing concerns over various societal issues regarding individual privacy, tools such as steganography can empower individuals to have greater control over their information. For example, activists in nations where encryption is criminalized may apply steganography in a non-secret medium to hide their messages. In this aspect, our work helps widen the available pool of steganographic medium an individual can hide their information on. But this raises concern to a potential risk based on the malicious uses of steganography, such as its applications as digital fingerprints in intelligence services. Hence, researches on steganography and its applications must be developed with these positive and negative effects in consideration.

Review summaries:
This paper presents a steganographic approach called Variable Length Variable Quality Audio Steganography (VLVQ) that encodes variable length audio data inside images with varying quality trade-offs. The method works by training a network to iteratively encode and decode the audio data from the container image. An optional conditional loss term is also proposed apart from the standard reconstruction loss. Experiments presented on two large datasets - ImageNet dataset for container images and AudioNet dataset for embedding the audio, shows that the approach is promising.   

The paper presents a method for steganography, that is to encode and decode an audio signal into an image in an unnoticeable way. The feature of the proposed method is that it can handle audio with variable length, and can flexibly trade-off between audio encoding quality and image modification unnoticeability in the inference stage. The method is to learn a pair of encoder and decoder to achieve both encoding and unnoticeability up to some trade-off, and the trade-off coefficient is also input into the model to avoid retraining for an arbitrary trade-off in inference.

This submission deals with steganography, i.e. the task of hiding a secret message inside a multimedia content. The authors in this submission choose images as cover content and audio signals as secret messages. They also ask their embedding algorithm to be flexible in the sense that variable length audios can be processed by the same algorithm. This is achieved by cleverly designing a neural network pipeline with recursions to tackle the variable length message issue.