HCISNet: Higher‐capacity invisible image steganographic network

IET Image ProcessingVolume 15, Issue 13 p. 3332-3346 ORIGINAL RESEARCH PAPEROpen Access HCISNet: Higher-capacity invisible image steganographic network Yafeng Li, Yafeng Li School of Information Science and Engineering, Shandong University, Qingdao, ChinaSearch for more papers by this authorJu Liu, Corresponding Author Ju Liu juliu@sdu.edu.cn orcid.org/0000-0003-3160-9233 School of Information Science and Engineering, Shandong University, Qingdao, China Correspondence Ju Liu, School of Information Science and Engineering, Shandong University, Qingdao, 266237, China, E-mail: juliu@sdu.edu.cnSearch for more papers by this authorXiaoxi Liu, Xiaoxi Liu School of Information Science and Engineering, Shandong University, Qingdao, ChinaSearch for more papers by this authorXuejing Wang, Xuejing Wang School of Information Science and Engineering, Shandong University, Qingdao, ChinaSearch for more papers by this authorXuesong Gao, Xuesong Gao School of Information Science and Engineering, Shandong University, Qingdao, China State Key Laboratory of Digital Multimedia Technology, Hisense, Qingdao, China College of Intelligence and Computing, Tianjin University, Tianjin, ChinaSearch for more papers by this authorYuyi Zhang, Yuyi Zhang State Key Laboratory of Digital Multimedia Technology, Hisense, Qingdao, ChinaSearch for more papers by this author Yafeng Li, Yafeng Li School of Information Science and Engineering, Shandong University, Qingdao, ChinaSearch for more papers by this authorJu Liu, Corresponding Author Ju Liu juliu@sdu.edu.cn orcid.org/0000-0003-3160-9233 School of Information Science and Engineering, Shandong University, Qingdao, China Correspondence Ju Liu, School of Information Science and Engineering, Shandong University, Qingdao, 266237, China, E-mail: juliu@sdu.edu.cnSearch for more papers by this authorXiaoxi Liu, Xiaoxi Liu School of Information Science and Engineering, Shandong University, Qingdao, ChinaSearch for more papers by this authorXuejing Wang, Xuejing Wang School of Information Science and Engineering, Shandong University, Qingdao, ChinaSearch for more papers by this authorXuesong Gao, Xuesong Gao School of Information Science and Engineering, Shandong University, Qingdao, China State Key Laboratory of Digital Multimedia Technology, Hisense, Qingdao, China College of Intelligence and Computing, Tianjin University, Tianjin, ChinaSearch for more papers by this authorYuyi Zhang, Yuyi Zhang State Key Laboratory of Digital Multimedia Technology, Hisense, Qingdao, ChinaSearch for more papers by this author First published: 01 September 2021 https://doi.org/10.1049/ipr2.12329AboutSectionsPDF ToolsRequest permissionExport citationAdd to favoritesTrack citation ShareShare Give accessShare full text accessShare full-text accessPlease review our Terms and Conditions of Use and check box below to share full-text version of article.I have read and accept the Wiley Online Library Terms and Conditions of UseShareable LinkUse the link below to share a full-text version of this article with your friends and colleagues. Learn more.Copy URL Share a linkShare onFacebookTwitterLinked InRedditWechat Abstract Information hiding as a crucial method for multimedia security and privacy protection, has received a substantial amount of attention. However, the most existing methods focus on the resistance of steganalysis and the robustness of information, resulting in low capacity. In this paper, to further increase the capacity of information hiding, a high-capacity adversarial image steganography model with end-to-end manner, termed as HCISNet is proposed. First, an enhanced Dense Atrous Spatial Pyramid Pooling module is presented to learn more semantic information in cover image, which can adaptively embed more message in redundant regions with rich textures. Then, a modified discriminator network with lightweight residual block is employed to assist the encoder network with optimal high-capacity solution. Meanwhile, the multiple objective function with the perceptual loss and several training tricks are developed, which can improve the visual and perceptual consistency of the original and the steganographic image, further enhancing the capacity. Finally, experiments and analysis are conducted on three public datasets, where this model has better imperceptibility, higher security and greater capacity. Under same experimental conditions, the cover image can embed 5.68 bits per pixel (BPP), far exceeding the previous highest value of 4.4 BPP in state-of-the-art methods in the authors' knowledge. 1 INTRODUCTION Recently, the acquisition, modification and dissemination of multimedia data has become easier and faster, which will involve information security and privacy issues. To cope with these issues, researchers first dedicate to leveraging encryption and digital signatures to protect multimedia data [1-6]. However, cryptography has some limitations: (1) The information represented by cipher text is easier to be identified by stealers. It is more vulnerable to be attacked and only can be used by receiver. (2) Encryption only protects the communication and transmission process of information. Once the decryption conversion is performed, it is difficult to determine whether the data is distributed and copied. (3) With the rapid development of decryption approaches, many cryptography methods are being cracked by high-performance computing, especially in the big data era. Information hiding is another emerging practice for information security and privacy protection [7]. The visual redundancy information is the image resolution changes that cannot be captured in human visual system. Information hiding embeds information into the redundant regions of cover image, ensuring the normal transmission and use of the cover image while people cannot perceive its existence [8]. Compared with cryptography, information hiding can simultaneously hide its content and existence [9, 10], rather than only hiding the content of message. Depending on the purpose of embedded message, information hiding can be divided into steganography and digital watermarking [11]. Among them, Steganography leverages multimedia data as carriers, and embeds secret information into carrier signals [12]. Without being noticed and perceived by the outside world, both parties in communication can use public channels for information transmission. The embedded message in blind digital watermarking is the copyright labelling for digital products, such as copyright owner and copyright user [13, 14]. In contrast, when ensuring the invisibility and undetectability of the embedded information, steganography requires large hiding capacity to improve communication efficiency, so robustness is usually ignored. Compared with steganography, digital watermarking pays more attention to robustness [15, 16], rather than high capacity [17]. 1.1 Motivations Early in the field of information hiding, traditional machine learning methods occupied a dominant position, but limited in speed, poorly scalable and difficult to deploy. Under the wave of deep learning, some pioneering approaches leveraged deep networks to hide and extract information with robustness and information capacity. Mo et al. [18] proposed a generic network against embedding to improve the security and effectiveness of JPEG steganography, which can resist detection by steganalysers. Liu et al. [19] developed a two-stage deep learning network for blind watermarking, using the encoder-decoder network to embed the copyright information, which can achieve resistance to high-intensity noise and enhance network stability. Then, Zhu et al. [20] proposed an end-to-end model framework that can trade off capacity and robustness, which is extensible and easy to train. Also, a deep model watermarking algorithm proposed by Zhang et al. [21] can realise the copyright protection and traceability of the AI model, which hides the information into an image processing CNN model. Although these methods possess good robustness, it is still worthy of exploration in improving the information capacity of cover image. Nowadays, generative adversarial networks (GAN) [22] have made great achievements in academia and industry, meanwhile gained attention in multimedia security and privacy protection. Compared with Zhu et al., the SteganoGAN [23] proposed by Zhang et al. focuses on exploring how to improve the information capacity of the cover image. Besides, it can also resist the detection of steganalysers while ensuring the quality of the generated image. However, SteganoGAN lacks the capability of high-level features extraction, which cannot make full use of semantic context to improve the information capacity. 1.2 Contributions Inspired by the above, in this paper, we propose an end-to-end high-capacity information hiding network model based on the SteganoGAN, termed as HCISNet. First of all, to further strengthen the effectiveness of features at different levels and enhance the payload, we introduce the idea of Dense atrous spatial pyramid pooling (DenseASPP) module [24]. We enhanced this module and employ dilated convolutions [25-27] with different dilation rates to improve the receptive field of features, which can obtain visual redundant information and semantic features in a wider range. Furthermore, we propose a modified discriminator with lightweight residual block [28] to realise fine-tuning and optimization of the encoder network. In addition, in order to optimise the generation of the adversarial network and enhance the stability of the network, we add multiple loss functions with perceptual loss [29] and dual-scale update tricks in the training process to speed up the network convergence. The multiple loss functions can further assist the HCISNet with high-payload performance. Experiments demonstrate that our proposed method can hide arbitrary binary data in natural scene RGB images, which achieves the latest embedding amount of 5.68 bits per pixel (BPP). At the same time, it ensures the non-distortion of the cover image and the invisibility of information. The main contributions of our proposed method are summarised as follows: We construct the enhanced DenseASPP module, which can greatly improve the feature receptive field without increasing model parameters, obtaining more redundant information and semantic features. We argue that the enhanced DenseASPP module can improve image information capacity and accuracy of information extraction. We propose a modified discriminator network with lightweight residual block to classify the original and generated image. In the training process, our proposed module can realise fine-tuning and optimization of the encoder network, meanwhile improve the quality of steganographic images. To enhance the performance and stability of our method, we design a multiple loss function with perceptual loss to train our model. It can also mitigate the gradient dispersion phenomenon. Meanwhile, we optimise the dual-scale update tricks to obtain the high-payload optimal solution of our model. We develop an end-to-end high-capacity information hiding network based on the SteganoGAN, termed as HCISNet. To our best knowledge, HCISNet achieves the best performance on BPP among the current state-of-the-art methods, which can reach a payload of 5.68 BPP. The ablation experiments and visualization can further prove the effectiveness of our method. Furthermore, we leverage three common steganalysers to analyse the steganographic images generated by HCISNet. The results of steganalysers shows that HCISNet can avoid the detection of steganalysis tools better. The rest of this paper is organised as follows. Section 2 briefly introduces the related work of information hiding. Section 3 presents the proposed method and describes our structure. Section 4 explains the important indicators for evaluating model performance, describes the implementation details and ablation experiments. Section 5 is the summary of our work. 2 RELATED WORK 2.1 Watermarking Digital watermarking embeds copyright and other information in multimedia data, which can be accurately extracted even after being modified. Therefore, more attention is paid to the robustness of watermarking [30]. There is a plethora of traditional work on invisible blind watermarking, which is embedded in the spatial domain or the transform domain [31, 32]. To name a few, Fang et al. [33] designed a watermarking scheme based on scale-invariant feature transformation, which considered embedding digital watermark at key points of images. This scheme has strong resistance to multiple screen distortions, and can realise the protection of image information under screen capture and camera shooting operations. Inspired by the great success of deep learning, GAN has been applied for watermarking [34-36]. Luo et al. [34] proposed a distortion-agnostic watermarking framework, which uses adversarial training and channel coding to improve the robustness of the watermark. Zhang et al. [35] first proposed a model watermarking framework, which used a spatially invisible watermarking mechanism to protect the image processing network and has high robustness and resistance to different proxy models. As above, watermarking is generally applied for digital forensics and copyright protection. It has high robustness and tamper resistance, but ignoring the requirements for information capacity. 2.2 Steganography In contrast, when ensuring the invisibility and undetectability of information, steganography requires considerable hiding capacity to improve communication efficiency. Generally, it does not consider robustness. In earlier period research, there are many machine learning methods for image steganography [37]. Not only embedding message in the least significant bit, Shumeet [38] encodes the secret information in all available pixel bits of the cover image. Therefore, it can extract the information from the generated image even after being compressed and propagated. Yu et al. [39] proposed a batch adaptive steganography embedding strategy, which is generic in the spatial and JPEG domain. Tomáš Pevný et al. [40] introduced a steganography method based on cost function, which realised the embedding of information by minimizing the proposed loss. The traditional steganography scheme based on transform domain provides steganographic images with good visual quality and can resist steganalysis, but the embedding ability is limited. As the embedding capacity increases, the quality of steganographic images begins to decrease. Recently, deep learning has demonstrated powerful performance in steganography. To enhance steganographic security, Tang et al. [41] present a steganographic scheme with adversarial embedding operation. Further, Zhong et al. [42] propose a novel strategy of batch steganography exploiting a generative network. Through this method, different payloads can be allocated to different images. Tang et al. [43] propose an adaptive fuzzy inference approach for image steganography, and least significant bit substitution is used to hide the data adaptively. Agrawal et al. [44] propose a Compressed Sensing Image Steganography (CSIS) scheme. It proposes a novel data extraction method to recover secret data lossless and achieves resistant to steganalysis attacks while embedding binary data in images. The above method has good robustness and resistance, but the quality of the constructed secret image has decreased. Moreover, the training is more complicated and difficult. 2.3 Steganography with GAN Besides, some pioneers leverage GAN to study information hiding methods. To name a few, Hayes et al. [45] first introduce the steganographic method through a novel adversarial training scheme. Then, many subsequent studies carried out. Volkhonskiy et al. [46] open a field for steganography application of GAN, which can deceive the steganography analyser. Tang et al. [47] first propose an automatic steganographic distortion learning framework, in which the generator produces stego images according to the learnt distortion function. Ma et al. [48] present a novel adversarial embedding scheme named Steganalytic Feature based Adversarial Embedding (SFAE), which effectively improves the ability to resist steganalysis tools. Li et al. [49] proposed Adversarial image Steganography with adversarial networks (AdvSGAN), which learns to perform steganography within adversarial neural networks while fooling the target steganalyser. Also, Zhang et al. [50] propose an invisible steganography architecture based on GAN to conceal a secret grey image into the RGB one. These methods use adversarial examples for training, which can deceive the steganalyser, but the embedding capacity is very low. In contrast, HiDDeN [20] can achieve a trade-off between capacity and robustness by changing the parameters during training. Similar to HiDDeN, the SteganoGAN [23] method proposed by Zhang et al. focuses on information capacity and invisibility, which employs adversarial training for steganographic tasks and achieves high-capacity hiding. At the same time, Wang et al. [51] improves the network based on HiDDeN and proposes HidingGAN, which can realise high-capacity information embedding and avoid steganalysis. However, these algorithms do not consider the impact of steganographic images at the feature level, and the accuracy of information extraction needs to be further improved. Inspired by the previous works, we propose a novel generative adversarial information hiding method based on SteganoGAN, termed as HCISNet. SteganoGAN only uses basic classification network modules to embed secret message, extract information from stego image, and utilises stacked convolutional layers to discriminate images. HCISNet greatly improves the model performance of payload by only increasing a few network parameters. We develop the enhanced DenseASPP module as the backbone of encoder-decoder network to improve the image information capacity and information extraction accuracy. Meanwhile, we propose a modified discriminator network with lightweight residual block to assist optimization of the encoder network. Also, we applying the multiple loss functions with perceptual loss and dual-scale update tricks to train the model, leveraging cosine annealing to adjust the learning rate. Specially, similar to the HiDDeN framework, our method is an end-to-end deep learning framework. The difference is that the generic network in the target classification field is modified as our model backbone to improve the ability of feature extraction. Compared with the SteganoGAN framework, we first attach the modified DenseASPP module to optimise the network and enhance the network's ability to learn visual redundant information. 3 THE PROPOSED METHOD Our motivation is to design a learnable, unsupervised, high-capacity and end-to-end generative adversarial steganography model. In this paper, our HCISNet can automatically identify the visual redundancy of the image, embed more secret message into the redundant region, thereby increasing the information capacity. Meanwhile, it can accurately recover the embedded message without prior knowledge, means that we only need a steganographic image as input to the decoder network, without any conditional information about the secret message. In this section, we first give the overall descriptions of the proposed HCISNet. Then we introduce the DenseNet module and construct the backbone network based on it. Next, we develop the enhanced DenseASPP and modify discriminator with lightweight residual block to our HCISNet. Finally, we describe our effective training tricks and objective function. In order to improve the generalization ability of the model, we randomly generate binary vectors as secret information during training and testing. In practical applications, we can use encoded text or binary images as secret information. In addition, we define the combination of convolution (Conv), batch normalization (BN), and Leaky ReLU (LReLU) activation function as the Conv-BN-LReLU module. As shown in Figure 1a, the HCISNet generally can be divided into three parts. We utilise the encoder module E ( • ) to encode the cover image I c o v e r and secret information M i n to generate a steganographic image I e n c o d e . I c o v e r and I e n c o d e all select RGB images whose the size is C × H × W , and the hidden message M i n is a randomly generated binary message with a size of D × H × W , where D is the relative payload depth, D ∈ [ 1 , 6 ] . Similarly, the decoder module D ( • ) is utilised to accurately extract the embedded information M o u t from the generated image I e n c o d e . Finally, the discriminator module A ( • ) can evaluate the cover image I c o v e r and the steganographic image I e n c o d e . FIGURE 1Open in figure viewerPowerPoint The structure and algorithms of the HCISNet. (a) The model architecture of the proposed, (b) the embedding and extraction algorithms of the proposed In this paper, the encoder and decoder are regarded as the generator network. As shown in Figure 1b, during the training and testing stages, we employ the generator and discriminator networks, and use adversarial training to improve the embedded information capacity, stego image quality, and information extraction accuracy. In the inference stage, only the encoder module of the generator used to embed information in the cover image, and the decoder module is used to extract the secret information. Specifically, we will explain the encoder, decoder and discriminator in the following subsections. 3.1 Encoder network Encoder module as shown in Figure 1b, given the cover image I c o v e r and the secret message M i n as input, we extract features of the cover image I c o v e r leveraging the Conv-BN-LReLU module and concat the secret message M i n in the channel dimension. Then, we develop the dense backbone to reduce the losses between deep layers. After that, we present the enhanced DenseASPP block to further explore the multi-scale and multi-dilation rate image features. Finally, the features obtained by dense backbone are added to the cover image bitwise as the stego image. 3.1.1 Dense backbone DenseNet is proposed to improve the accuracy of image classification and target detection, rely solely on the dense connection method [52]. It enables the network to reduce the disappearance of gradients and strengthen the forward propagation of features even though in the deep network. As shown in Figure 2, drawing on the DenseNet, we densely connect each layer of convolutional neural network as the encoding backbone, termed as dense encoder backbone, which can reduce the loss of information between layers. Then, in order to alleviate the possibility of neuron deactivation by ReLU, we choose the Leaky ReLU activation function after the convolutional layer and batch normalization. Meanwhile, to maintain consistent resolution of the input and output of each layer and avoid the loss of important information and features, no pooling layer is applied in our dense backbone. Different from the DenseNet used in classification tasks, we remove the final global pooling layer and fully connected layer, and only the transition layer is applied to compress the channel dimensions for fitting the Enhanced DenseASPP module. FIGURE 2Open in figure viewerPowerPoint The backbone structure of encoding module 3.1.2 Enhanced DenseASPP module Motivated by the DenseASPP module for image semantic segmentation, employing different rate of dilated convolution can expand the receptive field without sacrificing feature spatial resolution. Besides, it utilises the advantages of the dilated convolution [27], and employs a dense connection method to achieve the fusion of multi-scale and multi-dilation rate image features. Although dense connection reduces the amount of network parameters, it consumes a lot of memory during the training process. Moreover, as the dilation rate exceeds the threshold, the features obtained by the dilated convolution are meaningless to the network. Also, the ReLU activation function applied in DenseASPP will make the network training very fragile and cannot quickly converge to the optimal solution. Thus, as shown in Figure 3, we modify DenseASPP module to be suitable for information hiding tasks. In detail, we remove the last two layers of DenseASPP and 1×1 convolutional layer. Meanwhile, we apply the Leaky ReLU activation function after each BN layer, which can speed up the network convergence. The enhanced DenseASPP module, while ensuring the invisibility of embedded information and the fidelity of the cover image, integrates the features of multi-scale and different receptive fields. It can better obtain the image redundancy in structure and visual, which can greatly increase the information capacity. FIGURE 3Open in figure viewerPowerPoint The structure of the Enhanced DenseASPP network structure. Considering the size of the feature map, we select the dilation rate d is 3, 6, 12 and 18 3.2 Decoder network HiDDeN [20] adopts the cascaded convolutional layer to extract the features of steganographic images I e n c o d e , then utilises the pooling and fully connected layers to obtain the hidden information M i n . But this can only extract a small amount of message while the structure is not easy to expand. SteganoGAN [23] simply connects the common convolution layers with dense method for information decoding, which lacks the acquisition of high-level semantic features. The structure of the above two networks needs to be optimised and adjusted. Therefore, we construct a structure similar to the encoder network E ( • ) to decode the message, and remove the residual connection of the last layer. As shown in Figure 1b, without resorting to other prior conditions, the designed decoder network D ( • ) only needs to receive the generated steganographic image I e n c o d e as input, which can extract message through the dense backbone and the enhanced DenseASPP module introduced in Section 3.2. The results show that the proposed decoding network D ( • ) can enhance the representation capability of image and visual redundancy information. It will further obtain the semantic features and detailed representations of hidden information, successfully decoding the embedded message M o u t . 3.3 Discriminator network In the previous steganography work, most of the discriminator cascaded a small number of convolutional layers, followed by pooling or fully connected layers. This method only simply obtains low-level features, and loses a lot of valuable information. The post-processing of features is too rough, which makes little contribution to the generator network even has a counterproductive effect. To this end, we introduce the idea of residual block for network designing. In this paper, the original image I c o v e r and the steganographic image I e n c o d e are respectively input to the discriminator A ( • ) . As shown in Table 1, a simple 3×3 convolutional layer and three residual blocks are deployed to extract richer features from the input image firstly, where the symbol N is the batch size at each iteration. After that, the pooling layer is devised to down-sample the features, which can expand the receptive field. Most importantly, the output features are flattened and exploited as the input of the fully connected layer to produce an output discriminant score, which is performed for adversarial training. Specifically, the specific process and network structure of discriminator are shown in Figure 1b and Table 1. TABLE 1. Network structure of discriminator Stage Kernel/ stride/ padding Output sizes N×D×H×W Conv1 16×3×3/ 1/ 1 N×16×360×360 Residual block1 16×3×3/ 1/ 1 N×16×360×360 16×3×3/ 1/ 1 Residual block2 32×3×3/ 2/ 1 N×32×180×180 32×3×3/ 1/ 1 Residual block3 64×3×3/ 2/ 1 N×64×90×90 64×3×3/ 1/ 1 Pooling Adaptive average pooling N×64×3×3 Linear 576×1 N×1 3.4 Loss function For improving the network performance of high-payload, the objective function of our HCISNet can be divided into information embedding and extraction loss, which can be defined as follows: L a l l = L e m b e d + α × L e x t r a c t (1)where L a l l is the overall loss function of the network, L e m b e d and L e x t r a c t are the embedding and extraction loss of information hiding respectively. α is a hyperparameter to balance the two components of loss. Next, we will introduce and explain these two components of losses in detail. 3.4.1 Embedding loss In order to ensure the visual consistency and fidelity of the cover image and the generated image in the high-payload condition, we consider the following three types of loss in the embedding stage: pixel-level mean square error (MSE) loss L E , feature-level perceptual error (PLE) loss L F , and adversarial loss L A . L e m b e d = λ 1 × L E + λ 2 × L F + λ 3 × L A (2)where λ 1 , λ 2 , λ 3 are the weight hyperparameters of each loss function in L e m b e d . In this paper, the mean square error loss L E is leveraged to measure the pixel value error between the original cover image I c o v e r and the steganographic generated image I e n c o d e , ensuring the visual consistency of I c o v e r and I e n c o d e . S and S ′ are the set of cover image and corresponding steganographic images, respectively. The loss L E is defined as follows: L E = M S E I c o v e r , I e n c o d e = ∑ I c o v e r ∈