# Domain Transferred Image Recognition via Generative Adversarial Network

Authors: Haoqi Hu; Sheng Li; Zhenxing Qian; Xinpeng Zhang
Journal: Security and Communication Networks
 (2022)

---

## Abstract

Recent studies have demonstrated that neural networks exhibit excellent performance in information hiding and image domain transfer. Considering the tremendous progress that deep learning has made in image recognition, we explore whether neural networks can recognize the imperceptible image in the transferred domain. Our target is to transfer natural images into images that belong to a different domain, while at the same time, the attribute of natural images can be recognized on domain transferred images directly. To address this issue, we proposed domain transferred image recognition to achieve image recognition directly on the transferred images without the original images. In our proposed system, a generator is designed for the domain transfer and a recognizer is responsible for image recognition. To be flexible for the natural image restoration in some cases, we also incorporate an additional generator in our method. In addition, a discriminator will play an indispensable role in the image domain transfer. Finally, we demonstrate that our method can successfully identify the natural images on transferred images without access to original images.


---

## Body

## 1. Introduction

Recently, there have emerged numerous methods for privacy protection-awareness [1–3]. At the same time, deep learning has made great breakthroughs in speech, image, and text recognition [4–6]. However, training these networks requires a large amount of data, which makes some giant companies such as Google, Microsoft, and Amazon or personalized customization organizations which try every means to collect personal data of their users for training deep models [7]. Despite of great performance of these well-trained deep networks, they bring huge privacy risks [8, 9].Most of small businesses or individuals use the cloud services provided by giant companies for deep learning tasks since they are limited to the local storage capacity and GPU resources. However, the data collected by these organizations can be reused repeatedly, making users difficult to delete. Besides, these sensitive data may contain unique personal identical information such as faces and voices, which inevitably bring risks when stolen by malicious attackers and used for illegal benefits [10, 11].To securely perform image recognition, information hiding, which conceals important secret information in the carrier (image, video, audio, etc.), can solve the issue of privacy leakage elegantly and flexibly [12, 13]. However, information hiding mainly focuses on protecting secret information from being leaked during transmission. As shown in Figure 1, when image recognition is required, it has to restore the secret images, which increases the risk of information leakage. Moreover, to hide abundant secret information, it is necessary to select an appropriate carrier with large redundant room, which is time-consuming and will inevitably increase the risk of information leakage once the carrier is intercepted. To this end, we propose a method that can achieve image recognition in the transferred domain directly, which can recognize the attributes of secret images on transferred images.Figure 1
The comparison of application processes. For a system that requires image recognition in the cloud server, to protect the image content privacy, other methods based on information hiding will first select a textured carrier/cover image and then apply a special embedding algorithm to hide the secret image in a carrier image and send it to the cloud end. The receiver will apply the corresponding extracting method to obtain the secret image for recognition. Differently, our method is free from carrier image by transferring the secret image to another domain and the recognition process can be performed on the transferred domain directly.Specially, our detailed application scenario for real world is shown in Figure2; a giant company deploys an image recognition service in the cloud for profits. To avoid data leakage of users, we train a series of models ahead to reach this goal. In the real service deployment, the service provider owns/deploys only the well-trained recognizer in its cloud, and the domain transfer generator is deployed in a trustworthy party.Figure 2
The conception of our system. The models are trained under supervision, where the domain transfer generator is used to protect the content privacy of the user's secret images and a recognizer network which is used to classify secret images in the transferred domain. In the cloud recognition application scenario, the domain transfer generator is handed over to a trusted party, and the recognizer is controlled by an organization which is deployed on a cloud service.Therefore, a user uploads a secret image to the trustworthy party to transfer his/her image into another image which has different content/style and then delivers the transferred image to the cloud for image recognition.To perform image recognition in the transferred domain, we designed a transfer generator to accomplish the process of domain transfer. Same as the classic generative adversarial networks, an indispensable discriminator aids the generator to generate a high-quality image. Besides, a classifier is responsible for the image recognition task. Although our system is able to recognize the attributes of the original images through the transferred images directly, to facilitate the original image restoration in some cases, we also incorporate an additional restore generator to recover the secret image in our method. In summary, our contributions are three-fold:(i)
To achieve image recognition as well as avoid private information leakage issue, we proposed a framework to perform image recognition directly on domain transferred images(ii)
Our image recognition method is performed on the transferred domain, which decreases the exposure risk of the source image and omits the process of carrier selection(iii)
Experiments demonstrate the availability of our method in terms of classification accuracy and visual effect of the transferred images


## 2. Related Work

In this section, we will first review some secure inference and visual information protection strategies.


### 2.1. Encryption

Encryption technique is used for data privacy protection. Its basic target is to hide the content of the data, which encrypts the raw data into the ciphertext data and decrypts it back to the original version with the corresponding decryption algorithm.Homomorphic encryption proposed by Craig [14] can support arbitrary computations on the encrypted data and the final calculation result can be obtained after decryption. However, this technique, which is calculated in the encrypted domain, is computationally expensive compared to plaintext calculation. For example, recently, Sanyal [15] proposed a homomorphic encryption-based image classification algorithm to protect image privacy. However, it takes nearly 2 hours to classify encrypted MNIST images on a 16-core workstation, which is impractical in reality. Moreover, the encrypted images can raise the awareness of attackers due to its unreasonable ciphertext.Secure multiparty computation is an important branch of cryptography, which aims to solve the problem of collaborative computing that protects privacy among a group of untrusted parties. Implementations of predicting encrypted data based on secure multiparty computing have flourished [16, 17]. However, these methods require the data owner to encrypt the inputs and constantly interact and communicate with each other. Besides, as mentioned in [18], most of the existing multiparty computation-based secure inferences rely on customized protocols that are highly optimized for particular activation functions. For example, XONN [19] is currently the most efficient solution for 2-party protocol, but XONN only works with Sign as the activation function. Implementing exponential function (Sigmoid) or max function (ReLU) requires heavy computations and communications in SMPC-based solutions.


### 2.2. Information Hiding

Information hiding is one of the most important ways to protect secret data, which has been well researched in the past decades [20–22]. This technique can be roughly classified into two categories: digital watermarking and steganography. Recently, many deep learning algorithms related to information hiding have been developed. Usually, digital watermarking technology hides a particular bit string in inconspicuous places to protect the copyright of images, models, etc. Uchida et al. [23] embed a watermark in model parameters using a regularizer to protect the intellectual property of trained models, with the performance of trained model hardly affected. Rouhani et al. [24] embed watermarking in the weight distribution of convolutional layers in trained models to protect deep learning models. Baluja [25] successfully hides a full-size color image into another image of the same size based on a deep image encoder and decoder network to realize image steganography. However, these methods focus on protecting the security of hidden information during transmission. These methods have to go through the process of extracting hidden secret information when using these information, which may lead to hidden secret information leakage after restoration.


### 2.3. Image Domain Transfer

Image domain transfer refers to the process of generating another image according to one image, that is, transfer one image from one domain to another. Pix2pix proposed by Isola et al. [26] provides a concise and elegant general framework for solving a series of image domain transfer tasks, and the author proves that the method has good performance in tasks such as segmentation map to street view map, grayscale map to color map, and clothing outline sketch to color map. In addition to supervised image domain tasks (with paired training samples), for unsupervised image domain transfer tasks without paired images as training samples, Zhu et al. [27] designed CycleGAN to complete the image domain transfer process from one dataset to another dataset. Moreover, StarGAN [28] proposed by Choi et al. can complete various attribute conversions of face images, such as gender, age, skin color, and emotions, and Tang et al. [29] proposed a method that can transfer an image from a source to a target domain guided by controllable structures. However, these GAN-based image domain transfer methods simply focus on approximating the distribution of generated images with the distribution of the target domain. Inspired by the success of deep learning in multitasking [30, 31], we turn our attention to the image recognition task in the transferred domain that is free from image content exposure.


## 2.1. Encryption

Encryption technique is used for data privacy protection. Its basic target is to hide the content of the data, which encrypts the raw data into the ciphertext data and decrypts it back to the original version with the corresponding decryption algorithm.Homomorphic encryption proposed by Craig [14] can support arbitrary computations on the encrypted data and the final calculation result can be obtained after decryption. However, this technique, which is calculated in the encrypted domain, is computationally expensive compared to plaintext calculation. For example, recently, Sanyal [15] proposed a homomorphic encryption-based image classification algorithm to protect image privacy. However, it takes nearly 2 hours to classify encrypted MNIST images on a 16-core workstation, which is impractical in reality. Moreover, the encrypted images can raise the awareness of attackers due to its unreasonable ciphertext.Secure multiparty computation is an important branch of cryptography, which aims to solve the problem of collaborative computing that protects privacy among a group of untrusted parties. Implementations of predicting encrypted data based on secure multiparty computing have flourished [16, 17]. However, these methods require the data owner to encrypt the inputs and constantly interact and communicate with each other. Besides, as mentioned in [18], most of the existing multiparty computation-based secure inferences rely on customized protocols that are highly optimized for particular activation functions. For example, XONN [19] is currently the most efficient solution for 2-party protocol, but XONN only works with Sign as the activation function. Implementing exponential function (Sigmoid) or max function (ReLU) requires heavy computations and communications in SMPC-based solutions.


## 2.2. Information Hiding

Information hiding is one of the most important ways to protect secret data, which has been well researched in the past decades [20–22]. This technique can be roughly classified into two categories: digital watermarking and steganography. Recently, many deep learning algorithms related to information hiding have been developed. Usually, digital watermarking technology hides a particular bit string in inconspicuous places to protect the copyright of images, models, etc. Uchida et al. [23] embed a watermark in model parameters using a regularizer to protect the intellectual property of trained models, with the performance of trained model hardly affected. Rouhani et al. [24] embed watermarking in the weight distribution of convolutional layers in trained models to protect deep learning models. Baluja [25] successfully hides a full-size color image into another image of the same size based on a deep image encoder and decoder network to realize image steganography. However, these methods focus on protecting the security of hidden information during transmission. These methods have to go through the process of extracting hidden secret information when using these information, which may lead to hidden secret information leakage after restoration.


## 2.3. Image Domain Transfer

Image domain transfer refers to the process of generating another image according to one image, that is, transfer one image from one domain to another. Pix2pix proposed by Isola et al. [26] provides a concise and elegant general framework for solving a series of image domain transfer tasks, and the author proves that the method has good performance in tasks such as segmentation map to street view map, grayscale map to color map, and clothing outline sketch to color map. In addition to supervised image domain tasks (with paired training samples), for unsupervised image domain transfer tasks without paired images as training samples, Zhu et al. [27] designed CycleGAN to complete the image domain transfer process from one dataset to another dataset. Moreover, StarGAN [28] proposed by Choi et al. can complete various attribute conversions of face images, such as gender, age, skin color, and emotions, and Tang et al. [29] proposed a method that can transfer an image from a source to a target domain guided by controllable structures. However, these GAN-based image domain transfer methods simply focus on approximating the distribution of generated images with the distribution of the target domain. Inspired by the success of deep learning in multitasking [30, 31], we turn our attention to the image recognition task in the transferred domain that is free from image content exposure.


## 3. Domain Transferred Image Recognition

In order to transfer the secret image from the source domain to a target domain, an image generator is indispensable. Besides, a necessary classifier/recognizer will be responsible for image recognition in the transferred domain. As mentioned before, the domain transfer will alter the distribution of source images, in which way the privacy of images can be protected. However, changes in the distribution of data will hamper the image recognition task. Therefore, the generator and classifier modules will be trained in turn, just as the way of training a generative adversarial network. Besides, an alternative restore generator can also be incorporated to restore the secret image in some cases. Therefore, as shown in Figure3, our whole framework mainly contains four modules: two generators, one discriminator, and one recognizer.Figure 3
The schematic structure of our proposed system. For a secret image, it is firstly transferred to another image with a different style using the domain transfer generator. The transferred image serves as the input of the discriminator, recognizer, and restore generator. The recognizer can identify the attributes of the secret images on the transferred images. The restore generator is responsible for reconstructing the secret images based on transferred images. The discriminator aims to distinguish images in the target domain from images generated by the transfer generator.In order to describe the proposed method more clearly, we define the relevant concepts and variables as follows. The secret image dataset/domain is denoted asXnatural, where the image in it is represented by Ise. Similarly, the target dataset/domain is defined as Xtarget, where the image in it is denoted as Ita. Itr and Ire respectively indicate the transferred images output by the domain transfer generator and the restored images output by the restore generator. Gt and Gr represent the generator for the domain transfer and the generator for original secret image restoration, respectively. The discriminator is denoted by D, and the recognizer ℛ can identify secret image attributes on the transferred domain images Itr.


### 3.1. Domain Transfer Generator

Given a secret imageIse from the dataset Xnatural to be identified, the domain transfer generator (corresponding to Gt in Figure 3) aims to transfer the secret image into an image in the target domain with a different style such as a cartoon face style and an animal face style. In other words, the transfer generator Gt should extract the feature of the secret image as much as possible and transform this feature into an image that conforms to the distribution of the target domain. Besides, the transferred image must fool the discriminator to make it fail to tell whether this image is generated by Gt or comes from Xtarget. When the whole system is fully trained, the generator Gt can be used as an independent module to perform image transfer from Xnatural domain to Xtarget domain.The domain transfer process can be written as(1)Itr=GtIse.As for the design of our domain transfer generator, given that the convolutional neural network is experimentally proven to have excellent feature learning and extraction abilities for images and that a large amount of autoencoders have shown extraordinary performance in image generation, we adopt the combination of convolutional neural network and autoencoder and design the generatorGt with a ResNet [32] structure. Specifically, as shown in Figure 4, we first stack 3 sets of convolutional layers, instance normalization layers, and ReLU activation, followed by 9 residual blocks, with each residual block containing two convolutional layers, two instance normalization layers, and one ReLU activation. The input of each residual block and the output of the latter instance normalization will be added as the output of each residual block. Finally, 2 sets of deconvolution layers, ReLU activation layers, and instance normalization layers are followed to make the size of the final generated image consistent with that of the original image. By the way, the last activation layer will be tanh to cover the full pixel range.Figure 4
Structure of domain transfer generatorGt and restore generator Gr.


### 3.2. Discriminator

For deep models based on generative adversarial networks, the discriminatorD is indispensable in ensuring the quality of the generated images. The discriminator is expected to learn features that can distinguish the image Ita in the target domain from the generated image Itr in the transferred domain. When the input image is from the target domain Xtarget, the discriminator should identify it as a “real” image, and when the input image is generated by the generator Gt, the discriminator should identify it as a “fake” image, which can be mathematically expressed as(2)t=DIin,where t represents the output of discriminator D, t∈ {“real”: image from target domain, “fake”: image generated by Gt}, and Iin represents the input image of D.As an auxiliary module of the generatorGt, for each input image, the discriminator D needs to judge whether it is “true” or “false.” But, unlike generators which have to perform complex image generation tasks, the discriminator only makes a binary decision. Therefore, our framework only contains one discriminator, a very “shallow” network. The specific structure is shown in Figure 5(a). First, we stack a convolutional layer and a ReLU activation layer and then go through 3 convolutional blocks, each of which contains one convolutional layer, one instance normalization layer, and one ReLU activation layer, finally appending one convolutional layer as the end of the discriminator.Figure 5
Structures of (a) discriminator and (b) recognizer.
(a)(b)


### 3.3. Recognizer

For the recognition networkℛ, its main task is to be able to identify the attribute information of the original secret image Ise from the image Itr generated by the domain transfer generator Gt, that is,(3)l=ℛItr,where l is the predicted result.Through the combination of several proposed modules, for a secret imageIse, the generator Gt firstly transforms the secret image to the target domain Xtarget and retains features that can characterize its attributes. Then, the recognizer ℛ is able to extract this feature from the generated image and maps it to the attribute label representing the secret image. This process can be expressed as(4)ℛ:Gt:Ise⟶Xtarget⟶l.There are currently some popular network structures that perform very well in image recognition tasks, such as VGG [33] and ResNet [32]. Therefore, without loss of generality, we adopt ResNet structure as backbone for the recognizer in our framework and design different model heads according to the specific recognition task. Specifically, we take ResNet50 [32] as the backbone of our recognizer and replace the fully connected layer at the end of the model. The final number of output logits of the recognizer equals to the number of categories. The specific structure of the network is shown in Figure 5(b). After the input image passes through the backbone of ResNet50, the obtained features are flattened, passed through a fully connected layer, and activated by the “softmax” function. Finally, the predicted label corresponds to the category label with the highest confidence.


### 3.4. Restore Generator

The recognizer designed in this paper can already recognize images, but in some scenarios that require more flexible authority certification or image processing operation, it may be necessary to recover the original secret image. Therefore, in addition to the recognizer for image recognition/authority certification, we also provide an additional restore generatorGt for image restoration. The restore generator receives the image generated by the domain transfer generator Gt as input and outputs an image as identical to the original secret image as possible. Since the task of the restore generator Gr is also to generate images, in order to reduce the complexity of the whole system, we set the structure of the restore generator Gr the same as the domain transfer generator Gt, which is shown in Figure 4.


### 3.5. Objective Loss Function

For a given secret imageIse∈Xnatural and its label and an image in the target domain Ist∈Xtarget, the designed network will be trained using the following adversarial loss.For the discriminatorD, its input is the image Itr generated by the generator or the image Ita in the target domain Xtarget, and its task is to be able to distinguish between these two kinds of images, which is a binary classification problem. In our framework, the images generated by the domain transfer generator are regarded as negative samples, and the images from the target domain are regarded as positive samples. Therefore, the discriminator will generate the following loss:(5)ℒdis=EIstlogDIst+EIselog1−DGIse.For the recognizerℛ, its purpose is to predict the attribute label of the secret image. Therefore, the cross-entropy loss function, most frequently used in image recognition, urges the recognizer to make correct prediction to the target label:(6)ℒcls=−∑i=1KyilogℛItr,where K is the number of image categories, y is the image label, and ℛItr is the predicted probability of the recognition model on the domain transferred image Itr.At the same time, the proposed framework also provides the function of restoring the original secret image. That is to say, the output of the restore generatorGr should be as same as possible to the secret image. To this end, the following loss function controls the similarity between Ise and Ire:(7)ℒres=Ise−Ire2.Finally, we will obtain a total loss as follows:(8)ℒGr,Gt,G,ℛ=ℒdis+ℒcls+ℒres.


## 3.1. Domain Transfer Generator

Given a secret imageIse from the dataset Xnatural to be identified, the domain transfer generator (corresponding to Gt in Figure 3) aims to transfer the secret image into an image in the target domain with a different style such as a cartoon face style and an animal face style. In other words, the transfer generator Gt should extract the feature of the secret image as much as possible and transform this feature into an image that conforms to the distribution of the target domain. Besides, the transferred image must fool the discriminator to make it fail to tell whether this image is generated by Gt or comes from Xtarget. When the whole system is fully trained, the generator Gt can be used as an independent module to perform image transfer from Xnatural domain to Xtarget domain.The domain transfer process can be written as(1)Itr=GtIse.As for the design of our domain transfer generator, given that the convolutional neural network is experimentally proven to have excellent feature learning and extraction abilities for images and that a large amount of autoencoders have shown extraordinary performance in image generation, we adopt the combination of convolutional neural network and autoencoder and design the generatorGt with a ResNet [32] structure. Specifically, as shown in Figure 4, we first stack 3 sets of convolutional layers, instance normalization layers, and ReLU activation, followed by 9 residual blocks, with each residual block containing two convolutional layers, two instance normalization layers, and one ReLU activation. The input of each residual block and the output of the latter instance normalization will be added as the output of each residual block. Finally, 2 sets of deconvolution layers, ReLU activation layers, and instance normalization layers are followed to make the size of the final generated image consistent with that of the original image. By the way, the last activation layer will be tanh to cover the full pixel range.Figure 4
Structure of domain transfer generatorGt and restore generator Gr.


## 3.2. Discriminator

For deep models based on generative adversarial networks, the discriminatorD is indispensable in ensuring the quality of the generated images. The discriminator is expected to learn features that can distinguish the image Ita in the target domain from the generated image Itr in the transferred domain. When the input image is from the target domain Xtarget, the discriminator should identify it as a “real” image, and when the input image is generated by the generator Gt, the discriminator should identify it as a “fake” image, which can be mathematically expressed as(2)t=DIin,where t represents the output of discriminator D, t∈ {“real”: image from target domain, “fake”: image generated by Gt}, and Iin represents the input image of D.As an auxiliary module of the generatorGt, for each input image, the discriminator D needs to judge whether it is “true” or “false.” But, unlike generators which have to perform complex image generation tasks, the discriminator only makes a binary decision. Therefore, our framework only contains one discriminator, a very “shallow” network. The specific structure is shown in Figure 5(a). First, we stack a convolutional layer and a ReLU activation layer and then go through 3 convolutional blocks, each of which contains one convolutional layer, one instance normalization layer, and one ReLU activation layer, finally appending one convolutional layer as the end of the discriminator.Figure 5
Structures of (a) discriminator and (b) recognizer.
(a)(b)


## 3.3. Recognizer

For the recognition networkℛ, its main task is to be able to identify the attribute information of the original secret image Ise from the image Itr generated by the domain transfer generator Gt, that is,(3)l=ℛItr,where l is the predicted result.Through the combination of several proposed modules, for a secret imageIse, the generator Gt firstly transforms the secret image to the target domain Xtarget and retains features that can characterize its attributes. Then, the recognizer ℛ is able to extract this feature from the generated image and maps it to the attribute label representing the secret image. This process can be expressed as(4)ℛ:Gt:Ise⟶Xtarget⟶l.There are currently some popular network structures that perform very well in image recognition tasks, such as VGG [33] and ResNet [32]. Therefore, without loss of generality, we adopt ResNet structure as backbone for the recognizer in our framework and design different model heads according to the specific recognition task. Specifically, we take ResNet50 [32] as the backbone of our recognizer and replace the fully connected layer at the end of the model. The final number of output logits of the recognizer equals to the number of categories. The specific structure of the network is shown in Figure 5(b). After the input image passes through the backbone of ResNet50, the obtained features are flattened, passed through a fully connected layer, and activated by the “softmax” function. Finally, the predicted label corresponds to the category label with the highest confidence.


## 3.4. Restore Generator

The recognizer designed in this paper can already recognize images, but in some scenarios that require more flexible authority certification or image processing operation, it may be necessary to recover the original secret image. Therefore, in addition to the recognizer for image recognition/authority certification, we also provide an additional restore generatorGt for image restoration. The restore generator receives the image generated by the domain transfer generator Gt as input and outputs an image as identical to the original secret image as possible. Since the task of the restore generator Gr is also to generate images, in order to reduce the complexity of the whole system, we set the structure of the restore generator Gr the same as the domain transfer generator Gt, which is shown in Figure 4.


## 3.5. Objective Loss Function

For a given secret imageIse∈Xnatural and its label and an image in the target domain Ist∈Xtarget, the designed network will be trained using the following adversarial loss.For the discriminatorD, its input is the image Itr generated by the generator or the image Ita in the target domain Xtarget, and its task is to be able to distinguish between these two kinds of images, which is a binary classification problem. In our framework, the images generated by the domain transfer generator are regarded as negative samples, and the images from the target domain are regarded as positive samples. Therefore, the discriminator will generate the following loss:(5)ℒdis=EIstlogDIst+EIselog1−DGIse.For the recognizerℛ, its purpose is to predict the attribute label of the secret image. Therefore, the cross-entropy loss function, most frequently used in image recognition, urges the recognizer to make correct prediction to the target label:(6)ℒcls=−∑i=1KyilogℛItr,where K is the number of image categories, y is the image label, and ℛItr is the predicted probability of the recognition model on the domain transferred image Itr.At the same time, the proposed framework also provides the function of restoring the original secret image. That is to say, the output of the restore generatorGr should be as same as possible to the secret image. To this end, the following loss function controls the similarity between Ise and Ire:(7)ℒres=Ise−Ire2.Finally, we will obtain a total loss as follows:(8)ℒGr,Gt,G,ℛ=ℒdis+ℒcls+ℒres.


## 4. Experiments

In this section, we will first introduce our datasets and evaluation metrics and experimental details. Then, we will demonstrate the effectiveness of our method on varied datasets.


### 4.1. Dataset

Since the proposed method is to transform images in one dataset/domain to another dataset/domain, for a complete domain transfer, two datasets are required, namely, the secret image dataset/domainXnatural and the target dataset/domain Xtarget. We select face images indicating strong privacy as our source dataset/domain, and we adopt CelebA [34] and Pubfig [35] dataset in the experiment.CelebA [34] is a large-scale face image database containing 202,599 images with 40 categories collected from the Internet by the Chinese University of Hong Kong. Each of these images has 40 binary attributes, such as gender, attractive or not, and young or not. Without loss of generality, we use the attribute “gender” as our prediction attribute, and all the images will be resized to resolution of 256×256. The first 10K images and the subsequent 2K images are respectively used as the training dataset and the test dataset in the experiment.Pubfig [35] is a face image dataset with 58,797 images of 200 categories collected from the Internet. Each category has an average of 300 face images of one person. However, due to the copyright and privacy issue, most of the image links provided by the paper [35] are invalid now. As an alternative, we use the version published by other user on the Kaggle platform [36] including only 11,640 images with 150 categories. Specifically, we randomly choose 80% images as training dataset and the remain 20% images as testing dataset.For the target dataset/domainXtarget, we mainly use Bitmoji-style cartoon face images. The Bitmoji [37] dataset is a cartoon style face downloaded directly from the mobile app. The Bitmoji [37] dataset contains 4085 faces with the resolution of 384×384. In the experiments, all the Bitmoji images are resized to the same size as the source domain Xnatural, with the resolution of 256×256. Figure 6 illustrates some examples of the Bitmoji dataset.Figure 6
Examples of Bitmoji images. The Bitmoji images contain cartoon faces of different genders, different hair colors, etc., all of which are centered and surrounded by white space.


### 4.2. Evaluation

Since our framework is to perform image recognition in the transferred domain, the accuracy of image recognition is one of our goals. At the same time, we also provide a restore generatorGr to recover the original secret image. Therefore, the PSNR and SSIM metrics, which are most commonly used in digital image processing, are used to measure the quality of the recovered image. Given a reference image I and a test image K, both with size m×n, the PSNR between I and K is defined as(9)PSNR=10⋅log10L2MSE,where L is the dynamic range of allowable image pixel intensities (usually takes 255). For our 3-channel color image, we first calculate the MSE value of each channel and then calculate the average to get the MSE in equation (9).(10)MSE=1mn∑i=0m−1∑j=0n−1Ii,j−Ki,j2.The SSIM is given by(11)SSIM=2μIμK+c12σΙΚ+c2μΙ2+μΚ2+c1σΙ2+σΚ2+c1,where μI and μK are the local means, σΙ and σΚ are the standard deviations and σΙΚ is the cross-covariance for images I and K sequentially, and c1 and c2 are 6.50 and 58.5, respectively, by default.


### 4.3. Implementation Details

The implementation is based on Keras with TensorFlow as the backend. In our experiments, we use Adam [38] optimizer with a learning rate of 0.001 and linearly decay it to 0 after 50 training epochs. Our training batch size is set 4 and it takes our 4 days for training about 200 epochs on one single NVIDIA RTX 1080 Ti GPU.


### 4.4. Experimental Results

We first take the CelebA dataset as the domainXnatural, the Bitmoji dataset as the domain Xtarget, and the “gender” attribute in the CelebA dataset as the recognized attribute. After sufficient training, the obtained domain transferred images and the restored images are shown in Figure 7, where images in the first row belong to the CelebA dataset, images in the second row are transferred by the domain transfer generator Gt, and images in the last row are the recovered images. Visually, the transferred face image is similar to the cartoon face image in the Bitmoji dataset, which are all centered, frontal, and surrounded by white space. It is difficult to distinguish between these two kinds of images visually.Figure 7
Visualization of domain transferred CelebA images and the corresponding restored images. From top to last row are original secret imagesIse, domain transferred images Itr, and recovered secret images Ire, respectively.In order to verify the generalization of our method, we also use Pubfig dataset and the Bitmoji dataset asXnatural and Xtarget respectively for training, where the identity of Pubfig images is used as the prediction label. The results are shown in Figure 8.Figure 8
Visualization of domain transferred Pubfig images and the corresponding restored images. From top to last row are original secret imagesIse, domain transferred images Itr, and recovered secret images Ire, respectively.


#### 4.4.1. Comparison of Other Methods

Table1 shows the comparison of our method with other related works from various aspects. Tao et al. [39] and Baluja [25] focus on the secure secret message communication but failed in image recognition. The domain transfer methods [40–42] are free from carrier selection but none of them take secure image recognition into consideration. Our method can not only support secret image recovery and direct image recognition but also is free from carrier selection. Besides, the difference between our secret images and transferred images is enough (middle degree) to prevent the adversary from inferring the image content.Table 1
Comparison of relevant studies from different aspects. Difference means the difference between the secret image and the stego/generated/transferred image. STC means syndrome trellis coding; CNN and GAN mean convolutional neural network and generative adversarial networks, respectively.
Ref.Carrier requiredDifferenceTechnology adoptedSupport secret images recoverySupport direct image recognitionTao et al. [39]YesLargeSTCYesNoBaluja [25]YesLargeCNNYesNoKim et al. [40]NoMiddleGANYesNoChen et al. [41]NoMiddleGANYesNoLiu et al. [42]NoSmallGANNoNoOursNoMiddleGANYesYes


#### 4.4.2. Visualization of Domain Transfer

Since the proposed model needs to transfer the image from one domain to another domain for “camouflage,” the features of images obtained by domain transfer should be as close to the target domainXtarget as possible. In the field of machine learning, there are some classic feature compression/dimension reduction methods, such as PCA [43], t-SNE [44], and LLE [45]. In order to explicitly portray the domain transfer process of our method, we visualize the feature distribution of the dataset Xnatural before and after domain transfer and the feature distribution of target dataset Xtarget using t-SNE. Specifically, we first flatten all the CelebA and Bitmoji images from 256×256 to 1×256∗256∗3 and then directly apply the t-SNE function in the Sklearn [46] library to compress them into 1×2 and characterize these images on a 2D plane. The visualization of the dataset is shown in Figure 9. From Figure 9(a), we can see that the boundary between CelebA and Bitmoji is very obvious. But after transferring images in CelebA, the distribution of transferred CelebA and Bitmoji datasets is very similar, shown by the overlapping between the red dots and blue dots in Figure 9(b).Figure 9
Visualization of the dataset after reducing the images to 2 dimensions using the t-SNE [44] algorithm. (a) The distribution of the CelebA dataset and Bitmoji dataset. The red dot means images in the CelebA dataset, and the blue dot means images in the Bitmoji dataset. (b) The distribution of the domain transferred CelebA dataset and Bitmoji dataset.


#### 4.4.3. Recognition of Domain Transferred Image

In our proposed framework, after the original secret face image is transformed by the domain transfer generatorGt, the corresponding recognizer ℛ should be able to directly identify the original secret image according to the transferred image. To examine the performance of the trained recognizer ℛ, we also train a recognition network directly on the original natural face as the best recognizer for comparison. Experiments are shown in Table 2. From the table, we can see that even if the image is transferred into other domains, our image recognition accuracy hardly decreases compared to the highest untransferred image recognition accuracy. For CelebA images, the recognition accuracy drops from 92.4%⟶92.1% and Pubfig images drop from 89.7%⟶88.4%.Table 2
Recognition accuracy comparison. The ACC in the “CelebA” column and “Pubfig” column means the top accuracy obtained by recognizers trained by natural, unaltered face images. The ACC in the “CelebA-T” column and “Pubfig-T” column means recognition accuracy obtained on transferred images.
DatasetCelebACelebA-TPubfigPubfig-TACC (%)92.492.189.788.4


#### 4.4.4. Recognition of Reconstructed Images

As mentioned before, in order to use our proposed method more flexibly in some scenarios where the original secret image needs to be recovered, we also include a restore generatorGr for recovering the original secret image, and the usage of the restored image should not be affected. In order to test the effect of domain transfer on secret images, we test the image recognition performance of restored images. Specifically, we train several recognizers on natural face images with different kinds of attributes and use these trained recognizers to test the image recognition accuracy on restored images. Without loss of generality, we select four attributes of “Male,” “Bald,” “Heavy_Makeup,” and “Attractive” on the CelebA dataset to test the recognition performance of reconstructed images. The experimental results are shown in Table 3.Table 3
The recognition accuracy of CelebA and reconstructed CelebA images. ACC(Base) means the best accuracy obtained by the model trained on natural images. ACC(Res) means the recognition accuracy obtained on the restored images.
AttributesMaleBaldHeavy_MakeupAttractiveAverageACC(Base)92.498.290.080.290.2ACC(Res)92.298.287.779.789.5As can be seen from Table3, when the network is trained directly with the original CelebA dataset, the recognition accuracies of “Male,” “Bald,” “Heavy_Makeup,” and “Attractive” attributes are 92.4%, 98.2%, 90.0%, and 80.2%, respectively. Even if the original image has gone through the domain transfer, the recognition accuracy of the restored image is hardly affected, with the accuracy of “Male,” “Bald,” “Heavy_Makeup,” and “Attractive” attributes being 92.2%, 98.2%, 87.7.0%, and 79.7%, respectively. Compared with the highest recognition accuracy of plaintext/baseline, the average value of the recognition accuracy has only dropped by less than one point (90.2% ⟶ 89.5%), indicating that the image reconstructed in our method can still be used with little performance penalty.


#### 4.4.5. Visual Quality of Reconstructed Images

From the perspective of secret information transfer, the process of recovering the original secret image in our method can also be used for secret communication. The method of hiding images within images mentioned in [25] is an advanced method based on neural networks for large capacity information hiding, where the author pointed out that if the natural image is directly fed into the proposed neural network, the hidden image will be exposed in the residual image. To eliminate the traces of hidden image content in residual images between cover images and stego images, a simple way is to permute the pixels of hidden images before they are passed to the preparation network. Following [25], we retrained our whole network to hide images without the spatial coherence of natural images. As shown in Table 4, although the average values of PSNR and SSIM of the restored images are slightly inferior to the shuffled version, the recognition of restored images will be little affected. One thing that needs to noted is that we are only sacrificing a little bit of image quality to enable image recognition while preserving privacy. Besides, our method is free from cover image selection hence decreasing the complexity of the system during the usage/inference phase.Table 4
Quantitative visual quality of recovered images.
MethodBaluja [25]Baluja (shuffled) [25]OursPSNR34.231.626.4SSIM0.9620.9430.948


## 4.1. Dataset

Since the proposed method is to transform images in one dataset/domain to another dataset/domain, for a complete domain transfer, two datasets are required, namely, the secret image dataset/domainXnatural and the target dataset/domain Xtarget. We select face images indicating strong privacy as our source dataset/domain, and we adopt CelebA [34] and Pubfig [35] dataset in the experiment.CelebA [34] is a large-scale face image database containing 202,599 images with 40 categories collected from the Internet by the Chinese University of Hong Kong. Each of these images has 40 binary attributes, such as gender, attractive or not, and young or not. Without loss of generality, we use the attribute “gender” as our prediction attribute, and all the images will be resized to resolution of 256×256. The first 10K images and the subsequent 2K images are respectively used as the training dataset and the test dataset in the experiment.Pubfig [35] is a face image dataset with 58,797 images of 200 categories collected from the Internet. Each category has an average of 300 face images of one person. However, due to the copyright and privacy issue, most of the image links provided by the paper [35] are invalid now. As an alternative, we use the version published by other user on the Kaggle platform [36] including only 11,640 images with 150 categories. Specifically, we randomly choose 80% images as training dataset and the remain 20% images as testing dataset.For the target dataset/domainXtarget, we mainly use Bitmoji-style cartoon face images. The Bitmoji [37] dataset is a cartoon style face downloaded directly from the mobile app. The Bitmoji [37] dataset contains 4085 faces with the resolution of 384×384. In the experiments, all the Bitmoji images are resized to the same size as the source domain Xnatural, with the resolution of 256×256. Figure 6 illustrates some examples of the Bitmoji dataset.Figure 6
Examples of Bitmoji images. The Bitmoji images contain cartoon faces of different genders, different hair colors, etc., all of which are centered and surrounded by white space.


## 4.2. Evaluation

Since our framework is to perform image recognition in the transferred domain, the accuracy of image recognition is one of our goals. At the same time, we also provide a restore generatorGr to recover the original secret image. Therefore, the PSNR and SSIM metrics, which are most commonly used in digital image processing, are used to measure the quality of the recovered image. Given a reference image I and a test image K, both with size m×n, the PSNR between I and K is defined as(9)PSNR=10⋅log10L2MSE,where L is the dynamic range of allowable image pixel intensities (usually takes 255). For our 3-channel color image, we first calculate the MSE value of each channel and then calculate the average to get the MSE in equation (9).(10)MSE=1mn∑i=0m−1∑j=0n−1Ii,j−Ki,j2.The SSIM is given by(11)SSIM=2μIμK+c12σΙΚ+c2μΙ2+μΚ2+c1σΙ2+σΚ2+c1,where μI and μK are the local means, σΙ and σΚ are the standard deviations and σΙΚ is the cross-covariance for images I and K sequentially, and c1 and c2 are 6.50 and 58.5, respectively, by default.


## 4.3. Implementation Details

The implementation is based on Keras with TensorFlow as the backend. In our experiments, we use Adam [38] optimizer with a learning rate of 0.001 and linearly decay it to 0 after 50 training epochs. Our training batch size is set 4 and it takes our 4 days for training about 200 epochs on one single NVIDIA RTX 1080 Ti GPU.


## 4.4. Experimental Results

We first take the CelebA dataset as the domainXnatural, the Bitmoji dataset as the domain Xtarget, and the “gender” attribute in the CelebA dataset as the recognized attribute. After sufficient training, the obtained domain transferred images and the restored images are shown in Figure 7, where images in the first row belong to the CelebA dataset, images in the second row are transferred by the domain transfer generator Gt, and images in the last row are the recovered images. Visually, the transferred face image is similar to the cartoon face image in the Bitmoji dataset, which are all centered, frontal, and surrounded by white space. It is difficult to distinguish between these two kinds of images visually.Figure 7
Visualization of domain transferred CelebA images and the corresponding restored images. From top to last row are original secret imagesIse, domain transferred images Itr, and recovered secret images Ire, respectively.In order to verify the generalization of our method, we also use Pubfig dataset and the Bitmoji dataset asXnatural and Xtarget respectively for training, where the identity of Pubfig images is used as the prediction label. The results are shown in Figure 8.Figure 8
Visualization of domain transferred Pubfig images and the corresponding restored images. From top to last row are original secret imagesIse, domain transferred images Itr, and recovered secret images Ire, respectively.


### 4.4.1. Comparison of Other Methods

Table1 shows the comparison of our method with other related works from various aspects. Tao et al. [39] and Baluja [25] focus on the secure secret message communication but failed in image recognition. The domain transfer methods [40–42] are free from carrier selection but none of them take secure image recognition into consideration. Our method can not only support secret image recovery and direct image recognition but also is free from carrier selection. Besides, the difference between our secret images and transferred images is enough (middle degree) to prevent the adversary from inferring the image content.Table 1
Comparison of relevant studies from different aspects. Difference means the difference between the secret image and the stego/generated/transferred image. STC means syndrome trellis coding; CNN and GAN mean convolutional neural network and generative adversarial networks, respectively.
Ref.Carrier requiredDifferenceTechnology adoptedSupport secret images recoverySupport direct image recognitionTao et al. [39]YesLargeSTCYesNoBaluja [25]YesLargeCNNYesNoKim et al. [40]NoMiddleGANYesNoChen et al. [41]NoMiddleGANYesNoLiu et al. [42]NoSmallGANNoNoOursNoMiddleGANYesYes


### 4.4.2. Visualization of Domain Transfer

Since the proposed model needs to transfer the image from one domain to another domain for “camouflage,” the features of images obtained by domain transfer should be as close to the target domainXtarget as possible. In the field of machine learning, there are some classic feature compression/dimension reduction methods, such as PCA [43], t-SNE [44], and LLE [45]. In order to explicitly portray the domain transfer process of our method, we visualize the feature distribution of the dataset Xnatural before and after domain transfer and the feature distribution of target dataset Xtarget using t-SNE. Specifically, we first flatten all the CelebA and Bitmoji images from 256×256 to 1×256∗256∗3 and then directly apply the t-SNE function in the Sklearn [46] library to compress them into 1×2 and characterize these images on a 2D plane. The visualization of the dataset is shown in Figure 9. From Figure 9(a), we can see that the boundary between CelebA and Bitmoji is very obvious. But after transferring images in CelebA, the distribution of transferred CelebA and Bitmoji datasets is very similar, shown by the overlapping between the red dots and blue dots in Figure 9(b).Figure 9
Visualization of the dataset after reducing the images to 2 dimensions using the t-SNE [44] algorithm. (a) The distribution of the CelebA dataset and Bitmoji dataset. The red dot means images in the CelebA dataset, and the blue dot means images in the Bitmoji dataset. (b) The distribution of the domain transferred CelebA dataset and Bitmoji dataset.


### 4.4.3. Recognition of Domain Transferred Image

In our proposed framework, after the original secret face image is transformed by the domain transfer generatorGt, the corresponding recognizer ℛ should be able to directly identify the original secret image according to the transferred image. To examine the performance of the trained recognizer ℛ, we also train a recognition network directly on the original natural face as the best recognizer for comparison. Experiments are shown in Table 2. From the table, we can see that even if the image is transferred into other domains, our image recognition accuracy hardly decreases compared to the highest untransferred image recognition accuracy. For CelebA images, the recognition accuracy drops from 92.4%⟶92.1% and Pubfig images drop from 89.7%⟶88.4%.Table 2
Recognition accuracy comparison. The ACC in the “CelebA” column and “Pubfig” column means the top accuracy obtained by recognizers trained by natural, unaltered face images. The ACC in the “CelebA-T” column and “Pubfig-T” column means recognition accuracy obtained on transferred images.
DatasetCelebACelebA-TPubfigPubfig-TACC (%)92.492.189.788.4


### 4.4.4. Recognition of Reconstructed Images

As mentioned before, in order to use our proposed method more flexibly in some scenarios where the original secret image needs to be recovered, we also include a restore generatorGr for recovering the original secret image, and the usage of the restored image should not be affected. In order to test the effect of domain transfer on secret images, we test the image recognition performance of restored images. Specifically, we train several recognizers on natural face images with different kinds of attributes and use these trained recognizers to test the image recognition accuracy on restored images. Without loss of generality, we select four attributes of “Male,” “Bald,” “Heavy_Makeup,” and “Attractive” on the CelebA dataset to test the recognition performance of reconstructed images. The experimental results are shown in Table 3.Table 3
The recognition accuracy of CelebA and reconstructed CelebA images. ACC(Base) means the best accuracy obtained by the model trained on natural images. ACC(Res) means the recognition accuracy obtained on the restored images.
AttributesMaleBaldHeavy_MakeupAttractiveAverageACC(Base)92.498.290.080.290.2ACC(Res)92.298.287.779.789.5As can be seen from Table3, when the network is trained directly with the original CelebA dataset, the recognition accuracies of “Male,” “Bald,” “Heavy_Makeup,” and “Attractive” attributes are 92.4%, 98.2%, 90.0%, and 80.2%, respectively. Even if the original image has gone through the domain transfer, the recognition accuracy of the restored image is hardly affected, with the accuracy of “Male,” “Bald,” “Heavy_Makeup,” and “Attractive” attributes being 92.2%, 98.2%, 87.7.0%, and 79.7%, respectively. Compared with the highest recognition accuracy of plaintext/baseline, the average value of the recognition accuracy has only dropped by less than one point (90.2% ⟶ 89.5%), indicating that the image reconstructed in our method can still be used with little performance penalty.


### 4.4.5. Visual Quality of Reconstructed Images

From the perspective of secret information transfer, the process of recovering the original secret image in our method can also be used for secret communication. The method of hiding images within images mentioned in [25] is an advanced method based on neural networks for large capacity information hiding, where the author pointed out that if the natural image is directly fed into the proposed neural network, the hidden image will be exposed in the residual image. To eliminate the traces of hidden image content in residual images between cover images and stego images, a simple way is to permute the pixels of hidden images before they are passed to the preparation network. Following [25], we retrained our whole network to hide images without the spatial coherence of natural images. As shown in Table 4, although the average values of PSNR and SSIM of the restored images are slightly inferior to the shuffled version, the recognition of restored images will be little affected. One thing that needs to noted is that we are only sacrificing a little bit of image quality to enable image recognition while preserving privacy. Besides, our method is free from cover image selection hence decreasing the complexity of the system during the usage/inference phase.Table 4
Quantitative visual quality of recovered images.
MethodBaluja [25]Baluja (shuffled) [25]OursPSNR34.231.626.4SSIM0.9620.9430.948


## 4.4.1. Comparison of Other Methods

Table1 shows the comparison of our method with other related works from various aspects. Tao et al. [39] and Baluja [25] focus on the secure secret message communication but failed in image recognition. The domain transfer methods [40–42] are free from carrier selection but none of them take secure image recognition into consideration. Our method can not only support secret image recovery and direct image recognition but also is free from carrier selection. Besides, the difference between our secret images and transferred images is enough (middle degree) to prevent the adversary from inferring the image content.Table 1
Comparison of relevant studies from different aspects. Difference means the difference between the secret image and the stego/generated/transferred image. STC means syndrome trellis coding; CNN and GAN mean convolutional neural network and generative adversarial networks, respectively.
Ref.Carrier requiredDifferenceTechnology adoptedSupport secret images recoverySupport direct image recognitionTao et al. [39]YesLargeSTCYesNoBaluja [25]YesLargeCNNYesNoKim et al. [40]NoMiddleGANYesNoChen et al. [41]NoMiddleGANYesNoLiu et al. [42]NoSmallGANNoNoOursNoMiddleGANYesYes


## 4.4.2. Visualization of Domain Transfer

Since the proposed model needs to transfer the image from one domain to another domain for “camouflage,” the features of images obtained by domain transfer should be as close to the target domainXtarget as possible. In the field of machine learning, there are some classic feature compression/dimension reduction methods, such as PCA [43], t-SNE [44], and LLE [45]. In order to explicitly portray the domain transfer process of our method, we visualize the feature distribution of the dataset Xnatural before and after domain transfer and the feature distribution of target dataset Xtarget using t-SNE. Specifically, we first flatten all the CelebA and Bitmoji images from 256×256 to 1×256∗256∗3 and then directly apply the t-SNE function in the Sklearn [46] library to compress them into 1×2 and characterize these images on a 2D plane. The visualization of the dataset is shown in Figure 9. From Figure 9(a), we can see that the boundary between CelebA and Bitmoji is very obvious. But after transferring images in CelebA, the distribution of transferred CelebA and Bitmoji datasets is very similar, shown by the overlapping between the red dots and blue dots in Figure 9(b).Figure 9
Visualization of the dataset after reducing the images to 2 dimensions using the t-SNE [44] algorithm. (a) The distribution of the CelebA dataset and Bitmoji dataset. The red dot means images in the CelebA dataset, and the blue dot means images in the Bitmoji dataset. (b) The distribution of the domain transferred CelebA dataset and Bitmoji dataset.


## 4.4.3. Recognition of Domain Transferred Image

In our proposed framework, after the original secret face image is transformed by the domain transfer generatorGt, the corresponding recognizer ℛ should be able to directly identify the original secret image according to the transferred image. To examine the performance of the trained recognizer ℛ, we also train a recognition network directly on the original natural face as the best recognizer for comparison. Experiments are shown in Table 2. From the table, we can see that even if the image is transferred into other domains, our image recognition accuracy hardly decreases compared to the highest untransferred image recognition accuracy. For CelebA images, the recognition accuracy drops from 92.4%⟶92.1% and Pubfig images drop from 89.7%⟶88.4%.Table 2
Recognition accuracy comparison. The ACC in the “CelebA” column and “Pubfig” column means the top accuracy obtained by recognizers trained by natural, unaltered face images. The ACC in the “CelebA-T” column and “Pubfig-T” column means recognition accuracy obtained on transferred images.
DatasetCelebACelebA-TPubfigPubfig-TACC (%)92.492.189.788.4


## 4.4.4. Recognition of Reconstructed Images

As mentioned before, in order to use our proposed method more flexibly in some scenarios where the original secret image needs to be recovered, we also include a restore generatorGr for recovering the original secret image, and the usage of the restored image should not be affected. In order to test the effect of domain transfer on secret images, we test the image recognition performance of restored images. Specifically, we train several recognizers on natural face images with different kinds of attributes and use these trained recognizers to test the image recognition accuracy on restored images. Without loss of generality, we select four attributes of “Male,” “Bald,” “Heavy_Makeup,” and “Attractive” on the CelebA dataset to test the recognition performance of reconstructed images. The experimental results are shown in Table 3.Table 3
The recognition accuracy of CelebA and reconstructed CelebA images. ACC(Base) means the best accuracy obtained by the model trained on natural images. ACC(Res) means the recognition accuracy obtained on the restored images.
AttributesMaleBaldHeavy_MakeupAttractiveAverageACC(Base)92.498.290.080.290.2ACC(Res)92.298.287.779.789.5As can be seen from Table3, when the network is trained directly with the original CelebA dataset, the recognition accuracies of “Male,” “Bald,” “Heavy_Makeup,” and “Attractive” attributes are 92.4%, 98.2%, 90.0%, and 80.2%, respectively. Even if the original image has gone through the domain transfer, the recognition accuracy of the restored image is hardly affected, with the accuracy of “Male,” “Bald,” “Heavy_Makeup,” and “Attractive” attributes being 92.2%, 98.2%, 87.7.0%, and 79.7%, respectively. Compared with the highest recognition accuracy of plaintext/baseline, the average value of the recognition accuracy has only dropped by less than one point (90.2% ⟶ 89.5%), indicating that the image reconstructed in our method can still be used with little performance penalty.


## 4.4.5. Visual Quality of Reconstructed Images

From the perspective of secret information transfer, the process of recovering the original secret image in our method can also be used for secret communication. The method of hiding images within images mentioned in [25] is an advanced method based on neural networks for large capacity information hiding, where the author pointed out that if the natural image is directly fed into the proposed neural network, the hidden image will be exposed in the residual image. To eliminate the traces of hidden image content in residual images between cover images and stego images, a simple way is to permute the pixels of hidden images before they are passed to the preparation network. Following [25], we retrained our whole network to hide images without the spatial coherence of natural images. As shown in Table 4, although the average values of PSNR and SSIM of the restored images are slightly inferior to the shuffled version, the recognition of restored images will be little affected. One thing that needs to noted is that we are only sacrificing a little bit of image quality to enable image recognition while preserving privacy. Besides, our method is free from cover image selection hence decreasing the complexity of the system during the usage/inference phase.Table 4
Quantitative visual quality of recovered images.
MethodBaluja [25]Baluja (shuffled) [25]OursPSNR34.231.626.4SSIM0.9620.9430.948


## 5. Conclusion

In this paper, we proposed a technique for image recognition while protecting the privacy of image content. First, we point out that our method is free from not only complex computation such as encryption algorithms but also carrier selection like information hiding-based method. Second, we designed and trained a combined network containing two generators, one recognizer, and one discriminator, where these two generators are responsible for domain transfer and image reconstruction, respectively, and the recognizer for image recognition on domain transferred images. Experiments are conducted on several standard datasets and the results have validated the effectiveness of our proposed method.



---

*Source: 4466426-2022-04-26.xml*