# Zero-Watermarking Algorithm for Medical Image Based on VGG19 Deep Convolution Neural Network

Authors: Baoru Han; Jinglong Du; Yuanyuan Jia; Huazheng Zhu
Journal: Journal of Healthcare Engineering
 (2021)

---

## Abstract

Aiming at the security issues in the storage and transmission of medical images in the medical information system, combined with the special requirements of medical images for the protection of lesion areas, this paper proposes a robust zero-watermarking algorithm for medical images’ security based on VGG19. First, the pretrained VGG19 is used to extract deep feature maps of medical images, which are fused into the feature image. Second, the feature image is transformed by Fourier transform, and low-frequency coefficients of the Fourier transform are selected to construct the feature matrix of the medical image. Then, based on the low-frequency part of the feature matrix of the medical image, the mean-perceptual hashing algorithm is used to achieve a set of 64-bit binary perceptual hashing values, which can effectively resist local nonlinear geometric attacks. Finally, the algorithm adopts a watermarking image after scrambling and the 64-bit binary perceptual hashing value to obtain robust zero-watermarking. At the same time, the proposed algorithm utilizes Hermite chaotic neural network to scramble the watermarking image for secondary protection, which enhances the security of the algorithm. Compared with the existing related works, the proposed algorithm is simple to implement and can effectively resist local nonlinear geometric attacks, with good robustness, security, and invisibility.


---

## Body

## 1. Introduction

With the construction of hospital normalization, digitization is widely applied in the medical field [1]. Modern medical equipment produces lots of digital medical images every day [2, 3]. Owing to the rapid development of information technology, intelligent medicine and remote diagnosis are becoming more and more mature [4–6]. A large number of medical images are often transmitted through the Internet [7]. Unauthorized persons can easily obtain, store, use, and tamper with medical images on the network [8]. In this scene, sensitive information, such as patient information recorded in medical images, is also easy to leak. With the development of the medical information system, medical image sharing and remote diagnosis technology are becoming more mature [9]. In the application of these techniques, it becomes more and more urgent to protect patient’s information, such as personal information in magnetic resonance imaging (MRI) and other medical images, as well as electronic medical record data from being leaked [10, 11]. Therefore, the problem of medical information security has become an urgent problem.Medical image watermarking can effectively solve the above problem and provide safe and reliable protection for medical information. Current information security of the medical information system relies on the development of modern information technology [12]. Traditional encryption methods of cryptography have great limitations and high risks. It is unable to identify the copyright of data at the technical level, and it has been unable to adapt to the changing information environment. Digital watermarking is a kind of information security technology, which can be applied to image integrity authentication and image copyright protection [13, 14]. Originally, digital watermarking was used for copyright protection of digital multimedia on the Internet. With the increasing demand for information security in the medical field, the invisibility and robustness of digital watermarking are used to hide patient’s personal information in the medical image to secure transmission on the Internet. By means of digital watermarking, specific identification information is embedded into carrier images, so that the authenticity and integrity authentication of medical images, Electronic Patient Record (EPR) hiding, and copyright protection can be realized. Therefore, medical image watermarking has an extremely important application value, which attracts continuous attention from researchers [15, 16].Medical image watermarking generally chooses different watermarking algorithms according to invisibility, robustness, and embedding capacity. The invisibility, robustness, and capacity of watermarking restrict each other, but most of the current medical image watermarking algorithms only focus on one of them. Medical images are mostly single channel gray images, and the details contained in the pixels are very important. Any changes may cause distortion, which will affect doctor’s diagnosis. However, the application of traditional image watermarking to medical images can easily cause the distortion of medical images, which may lead to potential misdiagnosis. In order to reduce the influence of watermarking on the original information of the medical image, it is very important to design a medical image watermarking algorithm which is lossless to the original image. Since the robust zero-watermarking adopts the way of zero-embedding, the original medical image is not modified. While realizing protection, the robust zero-watermarking takes into account the robustness and capacity of the watermarking and will not destroy the integrity of the medical image. Therefore, it is very suitable for the medical image [17–19].Robust medical image zero-watermarking mainly focuses on the ability of the algorithm to resist various attacks, that is, the algorithm can completely extract the zero-watermarking information after various attacks [20–22]. The geometric attack will change the position relationship between the attacked medical image and the original medical image, resulting in serious out of synchronization between zero-watermarking information and medical images, which makes it is extremely difficult to extract zero-watermarking. Therefore, the ability of the algorithm to resist geometric attacks has become a research hotspot in the field of medical image zero-watermarking [1, 12]. But these geometric attacks mainly refer to global geometric transformation (rotation, scaling, translation, etc.), which is a kind of linear transformation. However, in practical applications, there is usually another type of geometric attack, which belongs to the local nonlinear geometric attack. At present, there are a few watermarking algorithms that can resist local nonlinear geometric attacks, which make almost all watermarking algorithms invalid. Compared with the global geometric attack, the local nonlinear geometric attack is more difficult to implement.To solve this problem, a zero-watermarking algorithm based on VGG19 against local nonlinear geometric attacks is proposed in this paper. In this algorithm, VGG19 can extract complex high-level features using a deep structure with powerful representation and learning ability. At the same time, the network structure has good geometric invariance and can extract image features with high and good robustness. The algorithm uses a perceptual hashing algorithm based on deep features to construct zero-watermarking, which has good robustness against local nonlinear geometric attacks. Hermite chaotic neural network is used to scramble the watermarking image for secondary protection, and it can enhance the security of the algorithm. Experimental results show that the proposed algorithm can effectively resist local nonlinear geometric attacks and display the superior performance of the proposed algorithm compared with other watermarking algorithms.


## 2. The Fundamental Theory

### 2.1. Feature Extraction Based on VGGnet

VGGnet is a representative type of deep convolutional neural network (CNN), which is often used in feature extraction and transfer learning [23]. The most widely used VGGnet is VGG19, which consists of 19 hidden layers (16 convolution layers and 3 fully connected layers), as shown in Figure 1. VGG19 uses a series of 3 × 3 convolution kernels to extract image features and expands the number of feature channels through convolution layers. Let Wi and bi represent the weights and bias of the ith convolution layer, the feature can be extracted by(1)Xiout=σWi∗Xiin+bi,where Xiout and Xiin, respectively, denote the input and output feature maps and σ⋅ is the rectified linear unit (ReLU). In each convolution layer, the stride is set to 1. To avoid the explosion of calculation, VGG19 uses max pooling layers to reduce the size of the feature maps.Figure 1
The structure of the VGG19 network.In fully connected layers, each node of the given layer is connected with all nodes of its previous layers, which can map the distributed feature representation to the sample label space by(2)Y=FC3FC2FC1PX16out,where FC⋅ denotes the operation of the fully connected layer and P⋅ refers to the max pooling operation.At the end of VGG19, a softmax layer produces the classification result of the image:(3)Yj=ezj∑c=1Cezc,where Yj is the probability of the jth node and zj and C, respectively, denote the output of jth node and the number of the classification.Compared with other types of CNNs, VGG19 improves the depth of the network and adopts the alternate structure of many convolution layers and nonlinear activation layers, which is beneficial to extract accurate features. In this work, different from image classification tasks, we just use the convolution layers and max pooling layers from the pretrained VGG19 as the preprocessing method to extract deep feature maps from medical images. Unlike other medical zero-watermarking algorithms, our method can extract abstract high-level features from medical images to improve the antigeometric attack ability of zero-watermarking.


### 2.2. Discrete Fourier Transform

Discrete Fourier transform plays an important role in the development of signal analysis and processing. Because of its definite physical meaning, discrete Fourier transform is widely used in many fields of signal analysis and processing.


#### 2.2.1. One-Dimensional Discrete Fourier Transform

Letfx denote a time domain function of x, where x represents a time domain variable and u is a frequency domain variable. When fx reaches the Dirichlet condition, the discrete Fourier transform is formula (4) and the inverse transform is formula (5), where Fu is a frequency domain function:(4)Fu=∑x=0N−1fxe−j2πux/N,u=0,1,…,N−1,(5)fx=1N∑u=0N−1Fuej2πux/N,x=0,1,…,N−1.


#### 2.2.2. Two-Dimensional Discrete Fourier Transform

For a given image with the size ofM×N, when it satisfies the Dirichlet condition, the two-dimensional discrete Fourier positive transform is(6)Fu,v=∑x=0M−1∑y=0N−1fx,ye−j2π/Mxue−j2π/Nyv,u=0,1,…,M−1;v=0,1,…,N−1.Its inverse transformation formula is(7)fx,y=1MN∑u=0M−1∑v=0N−1Fu,vej2π/Mxuej2π/Nyv,x=0,1,…,M−1;y=0,1,…,N−1,where x and y are the values in the space domain and u and v are the values in the frequency domain.Two-dimensional discrete Fourier transform transforms an image from the space domain to the frequency domain, which has a clear physical meaning. The frequency of an image is an index that characterizes the intensity of the gray level change in the image and is the gradient of the gray level in the plane space. After the Fourier transform, the changes in the flat area of the image are represented by low-frequency coefficients, and the details of the image are represented by high-frequency coefficients. Figure2 shows the result of the Fourier transform of a medical image.Figure 2
Discrete Fourier transform of the medical image. (a) Original medical image. (b) Medical images in the Fourier domain.
(a)(b)After the two-dimensional discrete Fourier transform, the transform coefficient matrix of the image shows that if the origin of the transform matrix is set at the center, the spectral energy is concentrated near the center of the short transform coefficient matrix. If the origin of the two-dimensional discrete Fourier transform matrix is set in the upper left corner, the energy of the image will be concentrated in the four corners of the coefficient matrix. This is determined by the nature of the two-dimensional discrete Fourier transform itself. It also shows that the image energy is generally concentrated in the low-frequency area. This determines that the two-dimensional discrete Fourier transform is very suitable for image processing. In this paper, a two-dimensional discrete Fourier transform is used to transform the features of the medical image into the Fourier domain to construct zero-watermarking.


### 2.3. Perceptual Hashing Algorithm

Perceptual hashing is considered as a one-way mapping from the multimedia data set to the perceptual content hashing value; that is, a short digital digest uniquely represents multimedia data with the same perceptual content. This digital digest is called perceptual hashing value, so the mapping process is also called the process of hash value generation.Image perceptual hashing is usually called image digital fingerprint or image digital digest. It can map the image to a group of hash sequences, which greatly reduces the storage of the digital image and brings great convenience to image management and maintenance. It has become a research hotspot in the field of multimedia signal processing and security. Perceptual feature extraction is the core of the perceptual hashing algorithm. The effectiveness and reliability of perceptual feature extraction will directly affect the uniqueness and robustness of the perceptual hashing sequence.A mean-perceptual hashing algorithm is proposed in this paper. The mean-perceptual hashing algorithm uses the mean value of the elements of the medical image feature matrix to generate a hashing sequence. The construction of the feature image is realized by VGG19 and image fusion. Figure3 describes the flowchart of the mean-perceptual hashing algorithm.Figure 3
The flowchart of mean-perceptual hashing.


### 2.4. Hermite Chaotic Neural Network

In this work, a new Hermite chaotic neural network is used to scramble the watermarking image [24]. The structure of the network is shown in Figure 4, whose topology is m×n×1.Figure 4
The architecture of Hermite chaotic neural network.The hidden layer neuron input is(8)Oj=wjx,j=0,1,2,…,n−1,where wj is the weight between the input layer and hidden layer and cj is the weight between the hidden layer and output layer. The hidden layer neuron output is(9)HjOj,j=0,1,2,…,n−1,where Hj,j=0,1,2,…,n−1 represents Hermite orthogonal polynomial terms. The Hermite orthogonal polynomial formula is as follows:(10)H0x=1,H1x=2x,Hk+1x=2xHkx−2kHk−1x,k=1,2,…,x∈−∞,∞.The output of the Hermite chaotic neural network is(11)y=∑j=0n−1cjHjOj,where (Tt, dt), t = 1, 2, …, l is the training sample set and l is the number of training samples. Hermite chaotic neural network input is Tt=x1t,x2t,…,xmt, and its desired output is dt. The backpropagation (BP) algorithm is used to train the network.The weights of the network are trained according to the following formula:(12)et=dt−yt,E=12∑t=1let2,Δcj=−η∂E∂cj,Δwij=−η∂E∂wij,where t=1,2,…,l, i=1,2,…,m,j=1,2,…,n.The chaotic sequence produced by the logistic chaotic function is used as a sample set. The logistic chaotic function is as follows:(13)xn+1=μxn1−xn,where 3.5699456 ≤ u ≤ 4, x∈ (0, 1).In this work, we setμ=3.8, l=1000, E=10−12, η=0.03, n=3, and the maximum number of training to 1500 epochs. The training process of the Hermite chaotic neural network is shown in Figure 5. When the number of training times is 260, the error is 9.5169e−13, which has reached the expected error. When the initial value of the Hermite chaotic neural network is 0.66, the chaotic sequence generated by the network is shown in Figure 6. The chaotic sequence is used to scramble the watermarking image. The number of the chaotic sequence is determined by the size of the watermarking image.Figure 5
Training process.Figure 6
Chaotic sequence.


## 2.1. Feature Extraction Based on VGGnet

VGGnet is a representative type of deep convolutional neural network (CNN), which is often used in feature extraction and transfer learning [23]. The most widely used VGGnet is VGG19, which consists of 19 hidden layers (16 convolution layers and 3 fully connected layers), as shown in Figure 1. VGG19 uses a series of 3 × 3 convolution kernels to extract image features and expands the number of feature channels through convolution layers. Let Wi and bi represent the weights and bias of the ith convolution layer, the feature can be extracted by(1)Xiout=σWi∗Xiin+bi,where Xiout and Xiin, respectively, denote the input and output feature maps and σ⋅ is the rectified linear unit (ReLU). In each convolution layer, the stride is set to 1. To avoid the explosion of calculation, VGG19 uses max pooling layers to reduce the size of the feature maps.Figure 1
The structure of the VGG19 network.In fully connected layers, each node of the given layer is connected with all nodes of its previous layers, which can map the distributed feature representation to the sample label space by(2)Y=FC3FC2FC1PX16out,where FC⋅ denotes the operation of the fully connected layer and P⋅ refers to the max pooling operation.At the end of VGG19, a softmax layer produces the classification result of the image:(3)Yj=ezj∑c=1Cezc,where Yj is the probability of the jth node and zj and C, respectively, denote the output of jth node and the number of the classification.Compared with other types of CNNs, VGG19 improves the depth of the network and adopts the alternate structure of many convolution layers and nonlinear activation layers, which is beneficial to extract accurate features. In this work, different from image classification tasks, we just use the convolution layers and max pooling layers from the pretrained VGG19 as the preprocessing method to extract deep feature maps from medical images. Unlike other medical zero-watermarking algorithms, our method can extract abstract high-level features from medical images to improve the antigeometric attack ability of zero-watermarking.


## 2.2. Discrete Fourier Transform

Discrete Fourier transform plays an important role in the development of signal analysis and processing. Because of its definite physical meaning, discrete Fourier transform is widely used in many fields of signal analysis and processing.


### 2.2.1. One-Dimensional Discrete Fourier Transform

Letfx denote a time domain function of x, where x represents a time domain variable and u is a frequency domain variable. When fx reaches the Dirichlet condition, the discrete Fourier transform is formula (4) and the inverse transform is formula (5), where Fu is a frequency domain function:(4)Fu=∑x=0N−1fxe−j2πux/N,u=0,1,…,N−1,(5)fx=1N∑u=0N−1Fuej2πux/N,x=0,1,…,N−1.


### 2.2.2. Two-Dimensional Discrete Fourier Transform

For a given image with the size ofM×N, when it satisfies the Dirichlet condition, the two-dimensional discrete Fourier positive transform is(6)Fu,v=∑x=0M−1∑y=0N−1fx,ye−j2π/Mxue−j2π/Nyv,u=0,1,…,M−1;v=0,1,…,N−1.Its inverse transformation formula is(7)fx,y=1MN∑u=0M−1∑v=0N−1Fu,vej2π/Mxuej2π/Nyv,x=0,1,…,M−1;y=0,1,…,N−1,where x and y are the values in the space domain and u and v are the values in the frequency domain.Two-dimensional discrete Fourier transform transforms an image from the space domain to the frequency domain, which has a clear physical meaning. The frequency of an image is an index that characterizes the intensity of the gray level change in the image and is the gradient of the gray level in the plane space. After the Fourier transform, the changes in the flat area of the image are represented by low-frequency coefficients, and the details of the image are represented by high-frequency coefficients. Figure2 shows the result of the Fourier transform of a medical image.Figure 2
Discrete Fourier transform of the medical image. (a) Original medical image. (b) Medical images in the Fourier domain.
(a)(b)After the two-dimensional discrete Fourier transform, the transform coefficient matrix of the image shows that if the origin of the transform matrix is set at the center, the spectral energy is concentrated near the center of the short transform coefficient matrix. If the origin of the two-dimensional discrete Fourier transform matrix is set in the upper left corner, the energy of the image will be concentrated in the four corners of the coefficient matrix. This is determined by the nature of the two-dimensional discrete Fourier transform itself. It also shows that the image energy is generally concentrated in the low-frequency area. This determines that the two-dimensional discrete Fourier transform is very suitable for image processing. In this paper, a two-dimensional discrete Fourier transform is used to transform the features of the medical image into the Fourier domain to construct zero-watermarking.


## 2.2.1. One-Dimensional Discrete Fourier Transform

Letfx denote a time domain function of x, where x represents a time domain variable and u is a frequency domain variable. When fx reaches the Dirichlet condition, the discrete Fourier transform is formula (4) and the inverse transform is formula (5), where Fu is a frequency domain function:(4)Fu=∑x=0N−1fxe−j2πux/N,u=0,1,…,N−1,(5)fx=1N∑u=0N−1Fuej2πux/N,x=0,1,…,N−1.


## 2.2.2. Two-Dimensional Discrete Fourier Transform

For a given image with the size ofM×N, when it satisfies the Dirichlet condition, the two-dimensional discrete Fourier positive transform is(6)Fu,v=∑x=0M−1∑y=0N−1fx,ye−j2π/Mxue−j2π/Nyv,u=0,1,…,M−1;v=0,1,…,N−1.Its inverse transformation formula is(7)fx,y=1MN∑u=0M−1∑v=0N−1Fu,vej2π/Mxuej2π/Nyv,x=0,1,…,M−1;y=0,1,…,N−1,where x and y are the values in the space domain and u and v are the values in the frequency domain.Two-dimensional discrete Fourier transform transforms an image from the space domain to the frequency domain, which has a clear physical meaning. The frequency of an image is an index that characterizes the intensity of the gray level change in the image and is the gradient of the gray level in the plane space. After the Fourier transform, the changes in the flat area of the image are represented by low-frequency coefficients, and the details of the image are represented by high-frequency coefficients. Figure2 shows the result of the Fourier transform of a medical image.Figure 2
Discrete Fourier transform of the medical image. (a) Original medical image. (b) Medical images in the Fourier domain.
(a)(b)After the two-dimensional discrete Fourier transform, the transform coefficient matrix of the image shows that if the origin of the transform matrix is set at the center, the spectral energy is concentrated near the center of the short transform coefficient matrix. If the origin of the two-dimensional discrete Fourier transform matrix is set in the upper left corner, the energy of the image will be concentrated in the four corners of the coefficient matrix. This is determined by the nature of the two-dimensional discrete Fourier transform itself. It also shows that the image energy is generally concentrated in the low-frequency area. This determines that the two-dimensional discrete Fourier transform is very suitable for image processing. In this paper, a two-dimensional discrete Fourier transform is used to transform the features of the medical image into the Fourier domain to construct zero-watermarking.


## 2.3. Perceptual Hashing Algorithm

Perceptual hashing is considered as a one-way mapping from the multimedia data set to the perceptual content hashing value; that is, a short digital digest uniquely represents multimedia data with the same perceptual content. This digital digest is called perceptual hashing value, so the mapping process is also called the process of hash value generation.Image perceptual hashing is usually called image digital fingerprint or image digital digest. It can map the image to a group of hash sequences, which greatly reduces the storage of the digital image and brings great convenience to image management and maintenance. It has become a research hotspot in the field of multimedia signal processing and security. Perceptual feature extraction is the core of the perceptual hashing algorithm. The effectiveness and reliability of perceptual feature extraction will directly affect the uniqueness and robustness of the perceptual hashing sequence.A mean-perceptual hashing algorithm is proposed in this paper. The mean-perceptual hashing algorithm uses the mean value of the elements of the medical image feature matrix to generate a hashing sequence. The construction of the feature image is realized by VGG19 and image fusion. Figure3 describes the flowchart of the mean-perceptual hashing algorithm.Figure 3
The flowchart of mean-perceptual hashing.


## 2.4. Hermite Chaotic Neural Network

In this work, a new Hermite chaotic neural network is used to scramble the watermarking image [24]. The structure of the network is shown in Figure 4, whose topology is m×n×1.Figure 4
The architecture of Hermite chaotic neural network.The hidden layer neuron input is(8)Oj=wjx,j=0,1,2,…,n−1,where wj is the weight between the input layer and hidden layer and cj is the weight between the hidden layer and output layer. The hidden layer neuron output is(9)HjOj,j=0,1,2,…,n−1,where Hj,j=0,1,2,…,n−1 represents Hermite orthogonal polynomial terms. The Hermite orthogonal polynomial formula is as follows:(10)H0x=1,H1x=2x,Hk+1x=2xHkx−2kHk−1x,k=1,2,…,x∈−∞,∞.The output of the Hermite chaotic neural network is(11)y=∑j=0n−1cjHjOj,where (Tt, dt), t = 1, 2, …, l is the training sample set and l is the number of training samples. Hermite chaotic neural network input is Tt=x1t,x2t,…,xmt, and its desired output is dt. The backpropagation (BP) algorithm is used to train the network.The weights of the network are trained according to the following formula:(12)et=dt−yt,E=12∑t=1let2,Δcj=−η∂E∂cj,Δwij=−η∂E∂wij,where t=1,2,…,l, i=1,2,…,m,j=1,2,…,n.The chaotic sequence produced by the logistic chaotic function is used as a sample set. The logistic chaotic function is as follows:(13)xn+1=μxn1−xn,where 3.5699456 ≤ u ≤ 4, x∈ (0, 1).In this work, we setμ=3.8, l=1000, E=10−12, η=0.03, n=3, and the maximum number of training to 1500 epochs. The training process of the Hermite chaotic neural network is shown in Figure 5. When the number of training times is 260, the error is 9.5169e−13, which has reached the expected error. When the initial value of the Hermite chaotic neural network is 0.66, the chaotic sequence generated by the network is shown in Figure 6. The chaotic sequence is used to scramble the watermarking image. The number of the chaotic sequence is determined by the size of the watermarking image.Figure 5
Training process.Figure 6
Chaotic sequence.


## 3. Zero-Watermarking Embedding and Extraction Algorithm

### 3.1. Embedding Algorithm

Choosing an image with a specific meaning, our algorithm uses it as the original watermarking image in the experiment. It is recorded asWi,j=wi,j|wi,j=0,1;1≤i≤M1,1≤j≤M2. The gray value of the original watermarking image is represented as wi,j. The original medical image is recorded as Fi,j=fi,j|fi,j∈R,1;1≤i≤N1,1≤j≤N2. Here, the pixel gray value of the original medical image is represented by fi,j. For the convenience of calculation, let M1=M2=64, N1=N2. Figure 7 shows the watermarking embedding procedure.(1)
Hermite chaotic neural network is used to scramble the position of pixels in the original watermarking imageWi,j to get the scrambled watermarking image BWi,j.(2)
The pretrained VGG19 is used to extract the deep feature mapsFMk,l,p of the original medical image Fi,j:(14)Fi,j⟶VGG19⟶FMk,l,p,where 1 ≤k ≤ 8, 1 ≤ l ≤ 8, and 1 ≤ p ≤ 512.(3)
The deep feature mapsFMk,l,p are fused to generate feature image FIk,l:(15)FIk,l=∑p=1512FMk,l,p.(4)
Two-dimensional discrete Fourier transform is used to transform the feature imageFIk,l, and the transform coefficients FIFk,l are obtained to construct the feature matrix of medical image FIMk,l:(16)FIFk,l=2D−DFTFIk,l,FIFk,l⟶FIMk,l.(5)
The hashing sequence of medical image feature matrixFIMk,l is extracted by the mean-perceptual hashing algorithm to generate 64-bit binary hashing sequence PHq:(17)PHq=mHashFIMk,l,where1≤q≤64.(6)
The 64-bit binary hashing sequencePHq is XORed with the scrambled watermarking image BWi,j. And, the watermarking extraction key Keyi,j is generated to extract the watermarking image:(18)Keyi,j=PHq⊕BWi,j,where 1≤q≤64, 1≤i≤64,and1≤j≤64. The watermarking extraction key Keyi,j can be saved in a third party for later watermarking extraction.Figure 7
Watermarking embedding procedure.


### 3.2. Extraction Algorithm

The medical image to be tested is recorded asF′i,j=f′i,j|f′i,j∈R,1;1≤i≤N1,1≤j≤N2. The extraction algorithm is similar to the embedding algorithm, and the specific steps are as follows. Figure 8 shows watermarking extraction procedure.(1)
The pretrained VGG19 is used to extract the deep feature mapsFM′k,l,p of the medical image to be tested F′i,j:(19)F′i,j⟶VGG19⟶FM′k,l,p,where1≤k≤8,1≤l≤8,1≤p≤512.(2)
The deep feature mapsFM′k,l,p are fused to generate feature image FI′k,l:(20)FI′k,l=∑p=1512FM′k,l,p.(3)
Two-dimensional discrete Fourier transform is used to transform the feature imageFI′k,l, and the transform coefficients FIF′k,l are obtained to construct the feature matrix of medical image FIM′k,l:(21)FIF′k,l=2D−DFTFI′k,l,FIF′k,l⟶FIM′k,l.(4)
The hashing sequence of medical image feature matrixFIM′k,l is extracted by the mean-perceptual hashing algorithm to generate 64-bit binary hashing sequence PH′q:(22)PH′q=mHashFIM′k,l,where1≤q≤64.(5)
The 64-bit binary hashing sequencePH′q is XORed with the watermarking extraction key Keyi,j to extract the watermarking image BW′i,j from the medical image to be tested F′i,j:(23)BW′i,j=PH′q⊕Keyi,j,where1≤q≤64, 1≤i≤64,1≤j≤64.(6)
The Hermite chaotic neural network is used to inversely scramble the extracted watermarking imageBW′i,j to obtain the restored watermarking image W′i,j.(7)
The normalized correlation coefficient (NC) is used to detect the restored watermarking imageW′i,j obtained from the medical image to be tested F′i,j. The normalized csrrelation coefficient formula is shown in the following formula:(24)NC=∑i=1M1∑j=1M2wi,jW′i,j∑i=1M1∑j=1M2wi,jwi,j.Figure 8
Watermarking extraction procedure.By comparing the NC value between the original watermarking imageWi,j and the restored watermarking image W′i,j, the restored watermarking image W′i,j obtained from the medical image to be tested is evaluated. The larger the NC value, the greater the correlation between the original watermarking image Wi,j and the restored watermarking image W′i,j.The peak signal-to-noise ratio (PSNR) was used to evaluate the quality of the medical image to be tested. The PSNR formula is as follows:(25)PSNR=10lgMAX2MSE,MSE=1N1×N2∑i=1N2∑j=1N1Fi,j−F′i,j2,where MAX is the maximum gray value of the image pixel and MSE is the mean square error between the original image and the test image.


## 3.1. Embedding Algorithm

Choosing an image with a specific meaning, our algorithm uses it as the original watermarking image in the experiment. It is recorded asWi,j=wi,j|wi,j=0,1;1≤i≤M1,1≤j≤M2. The gray value of the original watermarking image is represented as wi,j. The original medical image is recorded as Fi,j=fi,j|fi,j∈R,1;1≤i≤N1,1≤j≤N2. Here, the pixel gray value of the original medical image is represented by fi,j. For the convenience of calculation, let M1=M2=64, N1=N2. Figure 7 shows the watermarking embedding procedure.(1)
Hermite chaotic neural network is used to scramble the position of pixels in the original watermarking imageWi,j to get the scrambled watermarking image BWi,j.(2)
The pretrained VGG19 is used to extract the deep feature mapsFMk,l,p of the original medical image Fi,j:(14)Fi,j⟶VGG19⟶FMk,l,p,where 1 ≤k ≤ 8, 1 ≤ l ≤ 8, and 1 ≤ p ≤ 512.(3)
The deep feature mapsFMk,l,p are fused to generate feature image FIk,l:(15)FIk,l=∑p=1512FMk,l,p.(4)
Two-dimensional discrete Fourier transform is used to transform the feature imageFIk,l, and the transform coefficients FIFk,l are obtained to construct the feature matrix of medical image FIMk,l:(16)FIFk,l=2D−DFTFIk,l,FIFk,l⟶FIMk,l.(5)
The hashing sequence of medical image feature matrixFIMk,l is extracted by the mean-perceptual hashing algorithm to generate 64-bit binary hashing sequence PHq:(17)PHq=mHashFIMk,l,where1≤q≤64.(6)
The 64-bit binary hashing sequencePHq is XORed with the scrambled watermarking image BWi,j. And, the watermarking extraction key Keyi,j is generated to extract the watermarking image:(18)Keyi,j=PHq⊕BWi,j,where 1≤q≤64, 1≤i≤64,and1≤j≤64. The watermarking extraction key Keyi,j can be saved in a third party for later watermarking extraction.Figure 7
Watermarking embedding procedure.


## 3.2. Extraction Algorithm

The medical image to be tested is recorded asF′i,j=f′i,j|f′i,j∈R,1;1≤i≤N1,1≤j≤N2. The extraction algorithm is similar to the embedding algorithm, and the specific steps are as follows. Figure 8 shows watermarking extraction procedure.(1)
The pretrained VGG19 is used to extract the deep feature mapsFM′k,l,p of the medical image to be tested F′i,j:(19)F′i,j⟶VGG19⟶FM′k,l,p,where1≤k≤8,1≤l≤8,1≤p≤512.(2)
The deep feature mapsFM′k,l,p are fused to generate feature image FI′k,l:(20)FI′k,l=∑p=1512FM′k,l,p.(3)
Two-dimensional discrete Fourier transform is used to transform the feature imageFI′k,l, and the transform coefficients FIF′k,l are obtained to construct the feature matrix of medical image FIM′k,l:(21)FIF′k,l=2D−DFTFI′k,l,FIF′k,l⟶FIM′k,l.(4)
The hashing sequence of medical image feature matrixFIM′k,l is extracted by the mean-perceptual hashing algorithm to generate 64-bit binary hashing sequence PH′q:(22)PH′q=mHashFIM′k,l,where1≤q≤64.(5)
The 64-bit binary hashing sequencePH′q is XORed with the watermarking extraction key Keyi,j to extract the watermarking image BW′i,j from the medical image to be tested F′i,j:(23)BW′i,j=PH′q⊕Keyi,j,where1≤q≤64, 1≤i≤64,1≤j≤64.(6)
The Hermite chaotic neural network is used to inversely scramble the extracted watermarking imageBW′i,j to obtain the restored watermarking image W′i,j.(7)
The normalized correlation coefficient (NC) is used to detect the restored watermarking imageW′i,j obtained from the medical image to be tested F′i,j. The normalized csrrelation coefficient formula is shown in the following formula:(24)NC=∑i=1M1∑j=1M2wi,jW′i,j∑i=1M1∑j=1M2wi,jwi,j.Figure 8
Watermarking extraction procedure.By comparing the NC value between the original watermarking imageWi,j and the restored watermarking image W′i,j, the restored watermarking image W′i,j obtained from the medical image to be tested is evaluated. The larger the NC value, the greater the correlation between the original watermarking image Wi,j and the restored watermarking image W′i,j.The peak signal-to-noise ratio (PSNR) was used to evaluate the quality of the medical image to be tested. The PSNR formula is as follows:(25)PSNR=10lgMAX2MSE,MSE=1N1×N2∑i=1N2∑j=1N1Fi,j−F′i,j2,where MAX is the maximum gray value of the image pixel and MSE is the mean square error between the original image and the test image.


## 4. Experiment and Analysis

In this work, several experiments are conducted by applying local nonlinear geometric attacks to investigate the performance and effectiveness of the proposed zero-watermarking algorithm. Five original medical images of different parts of the human body produced by different medical equipment are used to test the effectiveness of the proposed zero-watermarking algorithm. The original medical images with the size of 128 × 128 are shown in Figure9. The original 64 × 64 watermarking image and the scrambled watermarking image are shown in Figure 10.Figure 9
The original medical images.
(a)(b)(c)(d)(e)Figure 10
The watermarking images. (a) The original watermarking image. (b) The scrambled watermarking image.
(a)(b)The following experiments test the ability of the watermarking algorithm to resist local nonlinear geometric attacks. For the convenience and repeatability of the experiment, we use the filter function of Adobe Photoshop software to implement the following common local distortion attacks. There are four types of local distortion attacks: ripple distortion attack, extrusion distortion attack, spherical distortion attack, and rotation distortion attack.


### 4.1. Ripple Distortion Attack

Ripple distortion attack is a common local nonlinear geometric attack, which transforms the coordinates of image pixels according to different functions. Table1 shows the experimental results under the ripple distortion attack when the distortion quantity increases from 150% to 750%. Table 2 gives the restored watermarking images with the smallest NC value and their corresponding test medical image. The restored watermarking images can be clearly identified from Table 2, which shows that the proposed algorithm is very effective against ripple distortion attack.Table 1
Experimental results under ripple distortion attack.
Distortion quantity (%)150300450600750Medical image APSNR (dB)12.788911.903611.576511.264811.0596NC0.875950.810210.810210.811060.74818Medical image BPSNR (dB)16.637812.809212.809211.758811.0855NC0.875950.875950.875950.689290.75075Medical image CPSNR (dB)21.826717.874515.837814.647313.8828NC0.813630.749890.811630.658420.59668Medical image DPSNR (dB)21.345617.758616.168714.892714.0789NC0.875090.871380.810210.813920.81392Medical image EPSNR (dB)28.182225.202323.316521.945720.5296NC0.878230.751040.689870.688720.68872Table 2
The images under ripple distortion attack.
Medical image AMedical image BMedical image CMedical image DMedical image EThe smallest NC value0.748180.689290.596680.810210.68872The restored watermarking imageMedical image to be tested


### 4.2. Extrusion Distortion Attack

For extrusion distortion attack, distortion quantity is varied between 10% and 90% for simulation. The experimental results are shown in Table3. All NC values are greater than 0.74. In Table 4, it is obvious that the restored watermarking image is very clear. The experimental results show that the proposed watermarking algorithm has good robustness against extrusion distortion attacks.Table 3
Experimental results under extrusion distortion attack.
Distortion quantity (%)1030507090Medical image APSNR (dB)13.049411.487310.662110.26159.9121NC1.01.01.00.93540.9354Medical image BPSNR (dB)21.723916.486214.185512.708111.6212NC1.00.842220.78150.810490.81278Medical image CPSNR (dB)26.56119.834217.175815.69514.9073NC0.938260.938260.748180.748180.81163Medical image DPSNR (dB)24.451118.079915.882214.69214.0065NC0.936260.810210.780760.780760.87395Medical image EPSNR (dB)32.018526.22923.581321.829620.504NC0.93940.93940.93940.875660.87766Table 4
The images under extrusion distortion attack.
Medical image AMedical image BMedical image CMedical image DMedical image EThe smallest NC value0.93540.78150.748180.780760.87566The restored watermarking imageMedical image to be tested


### 4.3. Spherical Distortion Attack

Then, we test the robustness of the proposed method on image spherical distortion. The test medical images were attacked by spherical distortions of 10%, 30%, 50%, 70%, and 90% in turn. Experimental results are listed in Table5. It can be observed from Table 5 that most of the NC values are close to 1.0. The restored watermarking image in Table 6 further proves that the proposed algorithm has excellent performance under the spherical distortion attack.Table 5
Experimental results under spherical distortion attack.
Distortion quantity (%)1030507090Medical image APSNR (dB)12.858511.491711.145911.303511.3435NC1.01.00.871660.871660.87166Medical image BPSNR (dB)18.498514.442713.009811.752410.5874NC0.936540.873660.812210.812210.74704Medical image CPSNR (dB)27.25919.87216.602814.508113.0453NC0.935121.01.01.00.93997Medical image DPSNR (dB)23.915117.113114.585213.254212.4597NC11110.93769Medical image EPSNR (dB)30.6425.448722.9120.915619.2908NC0.9394110.817630.62984Table 6
The images under spherical distortion attack.
Medical image AMedical image BMedical image CMedical image DMedical image EThe smallest NC value0.871660.747040.935120.937690.62984The restored watermarking imageMedical image to be tested


### 4.4. Rotation Distortion Attack

We also evaluate the robustness of the proposed algorithm under rotation distortion attack. Table7 shows that when the distortion quantity increases, the NC value and the PSNR values gradually decrease. As can be seen from Table 8, even if the NC value is 0.5621, the watermark image can still be accurately extracted. These results demonstrate that the proposed algorithm has better robustness against the rotation distortion attack.Table 7
Experimental results under rotation distortion attack.
Distortion quantity (%)1030507090Medical image APSNR (dB)13.873813.275112.813812.520212.2938NC1.01.01.01.00.94054Medical image BPSNR (dB)21.554116.33614.186912.710211.5936NC0.938830.810490.875090.626980.62927Medical image CPSNR (dB)23.259316.261114.421213.616613.1196NC0.936260.936260.936260.936260.5621Medical image DPSNR (dB)23.647117.993716.195515.190914.7219NC0.940540.940540.906820.906820.90682Medical image EPSNR (dB)32.992727.834625.452123.764122.4491NC0.93940.93940.878230.878230.8182Table 8
The images under rotation distortion attack.
Medical image AMedical image BMedical image CMedical image DMedical image EThe smallest NC value0.940540.626980.56210.906820.8182The restored watermarking imageMedical image to be testedThe above experimental results indicate that the proposed algorithm has strong capability against all the four types of local distortion attacks. Because it is a local nonlinear attack, the NC value does not necessarily decrease with the increase of the distortion quantity, and the change of the NC value is not the same as the change of the PSNR value.


## 4.1. Ripple Distortion Attack

Ripple distortion attack is a common local nonlinear geometric attack, which transforms the coordinates of image pixels according to different functions. Table1 shows the experimental results under the ripple distortion attack when the distortion quantity increases from 150% to 750%. Table 2 gives the restored watermarking images with the smallest NC value and their corresponding test medical image. The restored watermarking images can be clearly identified from Table 2, which shows that the proposed algorithm is very effective against ripple distortion attack.Table 1
Experimental results under ripple distortion attack.
Distortion quantity (%)150300450600750Medical image APSNR (dB)12.788911.903611.576511.264811.0596NC0.875950.810210.810210.811060.74818Medical image BPSNR (dB)16.637812.809212.809211.758811.0855NC0.875950.875950.875950.689290.75075Medical image CPSNR (dB)21.826717.874515.837814.647313.8828NC0.813630.749890.811630.658420.59668Medical image DPSNR (dB)21.345617.758616.168714.892714.0789NC0.875090.871380.810210.813920.81392Medical image EPSNR (dB)28.182225.202323.316521.945720.5296NC0.878230.751040.689870.688720.68872Table 2
The images under ripple distortion attack.
Medical image AMedical image BMedical image CMedical image DMedical image EThe smallest NC value0.748180.689290.596680.810210.68872The restored watermarking imageMedical image to be tested


## 4.2. Extrusion Distortion Attack

For extrusion distortion attack, distortion quantity is varied between 10% and 90% for simulation. The experimental results are shown in Table3. All NC values are greater than 0.74. In Table 4, it is obvious that the restored watermarking image is very clear. The experimental results show that the proposed watermarking algorithm has good robustness against extrusion distortion attacks.Table 3
Experimental results under extrusion distortion attack.
Distortion quantity (%)1030507090Medical image APSNR (dB)13.049411.487310.662110.26159.9121NC1.01.01.00.93540.9354Medical image BPSNR (dB)21.723916.486214.185512.708111.6212NC1.00.842220.78150.810490.81278Medical image CPSNR (dB)26.56119.834217.175815.69514.9073NC0.938260.938260.748180.748180.81163Medical image DPSNR (dB)24.451118.079915.882214.69214.0065NC0.936260.810210.780760.780760.87395Medical image EPSNR (dB)32.018526.22923.581321.829620.504NC0.93940.93940.93940.875660.87766Table 4
The images under extrusion distortion attack.
Medical image AMedical image BMedical image CMedical image DMedical image EThe smallest NC value0.93540.78150.748180.780760.87566The restored watermarking imageMedical image to be tested


## 4.3. Spherical Distortion Attack

Then, we test the robustness of the proposed method on image spherical distortion. The test medical images were attacked by spherical distortions of 10%, 30%, 50%, 70%, and 90% in turn. Experimental results are listed in Table5. It can be observed from Table 5 that most of the NC values are close to 1.0. The restored watermarking image in Table 6 further proves that the proposed algorithm has excellent performance under the spherical distortion attack.Table 5
Experimental results under spherical distortion attack.
Distortion quantity (%)1030507090Medical image APSNR (dB)12.858511.491711.145911.303511.3435NC1.01.00.871660.871660.87166Medical image BPSNR (dB)18.498514.442713.009811.752410.5874NC0.936540.873660.812210.812210.74704Medical image CPSNR (dB)27.25919.87216.602814.508113.0453NC0.935121.01.01.00.93997Medical image DPSNR (dB)23.915117.113114.585213.254212.4597NC11110.93769Medical image EPSNR (dB)30.6425.448722.9120.915619.2908NC0.9394110.817630.62984Table 6
The images under spherical distortion attack.
Medical image AMedical image BMedical image CMedical image DMedical image EThe smallest NC value0.871660.747040.935120.937690.62984The restored watermarking imageMedical image to be tested


## 4.4. Rotation Distortion Attack

We also evaluate the robustness of the proposed algorithm under rotation distortion attack. Table7 shows that when the distortion quantity increases, the NC value and the PSNR values gradually decrease. As can be seen from Table 8, even if the NC value is 0.5621, the watermark image can still be accurately extracted. These results demonstrate that the proposed algorithm has better robustness against the rotation distortion attack.Table 7
Experimental results under rotation distortion attack.
Distortion quantity (%)1030507090Medical image APSNR (dB)13.873813.275112.813812.520212.2938NC1.01.01.01.00.94054Medical image BPSNR (dB)21.554116.33614.186912.710211.5936NC0.938830.810490.875090.626980.62927Medical image CPSNR (dB)23.259316.261114.421213.616613.1196NC0.936260.936260.936260.936260.5621Medical image DPSNR (dB)23.647117.993716.195515.190914.7219NC0.940540.940540.906820.906820.90682Medical image EPSNR (dB)32.992727.834625.452123.764122.4491NC0.93940.93940.878230.878230.8182Table 8
The images under rotation distortion attack.
Medical image AMedical image BMedical image CMedical image DMedical image EThe smallest NC value0.940540.626980.56210.906820.8182The restored watermarking imageMedical image to be testedThe above experimental results indicate that the proposed algorithm has strong capability against all the four types of local distortion attacks. Because it is a local nonlinear attack, the NC value does not necessarily decrease with the increase of the distortion quantity, and the change of the NC value is not the same as the change of the PSNR value.


## 5. Algorithm Comparison

In this section, the proposed algorithm is compared with two representative medical image watermarking algorithms. The details of these three algorithms are shown in Table9. These three algorithms are robust medical image watermarking algorithms. Every robust watermarking algorithm can resist all kinds of attacks with good robustness and security and can solve all kinds of information security problems of the medical image. The proposed algorithm and [25, 26] are zero-watermarking algorithms. Their watermarking is based on the important features of the medical image rather than modifying the content of the medical image. They are easy to implement, fast to embed and extract, and have the advantages of small amount of computation and high speed. Therefore, the proposed algorithm and [25, 26] are very suitable for medical images. In these algorithms, DWT, DCT, and DFT are all linear transformations. However, VGG19 is a multilayer perceptron specially designed for image recognition. Its feature extraction is essentially a nonlinear transformation.Table 9
Comparison between the proposed algorithm and other algorithms.
AlgorithmsObjectiveMethodMedical image contentTransform usedAlgorithm [25]Medical image securityZero-watermarkingNo changeDWT-DFTAlgorithm [26]Medical image securityZero-watermarkingNo changeDCTThe proposed algorithmMedical image securityZero-watermarkingNo changeVGG19-DFTIn order to further analyze the performance of the proposed algorithm, the algorithm [25, 26] has also carried out local nonlinear geometric attack experiments under the same conditions. Their results are compared with the experimental results of the proposed algorithm.The comparison of the algorithms under four types of local nonlinear geometric attacks is shown in Figure11. As the blue bar of Figure 11(a) shows, it is obvious that the ability of the proposed algorithm to resist ripple distortion attack is better than the other two algorithms [25, 26]. The comparison of the experimental results of the extrusion distortion attack is shown in Figure 11(b). Figure 11(b) shows that, with the increase of extrusion distortion quantity, the NC value of the algorithm is close to 1, which is obviously higher than that of the other two algorithms [25, 26]. Especially when the extrusion distortion quantity is more than 70%, the difference of NC value is more significant. Figure 11(c) shows the comparison under the spherical distortion attack. The NC values of the proposed algorithms are all greater than 0.87. This proves that the proposed algorithm is more resistant to the spherical distortion attack. It can be observed from Figure 11(d) that the three algorithms can effectively resist the rotation distortion attack, but the proposed algorithm has a stronger ability.Figure 11
Robustness comparison between the proposed algorithm and two algorithms [25, 26] against four types of local nonlinear geometric attacks. (a) Ripple distortion attack. (b) Extrusion distortion attack. (c) Spherical distortion attack. (d) Rotation distortion attack.
(a)(b)(c)(d)With the above comparison and analysis of the four local nonlinear geometric attacks, the proposed algorithm is obviously stronger than the other two algorithms in resisting local nonlinear geometric attacks. Therefore, the proposed algorithm has good robustness and can well resist local nonlinear geometric attacks.


## 6. Conclusions

In recent years, the medical image watermarking algorithm against geometric attacks has been a hot and difficult topic in the research of robust watermarking technology. In this paper, a zero-watermarking algorithm based on VGG19 is designed to resist local nonlinear geometric attacks. VGG19 is used to extract deep features of the medical image, and two-dimensional discrete Fourier transform and mean-perceptual hashing algorithm are used to generate zero-watermarking. The design process of the algorithm combines the concepts of deep convolution neural network, Fourier transform, perceptual hashing, cryptography, and zero-watermarking, which solves the problem of watermarking resisting local nonlinear geometric attacks. At the same time, the scrambling of the watermarking image ensures the security of the algorithm. It has high practical value for medical information protection.



---

*Source: 5551520-2021-07-02.xml*