# A Novel Improved Reversible Visible Image Watermarking Algorithm Based on Grad-CAM and JND

Authors: Jiasheng Qu; Wei Song; Xiangchun Liu; Lizhi Zhao; Xiaobing Zhao
Journal: Security and Communication Networks
 (2021)

---

## Abstract

With the rapid access convenience of content brought by 5G technology, the integrity protection of content becomes more important. The reversible visible watermarking algorithm has attracted more attention due to its effective content protection. In this paper, a novel improved reversible visible image watermarking scheme based on gradient-weighted class activation mapping (Grad-CAM) and the just noticeable difference (JND) model has been presented. The proposed region of interest (ROI) selection strategy is used to locate the main protected body of images for watermark embedding. Divide the watermark and ROI into nonoverlapping blocks in the same way and then embed the classified two types of watermark blocks into corresponding ROI blocks with the JND model. The optimal bit positions for watermark embedding can be selected adaptively with JND threshold and achieve the tradeoff between the watermark visibility and watermarked image quality. For lossless image recovery and watermark extraction, the recovery information is reversibly hidden into watermarked image. In the experiments, the same process of grayscale images is used to each channel separately for color images watermarking. Besides, there are six aspects in this paper to estimate the proposed scheme; with the comparison to other reversible visible watermarking schemes, experimental results demonstrate the effectiveness of our proposed scheme.


---

## Body

## 1. Introduction

Nowadays, the rapid development of 5G technologies is benefiting all aspects of our lives; at the same time, the multimedia data in the networks are becoming larger and larger. The security of multimedia data deserves more attention. Also, the digital images, as one of the most important carriers in the network information transmission, its security and privacy need more concerns from researchers. The visible watermarking technique which can embed copyright information into cover images has developed rapidly. There are two categories of visible watermarking: reversible and irreversible. In the past, many irreversible visible watermarking schemes have been proposed [1–4], but the original cover image can hardly be recovered. On the other hand, the reversible visible watermarking schemes can recover both the original image and the watermark perfectly. This paper focuses on the latter.Many reversible visible watermarking schemes are proposed in the past [5–10]. Hu and Jeon [5] proposed a scheme by modifying one significant bit plane of the pixels to achieve watermark visibility. Then, they compressed the altered bit plane values as recovery information which was embedded into the nonwatermarked image region for lossless recovery. However, the watermarked image often distorts rather significantly compared with the original image. In [6], the reversible watermarking scheme based on deterministic one-to-one compound mappings of image pixel values has been proposed. Chen et al. [7] proposed a scheme based on the conventional difference expansion technique. In this scheme, the cover image is divided into nonoverlapping blocks and each block is embedded with one watermark bit. But unfortunately, there is a large amount of overflowing or underflowing pixels in the watermarked image. Once exceeded, the watermarked image cannot be recovered perfectly. In the scheme of [8] by Mohammad et al., the pixel circular shift operation is utilized to embed the watermark into the block truncation coding-compressed (BTC-compressed) image. According to the parity of the bit plane, the watermark signal can be extracted reversibly. In the abovementioned reversible visible watermarking schemes, the watermark embedding region is specified, which is generally the center of the cover image. In [9], Qi et al. proposed a reversible visible scheme based on the human visual system (HVS) and region of interest (ROI) selection. For watermark embedding, HVS is adopted to modify the pixel values so as to get a better effect of watermark visibility. And, the ROI selection strategy is designed to find the flat regions with low or high luminance for watermark embedding. The chosen flat regions usually do not contain abundant information which can be easily cropped but without affecting the whole information of images. However, to maintain the copyright of the images better, the visible watermark should be embedded into the subject region of the image which contains the most important image information. Hence, the selected ROI for watermark embedding in [9] may be not suitable. To select the subject region of images, we proposed a novel ROI selection strategy based on Grad-CAM [11] for visible watermark embedding. With this novel ROI selection strategy, the visible watermark can be embedded in the subject region to protect the copyright of images. However, the visible watermark into the subject region can degrade the watermarked image quality. The key to solving this problem is to balance the watermark visibility and the watermarked image quality, which means the embedded watermark should not be so significant that the watermarked image details have been covered too much [2, 3]. Motivated by the scheme proposed by Yao et al. [10], the enhanced JND model [12] has been utilized in this paper to obtain the tradeoff between watermark visibility and watermarked image quality.Generally, a reversible visible watermarking technique is used to protect and maintain the copyright. The region of visible watermark embedding is vital. Assuming that the watermark embedding region is not the subject of an image, it can be easily modified or cropped for malicious use. However, there is no scheme proposed in the past to select the main body region as ROI for visible watermark embedding.In this paper, we have proposed an improved reversible visible watermarking scheme. In the proposed scheme, we concentrate on the novel ROI selection strategy to locate the subject region for watermark embedding and apply the JND model to balance between the watermark visibility and the marked image quality. To achieve reversibility, the reversible data hiding algorithm to embed the recovery information into the watermarked image is utilized so that the image can be recovered losslessly and the watermark can be extracted perfectly.In conclusion, the contribution of the proposed algorithm is listed as follows:(1)
A novel ROI selection strategy to locate the subject region of the image for watermark embedding is proposed(2)
Watermark embedding positions can be adaptively selected for the balance between watermark visibility and the marked image quality(3)
It can be lossless and perfect for image recovery and watermark extractionThe rest of the paper is organized as follows. In Section2, the gradient-weighted class activation mapping (Grad-CAM) for ROI selection strategy is presented. The proposed method is described in Section 3. In Section 4, the performance of the proposed framework is reported. Finally, Section 5 concludes the paper and discusses possible future work.


## 2. The Gradient-Weighted Class Activation Mapping

For most images, it contains at least one subject, which must belong to a certain category. By locating the class region, we can select the subject area as ROI for watermark embedding. Grad-CAM uses the gradients of a target concept which means the name of the class here, e.g., ‘wood rabbit’ and flow the feature maps into the final convolutional layer. Finally, the coarse localization map is obtained in which the subject region of the concept is highlighted [11]. And, we can obtain the ROI for watermark embedding by processing the coarse localization map.With a specific class name and a cover image as input, we can obtain the coarse localization mapLGrad−CAMc∈ℝu×v of width u and height v for class c with the class subject region highlighted. Figure 1 shows the network structure in detail. The gradient information flows into the last convolutional layer of the convolutional neural network (CNN) by assigning importance values to each neuron to locate the class. And, the feature map importance weights αkc for class c can be calculated by(1)αkc=1Z∑i∑j︷GAP∂yc∂Ai,jk︸GB,where k means the k-th feature map; yc is the score for class c before softmax; Ak is the feature map activations at the position i,j of the feature map; GAP means global average pooling; GB means gradients via backprop; and Z is a constant used through the global average pooling.Figure 1
Grad-CAM deep network structure.The weightαkc which represents a partial linearization of the deep network downstream from A can capture feature map k which means ‘important’ for the target class c[11]. To obtain the coarse localization map LGrad−CAMc∈ℝu×v, the weighted combination of forwarding activation maps is combined with function ReLU. LC is the linear combination.(2)LGrad−CAMc=ReLU∑kαkcAk︸LC.So far, we can get the coarse heatmap of classc after resizing itself, and Figures 2 (a1), (b1), (a2), (b2), (a3), and (b3) present the class localization of the image subject.Figure 2
Image subject localization via Grad-CAM and watermarking ROI selection: (a1), (a2), and (a3) original cover image, wood rabbit, tabby, and partridge; (b1), (b2), and (b3) subject localization heatmap; (c1), (c2), and (c3) binarized R channel of heatmap with central coordinate of ROI; (d1), (d2), and (d3) ROI for watermark embedding with (r) = 2.


## 3. Proposed Scheme

The improved reversible visible image watermarking scheme proposed in this paper includes four parts: the novel ROI selection, watermark embedding, reversible data hiding, and last, watermark extraction and image recovery. The flow chart of our proposed scheme is shown in Figure3. With the original cover image I, we propose a novel ROI selection strategy to locate the subject region for watermark embedding via Grad-CAM. Then, embed the visible binary watermark W into ROI using the JND model for balancing between the watermark visibility and the marked image quality, and also, we can obtain the recovery information D and watermarked image Iw meanwhile. To achieve the reversibility, the encoded recovery information De for compressing the bits of D should be embedded into the watermarked image Iw to obtain the reversible watermarked image Iw′. When receiving the reversible watermarked image Iw′, the extracted watermark WE and the recovered image IR can be obtained losslessly and perfectly with the extracted recovery information from Iw′. To improve the security of this scheme, we use logistic mapping to generate pseudorandom sequences of {0,1} with the secret key for watermark embedding and recovery information encryption. And, the original image cannot be recovered if the secret key is wrong. For our proposed reversible visible image watermarking scheme, the novel ROI selection strategy is discussed in Section 3.1. Section 3.2 presents the watermark embedding in detail. The reversible data hiding of recovery information is described briefly in Section 3.3. Finally, Section 3.4 shows the process of watermark extraction and image recovery.Figure 3
The flow chart of our proposed scheme.


### 3.1. The Novel ROI Selection Strategy

The region of watermark embedding also called ROI should cover the subject of images to the greatest extent. And, the image subject heatmap can be obtained via Grad-CAM with the dataset ImageNet [13]. The heatmap is quantized based on jet color; that is, for some regions, the redder, the closer to the subject. Hence, we can get the information on ROI by only processing the R channel of the subject localization heatmap. By binarizing the R channel, the connected region that approximates the main protected body of the image is obtained, shown in Figures 2 (c1), (c2), and (c3). Choose the connected region with the maximum area as the selected ROI processing region. When there is more than one salient subject in the image, the maximum ROI means more important and needs primary protect. To get the ROI for watermark embedding, we should obtain the watermark size and at least one coordinate in the cover image for embedding. Also, we can get the geometric center coordinate xo,yo of the connected area meanwhile. For cover images from ImageNet [13], we do not specify the watermark size because of themselves varying in size. Considering a cover image I with the size of H×W, we can obtain the watermark size by(3)h=w=MinHr⋅2l⋅2l,Wr⋅2l⋅2l,r=2,3,4,where h×w is the watermark size, r is the ratio of watermark size respect to cover images, l is the divided block size of ROI for watermark embedding, and ⋅ is the ceiling operator. For convenience, we use the top-left coordinate of watermark embedding in cover images calculated by(4)xw=xo−h2,yw=yo−w2,with the watermark size h×w and the coordinate xw,yw, and the novel ROI for watermark embedding is denoted by(5)ROI=i,j|xw≤i≤xw+h−1,yw≤j≤yw+w−1,where i,j is the position in the cover image I. Figures 2 (d1), (d2), and (d3) show the ROI in the heatmap with r=2, which can cover the subject of images.


### 3.2. Watermark Embedding

In this section, the watermark embedding strategy will be discussed in detail. Figure4 shows the framework of watermark embedding. With the embedding information, the resized watermark is embedded into the ROI of the image. The bit planes of the original image need to be substituted with corresponding watermark values for visibility. To obtain the tradeoff between the watermark visibility and the watermarked image quality, the enhanced JND model is utilized. Just noticeable difference (JND) usually means the limit threshold of the human visual system (HVS). And, HVS can only notice the image content change which is larger than the JND threshold. There are many efficient JND models proposed in the past [12, 14, 15]. The enhanced JND model with pattern complexity proposed by Wu et al. [12] is utilized for watermark embedding in this paper. Through the JND model, we can obtain the threshold of each pixel in the image. Figure 5 shows the cover image and its JND map for better understanding; more details of this JND model can be seen in [12]. And, to improve the security of this scheme, we use the logistic mapping to generate pseudorandom sequences with the secret key for watermark embedding. It is defined as(6)xn+1=λxn1−xn,where n is the iteration times of chaotic sequences and λ is the control parameter of the logistic mapping system.Figure 4
Framework of watermark embedding strategy.Figure 5
JND demonstration: (a) image wood rabbit and (b) JND map.
(a)(b)For anyx0∈0,1, the logistic mapping has chaotic characteristics when 3.99<λ<4.00[16, 17]. With the natural sequence Sn as input, a pseudorandom sequence Sr can be obtained using (6). Then, the pseudorandom sequence of 0,1 for watermark embedding is generated by(7)Sw=modSr,2.For instance, withx0=0.4,λ=3.991, a natural sequence whose length value is 9, Sn=0,1,2,3,4,5,6,7,8, and Sr=5,6,2,0,7,3,1,8,4 can be obtained using (6), and finally the pseudorandom sequence for embedding Sw=1,0,0,0,1,1,1,0,0 can be generated with (7).Considering a binary watermarkW with the size of h×w, which is equal to ROI, we divide W and ROI into nonoverlapping blocks in the same way, and both of them are corresponding to each other denoted by(8)ΩW=Wk|k=1,2,…,hl⋅wl,ΩR=Rk|k=1,2,…,hl⋅wl,and then we classify watermark blocks into two types as black block and white block using(9)t=1,ifφWk≥l2,0,else,where t means the type of binary watermark block, which refers to white block if the value is 1 and means black block if the value is 0. φWk=∑pl∑qlWkp,q, where Wkp,q is the watermark value at p,q of this block which is 0 or 1 in a binary watermark. The embedding of the visible watermark is essentially the change of the pixel value of the cover image according to the watermark bits. Given a pixel pi,j at i,j in the original cover image I, we can obtain the changed pixel p˜i,jθ after embedding the watermark bit value δx,y=0,1 into pi,j by(10)p˜i,jθ=pi,j−bi,jθ⋅2θ+δx,y⋅2θ,ifdi,j=1,pi,j−bi,jθ−1⋅2θ−1+δx,y⋅2θ−1,ifdi,j=0,where bi,jθ=modpi,j/2θ,2,θ=1,2,3,…,7,θ is the watermark embedding position of original pixel bit planes, di,j is the pseudorandom 0,1 generated from logistic mapping, and ⋅ is the flooring operator. Noticed that the coordinate in watermark W, x,y has a relationship with the position of the cover image I, i,j as follows:(11)x=i−xw+1,y=j−yw+1.Considering thek-th ROI block Rk, its visual distortion caused by watermark bit embedded can be calculated by(12)DRkθ=1l×l∑pi,j∈Rkp˜i,jθ−pi,j2.For a binary watermark, only a level needs to be visible usually, which depends on the owner’s choice. The white region of the binary watermark using in this paper needs to be visible, while the black region should be invisible. The selected position of bit planes for watermark embedding can affect both watermark visibility and the watermarked image quality. To balance these two conflicting factors, we operate two types of blocks of watermark separately. For a watermark blockWk embedding into a corresponding cover image block Rk, the optimal embedding position is obtained by(13)OP1,ift=1,OP2,ift=0,where(14)OP1:minimizeDRkθsubject to∑pi,j∈Rkgp˜i,jθ−pi,jαTJNDpi,j≥β⋅l⋅l,θ=1,2,3,…,7,OP2:minimizeDRkθgx=0,ifx≤1,1,ifx≥10,log10x,else.In (13), TJNDpi,j is the JND threshold at i,j in the cover image and αandβ are tuning parameters to adjust the visibility of watermark. Simply speaking, the process of watermark embedding is to let the black block embedded with the least distortion while the white block is embedded with the least distortion but meeting the condition of visibility first. Also, the optimal watermark embedding positions set Θ=θk|k=1,2,…,h/l⋅w/l can be constructed using (13). For reversibility, the optimal embedding positions θk and its corresponding pixel bit value bi,jθk need to be saved as a part of recovery information. Through the operations of watermark blocks embedding into corresponding cover image blocks, we can obtain the watermarked image Iw with the tradeoff between the watermark visibility and the marked image quality. The recovery information D which should be reversibly hidden in Iw consists of four parts as follows:(1)
The top-left coordinatexw,yw of watermark embedding which needs log2H+log2W bits(2)
The size of the watermarkh×w which needs log2H+log2W bits(3)
The optimal watermark embedding positionsΘ=θk|k=1,2,…,h/l⋅w/l which need 3⋅h/l⋅w/l bits(4)
The corresponding substituted pixel bitsΒ=bi,jθk|i,j∈ΩR,θk∈Θ of the embedding positions which need h×w bits


### 3.3. Reversible Data Hiding

For the reversibility of this scheme, the pairwise prediction-error expansion (PEE) for efficient reversible data hiding proposed by Ou et al. [18] is utilized in this paper. Compared with other reversible data hiding (RDH) schemes based on PEE [19–21], Ou et al. [18] consider the correlations among prediction-errors and use the sequence and the resulting 2D prediction-error histogram to make the RDH scheme more efficient.Noticed that, the size of the total bits of the recovery information is2⋅log2H+2⋅log2W+3⋅h/l⋅w/l+h×w, which the selection positions and the original bit values are the most. Hence, before RDH, to reduce the size of recovery information, the simple arithmetic coding is utilized to encode the positions and the original bits; as a result, the encoded recovery information De is obtained. Next, we use the EXCLUSIVE-OR operator to encrypt the encoded recovery information with the pseudorandom sequence generated by (6) and (7). Finally, using RDH to reversibly hide the information into a watermarked image Iw, the reversible watermarked image Iw′ is obtained. Details of the data hiding and extraction procedures can be seen in [18].


### 3.4. Watermark Extraction and Image Recovery

The original cover image can be losslessly recovered by removing the watermark with the correct keys. In addition, the binary watermark can also be extracted perfectly through the process. The procedures of watermark extraction and image recovery can be simply described as follows.Step 1.
Extract the encrypted encoded recovery information from the reversible watermarked imageIw′ via the RDH algorithm [18], and also the watermarked image Iw is obtained meanwhile.Step 2.
Decrypt the encoded informationDe with the correct encryption key.Step 3.
Obtain the recovery informationD including the watermark embedding coordinate xw,yw, the watermark size h×w, the selected positions Θ, and the original bit values Β by decoding the encoded information De.Step 4.
Recover the original image and extract the binary watermark losslessly with the recovery informationD and watermark embedding key.
Using the embedding coordinatexw,yw and the watermark size h×w, we can restore the watermark embedding region in the watermarked image Iw. Given the pixel p˜i,j at i,j in watermarked image Iw, the optimal embedding positions θk, and the corresponding original bit values bi,jθk , we can recover the original image and extract the binary watermark using (15) and (16) which denotes IR and WE, respectively.(15)pi,jθk=p˜i,j+bi,jθk⋅2θk−ωx,y⋅2θk,ifdi,j=1,p˜i,j+bi,jθk−1⋅2θk−1−ωx,y⋅2θk−1,ifdi,j=0,(16)ωx,y=modp˜i,j2θk,2,ifdi,j=1,modp˜i,j2θk−1,2,ifdi,j=0,where di,j is the pseudorandom number of {0,1} generated by the watermark embedding key using (6) and (7).


## 3.1. The Novel ROI Selection Strategy

The region of watermark embedding also called ROI should cover the subject of images to the greatest extent. And, the image subject heatmap can be obtained via Grad-CAM with the dataset ImageNet [13]. The heatmap is quantized based on jet color; that is, for some regions, the redder, the closer to the subject. Hence, we can get the information on ROI by only processing the R channel of the subject localization heatmap. By binarizing the R channel, the connected region that approximates the main protected body of the image is obtained, shown in Figures 2 (c1), (c2), and (c3). Choose the connected region with the maximum area as the selected ROI processing region. When there is more than one salient subject in the image, the maximum ROI means more important and needs primary protect. To get the ROI for watermark embedding, we should obtain the watermark size and at least one coordinate in the cover image for embedding. Also, we can get the geometric center coordinate xo,yo of the connected area meanwhile. For cover images from ImageNet [13], we do not specify the watermark size because of themselves varying in size. Considering a cover image I with the size of H×W, we can obtain the watermark size by(3)h=w=MinHr⋅2l⋅2l,Wr⋅2l⋅2l,r=2,3,4,where h×w is the watermark size, r is the ratio of watermark size respect to cover images, l is the divided block size of ROI for watermark embedding, and ⋅ is the ceiling operator. For convenience, we use the top-left coordinate of watermark embedding in cover images calculated by(4)xw=xo−h2,yw=yo−w2,with the watermark size h×w and the coordinate xw,yw, and the novel ROI for watermark embedding is denoted by(5)ROI=i,j|xw≤i≤xw+h−1,yw≤j≤yw+w−1,where i,j is the position in the cover image I. Figures 2 (d1), (d2), and (d3) show the ROI in the heatmap with r=2, which can cover the subject of images.


## 3.2. Watermark Embedding

In this section, the watermark embedding strategy will be discussed in detail. Figure4 shows the framework of watermark embedding. With the embedding information, the resized watermark is embedded into the ROI of the image. The bit planes of the original image need to be substituted with corresponding watermark values for visibility. To obtain the tradeoff between the watermark visibility and the watermarked image quality, the enhanced JND model is utilized. Just noticeable difference (JND) usually means the limit threshold of the human visual system (HVS). And, HVS can only notice the image content change which is larger than the JND threshold. There are many efficient JND models proposed in the past [12, 14, 15]. The enhanced JND model with pattern complexity proposed by Wu et al. [12] is utilized for watermark embedding in this paper. Through the JND model, we can obtain the threshold of each pixel in the image. Figure 5 shows the cover image and its JND map for better understanding; more details of this JND model can be seen in [12]. And, to improve the security of this scheme, we use the logistic mapping to generate pseudorandom sequences with the secret key for watermark embedding. It is defined as(6)xn+1=λxn1−xn,where n is the iteration times of chaotic sequences and λ is the control parameter of the logistic mapping system.Figure 4
Framework of watermark embedding strategy.Figure 5
JND demonstration: (a) image wood rabbit and (b) JND map.
(a)(b)For anyx0∈0,1, the logistic mapping has chaotic characteristics when 3.99<λ<4.00[16, 17]. With the natural sequence Sn as input, a pseudorandom sequence Sr can be obtained using (6). Then, the pseudorandom sequence of 0,1 for watermark embedding is generated by(7)Sw=modSr,2.For instance, withx0=0.4,λ=3.991, a natural sequence whose length value is 9, Sn=0,1,2,3,4,5,6,7,8, and Sr=5,6,2,0,7,3,1,8,4 can be obtained using (6), and finally the pseudorandom sequence for embedding Sw=1,0,0,0,1,1,1,0,0 can be generated with (7).Considering a binary watermarkW with the size of h×w, which is equal to ROI, we divide W and ROI into nonoverlapping blocks in the same way, and both of them are corresponding to each other denoted by(8)ΩW=Wk|k=1,2,…,hl⋅wl,ΩR=Rk|k=1,2,…,hl⋅wl,and then we classify watermark blocks into two types as black block and white block using(9)t=1,ifφWk≥l2,0,else,where t means the type of binary watermark block, which refers to white block if the value is 1 and means black block if the value is 0. φWk=∑pl∑qlWkp,q, where Wkp,q is the watermark value at p,q of this block which is 0 or 1 in a binary watermark. The embedding of the visible watermark is essentially the change of the pixel value of the cover image according to the watermark bits. Given a pixel pi,j at i,j in the original cover image I, we can obtain the changed pixel p˜i,jθ after embedding the watermark bit value δx,y=0,1 into pi,j by(10)p˜i,jθ=pi,j−bi,jθ⋅2θ+δx,y⋅2θ,ifdi,j=1,pi,j−bi,jθ−1⋅2θ−1+δx,y⋅2θ−1,ifdi,j=0,where bi,jθ=modpi,j/2θ,2,θ=1,2,3,…,7,θ is the watermark embedding position of original pixel bit planes, di,j is the pseudorandom 0,1 generated from logistic mapping, and ⋅ is the flooring operator. Noticed that the coordinate in watermark W, x,y has a relationship with the position of the cover image I, i,j as follows:(11)x=i−xw+1,y=j−yw+1.Considering thek-th ROI block Rk, its visual distortion caused by watermark bit embedded can be calculated by(12)DRkθ=1l×l∑pi,j∈Rkp˜i,jθ−pi,j2.For a binary watermark, only a level needs to be visible usually, which depends on the owner’s choice. The white region of the binary watermark using in this paper needs to be visible, while the black region should be invisible. The selected position of bit planes for watermark embedding can affect both watermark visibility and the watermarked image quality. To balance these two conflicting factors, we operate two types of blocks of watermark separately. For a watermark blockWk embedding into a corresponding cover image block Rk, the optimal embedding position is obtained by(13)OP1,ift=1,OP2,ift=0,where(14)OP1:minimizeDRkθsubject to∑pi,j∈Rkgp˜i,jθ−pi,jαTJNDpi,j≥β⋅l⋅l,θ=1,2,3,…,7,OP2:minimizeDRkθgx=0,ifx≤1,1,ifx≥10,log10x,else.In (13), TJNDpi,j is the JND threshold at i,j in the cover image and αandβ are tuning parameters to adjust the visibility of watermark. Simply speaking, the process of watermark embedding is to let the black block embedded with the least distortion while the white block is embedded with the least distortion but meeting the condition of visibility first. Also, the optimal watermark embedding positions set Θ=θk|k=1,2,…,h/l⋅w/l can be constructed using (13). For reversibility, the optimal embedding positions θk and its corresponding pixel bit value bi,jθk need to be saved as a part of recovery information. Through the operations of watermark blocks embedding into corresponding cover image blocks, we can obtain the watermarked image Iw with the tradeoff between the watermark visibility and the marked image quality. The recovery information D which should be reversibly hidden in Iw consists of four parts as follows:(1)
The top-left coordinatexw,yw of watermark embedding which needs log2H+log2W bits(2)
The size of the watermarkh×w which needs log2H+log2W bits(3)
The optimal watermark embedding positionsΘ=θk|k=1,2,…,h/l⋅w/l which need 3⋅h/l⋅w/l bits(4)
The corresponding substituted pixel bitsΒ=bi,jθk|i,j∈ΩR,θk∈Θ of the embedding positions which need h×w bits


## 3.3. Reversible Data Hiding

For the reversibility of this scheme, the pairwise prediction-error expansion (PEE) for efficient reversible data hiding proposed by Ou et al. [18] is utilized in this paper. Compared with other reversible data hiding (RDH) schemes based on PEE [19–21], Ou et al. [18] consider the correlations among prediction-errors and use the sequence and the resulting 2D prediction-error histogram to make the RDH scheme more efficient.Noticed that, the size of the total bits of the recovery information is2⋅log2H+2⋅log2W+3⋅h/l⋅w/l+h×w, which the selection positions and the original bit values are the most. Hence, before RDH, to reduce the size of recovery information, the simple arithmetic coding is utilized to encode the positions and the original bits; as a result, the encoded recovery information De is obtained. Next, we use the EXCLUSIVE-OR operator to encrypt the encoded recovery information with the pseudorandom sequence generated by (6) and (7). Finally, using RDH to reversibly hide the information into a watermarked image Iw, the reversible watermarked image Iw′ is obtained. Details of the data hiding and extraction procedures can be seen in [18].


## 3.4. Watermark Extraction and Image Recovery

The original cover image can be losslessly recovered by removing the watermark with the correct keys. In addition, the binary watermark can also be extracted perfectly through the process. The procedures of watermark extraction and image recovery can be simply described as follows.Step 1.
Extract the encrypted encoded recovery information from the reversible watermarked imageIw′ via the RDH algorithm [18], and also the watermarked image Iw is obtained meanwhile.Step 2.
Decrypt the encoded informationDe with the correct encryption key.Step 3.
Obtain the recovery informationD including the watermark embedding coordinate xw,yw, the watermark size h×w, the selected positions Θ, and the original bit values Β by decoding the encoded information De.Step 4.
Recover the original image and extract the binary watermark losslessly with the recovery informationD and watermark embedding key.
Using the embedding coordinatexw,yw and the watermark size h×w, we can restore the watermark embedding region in the watermarked image Iw. Given the pixel p˜i,j at i,j in watermarked image Iw, the optimal embedding positions θk, and the corresponding original bit values bi,jθk , we can recover the original image and extract the binary watermark using (15) and (16) which denotes IR and WE, respectively.(15)pi,jθk=p˜i,j+bi,jθk⋅2θk−ωx,y⋅2θk,ifdi,j=1,p˜i,j+bi,jθk−1⋅2θk−1−ωx,y⋅2θk−1,ifdi,j=0,(16)ωx,y=modp˜i,j2θk,2,ifdi,j=1,modp˜i,j2θk−1,2,ifdi,j=0,where di,j is the pseudorandom number of {0,1} generated by the watermark embedding key using (6) and (7).


## 4. Experimental Results and Performance Analysis

In this section, we conduct a series of analyses and take the schemes proposed by Hu and Jeon [5], Chen et al. [7], and Mohammad et al. [8] for comparison to demonstrate the effectiveness of our proposed scheme. In the experiments of our proposed scheme, all the subject localization heatmaps are generated via Grad-CAM with off-the-shelf pretrained VGG-16 [22], and the dataset for training and testing is ImageNet [17]. Also, the cover images in this scheme are from ImageNet [13] which are RGB color type; to be simple to describe the algorithm, we operate the grayscale images of themselves first, and then the same operation is used to each channel of RGB separately for color images. There are six aspects to estimate the proposed scheme: the novel ROI selection performance, the watermark visibility analysis with tuning parameters αandβ, the reversible watermarked images quality estimation by the metrics of PSNR and SSIM [23], watermark robustness, original image recovery and watermark extraction with the security discussion, and last, the performance on color images.As shown in Figures6 and 7, the binary watermarks [24], deer and horse, and the cover images, wood rabbit, tabby, and partridge, are used in the experiments. A detailed description of the cover images and the watermarks is given in Table 1. And, the watermark can be resized for embedding according to the size of cover images H×W and the ratio r using (3). Besides, the nonoverlapping block size l×l is 3×3.Figure 6
Watermark images used in the experiments: (a) deer and (b) horse.
(a)(b)Figure 7
Cover images used in the experiments: (a1) wood rabbit, (a2) tabby, (a3) partridge, (b1) grayscale wood rabbit, (b2) grayscale tabby, and (b3) grayscale partridge.Table 1
Description of test images in detail.
ImageTypeResolutionWood rabbitColor675 × 477TabbyColor500 × 357PartridgeColor750 × 500DeerBinary693 × 703HorseBinary315 × 266


### 4.1. ROI Selection Performance

As discussed in Section3.1, the ROI for visible watermark embedding should cover the image subject to the greatest extent. With the novel ROI selection strategy based on Grad-CAM, the watermark embedding information can be obtained. Table 2 shows the ROI information including the watermark size h×w and embedding coordinate xw,yw under different proportion r with the method proposed by Qi et al. [9] and ours. To intuitively perform our selected ROI which is the subject of the image, Figure 8 shows the watermarked images with different proportion values; besides, the parameters α=1andβ=1 here. From Figure 8, we can see that the watermark can be embedded into the main body of the images whatever the size is. Figure 9 shows the ROI selection with different methods. For Qi et al. [9], the ROI is the region which not contains abundant main information of images that can be easily cropped and tampered for malicious use so that the effect of watermark for protecting the copyright is degraded. On the contrary, the ROI for watermarking by ours approximates that the image subject can protect the main body information and copyright better.Table 2
Watermark embedding information for cover images.
Cover imageProportionrWatermark sizeCoordinateProposedQi et al. [9]Wood rabbit2240 × 240(70,191)(201,227)3162 × 162(109,230)(129,227)4120 × 120(130,251)(121,263)Tabby2180 × 180(91,222)(141,319)3120 × 120(121,252)(183,379)490 × 90(136,267)(136,267)Partridge2252 × 252(90,272)(247,3)3168 × 168(132,314)(309,57)4126 × 126(153,335)(329,99)Figure 8
Embedding the watermark deer into ROI of cover image wood rabbit with different proportionr: (a) original cover image, (b) (r) = 2, (c) (r) = 3, and (d) (r) = 4.
(a)(b)(c)(d)Figure 9
ROI selection with different strategies: (a1), (a2), and (a3) Qi et al. [9] with (r) = 2, 3, and 4, respectively; (b1), (b2), and (b3) proposed strategy with (r) = 2, 3, and 4, respectively.


### 4.2. Parameters of Watermark Visibility Analysis

The parametersαandβ in (13) can adjust the watermark visibility and the watermarked image quality. From the definition of (13), the parameter β∈0,1.0, and in our scheme, we set α∈0.5,1.0. To understand the tuning parameters how to affect watermark visibility and marked image quality, we make the line chart of α∈0.5,1.0andβ∈0,0.5 with step 0.1 each, shown in Figure 10. According to Figure 10, the PSNR values of the reversible watermarked image Iw′ are getting lower and lower with the increase in tuning parameters α and β. The tuning parameters can control the bit position selection, with the larger αandβ, the higher position is selected, and result in the increase in watermark visibility, and the decrease in watermarked image quality. Also, these two factors for tuning, parameter β is coarse, while parameter α is precise. With the change of αandβ , we can adjust the watermark visibility and the watermarked image quality to obtain the tradeoff between these two factors. Figure 11 shows some reversible watermarked image with all α=1 and different values of β.Figure 10
PSNR values of reversible watermarked image wood rabbit with watermark deer embedded with different tuning parameter combinations: (a) PSNR values versusα and (b) PSNR values versus β.
(a)(b)Figure 11
Reversible watermarked image wood rabbit embedded watermark deer with different parameterα and β, all α = 1: (a) original cover image, (b) β = 0.3 (PSNR = 34.6251 dB), (c) β = 0.5 (PSNR = 25.6902 dB), and (d) β = 1.0 (PSNR = 24.0495 dB).
(a)(b)(c)(d)


### 4.3. Reversible Watermarked Image Quality

In Section4.2, we have discussed the effect of parameters αandβ on watermark visibility. From reversible watermarked images with different combinations of parameters αandβ, we choose the one α=0.6andβ=0.5 as the representative of our proposed scheme in the experiments. Figure 12 shows the reversible watermarked image with α=0.6andβ=0.5 comparing to other schemes by Hu and Jeon [5], Chen et al. [7], and Mohammad et al. [8]. The watermark visibility among all the images in the scheme proposed by Hu and Jeon [5] is the most obvious by modifying one significant bit plane of the pixels. However, the distortion of the watermarked image is too much, and the details of the cover image are almost covered by the watermark. The result of Mohammad et al. [8] does not show the watermark background, but it still has a lot of distortion. Noticed that, the reversible watermarked images of our scheme have the same level as Chen et al. [7] on watermark visibility in general. However, for reversible watermarked image quality, the detail of Chen et al. [7] performs worse than ours, especially when the region of watermark embedding is rough instead of smooth, which can be seen in Figure 13, zooming in the region of watermark embedding. Also, to objectively assess the reversible watermarked image quality, the values of PSNR and SSIM are shown in Tables 3 and 4. Overall, almost all the values of these two metrics of our proposed scheme are higher than others. For cover image wood rabbit, the values of PSNR are much larger than those of others, e.g., when r = 2, with watermark horse embedded, the value of PSNR is 18.5095 dB, 24.3761 dB, 21.919 dB, and 25.3271 dB corresponding to the schemes proposed by Hu and Jeon [5], Chen et al. [7], Mohammad et al. [8], and ours. And, the larger r is, the higher difference of PSNR and SSIM is. For cover image tabby with the watermark horse embedded, though the value of PSNR of our scheme, 23.9950 dB, is not so much higher than that of Chen et al. [7], 23.9406 dB, the value of SSIM is much larger than that of Chen et al. [7] and Mohammad et al. [8], where ours is 0.9402 while Chen et al. [7] is 0.8702 and Mohammad et al. [8] is 0.8787. For cover image partridge, though the PSNR value is less than the value of Chen et al. [7] when r = 2, the SSIM value is much higher. And, with the smaller r, the PSNR and SSIM values are both higher than others.Figure 12
Reversible watermarked images with watermark horse embedded using different reversible visible watermarking schemes: (a1), (a2), and (a3) Hu and Jeon [5]; (b1), (b2), and (b3) Chen et al. [7]; (c1), (c2), and (c3) Mohammad et al. [8]; (d1), (d2), and (d3) proposed scheme with α = 0.6 and β = 0.5.Figure 13
Zoom in the reversible watermarked image tabby with watermark horse embedded using different reversible visible watermarking schemes: (a) Hu and Jeon [5], (b) Chen et al. [7], (c) Mohammad et al. [8], and (d) proposed scheme with α = 0.6 and β = 0.5.
(a)(b)(c)(d)Table 3
PSNR and SSIM values of reversible watermarked images with watermark deer andα = 0.6 and β = 0.5
Cover imageProportionPSNRSSIMrHu and Jeon [5]Chen et al. [7]Mohammad et al. [8]ProposedHu and Jeon [5]Chen et al. [7]Mohammad et al. [8]ProposedWood rabbit218.426624.336322.176926.02360.83450.88770.89110.9537321.712326.941025.040031.34280.92440.94080.92440.9833424.356029.174927.023834.82970.95910.96570.93740.9919Tabby218.979124.019021.891124.36740.83160.86810.88120.9414322.407927.733925.125628.27030.92260.93790.91200.9739425.084130.317526.998931.09780.95630.96450.92460.9858Partridge219.349225.363923.016324.36710.84380.90150.91440.9465322.540527.480926.379728.93740.93100.94390.94560.9783424.812929.175928.629832.06140.96110.96390.95710.9883Table 4
PSNR and SSIM values of reversible watermarked images with watermark horse andα = 0.6 and β = 0.5
Cover imageProportionPSNRSSIMrHu and Jeon [5]Chen et al. [7]Mohammad et al. [8]ProposedHu and Jeon [5]Chen et al. [7]Mohammad et al. [8]ProposedWood rabbit218.509524.376121.919025.32710.83420.88690.88800.9480321.873126.785624.947430.48800.92520.94100.92410.9811424.457429.203626.898234.34540.95870.96660.93760.9911Tabby219.179923.940621.576723.99500.83210.87020.87870.9402322.591827.553524.644827.84480.92240.93840.91120.9731424.972230.068826.872130.85240.95500.96490.92540.9849Partridge219.478025.360122.841624.24410.84370.89920.91110.9434322.636427.427226.099628.42930.93100.94400.94530.9763424.906529.183128.458031.68010.96100.96390.95740.9872Also, more images with their grayscale are tested in this experiment, showing in Figure14. The 100 cover images are randomly selected from the dataset ImageNet with different categories, like lion, goldfinch, and cicada, and the watermark is deer. From Figure 14, most images’ PSNR and SSIM values are higher than other schemes, especially the SSIM. And, for the averages of PSNR and SSIM, the proposed scheme is 26.9895 dB, Hu and Jeon [5] is 18.3870 dB, 24.4509 dB is from Chen et al. [7], and Mohammad et al. [8] is 21.6282 dB. Besides, 0.9520, 0.8232, 0.8746, and 0.8771 are the SSIM averages, respectively. The results demonstrate the stability of our proposed scheme.Figure 14
PSNR and SSIM values within 100 randomly grayscale cover images and watermark deer using different reversible visible watermark schemes.Therefore, from Figures12–14 and Tables 3 and 4, we can conclude that our proposed scheme performs more excellent than other schemes in most cases on watermark visibility and watermarked image quality.


### 4.4. Watermark Robustness

The reversible visible watermark is a sign of the copyright. Watermark robustness is an important property for the reversible visible watermark. In order to test the watermark robustness, some common attacks are used in the experiment, like histogram equalization, mean filtering, Laplacian sharpening, noise addition, and JPEG compression. The results are shown in Figure15 by taking the cover image wood rabbit embedded horse as an example. From Figure 15, the visible watermark can be recognized clearly and the watermark can survive suffered after attacks. So, we can conclude that the proposed visible watermarking algorithm is robust against common signal processing attacks.Figure 15
Watermark robustness against common attacks: (a) without attacks, (b) histogram equalization, (c) 3×3 mean filtering, (d) Laplacian sharpening, (e) Gaussian noise addition, and (f) JPEG compression with QF = 50.
(a)(b)(c)(d)(e)(f)


### 4.5. Image Recovery and Watermark Extraction with Security Discussion

The original image recovery and the watermark extraction can be achieved losslessly and perfectly in our proposed scheme with the RDH algorithm proposed by Ou et al. [18]. Three groups of image recovery and watermark extraction are shown in Figure 16; besides, the recovered images and extracted watermarks are the same as the original cover images and watermarks.Figure 16
Reversible watermarked image recovery and watermark extraction: (a1), (a2), and (a3) reversible watermarked image withα = 0.6 and β = 0.5; (b1), (b2), and (b3) recovered original image; (c1), (c2), and (c3) extracted watermark.In our proposed scheme, security including watermark embedding security and recovery information security is taken into account. During the process of watermark embedding, the pseudorandom number of{0,1} generated by the logistic mapping is utilized. Without the embedding secret key, even though the selected bit positions Θ are right, you cannot figure it out that the changed position θk is the current bit plane or the next. Moreover, the encrypted encoded recovery information cannot be decrypted correctly without the right encryption key. Hence, Θ , B, and other information cannot be correct, not mention recovering the image and extracting the watermark. In this experiment, the secret key including x0 and λ are uniform in these two security protections, but the generated pseudo-sequences are different because of their different length, which is l×l and the other depending on the length of encoded information. Figure 17 shows the results of the recovered image and extracted watermark with the correct and wrong secret key, respectively.Figure 17
Security performance on reversible watermarked image recovery and watermark extraction: (a1) and (a2) original cover image and resized watermark; (b1) and (b2) recovered image and extracted watermark with the correct key; (c1) and (c2) recovered image and extracted watermark with the wrong key.


### 4.6. Performance on Color Images

The watermark can be embedded into each channel of RGB to obtain the color reversible watermarked image. As mentioned before, to simply describe the algorithm, the grayscale images are used in the experiments above. In this subsection, the color reversible watermarked images performance will be discussed. And, the adopted measurement PSNRc [10] will be used to assess the color reversible watermarked image quality which is defined as(17)PSNRc=10log102552MSE,MSE=13⋅H⋅W∑k=13∑i=1H∑j=1Wpi,j,k−p˜i,j,k2,where pi,j,k is the pixel value at i,j in the k-th color channel of the original cover image and p˜i,j,k is the corresponding reversible watermarked pixel value. Figure 18 shows the results of different reversible visible watermarking schemes. And, for reversible watermarked image wood rabbit embedded with the watermark deer, the PSNRc values of Hu and Jeon [5], Chen et al. [7], Mohammad et al. [8] and our proposed scheme are 18.4826 dB, 24.3150 dB, 21.9525 dB, and 26.2173 dB. For tabby with deer, the PSNRc values are 19.0718 dB, 23.9677 dB, 21.5469 dB, and 25.2189 dB. Among all these PSNRc values for these two cover images, the scheme proposed by ours is the largest. Although for image partridge with deer, the PSNRc of Chen et al. [7] which is 25.2949 dB is larger than ours, 25.0240 dB, and the difference is only 0.2709 dB. According to the analysis of the grayscale image in Section 4.3, the color reversible watermarked image quality performing best can be explained using the following equations:(18)Dmin=∑iH∑jWR′i,j−Ri,j2+G′i,j−Gi,j2+B′i,j−Bi,j2min,(19)Cmin=∑iH∑jWC′i,j−Ci,j2min,where Ci,j is the original pixel value at i,j ,C′i,j is the corresponding changed pixel in a channel, and C means the channel of RGB. In other words, the minimum distortion of color images, which means the shortest Euclidean distance in RGB space, is equivalent to the shortest distance or the least distortion in each channel. Hence, we can obtain the least distorted reversible watermarked color images by processing each RGB channel as the grayscale separately.Figure 18
Reversible color watermarked images with watermark deer embedded using different reversible visible watermarking schemes: (a1), (a2), and (a3) Hu and Jeon [5] (18.4826 dB, 19.0718 dB, and 19.6423 dB); (b1), (b2), and (b3) Chen et al. [7] (24.3150 dB, 23.9677 dB, and 23.2949 dB); (c1), (c2), and (c3) Mohammad et al. [8] (21.9525 dB, 21.5649 dB, and 22.1987 dB); (d1), (d2), and (d3) proposed scheme with α = 0.6 and β = 0.5 (26.2173 dB, 25.2189 dB, and 25.0240 dB).Noticed that, in Figure18 (b1), for the result of Chen et al. [7], the embedded watermark is kind of ‘purple’, which leads to great color distortion. To make the watermark visible, the scheme proposed by Chen et al. [7] sets a threshold for watermark embedding, which causes the lighter grayscale image to have the shadow watermark embedded while the darker grayscale image has the brighter one. And in the experiment, the channel G of the color image has the shadow watermark embedded while R and B channels are contrary, shown in Figure 19. For tabby and partridge embedded watermark deer, all channels have the bright watermarks, but as discussed in subsection C, the details of each watermarked image channel perform are not good enough, resulting in poor performance of the final color image details, shown in Figures 18 (b2) and (b3).Figure 19
RGB channel watermarked image with watermark deer embedded of Chen et al. [7]: (a) R channel, (b) G channel, and (c) B channel.
(a)(b)(c)


## 4.1. ROI Selection Performance

As discussed in Section3.1, the ROI for visible watermark embedding should cover the image subject to the greatest extent. With the novel ROI selection strategy based on Grad-CAM, the watermark embedding information can be obtained. Table 2 shows the ROI information including the watermark size h×w and embedding coordinate xw,yw under different proportion r with the method proposed by Qi et al. [9] and ours. To intuitively perform our selected ROI which is the subject of the image, Figure 8 shows the watermarked images with different proportion values; besides, the parameters α=1andβ=1 here. From Figure 8, we can see that the watermark can be embedded into the main body of the images whatever the size is. Figure 9 shows the ROI selection with different methods. For Qi et al. [9], the ROI is the region which not contains abundant main information of images that can be easily cropped and tampered for malicious use so that the effect of watermark for protecting the copyright is degraded. On the contrary, the ROI for watermarking by ours approximates that the image subject can protect the main body information and copyright better.Table 2
Watermark embedding information for cover images.
Cover imageProportionrWatermark sizeCoordinateProposedQi et al. [9]Wood rabbit2240 × 240(70,191)(201,227)3162 × 162(109,230)(129,227)4120 × 120(130,251)(121,263)Tabby2180 × 180(91,222)(141,319)3120 × 120(121,252)(183,379)490 × 90(136,267)(136,267)Partridge2252 × 252(90,272)(247,3)3168 × 168(132,314)(309,57)4126 × 126(153,335)(329,99)Figure 8
Embedding the watermark deer into ROI of cover image wood rabbit with different proportionr: (a) original cover image, (b) (r) = 2, (c) (r) = 3, and (d) (r) = 4.
(a)(b)(c)(d)Figure 9
ROI selection with different strategies: (a1), (a2), and (a3) Qi et al. [9] with (r) = 2, 3, and 4, respectively; (b1), (b2), and (b3) proposed strategy with (r) = 2, 3, and 4, respectively.


## 4.2. Parameters of Watermark Visibility Analysis

The parametersαandβ in (13) can adjust the watermark visibility and the watermarked image quality. From the definition of (13), the parameter β∈0,1.0, and in our scheme, we set α∈0.5,1.0. To understand the tuning parameters how to affect watermark visibility and marked image quality, we make the line chart of α∈0.5,1.0andβ∈0,0.5 with step 0.1 each, shown in Figure 10. According to Figure 10, the PSNR values of the reversible watermarked image Iw′ are getting lower and lower with the increase in tuning parameters α and β. The tuning parameters can control the bit position selection, with the larger αandβ, the higher position is selected, and result in the increase in watermark visibility, and the decrease in watermarked image quality. Also, these two factors for tuning, parameter β is coarse, while parameter α is precise. With the change of αandβ , we can adjust the watermark visibility and the watermarked image quality to obtain the tradeoff between these two factors. Figure 11 shows some reversible watermarked image with all α=1 and different values of β.Figure 10
PSNR values of reversible watermarked image wood rabbit with watermark deer embedded with different tuning parameter combinations: (a) PSNR values versusα and (b) PSNR values versus β.
(a)(b)Figure 11
Reversible watermarked image wood rabbit embedded watermark deer with different parameterα and β, all α = 1: (a) original cover image, (b) β = 0.3 (PSNR = 34.6251 dB), (c) β = 0.5 (PSNR = 25.6902 dB), and (d) β = 1.0 (PSNR = 24.0495 dB).
(a)(b)(c)(d)


## 4.3. Reversible Watermarked Image Quality

In Section4.2, we have discussed the effect of parameters αandβ on watermark visibility. From reversible watermarked images with different combinations of parameters αandβ, we choose the one α=0.6andβ=0.5 as the representative of our proposed scheme in the experiments. Figure 12 shows the reversible watermarked image with α=0.6andβ=0.5 comparing to other schemes by Hu and Jeon [5], Chen et al. [7], and Mohammad et al. [8]. The watermark visibility among all the images in the scheme proposed by Hu and Jeon [5] is the most obvious by modifying one significant bit plane of the pixels. However, the distortion of the watermarked image is too much, and the details of the cover image are almost covered by the watermark. The result of Mohammad et al. [8] does not show the watermark background, but it still has a lot of distortion. Noticed that, the reversible watermarked images of our scheme have the same level as Chen et al. [7] on watermark visibility in general. However, for reversible watermarked image quality, the detail of Chen et al. [7] performs worse than ours, especially when the region of watermark embedding is rough instead of smooth, which can be seen in Figure 13, zooming in the region of watermark embedding. Also, to objectively assess the reversible watermarked image quality, the values of PSNR and SSIM are shown in Tables 3 and 4. Overall, almost all the values of these two metrics of our proposed scheme are higher than others. For cover image wood rabbit, the values of PSNR are much larger than those of others, e.g., when r = 2, with watermark horse embedded, the value of PSNR is 18.5095 dB, 24.3761 dB, 21.919 dB, and 25.3271 dB corresponding to the schemes proposed by Hu and Jeon [5], Chen et al. [7], Mohammad et al. [8], and ours. And, the larger r is, the higher difference of PSNR and SSIM is. For cover image tabby with the watermark horse embedded, though the value of PSNR of our scheme, 23.9950 dB, is not so much higher than that of Chen et al. [7], 23.9406 dB, the value of SSIM is much larger than that of Chen et al. [7] and Mohammad et al. [8], where ours is 0.9402 while Chen et al. [7] is 0.8702 and Mohammad et al. [8] is 0.8787. For cover image partridge, though the PSNR value is less than the value of Chen et al. [7] when r = 2, the SSIM value is much higher. And, with the smaller r, the PSNR and SSIM values are both higher than others.Figure 12
Reversible watermarked images with watermark horse embedded using different reversible visible watermarking schemes: (a1), (a2), and (a3) Hu and Jeon [5]; (b1), (b2), and (b3) Chen et al. [7]; (c1), (c2), and (c3) Mohammad et al. [8]; (d1), (d2), and (d3) proposed scheme with α = 0.6 and β = 0.5.Figure 13
Zoom in the reversible watermarked image tabby with watermark horse embedded using different reversible visible watermarking schemes: (a) Hu and Jeon [5], (b) Chen et al. [7], (c) Mohammad et al. [8], and (d) proposed scheme with α = 0.6 and β = 0.5.
(a)(b)(c)(d)Table 3
PSNR and SSIM values of reversible watermarked images with watermark deer andα = 0.6 and β = 0.5
Cover imageProportionPSNRSSIMrHu and Jeon [5]Chen et al. [7]Mohammad et al. [8]ProposedHu and Jeon [5]Chen et al. [7]Mohammad et al. [8]ProposedWood rabbit218.426624.336322.176926.02360.83450.88770.89110.9537321.712326.941025.040031.34280.92440.94080.92440.9833424.356029.174927.023834.82970.95910.96570.93740.9919Tabby218.979124.019021.891124.36740.83160.86810.88120.9414322.407927.733925.125628.27030.92260.93790.91200.9739425.084130.317526.998931.09780.95630.96450.92460.9858Partridge219.349225.363923.016324.36710.84380.90150.91440.9465322.540527.480926.379728.93740.93100.94390.94560.9783424.812929.175928.629832.06140.96110.96390.95710.9883Table 4
PSNR and SSIM values of reversible watermarked images with watermark horse andα = 0.6 and β = 0.5
Cover imageProportionPSNRSSIMrHu and Jeon [5]Chen et al. [7]Mohammad et al. [8]ProposedHu and Jeon [5]Chen et al. [7]Mohammad et al. [8]ProposedWood rabbit218.509524.376121.919025.32710.83420.88690.88800.9480321.873126.785624.947430.48800.92520.94100.92410.9811424.457429.203626.898234.34540.95870.96660.93760.9911Tabby219.179923.940621.576723.99500.83210.87020.87870.9402322.591827.553524.644827.84480.92240.93840.91120.9731424.972230.068826.872130.85240.95500.96490.92540.9849Partridge219.478025.360122.841624.24410.84370.89920.91110.9434322.636427.427226.099628.42930.93100.94400.94530.9763424.906529.183128.458031.68010.96100.96390.95740.9872Also, more images with their grayscale are tested in this experiment, showing in Figure14. The 100 cover images are randomly selected from the dataset ImageNet with different categories, like lion, goldfinch, and cicada, and the watermark is deer. From Figure 14, most images’ PSNR and SSIM values are higher than other schemes, especially the SSIM. And, for the averages of PSNR and SSIM, the proposed scheme is 26.9895 dB, Hu and Jeon [5] is 18.3870 dB, 24.4509 dB is from Chen et al. [7], and Mohammad et al. [8] is 21.6282 dB. Besides, 0.9520, 0.8232, 0.8746, and 0.8771 are the SSIM averages, respectively. The results demonstrate the stability of our proposed scheme.Figure 14
PSNR and SSIM values within 100 randomly grayscale cover images and watermark deer using different reversible visible watermark schemes.Therefore, from Figures12–14 and Tables 3 and 4, we can conclude that our proposed scheme performs more excellent than other schemes in most cases on watermark visibility and watermarked image quality.


## 4.4. Watermark Robustness

The reversible visible watermark is a sign of the copyright. Watermark robustness is an important property for the reversible visible watermark. In order to test the watermark robustness, some common attacks are used in the experiment, like histogram equalization, mean filtering, Laplacian sharpening, noise addition, and JPEG compression. The results are shown in Figure15 by taking the cover image wood rabbit embedded horse as an example. From Figure 15, the visible watermark can be recognized clearly and the watermark can survive suffered after attacks. So, we can conclude that the proposed visible watermarking algorithm is robust against common signal processing attacks.Figure 15
Watermark robustness against common attacks: (a) without attacks, (b) histogram equalization, (c) 3×3 mean filtering, (d) Laplacian sharpening, (e) Gaussian noise addition, and (f) JPEG compression with QF = 50.
(a)(b)(c)(d)(e)(f)


## 4.5. Image Recovery and Watermark Extraction with Security Discussion

The original image recovery and the watermark extraction can be achieved losslessly and perfectly in our proposed scheme with the RDH algorithm proposed by Ou et al. [18]. Three groups of image recovery and watermark extraction are shown in Figure 16; besides, the recovered images and extracted watermarks are the same as the original cover images and watermarks.Figure 16
Reversible watermarked image recovery and watermark extraction: (a1), (a2), and (a3) reversible watermarked image withα = 0.6 and β = 0.5; (b1), (b2), and (b3) recovered original image; (c1), (c2), and (c3) extracted watermark.In our proposed scheme, security including watermark embedding security and recovery information security is taken into account. During the process of watermark embedding, the pseudorandom number of{0,1} generated by the logistic mapping is utilized. Without the embedding secret key, even though the selected bit positions Θ are right, you cannot figure it out that the changed position θk is the current bit plane or the next. Moreover, the encrypted encoded recovery information cannot be decrypted correctly without the right encryption key. Hence, Θ , B, and other information cannot be correct, not mention recovering the image and extracting the watermark. In this experiment, the secret key including x0 and λ are uniform in these two security protections, but the generated pseudo-sequences are different because of their different length, which is l×l and the other depending on the length of encoded information. Figure 17 shows the results of the recovered image and extracted watermark with the correct and wrong secret key, respectively.Figure 17
Security performance on reversible watermarked image recovery and watermark extraction: (a1) and (a2) original cover image and resized watermark; (b1) and (b2) recovered image and extracted watermark with the correct key; (c1) and (c2) recovered image and extracted watermark with the wrong key.


## 4.6. Performance on Color Images

The watermark can be embedded into each channel of RGB to obtain the color reversible watermarked image. As mentioned before, to simply describe the algorithm, the grayscale images are used in the experiments above. In this subsection, the color reversible watermarked images performance will be discussed. And, the adopted measurement PSNRc [10] will be used to assess the color reversible watermarked image quality which is defined as(17)PSNRc=10log102552MSE,MSE=13⋅H⋅W∑k=13∑i=1H∑j=1Wpi,j,k−p˜i,j,k2,where pi,j,k is the pixel value at i,j in the k-th color channel of the original cover image and p˜i,j,k is the corresponding reversible watermarked pixel value. Figure 18 shows the results of different reversible visible watermarking schemes. And, for reversible watermarked image wood rabbit embedded with the watermark deer, the PSNRc values of Hu and Jeon [5], Chen et al. [7], Mohammad et al. [8] and our proposed scheme are 18.4826 dB, 24.3150 dB, 21.9525 dB, and 26.2173 dB. For tabby with deer, the PSNRc values are 19.0718 dB, 23.9677 dB, 21.5469 dB, and 25.2189 dB. Among all these PSNRc values for these two cover images, the scheme proposed by ours is the largest. Although for image partridge with deer, the PSNRc of Chen et al. [7] which is 25.2949 dB is larger than ours, 25.0240 dB, and the difference is only 0.2709 dB. According to the analysis of the grayscale image in Section 4.3, the color reversible watermarked image quality performing best can be explained using the following equations:(18)Dmin=∑iH∑jWR′i,j−Ri,j2+G′i,j−Gi,j2+B′i,j−Bi,j2min,(19)Cmin=∑iH∑jWC′i,j−Ci,j2min,where Ci,j is the original pixel value at i,j ,C′i,j is the corresponding changed pixel in a channel, and C means the channel of RGB. In other words, the minimum distortion of color images, which means the shortest Euclidean distance in RGB space, is equivalent to the shortest distance or the least distortion in each channel. Hence, we can obtain the least distorted reversible watermarked color images by processing each RGB channel as the grayscale separately.Figure 18
Reversible color watermarked images with watermark deer embedded using different reversible visible watermarking schemes: (a1), (a2), and (a3) Hu and Jeon [5] (18.4826 dB, 19.0718 dB, and 19.6423 dB); (b1), (b2), and (b3) Chen et al. [7] (24.3150 dB, 23.9677 dB, and 23.2949 dB); (c1), (c2), and (c3) Mohammad et al. [8] (21.9525 dB, 21.5649 dB, and 22.1987 dB); (d1), (d2), and (d3) proposed scheme with α = 0.6 and β = 0.5 (26.2173 dB, 25.2189 dB, and 25.0240 dB).Noticed that, in Figure18 (b1), for the result of Chen et al. [7], the embedded watermark is kind of ‘purple’, which leads to great color distortion. To make the watermark visible, the scheme proposed by Chen et al. [7] sets a threshold for watermark embedding, which causes the lighter grayscale image to have the shadow watermark embedded while the darker grayscale image has the brighter one. And in the experiment, the channel G of the color image has the shadow watermark embedded while R and B channels are contrary, shown in Figure 19. For tabby and partridge embedded watermark deer, all channels have the bright watermarks, but as discussed in subsection C, the details of each watermarked image channel perform are not good enough, resulting in poor performance of the final color image details, shown in Figures 18 (b2) and (b3).Figure 19
RGB channel watermarked image with watermark deer embedded of Chen et al. [7]: (a) R channel, (b) G channel, and (c) B channel.
(a)(b)(c)


## 5. Conclusions and Future Work

To embed the watermark into the subject in cover images for protecting the copyright better, the novel ROI selection strategy based on Grad-CAM is proposed in this paper. To obtain the tradeoff between the watermark visibility and the reversible watermarked image quality, the JND model is utilized for watermark embedding to select the optimal bit positions of the image pixel. And, the efficient RDH algorithm is used in this paper for reversibility, with the extracted recovery information, the original cover image can be recovered losslessly, and the embedded watermark can be extracted perfectly. According to the watermark visibility and the reversible watermarked image quality with less distortion, the proposed scheme performs much more excellent compared with other reversible visible watermarking schemes.In the proposed scheme, the visible watermark can survive against common attacks. However, image recovery and watermark extraction cannot be achieved reversibly. In the future, enhancing the watermark robustness for reversibility and designing a general metric of watermark visibility to choose the most suitable tuning parameters deserve further investigation.



---

*Source: 6652897-2021-01-25.xml*