# Combining Digital Watermarking and Fingerprinting Techniques to Identify Copyrights for Color Images

Authors: Shang-Lin Hsieh; Chun-Che Chen; Wen-Shan Shen
Journal: The Scientific World Journal
 (2014)

---

## Abstract

This paper presents a copyright identification scheme for color images that takes advantage of the complementary nature of watermarking and fingerprinting. It utilizes an authentication logo and the extracted features of the host image to generate a fingerprint, which is then stored in a database and also embedded in the host image to produce a watermarked image. When a dispute over the copyright of a suspect image occurs, the image is first processed by watermarking. If the watermark can be retrieved from the suspect image, the copyright can then be confirmed; otherwise, the watermark then serves as the fingerprint and is processed by fingerprinting. If a match in the fingerprint database is found, then the suspect image will be considered a duplicated one. Because the proposed scheme utilizes both watermarking and fingerprinting, it is more robust than those that only adopt watermarking, and it can also obtain the preliminary result more quickly than those that only utilize fingerprinting. The experimental results show that when the watermarked image suffers slight attacks, watermarking alone is enough to identify the copyright. The results also show that when the watermarked image suffers heavy attacks that render watermarking incompetent, fingerprinting can successfully identify the copyright, hence demonstrating the effectiveness of the proposed scheme.


---

## Body

## 1. Introduction

Many researchers [1–17] have been engaged in finding the solution to protecting copyrights of digital images, which may be duplicated and distributed over the Internet without the authors’ permission. Generally speaking, there are two approaches to discovering image copyright infringement. One is watermarking [1–11] and the other is fingerprinting [12–17]. The main idea of watermarking is to embed a piece of information (i.e., watermark) in the host image. If a similar watermark can be retrieved from a suspect image, it is then considered a duplicated one. On the other hand, the principle of fingerprinting is to extract unique features (i.e., fingerprints) from both the host image and the suspect one for comparison. If their fingerprints are similar, the ownership of the image can then be confirmed.There are some general considerations on the two techniques, including the processing time and robustness. In terms of processing time, watermarking is more efficient because fingerprinting needs extra time to compare the image’s fingerprint with those stored in the database. If the database is large, it will be very time consuming. On the other hand, fingerprinting is generally more robust [18, 19] because when a watermarked image suffers some image processing operations that modify the content of the image, the embedded watermark will usually be damaged or even destroyed. On the contrary, since normal image processing does not destroy the features of an image, the fingerprint of the image can therefore be preserved. In summary, fingerprinting is more robust whereas watermarking is more efficient. If the complementary natures of two approaches can be utilized properly, a robust and efficient scheme can then be developed to identify copyrights.This paper proposes a novel scheme that combines the two techniques to identify copyrights for color images. The proposed scheme generates from the image a fingerprint, which also serves as the watermark. The watermark is then embedded in the host image to produce a watermarked image. When there is a dispute over the copyright of a suspect image, the suspect image will first be processed by watermarking, which tries to retrieve the watermark from the suspect image. If the watermark is identified, the copyright is confirmed at this stage; otherwise, the image will then be processed by fingerprinting, which utilizes the retrieved watermark as the fingerprint and compares it with those stored in the database. If a match is found, then the suspect image will be considered a duplicated one.


## 2. Related Background

The proposed scheme utilizes a special technique called image secret sharing (ISS), whose details can be found in the paper [20] we published in 2008. The following briefly describes the main idea of the technique utilized by the proposed scheme.The ISS generates a share image from two images. In the proposed scheme, the two images are the logo image and the feature image (as depicted in Figure1). The logo image can be any identifiable image. The feature image is generated from the input image as follows. First, the input image is split into nonoverlapping 8 × 8 blocks. Then, the 2D DWT is applied to each block to generate four subbands, LL2, LH2, HL2, and HH2. An example of 2D DWT is shown in Figure 2. Finally, the coefficients in the LL2 subband of each DWT block are used to generate the feature image. The ISS then generates a share image from the logo image and feature image. The share image will be used as the fingerprint of the input image by the scheme. The share image also serves as the watermark to be embedded in the host image. The benefit of the ISS scheme is that performing the XOR operation on the feature image and the share image will restore the logo image, which can then be used to identify the copyright.Figure 1
The process of the ISS.Figure 2
2D DWT.


## 3. The Proposed Copyright Identification Scheme

The proposed scheme contains two phases: thefingerprint and watermarked image generation phase and theauthentication logo detection phase. The former phase extracts features from the host image, which, along with a logo image, is used to generate the fingerprint. The fingerprint also serves as the watermark, and the phase embeds it in the host image to produce a watermarked image. On the other hand, the latter phase extracts features and retrieves the watermark from the suspect image. The extracted features and the retrieved watermark are utilized to restore the logo image, which is used to identify the copyright. If it fails, the retrieved watermark then serves as the fingerprint and is compared with those in the database to determine if the suspect image is a duplicated one.Thefingerprint and watermarked image generation phase (shown in Figure 3) works as follows. In the beginning,feature extraction extracts the features of the host image and thenlogo scrambling disarranges the authentication logo to a scrambled logo image. After that,fingerprint generation takes as input the extracted features and the scrambled logo to generate the fingerprint. Finally, the fingerprint serves as a watermark and is embedded in the host image, which becomes a watermarked image. The fingerprint is also stored in a database for later use in the next phase.Figure 3
The phases of the proposed scheme.Theauthentication logo detection phase (shown in Figure 3) checks the watermark first and, if necessary, the fingerprint next. In the beginning,watermark retrieval regains the watermark from the suspect image. Next, the features of the suspect image are extracted byfeature extraction. After that,logo restoration takes as input the retrieved watermark (the expected fingerprint of the suspect image) and the extracted features to recover and rearrange the scrambled logo to restore the authentication logo. The phase ends if the accuracy rate of the restored logo determined bylogo comparison is high enough; otherwise, the process proceeds to retrieve the next available fingerprint from the database and then returns tologo restoration, which takes as input the retrieved fingerprint instead of the extracted watermark. The phase restores the logo from the retrieved fingerprint as well as the extracted features and proceeds tologo comparison. The looping process continues until the authentication logo is discovered or no fingerprint is available.


### 3.1. Fingerprint and Watermarked Image Generation Phase

The following paragraphs detail the stages in thefingerprint andwatermarked image generation phase, includingfeature extraction, logo scrambling, fingerprint generation, andwatermark embedding.


#### 3.1.1. Feature Extraction

Thefeature extraction stage takes a color image as input and then extracts its features. The stage has two substages,sampling andfeature generation. Duringsampling, the stage first transforms the RGB image to the YCbCr color space [21, 22]. Then, it partitions each of the three channels into several nonoverlapping blocks of size 8 × 8 (hence, a color image of size N
×
N will result in 3
×
N
/
8
×
N
/
8 nonoverlapping blocks). After partition, for each row of the corresponding blocks in the three channels, the stage takes the first four samples from the Y channel, the next two from the Cb, and the last two from the Cr (as depicted in Figure 4) to generate new packed blocks, each of size 8 × 8.Figure 4
Illustration ofsampling.Aftersampling, the stage enters its second substage,feature generation. For each of the packed blocks, the stage applies 2D DWT to the block, resulting in four coefficients in the LL2 subband. Next, the stage computes the average (denoted by A) of the four coefficients and then obtains a feature type T according to the relationship of the four coefficients and the average A as expressed in (1). Consider
(1)
T
=
{
1
,
if
only
one
coefficient
is
smaller
than
A
.
2
,
if
only
two
coefficients
are
smaller
than
A
.
3
,
if
only
one
coefficient
is
greater
than
A
.
4
,
if
all
of
the
four
coefficients
are
the
same
and
hence
all
equal
A
.According toT, A, and the mapping table shown in Table 1, a feature share (called FT-share) of size 2 × 2 is determined for each block. The FT-shares represent the features of the input color image. They are assembled to form the feature image.Table 1
The mapping table.
Feature typeT
Mean value position
White logo pixel
FT-shareXOR FP-share
Black logo pixel
FT-shareXOR FP-share
FT-share
FP-share
FT-share
FP-share
1
a
<
A
<
b
,
c
,
d
b
<
A
<
a
,
c
,
d
c
<
A
<
a
,
b
,
d
d
<
A
<
a
,
b
,
c
2
a
,
b
≤
A
<
c
,
d
c
,
d
≤
A
<
a
,
b
a
,
d
≤
A
<
b
,
c
b
,
c
≤
A
<
a
,
d
a
,
c
≤
A
<
b
,
d
b
,
d
≤
A
<
a
,
c
3
b
,
c
,
d
≤
A
<
a
a
,
c
,
d
≤
A
<
b
a
,
b
,
d
≤
A
<
c
a
,
b
,
c
≤
A
<
d
4
A
=
a
=
b
=
c
=
d
The coefficients of the LL2 subband (a is at the top left position, b the top right, c the bottom left, and d the bottom right)
FT-share: feature share; FP-share: fingerprint share.The steps of thefeature extraction stage are listed in Algorithm 1.Algorithm 1:Feature extraction.
Input:
A color image H(N× N).
Output:
A feature image FT (N/8 × N/8).
ConvertH to the YCbCr color space
Partition each of the Y, Cb, and Cr channels intoN/8 × N/8 non-overlapping blocks of size 8 × 8
For each corresponding block of the three channels
Take the first 4 samples from the Y channel, the next 2 from the Cb, and last 2 from the
Cr as depicted in Figure4 to form a packed block of size 8 × 8
End For
Apply 2D-DWT to each packed block to obtainN/8 × N/8 LL2blocks of size 2 × 2
For each of the LL2 blocks
Compute the averageA of the four coefficients
Obtain the feature typeT according to (1)
Determine the FT-share according toT, A, and Table 1
End For
Assemble the FT-shares to form the feature image FT


#### 3.1.2. Logo Scrambling

In order to disperse the intensity of attacks, the proposed scheme adopts Torus automorphism [23] to scramble the authentication logo. The stage uses a predetermined key, k, and the following equation to scramble the logo. Consider
(2)
(
x
t
y
t
)
=
(
1
1
k
k
+
1
)
(
x
t
-
1
y
t
-
1
)
mod
⁡
T
,
where (x
t, y
t) are the coordinates in statet and T is the coordinate size of the given image. Figure 5 shows an example of the authentication logo scrambled four times with k
=
1, T
=
128.Figure 5
Example of scrambling.


#### 3.1.3. Fingerprint Generation

The proposed scheme uses ISS (mentioned in Section2) to generate the fingerprint. It determines a FP-share for each FT-share (generated infeature extraction) according to the color of the corresponding pixel in the scrambled logo by looking Table 1 up. For example, if the feature type is 2 and the FT-share is the same as the one in Table 2, then the FP-share will be either one of the two FP-shares in Table 2 according to the color of the corresponding pixel in the scrambled logo. After every FP-share is determined, the stage gathers all the FP-shares to form the fingerprint image. The fingerprint also serves as the watermark in the next stage.Table 2
Partial table of Table1.
Feature typeT
Mean value position
White logo pixel
FT-shareXOR FP-share
Black logo pixel
FT-shareXOR FP-share
FT-share
FP-share
FT-share
FP-share
⋮
2
b
,
c
≤
A
<
a
,
d
⋮


#### 3.1.4. Watermark Embedding

Thewatermark embedding stage uses the resulting fingerprint image as a watermark and then embeds it in the host image. Generally speaking, for the RGB color space, the human visual system is more sensitive to the G channel than to the other two [22–28]. Therefore, the proposed scheme embeds the watermark into the less sensitive R and B channels of the host image. To be more specific, there are four areas: two in the R channel (I
1 and I
2) and two in the B channel (I
3 and I
4), used forwatermark embedding (see Figure 6). The stage applies the 2D DWT to the R and B channels of the host image and next applies the 1D DWT to the resulting HL2 and LH2 to obtain the four blocks, I
1, I
2, I
3, and I
4.Figure 6
TheI
1, I
2, I
3, and I
4 blocks used forwatermark embedding.The proposed scheme embeds the watermark into the image by adjusting the coefficients in theI
1 to I
4 blocks according to a predefined M. The value of M affects the robustness and the quality. The larger the M is, the more robust the embedded watermark is, but the worse the quality of the watermarked image is. Figure 7 illustrates the adjustment of the coefficients. If the watermark bit is 1 and the current coefficient is between M
×
(
i
-
1
) and M
×
i, then the coefficient will be adjusted to be M
×
(
i
-
1
)
-
M
/
4 or M
×
i
-
M
/
4, whichever is closer to the current coefficient. On the other hand, the adjusted coefficient will be M
×
(
i
-
1
)
+
M
/
4 or M
×
i
+
M
/
4 if the watermark bit is 0.Figure 7
Coefficient adjustment.The embedding process is described as follows. First, the stage uses the current coefficientc and the predefined M to calculateS, sign, and (
r
0
,
r
1
) according to (3), (4), and (5), respectively. In the equations, sign indicates that the c is positive or negative; r
0 is the remainder for watermark bit value 0 and, similarly, r
1 for watermark bit value 1. Then, the stage determines theC_Low andC_High according to the value of the watermark bit. If the value is 0, (6) will be used, otherwise, (7). Finally, the stage adjusts the coefficient c to c
′ according to (8). Consider
(3)
S
=
|
c
|
M
,
(4)
sign
⁡
=
{
1
,
if
c
≥
0
,
-
1
,
otherwise
,
(5)
(
r
0
,
r
1
)
=
{
(
M
4
,
3
M
4
)
,
if
c
≥
0
,
(
3
M
4
,
M
4
)
,
otherwise
,
(6)
C
_
Low
=
sign
⁡
×
(
S
×
M
+
r
0
)
,
C
_
High
=
sign
⁡
×
(
(
S
+
sign
⁡
)
×
M
+
r
0
)
,
(7)
C
_
Low
=
sign
⁡
×
(
(
S
-
sign
⁡
)
×
M
+
r
1
)
,
C
_
High
=
sign
⁡
×
(
S
×
M
+
r
1
)
,
(8)
c
′
=
{
C
_
Low
,
if
|
C
_
Low
-
c
|
≤
|
C
_
High
-
c
|
C
_
High
,
otherwise
.The steps of thewatermark embedding stage are listed in Algorithm 2.Algorithm 2:Watermark embedding.
Input:
A color image H(N
×
N) and a fingerprint image FP (N/4 × N/4).
Output:
A watermarked image (N
×
N).
Apply DWT to the R and B channels ofH to obtain the four embedding blocks I
1
,
…
,
I
4
For each coefficientc of the I
1
–
I
4 blocks and the corresponding pixel of FP
Compute thesign, S, and (r
0, r
1) by (3), (4), and (5), respectively
Compute theC_Low and C_High by (6) if watermark bit is 0, or (7) if otherwise
Adjust the coefficient toc
′ by (8)
End For
Perform inverse DWT to produce the watermarked image


### 3.2. Authentication Logo Detection Phase

The phase is activated when a dispute over the copyright of a suspect image occurs. The following details the stages in the phase, includingwatermark retrieval andlogo restoration.


#### 3.2.1. Watermark Retrieval

The stage regains the watermark from the suspect image. First, the stage applies the DWT to the R and B planes of the suspect image in the same way as that inwatermark embedding to obtain the embedding blocks, I
1 to I
4 (refer to Figure 6). Then, for each coefficient c of the I
1
–
I
4blocks, it obtains the watermark bit w according to
(9)
w
=
{
1
,
if
(
c
≥
0
,
c
mod
⁡
M
≥
M
2
)
,
or
(
c
<
0
,
|
c
|
mod
⁡
M
<
M
2
)
,
0
,
otherwise
,
where M is the same as that in (3).Finally, the watermark (which is supposed to be the fingerprint of the image) is restored by assembling everyw for each coefficient c in the I
1–I
4 blocks.Algorithm3 lists the steps of thewatermark retrieval stage.Algorithm 3:Watermark retrieval.
Input:
A suspect color image S (N
×
N).
Output:
A fingerprint (watermarked) image FP′ (N
/
4
×
N
/
4).
Apply DWT to the R and B channels ofS and obtain the embedding blocks I
1
–
I
4
For each coefficientc of the I
1
–
I
4 blocks
Obtainw according to (9)
End For
Assemble every watermark bitw to restore the fingerprint image FP′


#### 3.2.2. Logo Restoration

The stage restores the authentication logo. As shown in Figure8, it has four substages:scrambled logo recovery,logo unscrambling,logo enhancement, andlogo resizing.Figure 8
The four substages oflogo restoration.Scrambled Logo Recovery. After retrieving the fingerprint image (i.e., watermark) bywatermark retrieval and extracting the feature image byfeature extraction, the substage performs XOR operation on each pixel of the fingerprint image and the corresponding pixel of the feature image to retrieve the scrambled logo. Because both of the two images are black-and-white, each pixel is either 0 (black) or 1 (white). Therefore, the substage simply performs bitwise XOR operation on the two images and obtains a scrambled logo.Logo Unscrambling. As mentioned in Section 3.1.2, the logo was scrambled before it is used to generate fingerprint in thefingerprint and watermarked image generation phase. The substage adopts (10), which is the inverse equation of (2), to rearrange the scrambled logo and restore the logo. Consider
(10)
(
x
t
y
t
)
=
(
1
1
k
k
+
1
)
-
1
(
x
t
-
1
y
t
-
1
)
mod
⁡
T
.Logo Enhancement. The substage enhances the restored logo by erosion and dilation. Erosion removes pixels on object boundaries in an image and therefore can be used to remove smaller islands in the image; dilation, on the other hand, adds pixels to the boundaries of objects in an image and hence can be used to remove bright areas from the image. The substage performs erosion followed by dilation once, which is illustrated in Figure 9.Figure 9
An example of erosion and dilation.Erosion is performed by resetting each of the pixels according to (11). That is, if one of the neighbors is black, then the current pixel will be set to black; otherwise, it will be set to white. LetL(x,y) be the current pixel value to be determined and L
(
x
+
1
,
y
), L
(
x
+
1
,
y
+
1
), and L
(
x
,
y
+
1
) its neighbors; then
(11)
L
(
x
,
y
)
=
{
0
(
black
)
,
if
L
(
x
+
1
,
y
)
or
L
(
x
+
1
,
y
+
1
)
or
L
(
x
,
y
+
1
)
is
black
1
(
white
)
,
otherwise
.Dilation, on the contrary, resets the neighbors of each pixel rather than the pixel itself. IfL(x,y) is black; the substage sets all of its neighbors, L
(
x
+
1
,
y
), L
(
x
+
1
,
y
+
1
), and L
(
x
,
y
+
1
), to black; otherwise, the neighbors remain unchanged.Logo Resizing. The proposed scheme adopts ISS to retrieve the authentication logo, which causespixel expansion because one logo pixel is mapped to a share of four pixels (mentioned in Section 3.1.3). As a result, the retrieved logo from the previous stage will be larger than the original one. To resize the logo to its original size, the substage partitions the enhanced logo into several blocks of size 2 × 2, each of which is then reduced into one pixel with value L
(
x
,
y
) according to the following rule:
(12)
L
(
x
,
y
)
=
{
0
(
black
)
,
the
number
of
the
black
pixels
in
each
block
≥
3
,
1
(
white
)
,
otherwise
.


## 3.1. Fingerprint and Watermarked Image Generation Phase

The following paragraphs detail the stages in thefingerprint andwatermarked image generation phase, includingfeature extraction, logo scrambling, fingerprint generation, andwatermark embedding.


### 3.1.1. Feature Extraction

Thefeature extraction stage takes a color image as input and then extracts its features. The stage has two substages,sampling andfeature generation. Duringsampling, the stage first transforms the RGB image to the YCbCr color space [21, 22]. Then, it partitions each of the three channels into several nonoverlapping blocks of size 8 × 8 (hence, a color image of size N
×
N will result in 3
×
N
/
8
×
N
/
8 nonoverlapping blocks). After partition, for each row of the corresponding blocks in the three channels, the stage takes the first four samples from the Y channel, the next two from the Cb, and the last two from the Cr (as depicted in Figure 4) to generate new packed blocks, each of size 8 × 8.Figure 4
Illustration ofsampling.Aftersampling, the stage enters its second substage,feature generation. For each of the packed blocks, the stage applies 2D DWT to the block, resulting in four coefficients in the LL2 subband. Next, the stage computes the average (denoted by A) of the four coefficients and then obtains a feature type T according to the relationship of the four coefficients and the average A as expressed in (1). Consider
(1)
T
=
{
1
,
if
only
one
coefficient
is
smaller
than
A
.
2
,
if
only
two
coefficients
are
smaller
than
A
.
3
,
if
only
one
coefficient
is
greater
than
A
.
4
,
if
all
of
the
four
coefficients
are
the
same
and
hence
all
equal
A
.According toT, A, and the mapping table shown in Table 1, a feature share (called FT-share) of size 2 × 2 is determined for each block. The FT-shares represent the features of the input color image. They are assembled to form the feature image.Table 1
The mapping table.
Feature typeT
Mean value position
White logo pixel
FT-shareXOR FP-share
Black logo pixel
FT-shareXOR FP-share
FT-share
FP-share
FT-share
FP-share
1
a
<
A
<
b
,
c
,
d
b
<
A
<
a
,
c
,
d
c
<
A
<
a
,
b
,
d
d
<
A
<
a
,
b
,
c
2
a
,
b
≤
A
<
c
,
d
c
,
d
≤
A
<
a
,
b
a
,
d
≤
A
<
b
,
c
b
,
c
≤
A
<
a
,
d
a
,
c
≤
A
<
b
,
d
b
,
d
≤
A
<
a
,
c
3
b
,
c
,
d
≤
A
<
a
a
,
c
,
d
≤
A
<
b
a
,
b
,
d
≤
A
<
c
a
,
b
,
c
≤
A
<
d
4
A
=
a
=
b
=
c
=
d
The coefficients of the LL2 subband (a is at the top left position, b the top right, c the bottom left, and d the bottom right)
FT-share: feature share; FP-share: fingerprint share.The steps of thefeature extraction stage are listed in Algorithm 1.Algorithm 1:Feature extraction.
Input:
A color image H(N× N).
Output:
A feature image FT (N/8 × N/8).
ConvertH to the YCbCr color space
Partition each of the Y, Cb, and Cr channels intoN/8 × N/8 non-overlapping blocks of size 8 × 8
For each corresponding block of the three channels
Take the first 4 samples from the Y channel, the next 2 from the Cb, and last 2 from the
Cr as depicted in Figure4 to form a packed block of size 8 × 8
End For
Apply 2D-DWT to each packed block to obtainN/8 × N/8 LL2blocks of size 2 × 2
For each of the LL2 blocks
Compute the averageA of the four coefficients
Obtain the feature typeT according to (1)
Determine the FT-share according toT, A, and Table 1
End For
Assemble the FT-shares to form the feature image FT


### 3.1.2. Logo Scrambling

In order to disperse the intensity of attacks, the proposed scheme adopts Torus automorphism [23] to scramble the authentication logo. The stage uses a predetermined key, k, and the following equation to scramble the logo. Consider
(2)
(
x
t
y
t
)
=
(
1
1
k
k
+
1
)
(
x
t
-
1
y
t
-
1
)
mod
⁡
T
,
where (x
t, y
t) are the coordinates in statet and T is the coordinate size of the given image. Figure 5 shows an example of the authentication logo scrambled four times with k
=
1, T
=
128.Figure 5
Example of scrambling.


### 3.1.3. Fingerprint Generation

The proposed scheme uses ISS (mentioned in Section2) to generate the fingerprint. It determines a FP-share for each FT-share (generated infeature extraction) according to the color of the corresponding pixel in the scrambled logo by looking Table 1 up. For example, if the feature type is 2 and the FT-share is the same as the one in Table 2, then the FP-share will be either one of the two FP-shares in Table 2 according to the color of the corresponding pixel in the scrambled logo. After every FP-share is determined, the stage gathers all the FP-shares to form the fingerprint image. The fingerprint also serves as the watermark in the next stage.Table 2
Partial table of Table1.
Feature typeT
Mean value position
White logo pixel
FT-shareXOR FP-share
Black logo pixel
FT-shareXOR FP-share
FT-share
FP-share
FT-share
FP-share
⋮
2
b
,
c
≤
A
<
a
,
d
⋮


### 3.1.4. Watermark Embedding

Thewatermark embedding stage uses the resulting fingerprint image as a watermark and then embeds it in the host image. Generally speaking, for the RGB color space, the human visual system is more sensitive to the G channel than to the other two [22–28]. Therefore, the proposed scheme embeds the watermark into the less sensitive R and B channels of the host image. To be more specific, there are four areas: two in the R channel (I
1 and I
2) and two in the B channel (I
3 and I
4), used forwatermark embedding (see Figure 6). The stage applies the 2D DWT to the R and B channels of the host image and next applies the 1D DWT to the resulting HL2 and LH2 to obtain the four blocks, I
1, I
2, I
3, and I
4.Figure 6
TheI
1, I
2, I
3, and I
4 blocks used forwatermark embedding.The proposed scheme embeds the watermark into the image by adjusting the coefficients in theI
1 to I
4 blocks according to a predefined M. The value of M affects the robustness and the quality. The larger the M is, the more robust the embedded watermark is, but the worse the quality of the watermarked image is. Figure 7 illustrates the adjustment of the coefficients. If the watermark bit is 1 and the current coefficient is between M
×
(
i
-
1
) and M
×
i, then the coefficient will be adjusted to be M
×
(
i
-
1
)
-
M
/
4 or M
×
i
-
M
/
4, whichever is closer to the current coefficient. On the other hand, the adjusted coefficient will be M
×
(
i
-
1
)
+
M
/
4 or M
×
i
+
M
/
4 if the watermark bit is 0.Figure 7
Coefficient adjustment.The embedding process is described as follows. First, the stage uses the current coefficientc and the predefined M to calculateS, sign, and (
r
0
,
r
1
) according to (3), (4), and (5), respectively. In the equations, sign indicates that the c is positive or negative; r
0 is the remainder for watermark bit value 0 and, similarly, r
1 for watermark bit value 1. Then, the stage determines theC_Low andC_High according to the value of the watermark bit. If the value is 0, (6) will be used, otherwise, (7). Finally, the stage adjusts the coefficient c to c
′ according to (8). Consider
(3)
S
=
|
c
|
M
,
(4)
sign
⁡
=
{
1
,
if
c
≥
0
,
-
1
,
otherwise
,
(5)
(
r
0
,
r
1
)
=
{
(
M
4
,
3
M
4
)
,
if
c
≥
0
,
(
3
M
4
,
M
4
)
,
otherwise
,
(6)
C
_
Low
=
sign
⁡
×
(
S
×
M
+
r
0
)
,
C
_
High
=
sign
⁡
×
(
(
S
+
sign
⁡
)
×
M
+
r
0
)
,
(7)
C
_
Low
=
sign
⁡
×
(
(
S
-
sign
⁡
)
×
M
+
r
1
)
,
C
_
High
=
sign
⁡
×
(
S
×
M
+
r
1
)
,
(8)
c
′
=
{
C
_
Low
,
if
|
C
_
Low
-
c
|
≤
|
C
_
High
-
c
|
C
_
High
,
otherwise
.The steps of thewatermark embedding stage are listed in Algorithm 2.Algorithm 2:Watermark embedding.
Input:
A color image H(N
×
N) and a fingerprint image FP (N/4 × N/4).
Output:
A watermarked image (N
×
N).
Apply DWT to the R and B channels ofH to obtain the four embedding blocks I
1
,
…
,
I
4
For each coefficientc of the I
1
–
I
4 blocks and the corresponding pixel of FP
Compute thesign, S, and (r
0, r
1) by (3), (4), and (5), respectively
Compute theC_Low and C_High by (6) if watermark bit is 0, or (7) if otherwise
Adjust the coefficient toc
′ by (8)
End For
Perform inverse DWT to produce the watermarked image


## 3.1.1. Feature Extraction

Thefeature extraction stage takes a color image as input and then extracts its features. The stage has two substages,sampling andfeature generation. Duringsampling, the stage first transforms the RGB image to the YCbCr color space [21, 22]. Then, it partitions each of the three channels into several nonoverlapping blocks of size 8 × 8 (hence, a color image of size N
×
N will result in 3
×
N
/
8
×
N
/
8 nonoverlapping blocks). After partition, for each row of the corresponding blocks in the three channels, the stage takes the first four samples from the Y channel, the next two from the Cb, and the last two from the Cr (as depicted in Figure 4) to generate new packed blocks, each of size 8 × 8.Figure 4
Illustration ofsampling.Aftersampling, the stage enters its second substage,feature generation. For each of the packed blocks, the stage applies 2D DWT to the block, resulting in four coefficients in the LL2 subband. Next, the stage computes the average (denoted by A) of the four coefficients and then obtains a feature type T according to the relationship of the four coefficients and the average A as expressed in (1). Consider
(1)
T
=
{
1
,
if
only
one
coefficient
is
smaller
than
A
.
2
,
if
only
two
coefficients
are
smaller
than
A
.
3
,
if
only
one
coefficient
is
greater
than
A
.
4
,
if
all
of
the
four
coefficients
are
the
same
and
hence
all
equal
A
.According toT, A, and the mapping table shown in Table 1, a feature share (called FT-share) of size 2 × 2 is determined for each block. The FT-shares represent the features of the input color image. They are assembled to form the feature image.Table 1
The mapping table.
Feature typeT
Mean value position
White logo pixel
FT-shareXOR FP-share
Black logo pixel
FT-shareXOR FP-share
FT-share
FP-share
FT-share
FP-share
1
a
<
A
<
b
,
c
,
d
b
<
A
<
a
,
c
,
d
c
<
A
<
a
,
b
,
d
d
<
A
<
a
,
b
,
c
2
a
,
b
≤
A
<
c
,
d
c
,
d
≤
A
<
a
,
b
a
,
d
≤
A
<
b
,
c
b
,
c
≤
A
<
a
,
d
a
,
c
≤
A
<
b
,
d
b
,
d
≤
A
<
a
,
c
3
b
,
c
,
d
≤
A
<
a
a
,
c
,
d
≤
A
<
b
a
,
b
,
d
≤
A
<
c
a
,
b
,
c
≤
A
<
d
4
A
=
a
=
b
=
c
=
d
The coefficients of the LL2 subband (a is at the top left position, b the top right, c the bottom left, and d the bottom right)
FT-share: feature share; FP-share: fingerprint share.The steps of thefeature extraction stage are listed in Algorithm 1.Algorithm 1:Feature extraction.
Input:
A color image H(N× N).
Output:
A feature image FT (N/8 × N/8).
ConvertH to the YCbCr color space
Partition each of the Y, Cb, and Cr channels intoN/8 × N/8 non-overlapping blocks of size 8 × 8
For each corresponding block of the three channels
Take the first 4 samples from the Y channel, the next 2 from the Cb, and last 2 from the
Cr as depicted in Figure4 to form a packed block of size 8 × 8
End For
Apply 2D-DWT to each packed block to obtainN/8 × N/8 LL2blocks of size 2 × 2
For each of the LL2 blocks
Compute the averageA of the four coefficients
Obtain the feature typeT according to (1)
Determine the FT-share according toT, A, and Table 1
End For
Assemble the FT-shares to form the feature image FT


## 3.1.2. Logo Scrambling

In order to disperse the intensity of attacks, the proposed scheme adopts Torus automorphism [23] to scramble the authentication logo. The stage uses a predetermined key, k, and the following equation to scramble the logo. Consider
(2)
(
x
t
y
t
)
=
(
1
1
k
k
+
1
)
(
x
t
-
1
y
t
-
1
)
mod
⁡
T
,
where (x
t, y
t) are the coordinates in statet and T is the coordinate size of the given image. Figure 5 shows an example of the authentication logo scrambled four times with k
=
1, T
=
128.Figure 5
Example of scrambling.


## 3.1.3. Fingerprint Generation

The proposed scheme uses ISS (mentioned in Section2) to generate the fingerprint. It determines a FP-share for each FT-share (generated infeature extraction) according to the color of the corresponding pixel in the scrambled logo by looking Table 1 up. For example, if the feature type is 2 and the FT-share is the same as the one in Table 2, then the FP-share will be either one of the two FP-shares in Table 2 according to the color of the corresponding pixel in the scrambled logo. After every FP-share is determined, the stage gathers all the FP-shares to form the fingerprint image. The fingerprint also serves as the watermark in the next stage.Table 2
Partial table of Table1.
Feature typeT
Mean value position
White logo pixel
FT-shareXOR FP-share
Black logo pixel
FT-shareXOR FP-share
FT-share
FP-share
FT-share
FP-share
⋮
2
b
,
c
≤
A
<
a
,
d
⋮


## 3.1.4. Watermark Embedding

Thewatermark embedding stage uses the resulting fingerprint image as a watermark and then embeds it in the host image. Generally speaking, for the RGB color space, the human visual system is more sensitive to the G channel than to the other two [22–28]. Therefore, the proposed scheme embeds the watermark into the less sensitive R and B channels of the host image. To be more specific, there are four areas: two in the R channel (I
1 and I
2) and two in the B channel (I
3 and I
4), used forwatermark embedding (see Figure 6). The stage applies the 2D DWT to the R and B channels of the host image and next applies the 1D DWT to the resulting HL2 and LH2 to obtain the four blocks, I
1, I
2, I
3, and I
4.Figure 6
TheI
1, I
2, I
3, and I
4 blocks used forwatermark embedding.The proposed scheme embeds the watermark into the image by adjusting the coefficients in theI
1 to I
4 blocks according to a predefined M. The value of M affects the robustness and the quality. The larger the M is, the more robust the embedded watermark is, but the worse the quality of the watermarked image is. Figure 7 illustrates the adjustment of the coefficients. If the watermark bit is 1 and the current coefficient is between M
×
(
i
-
1
) and M
×
i, then the coefficient will be adjusted to be M
×
(
i
-
1
)
-
M
/
4 or M
×
i
-
M
/
4, whichever is closer to the current coefficient. On the other hand, the adjusted coefficient will be M
×
(
i
-
1
)
+
M
/
4 or M
×
i
+
M
/
4 if the watermark bit is 0.Figure 7
Coefficient adjustment.The embedding process is described as follows. First, the stage uses the current coefficientc and the predefined M to calculateS, sign, and (
r
0
,
r
1
) according to (3), (4), and (5), respectively. In the equations, sign indicates that the c is positive or negative; r
0 is the remainder for watermark bit value 0 and, similarly, r
1 for watermark bit value 1. Then, the stage determines theC_Low andC_High according to the value of the watermark bit. If the value is 0, (6) will be used, otherwise, (7). Finally, the stage adjusts the coefficient c to c
′ according to (8). Consider
(3)
S
=
|
c
|
M
,
(4)
sign
⁡
=
{
1
,
if
c
≥
0
,
-
1
,
otherwise
,
(5)
(
r
0
,
r
1
)
=
{
(
M
4
,
3
M
4
)
,
if
c
≥
0
,
(
3
M
4
,
M
4
)
,
otherwise
,
(6)
C
_
Low
=
sign
⁡
×
(
S
×
M
+
r
0
)
,
C
_
High
=
sign
⁡
×
(
(
S
+
sign
⁡
)
×
M
+
r
0
)
,
(7)
C
_
Low
=
sign
⁡
×
(
(
S
-
sign
⁡
)
×
M
+
r
1
)
,
C
_
High
=
sign
⁡
×
(
S
×
M
+
r
1
)
,
(8)
c
′
=
{
C
_
Low
,
if
|
C
_
Low
-
c
|
≤
|
C
_
High
-
c
|
C
_
High
,
otherwise
.The steps of thewatermark embedding stage are listed in Algorithm 2.Algorithm 2:Watermark embedding.
Input:
A color image H(N
×
N) and a fingerprint image FP (N/4 × N/4).
Output:
A watermarked image (N
×
N).
Apply DWT to the R and B channels ofH to obtain the four embedding blocks I
1
,
…
,
I
4
For each coefficientc of the I
1
–
I
4 blocks and the corresponding pixel of FP
Compute thesign, S, and (r
0, r
1) by (3), (4), and (5), respectively
Compute theC_Low and C_High by (6) if watermark bit is 0, or (7) if otherwise
Adjust the coefficient toc
′ by (8)
End For
Perform inverse DWT to produce the watermarked image


## 3.2. Authentication Logo Detection Phase

The phase is activated when a dispute over the copyright of a suspect image occurs. The following details the stages in the phase, includingwatermark retrieval andlogo restoration.


### 3.2.1. Watermark Retrieval

The stage regains the watermark from the suspect image. First, the stage applies the DWT to the R and B planes of the suspect image in the same way as that inwatermark embedding to obtain the embedding blocks, I
1 to I
4 (refer to Figure 6). Then, for each coefficient c of the I
1
–
I
4blocks, it obtains the watermark bit w according to
(9)
w
=
{
1
,
if
(
c
≥
0
,
c
mod
⁡
M
≥
M
2
)
,
or
(
c
<
0
,
|
c
|
mod
⁡
M
<
M
2
)
,
0
,
otherwise
,
where M is the same as that in (3).Finally, the watermark (which is supposed to be the fingerprint of the image) is restored by assembling everyw for each coefficient c in the I
1–I
4 blocks.Algorithm3 lists the steps of thewatermark retrieval stage.Algorithm 3:Watermark retrieval.
Input:
A suspect color image S (N
×
N).
Output:
A fingerprint (watermarked) image FP′ (N
/
4
×
N
/
4).
Apply DWT to the R and B channels ofS and obtain the embedding blocks I
1
–
I
4
For each coefficientc of the I
1
–
I
4 blocks
Obtainw according to (9)
End For
Assemble every watermark bitw to restore the fingerprint image FP′


### 3.2.2. Logo Restoration

The stage restores the authentication logo. As shown in Figure8, it has four substages:scrambled logo recovery,logo unscrambling,logo enhancement, andlogo resizing.Figure 8
The four substages oflogo restoration.Scrambled Logo Recovery. After retrieving the fingerprint image (i.e., watermark) bywatermark retrieval and extracting the feature image byfeature extraction, the substage performs XOR operation on each pixel of the fingerprint image and the corresponding pixel of the feature image to retrieve the scrambled logo. Because both of the two images are black-and-white, each pixel is either 0 (black) or 1 (white). Therefore, the substage simply performs bitwise XOR operation on the two images and obtains a scrambled logo.Logo Unscrambling. As mentioned in Section 3.1.2, the logo was scrambled before it is used to generate fingerprint in thefingerprint and watermarked image generation phase. The substage adopts (10), which is the inverse equation of (2), to rearrange the scrambled logo and restore the logo. Consider
(10)
(
x
t
y
t
)
=
(
1
1
k
k
+
1
)
-
1
(
x
t
-
1
y
t
-
1
)
mod
⁡
T
.Logo Enhancement. The substage enhances the restored logo by erosion and dilation. Erosion removes pixels on object boundaries in an image and therefore can be used to remove smaller islands in the image; dilation, on the other hand, adds pixels to the boundaries of objects in an image and hence can be used to remove bright areas from the image. The substage performs erosion followed by dilation once, which is illustrated in Figure 9.Figure 9
An example of erosion and dilation.Erosion is performed by resetting each of the pixels according to (11). That is, if one of the neighbors is black, then the current pixel will be set to black; otherwise, it will be set to white. LetL(x,y) be the current pixel value to be determined and L
(
x
+
1
,
y
), L
(
x
+
1
,
y
+
1
), and L
(
x
,
y
+
1
) its neighbors; then
(11)
L
(
x
,
y
)
=
{
0
(
black
)
,
if
L
(
x
+
1
,
y
)
or
L
(
x
+
1
,
y
+
1
)
or
L
(
x
,
y
+
1
)
is
black
1
(
white
)
,
otherwise
.Dilation, on the contrary, resets the neighbors of each pixel rather than the pixel itself. IfL(x,y) is black; the substage sets all of its neighbors, L
(
x
+
1
,
y
), L
(
x
+
1
,
y
+
1
), and L
(
x
,
y
+
1
), to black; otherwise, the neighbors remain unchanged.Logo Resizing. The proposed scheme adopts ISS to retrieve the authentication logo, which causespixel expansion because one logo pixel is mapped to a share of four pixels (mentioned in Section 3.1.3). As a result, the retrieved logo from the previous stage will be larger than the original one. To resize the logo to its original size, the substage partitions the enhanced logo into several blocks of size 2 × 2, each of which is then reduced into one pixel with value L
(
x
,
y
) according to the following rule:
(12)
L
(
x
,
y
)
=
{
0
(
black
)
,
the
number
of
the
black
pixels
in
each
block
≥
3
,
1
(
white
)
,
otherwise
.


## 3.2.1. Watermark Retrieval

The stage regains the watermark from the suspect image. First, the stage applies the DWT to the R and B planes of the suspect image in the same way as that inwatermark embedding to obtain the embedding blocks, I
1 to I
4 (refer to Figure 6). Then, for each coefficient c of the I
1
–
I
4blocks, it obtains the watermark bit w according to
(9)
w
=
{
1
,
if
(
c
≥
0
,
c
mod
⁡
M
≥
M
2
)
,
or
(
c
<
0
,
|
c
|
mod
⁡
M
<
M
2
)
,
0
,
otherwise
,
where M is the same as that in (3).Finally, the watermark (which is supposed to be the fingerprint of the image) is restored by assembling everyw for each coefficient c in the I
1–I
4 blocks.Algorithm3 lists the steps of thewatermark retrieval stage.Algorithm 3:Watermark retrieval.
Input:
A suspect color image S (N
×
N).
Output:
A fingerprint (watermarked) image FP′ (N
/
4
×
N
/
4).
Apply DWT to the R and B channels ofS and obtain the embedding blocks I
1
–
I
4
For each coefficientc of the I
1
–
I
4 blocks
Obtainw according to (9)
End For
Assemble every watermark bitw to restore the fingerprint image FP′


## 3.2.2. Logo Restoration

The stage restores the authentication logo. As shown in Figure8, it has four substages:scrambled logo recovery,logo unscrambling,logo enhancement, andlogo resizing.Figure 8
The four substages oflogo restoration.Scrambled Logo Recovery. After retrieving the fingerprint image (i.e., watermark) bywatermark retrieval and extracting the feature image byfeature extraction, the substage performs XOR operation on each pixel of the fingerprint image and the corresponding pixel of the feature image to retrieve the scrambled logo. Because both of the two images are black-and-white, each pixel is either 0 (black) or 1 (white). Therefore, the substage simply performs bitwise XOR operation on the two images and obtains a scrambled logo.Logo Unscrambling. As mentioned in Section 3.1.2, the logo was scrambled before it is used to generate fingerprint in thefingerprint and watermarked image generation phase. The substage adopts (10), which is the inverse equation of (2), to rearrange the scrambled logo and restore the logo. Consider
(10)
(
x
t
y
t
)
=
(
1
1
k
k
+
1
)
-
1
(
x
t
-
1
y
t
-
1
)
mod
⁡
T
.Logo Enhancement. The substage enhances the restored logo by erosion and dilation. Erosion removes pixels on object boundaries in an image and therefore can be used to remove smaller islands in the image; dilation, on the other hand, adds pixels to the boundaries of objects in an image and hence can be used to remove bright areas from the image. The substage performs erosion followed by dilation once, which is illustrated in Figure 9.Figure 9
An example of erosion and dilation.Erosion is performed by resetting each of the pixels according to (11). That is, if one of the neighbors is black, then the current pixel will be set to black; otherwise, it will be set to white. LetL(x,y) be the current pixel value to be determined and L
(
x
+
1
,
y
), L
(
x
+
1
,
y
+
1
), and L
(
x
,
y
+
1
) its neighbors; then
(11)
L
(
x
,
y
)
=
{
0
(
black
)
,
if
L
(
x
+
1
,
y
)
or
L
(
x
+
1
,
y
+
1
)
or
L
(
x
,
y
+
1
)
is
black
1
(
white
)
,
otherwise
.Dilation, on the contrary, resets the neighbors of each pixel rather than the pixel itself. IfL(x,y) is black; the substage sets all of its neighbors, L
(
x
+
1
,
y
), L
(
x
+
1
,
y
+
1
), and L
(
x
,
y
+
1
), to black; otherwise, the neighbors remain unchanged.Logo Resizing. The proposed scheme adopts ISS to retrieve the authentication logo, which causespixel expansion because one logo pixel is mapped to a share of four pixels (mentioned in Section 3.1.3). As a result, the retrieved logo from the previous stage will be larger than the original one. To resize the logo to its original size, the substage partitions the enhanced logo into several blocks of size 2 × 2, each of which is then reduced into one pixel with value L
(
x
,
y
) according to the following rule:
(12)
L
(
x
,
y
)
=
{
0
(
black
)
,
the
number
of
the
black
pixels
in
each
block
≥
3
,
1
(
white
)
,
otherwise
.


## 4. Experimental Results

Two kinds of experiments were conducted to prove the effectiveness of the proposed scheme. The first experiment shows the robustness of our scheme and the other demonstrates the capability of unique identification. In the experiments, the authentication logo used to generate the watermark (fingerprint) is shown in Figure10.Figure 10
The authentication logo (64 × 64).Two common measurements used to estimate the robustness of our scheme are described as below.(1)Peak Signal to Noise Ratio (PSNR). The measurement to estimate the color image quality after image processing is a variant version of normal PSNR [29]. The variant PSNR listed below does not consider the influence of the green channel because the channel is not modified by our scheme. Consider
(13)
PSNR
=
10
lo
g
10
25
5
2
(
MSE
(
R
)
+
MSE
(
B
)
)
/
2
dB
,
where MSE is themean square error between the original image and the modified image, which is defined as follows:
(14)
MSE
=
∑
i
=
1
N
∑
j
=
1
N
(
x
i
j
-
x
i
j
′
)
2
N
2
,
where x
i
j represents the original pixel value and x
i
j
′ denotes the modified pixel value.According to the definition of PSNR, the higher the value is, the better the quality of the modified image is. Generally, if the PSNR is greater than 30 dB, the quality of the modified image is acceptable.(2)Accuracy Rate (AR). The measurement shown below is used to evaluate the correctness of the logo after it has been restored. Consider
(15)
AR
(
%
)
=
CP
NP
×
100
,
where NP is the number of pixels in the original logo and CP is the number of correct pixels obtained by comparing the pixels of the original logo with the corresponding ones of the restored logo. Figure 11 shows the restored logos with different AR values. As can been seen, restored logos with AR higher than 81% (the ones in the upper row) still can be visually recognized whereas the restored logo with AR equal to 76% is hard to identify. However, according to the description in Section 4.2, if AR is higher than 75%, the scheme still can identify the copyrights of the image.Figure 11
The restored logo with theirAR values (%).


### 4.1. Robustness Experiments

The experiments proved our scheme is robust to different kinds of attacks. The test images used in the experiment, including “Lena,” “Mandrill,” “Sailboat,” and “Peppers,” are shown in Figure12. The commercial image processing software “Adobe Image Photoshop CS” was used to simulate several kinds of image attacks, some of which for “Lena” are shown in Figure 13.The test images used in the experiments: (a) Lena, (b) Mandrill, (c) Sailboat, and (d) Peppers.
(a)
(b)
(c)
(d)Figure 13
Some examples of "Lena" after different image attacks (PSNRs are listed in the last rows).The experimental results of the test images are shown in Table3. As mentioned above, if AR is less than 75%, the scheme cannot identify the copyrights of the image. Table 3 shows that our scheme failed to verify the copyrights for the images suffering the heavier attacks of JPEG, contrast, Gauss blurring, and scaling inwatermark verification. Nevertheless, the duplications of those images can all be determined infingerprint verification. In summary, our scheme can identify the copyrights of the suspect images under moderate attacks inwatermark verification and determine the duplications of those suffering the heavy attacks infingerprint verification.Table 3
The resulting AR values of the test images.
Attacks
AR (%)  (Watermarking/fingerprinting)
Lena
Mandrill
Sailboat
Peppers
Average
None
99.93/99.93
99.76/99.76
99.98/99.98
99.63/99.63
99.83/99.83
JPEG
(90)
88.31/99.71
86.84/99.44
88.04/99.88
88.16/99.49
87.84/99.63
(80)
79.76/99.73
76.17/99.02
77.29/99.58
77.91/99.12
77.78/99.36
(70)
74.58/99.46
72.12/98.73
72.61/99.54
73.05/99.17
73.09/99.23
(60)
71.58/99.63
70.14/98.1
70.19/99.44
69.95/98.83
70.47/99.00
(50)
71.48/99.41
70.09/97.68
70.43/99.24
69.8/98.88
70.45/98.80
Contrast
(10)
92.6/99.85
80.71/99.78
88.62/99.95
90.31/99.27
88.06/99.71
(20)
82.86/99.76
73.12/99.56
79.27/99.78
81.81/98.68
79.27/99.45
(30)
75.85/99.46
70.61/99.07
75.22/99.63
74.63/98.07
74.08/99.06
(40)
72.61/99.27
70.83/97.92
72.31/99.19
72.51/98.14
72.07/98.63
(50)
71.7/98.93
70.41/96.19
71.19/98.75
71.09/96.58
71.10/97.61
(60)
70.39/97.68
70.65/92.7
70.95/98.1
70.53/95.8
70.63/96.07
(70)
70.8/94.65
70.65/90.31
71.12/97.71
71.36/95
70.98/94.42
Gauss blurring
(0.5)
83.64/99.41
73.44/97.29
78.17/99.17
80.54/99.05
78.95/98.73
(1.0)
76.1/98.88
70.92/95.78
74.17/98.44
73.88/98.34
73.77/97.86
(1.5)
73.63/98.17
70.95/94.34
72.44/97.29
72.34/97.73
72.34/96.88
Scaling
w: 256, h: 256
71.66/97.92
70.75/92.53
71.22/96.83
70.73/95.8
71.09/95.77
w: 128, h: 128
72.83/96.02
73.41/86.04
73.49/94.04
73.41/92.99
73.29/92.27
Brightness
(10)
96.66/96.78
94.41/94.92
97.85/97.85
94.46/94.46
95.85/96.00
(20)
90.65/91.48
89.06/90.7
95.9/95.9
88.33/88.33
90.99/91.60
(30)
85.72/87.62
83.98/86.43
93.55/93.58
83.69/83.64
86.74/87.82
Cropping
w: 480, h: 480
89.62/92.92
88.99/92.46
91.48/95.04
88.77/91.87
89.72/93.07
w: 448, h: 448
82.71/86.82
81.71/85.67
83.79/88.89
82.03/85.82
82.56/86.80
w: 432, h: 432
79.88/83.08
79.25/83.01
81.13/86.23
79/83.3
79.82/83.91
w: 424, h: 432
78.88/81.76
78.49/81.96
80.52/85.86
78.59/83.06
79.12/83.16
w: 424, h: 424
78.22/81.2
78/81.49
80.25/85.33
78.1/82.42
78.64/82.61Moreover, there were 100 attacks in total and 53 of them resulted in an AR value higher than 75% inwatermarking verification. That is to say, the copyrights of the 53% of the attacked images can be successfully identified inwatermark verification, and hence only 47% of them needfingerprint verification.


### 4.2. Uniqueness Experiments

The experiment showed that our scheme has the capability of unique identification and is able to distinguish a copyrighted image from different ones. The four copyrighted images (with embedded watermarks) in Figure12 along with seven unwatermarked images (Figures 14(a)–14(g)) were processed by our scheme to identify the copyright. The stored fingerprints of the watermarked images were used infingerprint verification to restore the logos for all of the images.The other seven color images used for testing uniqueness.
(a)
(b)
(c)
(d)
(e)
(f)
(g)The results shown in Table4 demonstrated the extraordinary unique identification capability of our scheme. It can be clearly seen that all the restored logos except the ones of the copyrighted images (those on the rightmost side) are unrecognizable, which proves that our scheme is actually able to distinguish a copyrighted image from different ones. It will not mistakenly identify the copyrights of an unwatermarked image.Table 4
Experimental results of the uniqueness experiment.
(a)
Lena
Mandrill
Sailboat
Peppers
F-16
Opera
Watch
New York
Tiffany
Terraux
Fourviere
Lena
Watermark
AR (%)
73.07
73.69
73.07
69.80
70.39
70.09
71.34
70.26
69.78
70.26
99.93
Fingerprint
AR (%)
59.06
56.03
49.24
62.72
59.62
44.34
62.35
60.40
45.65
63.21
99.93
(b)
Mandrill
Lena
Sailboat
Peppers
F-16
Opera
Watch
New York
Tiffany
Terraux
Fourviere
Mandrill
Watermark
AR (%)
73.74
73.69
73.07
72.73
73.52
73.79
72.64
74.35
73.82
72.2
99.76
Fingerprint
AR (%)
58.2
57.81
62.08
57.84
59.45
60.03
59.64
54.35
59.01
60.38
99.76
(c)
Sailboat
Lena
Mandrill
Peppers
F-16
Opera
Watch
New York
Tiffany
Terraux
Fourviere
Sailboat
Watermark
AR (%)
73.74
73.07
73.07
72.73
73.52
73.79
72.64
74.35
73.82
72.2
99.98
Fingerprint
AR (%)
54.32
59.06
53.44
57.91
55.1
52.37
56.47
55.37
53.78
59.4
99.98
(d)
Peppers
Lena
Mandrill
Sailboat
F-16
Opera
Watch
New York
Tiffany
Terraux
Fourviere
Peppers
Watermark
AR (%)
73.74
73.07
73.69
72.73
73.52
73.79
72.64
74.35
73.82
72.2
99.63
Fingerprint
AR (%)
47.02
60.13
46.08
55.59
58.69
49.29
53.26
52.8
54.66
55.77
99.63Moreover, the resulting AR values of all the unwatermarked images are all lower than 75%. Hence, it is reasonable for our scheme to confirm the copyright when AR is higher than 75%.


### 4.3. Discussion and Comparison

The fingerprint extracted from an image is more robust than the watermark embedded in the image. This can be seen from Figure15, which shows the average AR values of the images suffering different attacks for watermarking and fingerprinting. The AR values for fingerprinting are all higher than those for watermarking. When the image undergoes image processing operations that heavily damage the embedded watermark, the extracted fingerprint still can survive the attacks. However, because linear comparison is a computationally intensive process, fingerprinting is costly in time if there are many fingerprints in the database.The averageAR values of test images: (a) JPEG quality, (b) contrast, (c) Gauss blurring, (d) scaling, (e) brightness, and (f) cropping.
(a)
(b)
(c)
(d)
(e)
(f)Table5 shows the processing time forwatermark extraction andlogo restoration and comparison in our scheme. The experiments were carried out on a computer equipped with the following hardware and software:CPU: 3.16 GHz Intel(R) Xeon(R) CPU E3120,
RAM: 4 GB,
OS: Windows 7,
Computing language and environment: MATLAB.Table 5
The processing time (ms) for watermark extraction and logo restoration and comparison.
Stage
Lena
Mandrill
Sailboat
Peppers
Average
Watermark extraction
218
209
200
213
210
Logo restoration and comparison
20
29
39
28
29
Watermark extraction + logo restoration and comparison
238
238
239
241
239As Table5 shows, it takes about 0.21 seconds to extract the watermark from the input image and 0.029 seconds to restore and compare the logo. Therefore, the total processing time T
p (in seconds) of our scheme can be expressed by
(16)
T
p
=
0.24
+
0.029
×
n
,
where n represents the number of the fingerprints retrieved from the database. If the copyright can be identified inwatermark verification, n is 0. That is, our scheme only needs 0.24 seconds in the best case. Otherwise, our scheme needs additional 0.029 seconds for each retrieved fingerprint in the database.Because our scheme combines watermarking and fingerprinting techniques, it can be as efficient as pure watermarking schemes and also as robust as pure watermarking schemes. A pure watermarking scheme is very efficient because it only needs to make one watermark comparison to verify the copyright. However, it may not be as robust as a pure fingerprinting scheme when dealing with images that have suffered heavy attacks. Table6 shows the comparison of our combined scheme and the other two pure schemes. If the copyright of the input image can be identified bywatermarking verification, our scheme can be as efficient as pure watermarking schemes. Ifwatermarking verification fails to identify the copyright, our scheme is still able to determine duplication infingerprinting verification, which makes our scheme as robust as pure fingerprinting schemes.Table 6
Comparison of our combined scheme and the other two pure schemes.
Purewatermarking scheme
Purefingerprinting scheme
Ourcombined scheme
Robustness
△
◯
◯
Efficiency
◯
△
◯
*
◯: good; △: fair.


## 4.1. Robustness Experiments

The experiments proved our scheme is robust to different kinds of attacks. The test images used in the experiment, including “Lena,” “Mandrill,” “Sailboat,” and “Peppers,” are shown in Figure12. The commercial image processing software “Adobe Image Photoshop CS” was used to simulate several kinds of image attacks, some of which for “Lena” are shown in Figure 13.The test images used in the experiments: (a) Lena, (b) Mandrill, (c) Sailboat, and (d) Peppers.
(a)
(b)
(c)
(d)Figure 13
Some examples of "Lena" after different image attacks (PSNRs are listed in the last rows).The experimental results of the test images are shown in Table3. As mentioned above, if AR is less than 75%, the scheme cannot identify the copyrights of the image. Table 3 shows that our scheme failed to verify the copyrights for the images suffering the heavier attacks of JPEG, contrast, Gauss blurring, and scaling inwatermark verification. Nevertheless, the duplications of those images can all be determined infingerprint verification. In summary, our scheme can identify the copyrights of the suspect images under moderate attacks inwatermark verification and determine the duplications of those suffering the heavy attacks infingerprint verification.Table 3
The resulting AR values of the test images.
Attacks
AR (%)  (Watermarking/fingerprinting)
Lena
Mandrill
Sailboat
Peppers
Average
None
99.93/99.93
99.76/99.76
99.98/99.98
99.63/99.63
99.83/99.83
JPEG
(90)
88.31/99.71
86.84/99.44
88.04/99.88
88.16/99.49
87.84/99.63
(80)
79.76/99.73
76.17/99.02
77.29/99.58
77.91/99.12
77.78/99.36
(70)
74.58/99.46
72.12/98.73
72.61/99.54
73.05/99.17
73.09/99.23
(60)
71.58/99.63
70.14/98.1
70.19/99.44
69.95/98.83
70.47/99.00
(50)
71.48/99.41
70.09/97.68
70.43/99.24
69.8/98.88
70.45/98.80
Contrast
(10)
92.6/99.85
80.71/99.78
88.62/99.95
90.31/99.27
88.06/99.71
(20)
82.86/99.76
73.12/99.56
79.27/99.78
81.81/98.68
79.27/99.45
(30)
75.85/99.46
70.61/99.07
75.22/99.63
74.63/98.07
74.08/99.06
(40)
72.61/99.27
70.83/97.92
72.31/99.19
72.51/98.14
72.07/98.63
(50)
71.7/98.93
70.41/96.19
71.19/98.75
71.09/96.58
71.10/97.61
(60)
70.39/97.68
70.65/92.7
70.95/98.1
70.53/95.8
70.63/96.07
(70)
70.8/94.65
70.65/90.31
71.12/97.71
71.36/95
70.98/94.42
Gauss blurring
(0.5)
83.64/99.41
73.44/97.29
78.17/99.17
80.54/99.05
78.95/98.73
(1.0)
76.1/98.88
70.92/95.78
74.17/98.44
73.88/98.34
73.77/97.86
(1.5)
73.63/98.17
70.95/94.34
72.44/97.29
72.34/97.73
72.34/96.88
Scaling
w: 256, h: 256
71.66/97.92
70.75/92.53
71.22/96.83
70.73/95.8
71.09/95.77
w: 128, h: 128
72.83/96.02
73.41/86.04
73.49/94.04
73.41/92.99
73.29/92.27
Brightness
(10)
96.66/96.78
94.41/94.92
97.85/97.85
94.46/94.46
95.85/96.00
(20)
90.65/91.48
89.06/90.7
95.9/95.9
88.33/88.33
90.99/91.60
(30)
85.72/87.62
83.98/86.43
93.55/93.58
83.69/83.64
86.74/87.82
Cropping
w: 480, h: 480
89.62/92.92
88.99/92.46
91.48/95.04
88.77/91.87
89.72/93.07
w: 448, h: 448
82.71/86.82
81.71/85.67
83.79/88.89
82.03/85.82
82.56/86.80
w: 432, h: 432
79.88/83.08
79.25/83.01
81.13/86.23
79/83.3
79.82/83.91
w: 424, h: 432
78.88/81.76
78.49/81.96
80.52/85.86
78.59/83.06
79.12/83.16
w: 424, h: 424
78.22/81.2
78/81.49
80.25/85.33
78.1/82.42
78.64/82.61Moreover, there were 100 attacks in total and 53 of them resulted in an AR value higher than 75% inwatermarking verification. That is to say, the copyrights of the 53% of the attacked images can be successfully identified inwatermark verification, and hence only 47% of them needfingerprint verification.


## 4.2. Uniqueness Experiments

The experiment showed that our scheme has the capability of unique identification and is able to distinguish a copyrighted image from different ones. The four copyrighted images (with embedded watermarks) in Figure12 along with seven unwatermarked images (Figures 14(a)–14(g)) were processed by our scheme to identify the copyright. The stored fingerprints of the watermarked images were used infingerprint verification to restore the logos for all of the images.The other seven color images used for testing uniqueness.
(a)
(b)
(c)
(d)
(e)
(f)
(g)The results shown in Table4 demonstrated the extraordinary unique identification capability of our scheme. It can be clearly seen that all the restored logos except the ones of the copyrighted images (those on the rightmost side) are unrecognizable, which proves that our scheme is actually able to distinguish a copyrighted image from different ones. It will not mistakenly identify the copyrights of an unwatermarked image.Table 4
Experimental results of the uniqueness experiment.
(a)
Lena
Mandrill
Sailboat
Peppers
F-16
Opera
Watch
New York
Tiffany
Terraux
Fourviere
Lena
Watermark
AR (%)
73.07
73.69
73.07
69.80
70.39
70.09
71.34
70.26
69.78
70.26
99.93
Fingerprint
AR (%)
59.06
56.03
49.24
62.72
59.62
44.34
62.35
60.40
45.65
63.21
99.93
(b)
Mandrill
Lena
Sailboat
Peppers
F-16
Opera
Watch
New York
Tiffany
Terraux
Fourviere
Mandrill
Watermark
AR (%)
73.74
73.69
73.07
72.73
73.52
73.79
72.64
74.35
73.82
72.2
99.76
Fingerprint
AR (%)
58.2
57.81
62.08
57.84
59.45
60.03
59.64
54.35
59.01
60.38
99.76
(c)
Sailboat
Lena
Mandrill
Peppers
F-16
Opera
Watch
New York
Tiffany
Terraux
Fourviere
Sailboat
Watermark
AR (%)
73.74
73.07
73.07
72.73
73.52
73.79
72.64
74.35
73.82
72.2
99.98
Fingerprint
AR (%)
54.32
59.06
53.44
57.91
55.1
52.37
56.47
55.37
53.78
59.4
99.98
(d)
Peppers
Lena
Mandrill
Sailboat
F-16
Opera
Watch
New York
Tiffany
Terraux
Fourviere
Peppers
Watermark
AR (%)
73.74
73.07
73.69
72.73
73.52
73.79
72.64
74.35
73.82
72.2
99.63
Fingerprint
AR (%)
47.02
60.13
46.08
55.59
58.69
49.29
53.26
52.8
54.66
55.77
99.63Moreover, the resulting AR values of all the unwatermarked images are all lower than 75%. Hence, it is reasonable for our scheme to confirm the copyright when AR is higher than 75%.


## 4.3. Discussion and Comparison

The fingerprint extracted from an image is more robust than the watermark embedded in the image. This can be seen from Figure15, which shows the average AR values of the images suffering different attacks for watermarking and fingerprinting. The AR values for fingerprinting are all higher than those for watermarking. When the image undergoes image processing operations that heavily damage the embedded watermark, the extracted fingerprint still can survive the attacks. However, because linear comparison is a computationally intensive process, fingerprinting is costly in time if there are many fingerprints in the database.The averageAR values of test images: (a) JPEG quality, (b) contrast, (c) Gauss blurring, (d) scaling, (e) brightness, and (f) cropping.
(a)
(b)
(c)
(d)
(e)
(f)Table5 shows the processing time forwatermark extraction andlogo restoration and comparison in our scheme. The experiments were carried out on a computer equipped with the following hardware and software:CPU: 3.16 GHz Intel(R) Xeon(R) CPU E3120,
RAM: 4 GB,
OS: Windows 7,
Computing language and environment: MATLAB.Table 5
The processing time (ms) for watermark extraction and logo restoration and comparison.
Stage
Lena
Mandrill
Sailboat
Peppers
Average
Watermark extraction
218
209
200
213
210
Logo restoration and comparison
20
29
39
28
29
Watermark extraction + logo restoration and comparison
238
238
239
241
239As Table5 shows, it takes about 0.21 seconds to extract the watermark from the input image and 0.029 seconds to restore and compare the logo. Therefore, the total processing time T
p (in seconds) of our scheme can be expressed by
(16)
T
p
=
0.24
+
0.029
×
n
,
where n represents the number of the fingerprints retrieved from the database. If the copyright can be identified inwatermark verification, n is 0. That is, our scheme only needs 0.24 seconds in the best case. Otherwise, our scheme needs additional 0.029 seconds for each retrieved fingerprint in the database.Because our scheme combines watermarking and fingerprinting techniques, it can be as efficient as pure watermarking schemes and also as robust as pure watermarking schemes. A pure watermarking scheme is very efficient because it only needs to make one watermark comparison to verify the copyright. However, it may not be as robust as a pure fingerprinting scheme when dealing with images that have suffered heavy attacks. Table6 shows the comparison of our combined scheme and the other two pure schemes. If the copyright of the input image can be identified bywatermarking verification, our scheme can be as efficient as pure watermarking schemes. Ifwatermarking verification fails to identify the copyright, our scheme is still able to determine duplication infingerprinting verification, which makes our scheme as robust as pure fingerprinting schemes.Table 6
Comparison of our combined scheme and the other two pure schemes.
Purewatermarking scheme
Purefingerprinting scheme
Ourcombined scheme
Robustness
△
◯
◯
Efficiency
◯
△
◯
*
◯: good; △: fair.


## 5. Conclusion

This paper presented a copyright identification scheme that takes advantage of the complementary nature of digital watermarking and fingerprinting. The experimental results showed that when the watermarked image suffers moderate attacks,watermarking verification alone is enough to identify the copyright, and there is no need forfingerprinting verification. In other words, the proposed scheme can identify the copyright efficiently in this situation. On the other hand, the experimental results also showed that when the watermarked image suffers heavy attacks that renderwatermarking verification incompetent,fingerprinting verification, although more time consuming, can successfully determine the duplication, hence demonstrating the robustness of the proposed scheme.One distinguishing characteristic of the proposed scheme is that it does not need a separate watermark forwatermarking verification and a separate fingerprint forfingerprinting verification. The proposed scheme extracts features from the input image to generate the fingerprint, which also serves as the watermark. Hence, only one piece of information is needed for bothwatermarking andfingerprinting verifications.To further improve the scheme, retrieving the stored fingerprint image from the database to restore the correct authentication logo more quickly is worth studying. When there are more than one fingerprint image in the database, the original fingerprint of the host image must be correctly retrieved; otherwise, the correct authentication logo cannot be restored. Retrieving every stored fingerprint image to restore an authentication logo for comparison is very time consuming. The scheme should provide a more efficient way that is able to find the proper one in less time, which is the future work of the research.



---

*Source: 454867-2014-07-08.xml*