Robust image hashing with visual attention model and invariant moments

IET Image ProcessingVolume 14, Issue 5 p. 901-908 Research ArticleFree Access Robust image hashing with visual attention model and invariant moments Zhenjun Tang, Corresponding Author tangzj230@163.com Guangxi Key Lab of Multi-Source Information Mining & Security, and Department of Computer Science, Guangxi Normal University, Guilin, 541004 People's Republic of ChinaSearch for more papers by this authorHanyun Zhang, Guangxi Key Lab of Multi-Source Information Mining & Security, and Department of Computer Science, Guangxi Normal University, Guilin, 541004 People's Republic of ChinaSearch for more papers by this authorChi-Man Pun, Department of Computer and Information Science, University of Macau, Macau, 999078 People's Republic of ChinaSearch for more papers by this authorMengzhu Yu, Guangxi Key Lab of Multi-Source Information Mining & Security, and Department of Computer Science, Guangxi Normal University, Guilin, 541004 People's Republic of ChinaSearch for more papers by this authorChunqiang Yu, Guangxi Key Lab of Multi-Source Information Mining & Security, and Department of Computer Science, Guangxi Normal University, Guilin, 541004 People's Republic of ChinaSearch for more papers by this authorXianquan Zhang, Guangxi Key Lab of Multi-Source Information Mining & Security, and Department of Computer Science, Guangxi Normal University, Guilin, 541004 People's Republic of ChinaSearch for more papers by this author Zhenjun Tang, Corresponding Author tangzj230@163.com Guangxi Key Lab of Multi-Source Information Mining & Security, and Department of Computer Science, Guangxi Normal University, Guilin, 541004 People's Republic of ChinaSearch for more papers by this authorHanyun Zhang, Guangxi Key Lab of Multi-Source Information Mining & Security, and Department of Computer Science, Guangxi Normal University, Guilin, 541004 People's Republic of ChinaSearch for more papers by this authorChi-Man Pun, Department of Computer and Information Science, University of Macau, Macau, 999078 People's Republic of ChinaSearch for more papers by this authorMengzhu Yu, Guangxi Key Lab of Multi-Source Information Mining & Security, and Department of Computer Science, Guangxi Normal University, Guilin, 541004 People's Republic of ChinaSearch for more papers by this authorChunqiang Yu, Guangxi Key Lab of Multi-Source Information Mining & Security, and Department of Computer Science, Guangxi Normal University, Guilin, 541004 People's Republic of ChinaSearch for more papers by this authorXianquan Zhang, Guangxi Key Lab of Multi-Source Information Mining & Security, and Department of Computer Science, Guangxi Normal University, Guilin, 541004 People's Republic of ChinaSearch for more papers by this author First published: 04 March 2020 https://doi.org/10.1049/iet-ipr.2019.1157Citations: 5AboutSectionsPDF ToolsRequest permissionExport citationAdd to favoritesTrack citation ShareShare Give accessShare full text accessShare full-text accessPlease review our Terms and Conditions of Use and check box below to share full-text version of article.I have read and accept the Wiley Online Library Terms and Conditions of UseShareable LinkUse the link below to share a full-text version of this article with your friends and colleagues. Learn more.Copy URL Share a linkShare onEmailFacebookTwitterLinked InRedditWechat Abstract Image hashing is an efficient technique of multimedia processing for many applications, such as image copy detection, image authentication, and social event detection. In this study, the authors propose a novel image hashing with visual attention model and invariant moments. An important contribution is the weighted DWT (discrete wavelet transform) representation by incorporating a visual attention model called Itti saliency model into LL sub-band. Since the Itti saliency model can efficiently extract saliency map reflecting regions of attention focus, perceptual robustness of the proposed hashing is achieved. In addition, as invariant moments are robust and discriminative features, hash construction with invariant moments extracted from the weighted DWT representation ensures good classification performance between robustness and discrimination. Extensive experiments with open image datasets are done to validate the performances of the proposed hashing. The results demonstrate that the proposed hashing is robust and discriminative. Performance comparisons with some hashing algorithms are also conducted, and the receiver operating characteristic results illustrate that the proposed hashing outperforms the compared hashing algorithms in classification performance between robustness and discrimination. 1 Introduction Wide applications of Internet and smart cellphone make our colourful lives, but also bring some new problems. For example, many users would like to take photos by the camera of smart cellphone and share their photos with friends via a social platform, such as Facebook or Wechat. In general, most photos are high resolutions and have big file sizes, which will increase the costs of image transmission and image storage. To overcome these issues, users often reduce resolutions of images or compress images by image processing tool (e.g. MeiTu App or ACDSee) before uploading them to the social platform. This introduces a practical problem called image copy detection. During image editing, users may save several versions of a photo for selection. In addition, some users may share the photos shared by their friends on the social platform. Therefore, several copies of a photo may be stored in the social platform. How to efficiently detect image copies of a given image is an important research topic of image processing community. In the past years, a useful technique called image hashing [1] is proposed to solve the problem of image copy detection [2]. Generally, image hashing should meet two fundamental properties [3, 4]: robustness and discrimination. The requirement of robustness property is that image hashing should generate a short representation called image hash based on the visual content of the input image. Specifically, if two images are visually similar (one may be a copy of the other), image hashing should map them to the same or similar hashes regardless of their resolutions and the bits of their files. This property ensures that image hashing can quickly find similar versions of an image. For the discrimination property, it demands that image hashing should produce different hashes for images with different contents. This means that image hashing can efficiently distinguish different images according to their visual contents. It is an important task to design an efficient image hashing algorithm with a good classification between robustness and discrimination. Image hashing can efficiently process image data due to the low costs of hash storage and hash similarity. Besides image copy detection, it has been successfully applied to many applications, such as copy-move forgery detection [5, 6], image forensics [7], image authentication [8, 9], image quality assessment [10], social event detection [11] and tamper detection [12]. In the literature, many useful techniques are exploited to design efficient image hashing algorithms for various applications. The typical early techniques include block histogram [1], two-dimensional DWT (discrete wavelet transform) [3], Radon transform [13], matrix factorisation [14], Fourier-Mellin transform (FMT) [15] and feature points [16]. In the past decade, many researchers contributed to the rapid progress of image hashing. For example, Khelifi and Jiang [17] used the theory of detecting virtual watermarks to design image hashing. Aiming at image authentication, Ahmed et al. [18] jointly exploited SHA and DWT to construct a hashing algorithm. To improve discrimination of hashing algorithm for colour images, Tang et al. [19] decomposed a colour image into six colour components with HSI colour space and YCbCr colour space and selected invariant moments of these components to construct robust hash. To improve rotation robustness, Li et al. [20] extracted rotation-invariant features with random Gabor filtering and conducted feature compression by vector quantisation. Zhao et al. [21] calculated Zernike moments and local features to make robust hashing for the application of image authentication. In [22], Tang et al. extracted colour image features by using a colour vector angle (CVA) and exploited 2D DWT to compress features. In another work, Li et al. [23] exploited quaternion polar cosine transform to derive rotation-invariant features for hash construction. To enhance hashing robustness against image rotation, Tang et al. [24] used the Canny operator to detect image edges and calculated statistical features based on CVAs and image edges. In another study, Tang et al. [25] divided the input image into some rings resistant to rotation and computed invariant distances between ring vectors to form image hash. This ring-based hashing can resist image rotation with any angle. Recently, Huang et al. [26] used a random walk on zigzag blocking to improve the security of image hashing. Qin et al. [27] exploited block truncation coding to design an image hashing algorithm with good perceptual robustness. Yan et al. [28] designed an efficient hashing algorithm with quaternion FMT for tampering detection. Karsh et al. [29] proposed a robust hashing algorithm by combining the spectral residual model, DWT, and singular value decomposition (SVD). Tang et al. [30] first proposed to use multidimensional scaling and log-polar transform to derive robust hash resilient to the rotation. In [31], Li and Guo exploited sparse coding to construct an image hash for content identification. In another work, Qin et al. [32] combined CVAs, Canny operator, and SVD to extract circle-based and block-based features for colour image hash. This hybrid features-based hashing can make a good trade-off between robustness and discrimination. Most existing image hashing algorithms have good performances in their target applications. However, they do not reach desirable classification performance between robustness and discrimination. In this paper, we propose a novel image hashing based on visual attention model and invariant moments. Compared with the existing algorithms, our proposed image hashing has two highlights as follows: (i) We extract a weighted DWT representation by incorporating visual attention model into LL sub-band. Since JPEG compression and image noises have a slight influence on the DWT coefficients of LL sub-band, hash construction based on LL sub-band can make our hashing resistant to these digital operations. The number of DWT coefficients in the LL-sub-band is about 1/4 of the pixel number of the input image. Consequently, the use of DWT achieves initial compression. In addition, a classical visual attention model called Itti saliency model is exploited to find a saliency map of the image. As saliency map can reflect the regions of attention focus in the human eye, hash extraction with the weighted DWT representation generated by saliency map and LL-sub-band can ensure perceptual robustness of the proposed algorithm. (ii) We calculate robust image hash by extracting invariant moments from the weighted DWT representation. The invariant moments are not only robust against geometric transform, but also can efficiently discriminate images with different contents. As the weighted DWT representation can indicate visual attention of the human eye, invariant moments extracted from the weighted representation can improve perceptual robustness and thus make our algorithm reach good classification performance between robustness and discrimination. Extensive experiments are conducted on the open benchmark image datasets. The results demonstrate that our image hashing is robust to many digital operations and has good discrimination. Comparisons illustrate that our image hashing outperforms some baseline algorithms in classification performance between robustness and discrimination. The rest of the paper is organised as follows. Section 2 explains the details of our proposed image hashing. Section 3 discusses experimental results, and Section 4 presents performance comparisons. Section 5 concludes this paper. 2 Proposed image hashing Schematic diagram of the proposed image hashing is illustrated in Fig. 1. Firstly, the input colour image is converted to a normalised image with a fixed size S × S by the bi-cubic interpolation. This is to ensure that our image hashing can resist the operation of image scaling and the hashes of different images with various resolutions have the same length. Next, a weighted image representation reflecting visual attention of human eye is calculated by incorporating saliency map into the LL sub-band of one level 2D DWT in YCbCr colour space. Finally, invariant moments extracted from the weighted representation are used to form image hash. The saliency map detection, weighted representation calculation, and moment extraction are detailed in the following sections. Fig. 1Open in figure viewerPowerPoint Schematic diagram of the proposed image hashing 2.1 Saliency map detection Visual attention model can efficiently detect the saliency map. Since the saliency map can reflect the regions of attention focus on the human eye, hash construction with saliency map can ensure perceptual robustness of the proposed image hashing. Itti model is a well-known visual attention model proposed by Itti et al. [33]. It has been successfully used in many applications, such as video object segmentation [34], image search [35], image classification [36], and image feature detection [37]. The Itti model exploits dyadic Gaussian pyramids, centre-surround operations, across-scale operations, and linear combination to find a saliency map. In general, it can be divided into four key steps as follows: (i) Saliency map of colours Mcolours is extracted by Gaussian pyramids, centre-surround operations, and across-scale combinations. (ii) Similarly, the saliency map of intensity Mintensity is detected through Gaussian pyramids, centre-surround operations, and across-scale combinations. (iii) Saliency map of orientations Morientations is also determined by Gaussian pyramids, centre-surround operations, and across-scale combinations. (iv) The final saliency map M is obtained by linearly combining the saliency maps of colours, intensity, and orientations as follows: (1)More details of the Itti model can be referred to [33]. In this paper, we select the Itti model to find the saliency map of the interpolated image. This is because it can reach a good balance between computational cost and saliency detection performance. Fig. 2 illustrates an instance of saliency map detection via the Itti model. Fig. 2a is an original image, Fig. 2b is the detected saliency map of colours, Fig. 2c is the detected saliency map of intensity, Fig. 2d is the detected saliency map of orientations and Fig. 2e is the final saliency map by combining Figs. 2b–d. Clearly, the final saliency map is closely matched with the focus of attention. Fig. 2Open in figure viewerPowerPoint Instance of saliency map detection (a) Original image, (b) Map of colours, (c) Map of intensity, (d) Map of orientations, (e) Final map 2.2 Weighted representation calculation Our weighted representation is calculated by incorporating the saliency map into the LL-sub-band of one level 2D DWT in YCbCr colour space. To do so, input colour image is first converted to YCbCr colour space, and the luminance component is taken for the input of 2D DWT. Conversion from RGB colour space to YCbCr colour space can be done by the following equation: (2)where R, G, and B are the red channel, green channel and blue channel of a pixel, respectively, Y is its luminance component, Cb and Cr are its blue-difference and red-difference chroma components, respectively. Next, one-level 2D DWT is applied to the luminance component of the input colour image. Fig. 3 demonstrates the schematic diagram of one-level 2D DWT. Clearly, the results of one-level 2D DWT include four sub-bands, i.e. LL sub-band, LH sub-band, HL sub-band, and HH sub-band. Note that the DWT coefficients in the LL sub-band are the approximation coefficients, those in the LH sub-band are the detail coefficients in the horizontal direction, those in the HL sub-band are the detail coefficients in the vertical direction, and those in the HH sub-band are the detail coefficients in a diagonal direction. Here, we take the DWT coefficients in the LL sub-band for representation. This is based on the following considerations. Compared with the coefficients in other sub-bands, those in the LL sub-band are slightly influenced by some digital operations, such as noise contamination, low-pass filtering, and JPEG compression. Therefore, the feature extracted from the LL sub-band can make robust hash. In addition, the number of coefficients in the LL sub-band is about 1/4 of the pixel number of the input image. Consequently, the use of DWT achieves initial compression. Let D be the matrix of the coefficients in the LL sub-band. Suppose that the size of D is N × N, where N = ⌊S/2⌋ and ⌊·⌋ is the downward rounding operation. To calculate a weighted representation based on the LL sub-band and saliency map, the saliency map M is downsampled to N × N size by bi-cubic interpolation since its size is S × S. Let J be the sampled version of M. Thus, the weighted representation I can be determined as follows: (3)where I(i,j), D(i,j), and J(i,j) are the elements of I, D, and J in the ith row and the jth column, respectively. Fig. 3Open in figure viewerPowerPoint Schematic diagram of one-level 2D DWT 2.3 Moment extraction Invariant moments are useful image features proposed by Hu [38]. They are geometric invariant and have been successfully used in many applications, such as image retrieval [39], image recognition [40, 41], and image classification [42]. In general, the seven classical invariant moments [38] are defined as follows: (4) (5) (6) (7) (8) (9) (10)where (p, q = 0, 1, 2, …) is the normalised central moment determined by (11)in which and is the (p + q)th-order central moment given by the following equation: (12)where is the grey pixel value of digital image sized m × n , and are defined as follows: (13) (14)where is the (p + q)th-order moment of a digital image as follows: (15)In this paper, we select invariant moments of the weighted representation I as image features and exploit them to generate the robust hash. More specifically, the weighted representation I is firstly divided into blocks sized . Thus, there are Q = (N/b)2 blocks in total. For each block, the first k moments (k = 1, 2, …, 7) are selected to form image hash. Let be the moments extracted from the ith block (i = 1, 2, …, Q). Thus, a short image representation v is available by concatenating the moments of all blocks as follows: (16)Clearly, there are L = kQ elements in v. Let v(i) be the ith element of v (1 ≤ i ≤ L). To reduce storage cost of the element, v(i) is quantised to an integer by the following equation: (17)where round[·] is the function returning the integer part of the input number. Finally, image hash is generated by the following equation: (18)Therefore, our hash length is L integers. Obviously, our hash length is closely related to the selected number of invariant moments of each block, i.e. the k value. Section 3.4 will discuss our hashing performances under different k values. Here, we choose invariant moments as image features instead of Zernike moments. This is because the invariant moments are better than the Zernike moments in making robust image hashing. Section 3.6 will validate this. 2.4 Similarity evaluation To evaluate the similarity of two image hashes h1 and h2, the well-known similarity metric called L2 norm is taken. Let h1 = [h1(1), h1(2), …, h1(L)] and h2 = [h2(1), h2(2), …, h2(L)]. Thus, their L2 norm d is calculated by the following equation: (19)In general, for a pair of similar images, the L2 norm of their hashes is expected to be small. For two different images, the L2 norm of their hashes is expected to be large. Therefore, if the L2 norm d is bigger than a threshold T, the images of input hashes are judged as visually different images. Otherwise, they are considered as similar images. 3 Experimental results In the experiments, the used parameters of our hashing are as follows. The input image is converted to a fixed size 512 × 512 and the block size is 32 × 32. The first three invariant moments of each block are used. In other words, S = 512, b = 32, and k = 3. Thus, N = ⌊S/2⌋ = 256 and Q = (N/b)2 = (256/32)2 = 64. Consequently, our hash length is L = kQ = 3 × 64 = 192 integers. Moreover, our hashing is coded with MATLAB 2016a and runs on a personal computer with the following configurations. The frequency of the Intel Core i7-7700 CPU is 3.60 GHz, and the size of RAM is 8 GB. Section 3.1 verifies robustness performance, and Section 3.2 discusses the discriminative result. Section 3.3 evaluates storage cost of our hash. The effect of the selecting number of invariant moments and the effect of block size are discussed in Sections 3.4 and 3.5, respectively. Finally, comparisons between invariant moments and Zernike moments are analysed in Section 3.6. 3.1 Robustness To verify the robustness of our hashing, a benchmark image dataset called Kodak Loss True Colour Image Suite [43] is used. This dataset contains 24 colour images, and the sizes of these images are 768 × 512 or 512 × 768. To produce similar versions of these 24 images, the well-known tools called Photoshop, MATLAB, and StirMark [44] are exploited to make robustness attacks. Specifically, Photoshop provides brightness adjustment and contrast adjustment (4 parameters for each operation). The MATLAB provides gamma correction (4 parameters), 3 × 3 Gaussian low-pass filtering (8 parameters), speckle noise (10 parameters), and salt and pepper noise (10 parameters). The StirMark provides JPEG compression (8 parameters), watermark embedding (10 parameters), image scaling (6 parameters), and the combinational attack of rotation, cropping, and rescaling (10 parameters). Therefore, the total number of our used robustness attacks is 74. This means that every original colour image has 74 similar versions after robustness attacks. Consequently, there are 24 × 74 = 1776 pairs of similar images and the total number of the used images reaches 1776 + 24 = 1800 in the robustness experiment. Table 1 presents the maximum value, the minimum value, the mean, and the standard deviation of hash distances under different robustness attacks. From Table 1, it is observed that the mean values of all robustness attacks are smaller than 20. In addition, the maximum values of all attacks are smaller than 18, except the combinational attack of rotation, cropping, and rescaling. The maximum value of the combinational attack is 26.34, which is much bigger than those of other attacks. This is because the combinational attack will cause more distortions than single robustness attack. Therefore, if the threshold is selected as T = 20, our hashing can correctly detect 94.03% similar images. When the threshold reaches T = 25, our hashing can correctly identify 99.89% similar images. A high correct detection rate illustrates good robustness of our hashing. Table 1. Statistics of hash distances under different robustness attacks Robustness attack Max Min Mean Standard deviation brightness adjustment 11.83 3.46 6.31 1.52 contrast adjustment 7.21 2.83 4.48 1.07 gamma correction 16.40 5.48 9.07 2.80 3 × 3 Gaussian low-pass filtering 11.62 0 5.25 2.60 speckle noise 14.59 1.73 6.72 2.38 salt and pepper noise 14.83 2.83 7.87 2.48 JPEG compression 13.49 6.08 9.17 1.60 watermark embedding 17.20 2.24 7.97 2.93 image scaling 13.56 6.16 9.44 1.67 rotation, cropping, and rescaling 26.34 11.18 19.17 3.24 3.2 Discrimination The open image dataset called UCID [45] is exploited to evaluate the discriminative capability of our hashing. This dataset contains 1338 colour images sized 512 × 384 or 384 × 512. To test discrimination, hashes of these 1338 colour images are firstly extracted. Then, for each image, the L2 norms between its hash and the hashes of other 1337 images are computed. Consequently, there are valid L2 norms in total. We calculate the distribution of these distances and obtain the results, as shown in Fig. 4. In this figure, the x-axis represents the L2 norm, and the y-axis is the frequency of L2 norm. It is found that the minimum L2 norm is 14.25, the maximum L2 norm is 121.84, the mean value is 45.40, and the standard deviation is 11.26. Obviously, the mean value of different images is much bigger than the mean values of similar images mentioned in Section 3.1. This implies that our hashing reaches good discrimination. In fact, our performances of robustness and discrimination are closely related to the selected threshold. Generally, a big threshold will improve robustness, but it will also decrease discrimination. Table 2 presents the robustness and discrimination performances under different thresholds, where the robustness is indicated by the correct detection rate (similar images detected as similar images) and the discrimination is described by the false recognition rate (different images recognised as similar images). From Table 2, it can be seen that, when T = 25, the total error rate reaches the minimum value of 0.204%. This means that we can select T = 25 to make optimum whole performances with respect to robustness and discrimination. Table 2. Our correct detection rates, false recognition rates and total error rates under different thresholds Threshold Correct detection rate (R1), % False recognition rate (R2), % Total error rate = (1 − R1) + R2), % 20 94.0300 0.0005 5.9705 21 95.5500 0.0018 4.4518 22 97.2400 0.0042 2.7642 23 99.3200 0.0113 0.6913 24 99.7200 0.0345 0.3145 25 99.8900 0.0942 0.2042 26 100 0.2200 0.2200 27 100 0.4900 0.4900 Fig. 4Open in figure viewerPowerPoint Distribution of L2 norms between hashes of different images 3.3 Hash storage To evaluate the storage cost of our hash, hashes extracted in discrimination test are selected as the dataset for analysis. Note that our hash includes 192 hash elements. Therefore, the total number of hash elements in the dataset is 1338 × 192 = 256,896. We calculate the distribution of these hash elements and obtain the results, as shown in Fig. 5. In this figure, the x-axis is the value of the hash element, and the y-axis is its frequency. It can be seen that these element values are distributed between 0 and 60. Since 6 bits can represent an integer in the interval [0, 63 = 26 − 1], our hash element just requires 6 bits for storage. Therefore, the storage cost of our hash is 192 × 6 = 1152 bits. Fig. 5Open in figure viewerPowerPoint Distribution of hash elements 3.4 Effect of the selected number of invariant moments on hash performances The classical invariant moments include seven moments. To determine the optimal number of invariant moments in our hashing, we discuss the selected number of invariant moments, i.e. the k value. In the experiments, the used values are k = 1, k = 2, k = 3, k = 4, k = 5, k = 6, and k = 7; other parameters are kept unchanged. The image datasets mentioned in Sections 3.1 and 3.2 are used here. To analyse classification performances between robustness and discrimination, a well-known tool called ROC (Receiver Operating Characteristic) graph [46] is taken. In the ROC graph, the x-axis is generally defined as FPR (False Positive Rate) measuring discrimination, and the y-axis is described as TPR (True Positive Rate) indicating robustness. The FPR P1 and TPR P2 are determined by the following equations: (20) (21)where N1 is the number of different images judged as similar images, N2 is the number of different images, N3 is the number of similar images classified as similar images, and N4 is the number of similar images. Note that the ROC curve is plotted by some points with the coordinates (P1, P2), which can be obtained by substituting different T values into (20) and (21). In general, if there are several ROC curves in the graph, the curve above the other curves has better classification performance. To make a quantitative comparison, the area under the ROC curve (AUC) [46] is often used. Note that the range of AUC is [0, 1], and a bigger AUC means better classification performance. Fig. 6 demonstrates the ROC curves of differentk values. It is observed that the curves of k = 3,k = 4, k = 5, k = 6, andk = 7 are overlapping and above the curves ofk = 1 and k = 2. In addition, the AUCs ofk = 1, k = 2, k = 3,k = 4, k = 5, k = 6, andk = 7 are 0.99980, 0.99989, 0.99999, 0.99997, 0.99998,0.99997, and 0.99997, respectively. Clearly, the AUC of k = 3is the biggest one, which means that our classification performance withk = 3 is better than those performances with otherk values. Moreover, the total run time of generating 1338colour images is recorded, and the hash length is also compared. Table 3 presents our hash performances underdifferent k values. Table 3. Our hash performances under different k values k AUC Hash length(integer) Total run time,s 1 0.99980 64 190.617 2 0.99989 128 190.686 3 0.99999 192 190.786 4 0.99997 256 191.330 5 0.99998 320 191.412 6 0.99997 384 191.677 7 0.99997 448 191.833 Fig. 6Open in figure v