Composite attacks‐based copy‐move image forgery detection using AKAZE and FAST with automatic contrast thresholding

IET Image ProcessingVolume 14, Issue 17 p. 4528-4542 Research ArticleFree Access Composite attacks-based copy-move image forgery detection using AKAZE and FAST with automatic contrast thresholding Anuja Dixit, Corresponding Author Anuja Dixit anu2010cse1@gmail.com orcid.org/0000-0002-8649-0463 Department of Computer Science and Engineering, Indian Institute of Technology (Indian School of Mines), Dhanbad, IndiaSearch for more papers by this authorSoumen Bag, Soumen Bag Department of Computer Science and Engineering, Indian Institute of Technology (Indian School of Mines), Dhanbad, IndiaSearch for more papers by this author Anuja Dixit, Corresponding Author Anuja Dixit anu2010cse1@gmail.com orcid.org/0000-0002-8649-0463 Department of Computer Science and Engineering, Indian Institute of Technology (Indian School of Mines), Dhanbad, IndiaSearch for more papers by this authorSoumen Bag, Soumen Bag Department of Computer Science and Engineering, Indian Institute of Technology (Indian School of Mines), Dhanbad, IndiaSearch for more papers by this author First published: 03 March 2021 https://doi.org/10.1049/iet-ipr.2020.1118Citations: 2AboutSectionsPDF ToolsRequest permissionExport citationAdd to favoritesTrack citation ShareShare Give accessShare full text accessShare full-text accessPlease review our Terms and Conditions of Use and check box below to share full-text version of article.I have read and accept the Wiley Online Library Terms and Conditions of UseShareable LinkUse the link below to share a full-text version of this article with your friends and colleagues. Learn more.Copy URL Share a linkShare onFacebookTwitterLinkedInRedditWechat Abstract Copy-move image forgery is one of the most popular image tampering technique which can be performed for vicious purposes. In this forgery technique, selected region is copied and pasted at different locations on the same image to produce a manipulated image. Such forgery is denigratory as it can alter the image content by hiding or appending visual information. In this study, the authors propose a novel keypoint-based technique to detect forged images sustaining composite attacks consisting of various combinations of geometrical and post-processing operations. In the authors' method, AKAZE and FAST techniques are used to extract keypoints from the image. Non-maximal value suppression with automatic contrast thresholding is performed during FAST keypoint extraction. SIFT and DAISY descriptors are computed corresponding to extracted keypoints. PCA is applied over SIFT and DAISY descriptors to discard lower components which are sensitive to distortions occurred in images. They apply a correlation-based nearest neighbour search technique to detect similarity among keypoint descriptors. HDBSCAN algorithm is applied to obtain matched keypoint clusters. Further, RANSAC algorithm is utilised for removal of keypoint outliers. In comparison to state-of-the-art techniques, their approach achieve high F-measure (%) and low FPR (%) for image-level as well as pixel-level copy-move forgery detection. 1 Introduction Due to intensified usage of imaging technology, various types of image processing softwares are developed. These softwares can alter the original image by manipulating the information conveyed by the image. The alterations performed by these softwares are done so exquisitely that they cannot be spotted by human eyes. Manipulated images can cause harmful consequences of raising negative disturbances in society. Therefore, assurance of authenticity of digital images is one of the significant subjects of research. In the field of digital image forensics [1], copy-move [2] is one of the well-known technique to forge an image. In this technique, a portion of the image is copied and pasted over different areas of the same image. This technique uses a replicated region of the image to add or hide significant objects present in the original image. The copied section of the image can be altered using several geometrical and post-processing attacks, as shown in Fig. 1 to make the detection of forged image h,arder. Broadly, copy-move forgery detection utilises two approaches: block-based and keypoint-based. In block-based approaches, image is partitioned into overlapping blocks. Block features are extracted and compared to identify forged images. In opposite to that, keypoint-based methods are applied over the complete image without imposing the requirement of division of an image into subregions for forgery detection analysis. Fig. 1Open in figure viewerPowerPoint Examples of copy-move forged images (a) Original images collected from Ardizzone et al. [3], CASIA ITDE V 2.0 [4], CoMoFoD [5], IMD [6], and USC-SIPI [7] data sets, (b) Copy-move forged images sustaining composite attacks where forged regions are outlined with a yellow colour rectangle Liu et al. [8] proposed a block-based approach. They employed circular blocks for image division. Further, they computed Hu moments as feature vectors for blocks. They compared these feature vectors for forgery detection. An improved approach for forgery detection was proposed in [9]. They employed Zernike moment computation with locality sensitive hashing (LSH) for efficient block matching. Cozzolino et al. [10] applied circular harmonic transform (CHT) to obtain image segment features. They used PatchMatch algorithm to find a dense approximate nearest neighbour matches between patches of the image. Their method failed to detect forged regions with large resizing operation. Dixit et al. [11] proposed a scheme using stationary wavelet transform (SWT) with singular value decomposition (SVD). They applied a novel approach using colour-based segmentation to achieve blur invariance. Hosny et al. [12] employed quaternion polar complex exponential transform moments (QPCETMs) for feature extraction from subimages. They computed the Euclidean distance to detect the matched duplicated regions. Although, the block-based methods can effectively detect the forged region in the case of a plain copy-move attack but these methods have several drawbacks, including a high number of computations due to the processing of a large number of blocks. The number of keypoints extracted in keypoint-based approaches are much smaller than the number of blocks obtained in block-based approaches. In addition, block-based methods can not detect forged regions with significant geometrical transformations which stimulated the development of keypoint-based methods. Li et al. [13] applied keypoint-based approach using the segmentation technique. They detected forgery using two phases approach. They utilised simple linear iterative clustering (SLIC) technique in the first phase to segment the input image so that meaningful irregular shaped superpixels can be obtained which resulted in segmentation of input image in semantically independent patches. In the second phase, they employed expectation-maximisation (EM) algorithm to iteratively estimate the transformation matrix. Pun et al. [14] utilised simple linear iterative clustering (SLIC) and Weber local descriptor (WLD) in the first stage for feature extraction. In the second stage, they employed discrete analytic Fourier–Mellin transform (DAFMT) with LSH for a forgery detection. Pun and Chung [15] used adaptive oversegmentation for the image partition. Further, they performed keypoint matching. Ardizzone et al. [3] used triangle of keypoints for forgery detection. Jin and Wan [16] utilised scale invariant feature transform (SIFT) and opponent SIFT for feature vector computation. Li and Zhou [17] used hierarchical feature point matching. Forged images bearing rotation with scaling attacks were identified using their approach. Selvaraj and Karuppiah [18] proposed a technique based on the SIFT keypoints extraction from image followed by sensitivity-based clustering of keypoints. 1.1 Motivation Previously, many researchers have worked upon the detection of copy-move forged images. We have studied the limitations present in the existing works which are as follows: M1: In keypoint-based approaches, several techniques [16, 19-22] have been suggested by the researchers for forgery detection, but they lack the capability of extraction of a sufficient number of keypoints from images in homogeneous regions. This limitation has proved to be an obstacle in identifying forged regions hidden behind homogeneous regions of the image. M2: In the literature [3, 8-13, 15-18], there are several studies related to detection of forged images sustaining individual geometrical and post-processing attacks. But there is no such previous study, which is mainly focused towards detection of copy-move forged images suffering from different composite attacks. M3: In the literature, several matching techniques have been employed to detect the similar blocks/keypoints in the images to identify the forged images, but these techniques [20-23] have obtained high false positive rate (FPR) which resulted in detection of even authentic images as tampered. M4: There is a lack of such techniques in the literature which can deal with the detection of forged images having different properties, such as variation in image size, variation in the size of the forged region, file format used to store the image, image category, etc. all at once. 1.2 Our contribution To overcome the limitations present in existing techniques of copy-move forgery detection, we propose a novel keypoint-based approach. The primary contributions of the proposed method are following: C1: Our approach utilises features from accelerated segment test (FAST) and accelerated KAZE (AKAZE) techniques to obtain sufficiently a large number of keypoints for the images having smooth regions which ensures the effective forgery detection even in large homogeneous regions. Our technique also utilises automatic contrast thresholding concept to enable the capability of detection of a sufficient number of keypoints even for the input images with varying contrast adjustment ranges. We also employ non-maximal value suppression to remove the insignificant keypoints. C2: Detection of forged images can be highly complex when tampered regions sustain composite attacks. Due to the manipulation in image regions their corresponding feature descriptors also get altered. Hence, we propose a technique utilising SIFT and DAISY descriptors which can effectively compute invariant descriptors even for the regions containing high level of composite attacks. C3: Our scheme utilises a correlation-based nearest neighbour search technique for keypoint similarity identification. Further, we utilise hierarchical density-based spatial clustering of applications with noise (HDBSCAN) algorithm for hierarchical clustering of similar keypoints. In forgery detection approaches, we are mainly interested in the regions of image having highly populated areas with similar keypoints. HDBSCAN provides significantly remarkable outcomes which are capable to declare sparse regions as noise while dense regions of keypoints as clusters. After HDBSCAN clustering, we utilise affine transformation estimation for outlier removal which contribute in achieving reduced FPR. C4: We formulate tampered images sustaining composite attacks using authentic images collected from Ardizzone et al. [3], CASIA ITDE V 2.0 [4], CoMoFoD [5], IMD [6], and USC-SIPI [7] data sets. We propose a technique which can effectively detect forged images belonging to miscellaneous categories, several file formats, different sizes, and varying forged region sizes. The rest of the paper is organised as follows. The proposed methodology is explained in Section 2. The experimental results and comparison with existing methods are illustrated in Section 3. Finally, this work is concluded in Section 4. 2 Proposed method To detect tampered images manipulated using composite geometrical and post-processing attacks, the proposed methodology is divided into six steps: keypoint extraction, descriptor computation, keypoint matching, clustering over keypoints, outlier removal, and correlation map generation for localisation of forged regions. These steps are discussed in the following sections. As pre-processing operation, RGB colour input image is converted into grey-scale image. 2.1 Keypoint extraction We have used two different keypoint extraction techniques, namely FAST and AKAZE to obtain a sufficient number of keypoints, even for the images with large homogeneous regions which are discussed in the following sections. 2.1.1 FAST keypoint extraction FAST keypoint extraction technique detects corner points by analysing the self-dissimilarity of the candidate pixel with their surrounding pixels. The FAST algorithm [24] works by perceiving circle of pixels around the candidate for the corner point. Candidate point is considered as the corner if several contiguous pixels corresponding to the surroundings are brighter or darker as compared to candidate pixel. Continuous brighter or darker pixels around a point show the low dissimilarity of the point with its neighbouring circular area, which determines the candidate point as corner point. For every pixel p, surrounding pixel values are stored in the form of a vector. Each of the surrounding pixels is classified into three states as follows: S x = darker , I x ≤ I p − th brighter , I x ≥ I p + th similar , I p − th < I x < I p + th (1) There exist various choices for the number of contiguous similar or dissimilar pixels ( L arc ) [24] as 9, 10, 11, and 12. Among these, we have employed FAST 9 − 16 technique, which provides most effective results. Hence, each pixel point around the surrounding area x ∈ { x 1 , x 2 , … , x 16 } belongs to one of the three states of S x , when the intensity value of the surrounding point I x is compared with the pixel values of candidate point I p . FAST detector utilises the proportion of circular arc length in the place of proportion of the surrounding area to compute self-dissimilarity of selected pixels. The maximum angle of identified corner point is based on the arc length ( L arc ). Large values of L arc increament the repeatability of corner detection, whereas the threshold (th) governs the number of corner points. High values of th produce few but strong corners, while small th provides more corners having smooth gradients. Non-maximal value suppression: Detection of several keypoints contiguous to one another is the problem related to the raw FAST algorithm. So, we have employed non-maximal value suppression technique over detected keypoints, which utilises the following three steps: A score function F n m s is determined for each of the detected interest points, which can be computed by finding the sum of absolute intensity difference between the detected keypoint and the keypoints present in contiguous arc ( L arc ) as follows: F n m s = max ∑ ( I x − I p ) , ( I x − I p > th ) ∑ ( I p − I x ) , ( I x − I p < th ) (2) F n m s values are compared between the keypoints detected in adjacent locations. Keypoints having lower F n m s value are discarded. Non-maximal suppression process compares the adjacent interest points and removes the comparatively insignificant keypoints. Automatic contrast threshold determination: FAST technique detects corner points using a contrast threshold (th), which affects the FAST keypoint extraction outcome for the images having different contrasts. In fact, grey-scale values as well as contrast of images deviate a lot due to different illumination conditions in which images are captured. There is no such threshold which can be uniformly used for all the images. We have performed two experiments, one of them is to observe the effect of varying threshold over the number of keypoints extracted, while another is to analyse the effect of change in contrast of an image over the number of FAST keypoints extracted when the fixed contrast threshold is utilised. In the first experiment, we have used an image with contrast range [ 0 , 1 ] and having a resolution of 800 × 507 pixels. The threshold between 0 to 90 is applied over the image with an interval of 10, as shown in Fig. 2a. For th = 20, the highest number of keypoints ( 15 , 100) is obtained while the number of keypoints gradually decreases for increasing value of threshold. For th = 90, only 3 , 100 keypoints are obtained. Fig. 2b shows the results of a number of keypoints extracted from an image with varying contrast where the threshold is fixed at 35. The pictorial results represented in Fig. 3 show that the number of keypoints detected depends on the contrast of the image. The number of keypoints is gradually decreasing for images having a small range of contrast. Hence, number of FAST keypoints are highly dependent on the image contrast and the threshold th. So, there is a firm requirement for selection of such threshold which is adaptable to different images. Fig. 2Open in figure viewerPowerPoint Variation in the number of keypoints extracted (a) For different thresholds, (b) For different contrast adjustment ranges Fig. 3Open in figure viewerPowerPoint Effect on the number of keypoints extracted using FAST technique with fixed threshold (th) = 35 from images having different contrast adjustment ranges. The images contain forged region bearing composite attack with downscaling: 70 % and shearing: 38 operations Different images have different contrasts so a universal threshold cannot be suitable for all the images, hence we have employed an automatic thresholding technique. Contrast of an image is based on the grey-scale distribution in the image. For narrow grey-scale distribution, contrast and entropy of the image are low but for even grey-scale distribution, contrast and entropy of the image are high. Therefore, entropy of an image can also describe its contrast. The normalised entropy of an image can be computed as follows: entropy norm = − ∑ i = 0 255 P i log 2 ⁡ ( P i ) log 2 ⁡ ( L ) (3) P i shows the proportion of pixels having grey-level value i, where i varies from 0 to 255. L represents the total number of grey levels in an image. As the normalised entropy varies according to the image contrast, so the following steps are performed to find out the adaptive threshold based on the entropy. At first, the reference threshold ( th 0 ) is determined. We have initialised th 0 as 35. The normalised entropy for the input image is computed. In the next step, an automatic contrast threshold ( t h auto ) is computed as follows: th auto = ( 2 entropy norm − 1 ) × th 0 (4) The exponential function facilitates the non-linear mapping of the threshold value in interval ( 0 , th 0 ). 2.1.2 AKAZE keypoint extraction FAST technique is resistant towards common image distortion attacks, but feature points located near image boundaries remain undetected due to consideration of circular arc for comparison of neighbouring pixel values during detection of FAST keypoints. So, we have also included AKAZE algorithm for keypoint detection. AKAZE algorithm is based on non-linear diffusion filtering, which formulates stable non-linear scale space to resolve the problem of boundaries and detail loss. AKAZE keypoint extraction technique meets the requirement of accuracy as well as high efficiency during keypoint extraction. By using two keypoint extraction techniques, namely FAST and AKAZE, we have obtained sufficient number of keypoints even for boundary as well as homogeneous regions. AKAZE technique [25] builds non-linear scale space. As there is no analytical solution for non-linear partial differential equation, so AKAZE algorithm utilises fast explicit diffusion (FED) algorithm to obtain an approximate solution and form the image pyramid. Interest points are detected by comparing pixels to their 26 neighbours for 3 × 3 regions in current as well as adjacent scales. By considering detected interest points as the centre, we identify the principal direction for the search radius using sampling step which ensures the rotation invariancy of interest points. For all pixels of the image, first-order differential values are weighted using Gaussian centred at the interest point. The resultant weighted values are considered as response value for image pixels. All the response values present in sliding window are added to the sector area. The entire circular area is traversed with sector area. The direction of the highest response value for a sector region shows main direction of the keypoints. 2.2 Keypoint descriptor computation To obtain invariancy towards geometrical and post-processing attacks, SIFT and DAISY descriptor computations are performed corresponding to the keypoints obtained using FAST and AKAZE techniques, which are discussed in the following sections. 2.2.1 SIFT descriptor computation SIFT descriptor [16] is calculated for keypoints obtained using the FAST algorithm which are distinct and robust against various distortions, such as variation in JPEG compression and noise addition. Keypoint descriptor is obtained by computing orientation histograms for 4 × 4 pixel region around keypoint. The orientation histograms possess relativity towards the orientation of keypoints and the orientation data, achieved from the Gaussian image which is closest to the scale of the keypoint. Each histogram contains 8 bins where each descriptor possesses a 4 × 4 array having 16 histograms around the selected keypoint which produces SIFT feature vector ( D sift ) having 128 ( 4 × 4 × 8 ) elements. The SIFT descriptor is resilient towards deformations caused due to perspective effects. The smooth manipulations in scale, space, and orientation do not cause radical changes over feature vector. 2.2.2 DAISY descriptor computation DAISY descriptor is utilised to compute descriptor corresponding to AKAZE keypoints. For DAISY descriptor computation, a primary orientation is allocated to every keypoint. For a keypoint coordinate, gradient magnitude, local histogram of neighbourhood centred at keypoint, and orientation for a circular region are computed. To identify the canonical orientation, the orientation of every pixel within a circular window is weighted and added to the histogram. Resultant histogram is quantised into 36 bins, hence each of bin covers about 10 °. Tola et al. [26] used R, Q, T, and H parameters, as shown in Table 1. The total number of grid points can be computed as S = Q × T + 1. The dimensionality of DAISY descriptor is computed using D = S × H. Hence, a DAISY descriptor is obtained which comprises histogram for D elements and considers the circular pixel area having radius R around the keypoint. DAISY descriptor possesses robustness against geometrical and post-processing attacks which causes precise detection of tampered regions within forged image. To obtain DAISY descriptor corresponding to AKAZE keypoints, we have initialised parameters as H = 8, Q = 3, and T = 4. DAISY descriptor produces keypoint descriptor ( D daisy ) of length 104. Table 1. Symbols used for computation of DAISY descriptor Parameter Description R distance of outermost grid point from the centre point Q number of rings across the centre grid point T number of grids in each ring H number of bins for a grid point S total number of grid points, S = Q × T + 1 D size of DAISY descriptor vector, D = H × S 2.2.3 Dimensionality reduction using PCA After achieving descriptors using SIFT and DAISY technique, we have applied PCA [27] over feature descriptors for dimensionality reduction. The PCA technique provides a less number of principal components or artificial variables which are responsible to represent high variance existing in the observed variables when a large number of observed variables are available. The principal components obtained are utilised as predictor variables during subsequent analysis. Basically, PCA utilises a q × p matrix denoted by M, where q and p shows the number of dimensions and total number of feature vectors, respectively. As a first step, a mean vector corresponding to each dimension is computed which is represented by V mean . Further, V mean is subtracted from each feature vector present in M. In the next step, covariance matrix M × M T is computed which is having dimension q × q. As the covariance matrix is a square matrix so q eigen-values and their corresponding q-dimensional eigen-vectors are obtained. As higher eigen-values contain a large quantity of information so eigenvectors are ordered based on the corresponding eigenvalues. PCA projection matrix M projection of the dimension q × q is achieved, which represent eigenvectors ordered from higher to lower based on the eigenvalues. Each row of M projection matrix shows an eigenvector. For x × q data matrix, which is denoted as M data , x represents the number of q-dimensional feature descriptors. The dimension of feature descriptors is reduced by projecting them over the first required features from each vector in M projection . The feature vector obtained after utilisation of PCA is significantly smaller than feature vector obtained using standard SIFT and DAISY techniques. We have initialised dimensionality of the feature space as PCA comp = 32. 2.3 Keypoint matching We have obtained feature keypoints { kp 1 , kp 2 , kp 3 , … , kp m } using FAST and AKAZE techniques. Feature descriptors { F 1 , F 2 , F 3 , … , F m } corresponding to keypoints are computed using SIFT and DAISY descriptors, where m represents the number of keypoints and their corresponding descriptors. After utilisation of PCA over SIFT and DAISY descriptors, we have obtained a reduced length of feature vector with more resilience towards composite attacks. We have employed correlation-based nearest neighbour search to find similar feature descriptors. In this matching technique, correlation is computed between feature descriptors as follows: corr = ∑ i = 1 n f ( F x i − F x ¯ ) ⋅ ( F y i − F y ¯ ) ∑ i = 1 n f ( F x i − F x ¯ ) 2 ⋅ ∑ i = 1 n f ( F y i − F y ¯ ) 2 (5) where F x i and F y i represent the components of the Fx and Fy feature descriptors, respectively. F x ¯ and F y ¯ denote mean of Fx and Fy, respectively. n f shows the length of the feature descriptor. For a descriptor, the sorted correlation values with other decsriptors are shown by the vector Corr = { corr 1 , corr 2 , corr 3 , … , corr m − 1 }, where m denotes the total number of feature descriptors. To identify nearest neighbour descriptors, a ratio is computed by dividing the nearest correlation value to the second nearest correlation value. The obtained ratio is compared with a pre-defined threshold value V t . The keypoints are considered similar for which corresponding descriptors satisfy the constraint as follows: corr j / corr j + 1 ≥ V t (6) where V t ∈ [ 0 , 1 ]. The nearest neighbour test includes multiple iterations between corr j and corr j + 1 until the ratio achieved is greater than or equal to V t , where 1 ≤ j ≤ m − 2. Iterative process takes several number of iterations to stop and the corresponding keypoints are considered as matched for the selected descriptors. We have used V t = 0.85 for experimentation. All possible matching keypoints are detected by comparing the corresponding descriptors using correlation-based nearest neighbour search technique. 2.4 Clustering over keypoints To obtain clusters of matched keypoints, we have employed HDBSCAN technique. This technique is able to achieve clusters of arbitrary shapes. It can also handle outliers and noisy data effectively by providing the separate class of outlier points which does not belong to any other clusters. This technique can deal with varying density data clusters and also provides the facility to extract the important clusters by allowing the user to decide the size of clusters. It can perform effective clustering over high-dimensional data. HDBSCAN technique is robust towards the selection of hyper parameters. HDBSCAN [28] introduces the concept of hierarchical clustering using the special treatment for shared boundary points. It merges the initially obtained clusters hierarchically. In such way, the HDBSCAN algorithm enhnces the inadequate clustering outcomes stimulated by unsuitable selection of input parameters. This clustering approach does not test and judge each and every keypoint, hence it also reduces computation overhead. It requires N number of points having D-dimensionality as well as distance metric d as input parameter. The algorithm is divided into different phases. At first, by using distance metric d, identification of Kth-nearest neighbour for each point is performed, where K represents a hyperparameter. The distance between Kth nearest neighbour and a point p is represented by a p . We used K = 10 for experimentation. For subsequent steps, HDBSCAN employs modified distance metric as follows: d ~ ( x , y ) = max ( a x , a y , d ( x , y ) )(7) The modified metric d ~ ( x , y ) is utilised to decrease sensitivity towards outliers. The minimum spanning tree is constructed for the selected points, where for all pairs of x and y there exists an edge joining them with weight d ~ ( x , y ). Different clusters are extracted by utilising minimum spanning trees. Cluster hierarchy is construc