Enhanced copy–paste forgery detection in digital images using scale‐invariant feature transform

IET Image ProcessingVolume 14, Issue 3 p. 462-471 Research ArticleFree Access Enhanced copy–paste forgery detection in digital images using scale-invariant feature transform Priyadharsini Selvaraj, Corresponding Author Priyadharsini Selvaraj priyadharsini@mepcoeng.ac.in Department of Computer Science & Engineering, Mepco Schlenk Engineering College, Sivakasi, Tamil Nadu, IndiaSearch for more papers by this authorMuneeswaran Karuppiah, Muneeswaran Karuppiah Department of Computer Science & Engineering, Mepco Schlenk Engineering College, Sivakasi, Tamil Nadu, IndiaSearch for more papers by this author Priyadharsini Selvaraj, Corresponding Author Priyadharsini Selvaraj priyadharsini@mepcoeng.ac.in Department of Computer Science & Engineering, Mepco Schlenk Engineering College, Sivakasi, Tamil Nadu, IndiaSearch for more papers by this authorMuneeswaran Karuppiah, Muneeswaran Karuppiah Department of Computer Science & Engineering, Mepco Schlenk Engineering College, Sivakasi, Tamil Nadu, IndiaSearch for more papers by this author First published: 20 January 2020 https://doi.org/10.1049/iet-ipr.2019.0842Citations: 5AboutSectionsPDF ToolsRequest permissionExport citationAdd to favoritesTrack citation ShareShare Give accessShare full text accessShare full-text accessPlease review our Terms and Conditions of Use and check box below to share full-text version of article.I have read and accept the Wiley Online Library Terms and Conditions of UseShareable LinkUse the link below to share a full-text version of this article with your friends and colleagues. Learn more.Copy URL Share a linkShare onFacebookTwitterLinkedInRedditWechat Abstract Image forgery detection and localisation is one of the principal problems in digital forensics. Copy–paste forgery in digital images is a type of forgery in which an image region is copied and pasted at another location within the same image. In this work, the authors propose a methodology to detect and localise copy-pasted regions in images based on scale-invariant feature transform (SIFT). Existing copy-paste forgery detection in images using SIFT and clustering techniques such as hierarchical agglomerative and density-based spatial clustering of applications with noise resulted many false pixel detections. They have introduced sensitivity-based clustering along with SIFT features to identify copy–pasted pixels and disregard the false pixels. Experimental evaluation on public image datasets MICC-F220, MICC-F2000 and MICC-F8 multi shows that the proposed method is showing improved performance in detecting and localising copy-paste forgeries in images than the existing works. Also the proposed work detects multiple copy–pasted regions in the images and is robust to attacks such as geometrical transformation of copied regions such as scaling and rotation. 1 Introduction Copy–paste/copy–move forgery is a popular image manipulation technique, in which, an image region is copied and pasted into another portion of the same image to create a feigned image. Also, the forger often applies geometric transformation, noise embedding, filtering etc. to satisfactorily achieve his aim. Copy–paste forgery comes in many flavours: copying an image region and pasting it in another location within the image; copying an image region and pasting at multiple zones within the image; copying multiple image regions and pasting at another (or multiple) locations of the same image. Copy–paste forgery detection system determines the portion(s) of the image, which exactly matches with some other portion(s) of the same image. Since images play a major role in courtrooms and other law and order places, it is necessary to ensure the authenticity of images. Generally, image forensic techniques can be classified into active techniques and passive or blind techniques. Typical active forensic techniques are watermarking [1] and digital signatures [2, 3]. These techniques embed specific validation data in visual media during their production, but these methods require prior knowledge about the original image and specialised hardware, limiting its application. The passive or blind detection schemes are emerging fields in digital forensics, which require nothing from the original images. It discovers inherent features in the image left by acquisition devices [4] or manipulation acts [5, 6]. Since images and videos are visual clues in criminal cases, high accuracy with good robustness and low false positive are the challenges of forgery detection system. Copy–paste forgery detection is classified into two broad categories: block-based and keypoint-based detection techniques. Block-based detection techniques divide the image into overlapping or non-overlapping blocks. Salient features extracted from individual blocks are matched against features from other blocks for similarity detection. Keypoint-based forgery detection techniques extract local features from the input image. The extracted features are represented by a set of descriptors and matching is performed among the feature descriptors to find the pasted regions. The organisation of our paper is as follows. The related work for copy–paste forgery detection in images is outlined in Section 2. In Section 3, we describe the preliminaries of this work. The proposed work and experimental evaluation are presented in Sections 4 and 5. Finally, the work is concluded in Section 6. 2 Related work In the last few years, there has been a lot of research on copy–paste forgery detection in digital images. Most of the existing copy–paste forgery detection approaches attempt to find similarity between the original image area and the pasted one. Block-based techniques find a similarity measure between the overlapping/non-overlapping blocks, whereas keypoint-based techniques find the correlation between the descriptors. Some of the block-based techniques in the existing work are illustrated here: Fridrich et al. [7] carried out the very first work of copy–paste forgery detection. The authors developed two algorithms for copy–paste forgery detection – one based on exact match and another uses a robust match for detection. The first method searches for an exact matching block from the forged image. Robust match uses quantised DCT coefficients for matching instead of pixel representation. Though they proposed that robust matching procedure has been producing efficient and accurate detection results, this procedure leads to many false matches when applied to large identical textured image regions. Luo et al. [8] divided the image into overlapping blocks and extracted seven characteristic features such as averages of red, green and blue components, features from Y channel along four different directions. From these features, the author obtained matched block pairs after performing similarity search. Li et al.[9] performed image blocking operation on low-frequency components of wavelet transformed image. Then each block is represented using singular value decomposition to obtain a reduced feature dimension. Then sorted neighbourhood approach is employed to detect duplicated image regions. Mahdian and Saic [10] proposed a method that tiles the image with overlapping blocks and uses blur invariants to represent them. The author performed principal component transformation for dimensionality reduction and used a k–d tree for block similarity analysis. Li et al. [11] proposed segmentation-based method to detect copy–move forgery detection. This method first segments the image into a number of non-overlapped patches. Patch-wise matching based on expectation-maximisation algorithm is performed to identify duplicated regions. Zhong et al. [12] proposed a block-based technique by dividing the image into circular blocks and extracting local and inner image features using Discrete Radial Harmonic Fourier Moments. As discussed most of the existing block-based forgery detection algorithms apply different feature extraction methods to extract the features from individual blocks and perform block similarity matches. Three main drawbacks of these block-based methods are: (i) blocking the image and finding matching block pairs exhaustively is computationally expensive as the size of the image increases; (ii) block-based methods fail when the copied regions undergo significant geometrical transformation; (iii) in most of the block-based methods, blocks are of regular shape which leads to low recall rate. Keypoint-based forgery detection is another category of copy–paste forgery detection. This method overcomes the drawbacks of the block-based forgery detection method. In this method, image keypoints are extracted and matched over the entire keypoints of the image to identify duplicated regions. Scale-invariant feature transform (SIFT) was one of the robust keypoint features introduced by Lowe [13] for object recognition. In [14], Lowe extracted invariant SIFT features for reliable matching. Pan and Lyu [15] used SIFT features to detect the duplicated regions in the images. However, this work is not able to identify small copy–pasted forgeries involved in the image. In addition, this method failed to produce quantitative analysis for estimating geometric transformation parameters. A SIFT keypoint distribution strategy was proposed by Bin et al. [16] and it seems that this technique is able to detect forgery in flat or smooth regions only. Amerini et al. [17] applied SIFT keypoint matching approach followed by hierarchical agglomerative clustering technique for forgery detection. Clustering of matched pixels aimed to remove false detections. However, unfortunately this clustering was not able to remove all false matched pixels. Soni et al. [18] proposed copy–move forgery detection using SIFT features and density-based spatial clustering of application with noise (DBSCAN) clustering algorithm. Though this work improved the results of [17] in terms of detection performance, still consists of many falsely identified pixels. We took his work [18] as the base work and improved the results by introducing a new clustering technique to remove the false matched pixels. Bo et al. [19] applied speeded up robust features (SURFs) [20] instead of SIFT to extract features. Although this method is very fast compared to SIFT, it is not robust to rotation and rescaling. Also, it cannot locate small forged regions very well. Wo et al. [21] detected the copied parts with large-scale rotation and scaling based on multi-radius polar complex exponential transform. This method resulted in high time complexity of multi-radius feature extraction. Also, this technique needs multi-threaded GPU acceleration technology to perform well in computing time. Mayer and Stamm [22] proposed a methodology to detect forged image regions based on detecting Lateral Chromatic Aberration (LCA). Recent research has shown that an image's LCA can be anti-forensically modified to avoid forgery detection. Soni et al. [23] proposed a combination of block-based and keypoint-based copy–move forgery detections. SURFs were extracted from each non-overlapping block and 2NN matching is performed against the blocks. Though this method is invariant to geometric transformations compared to other block-based methods, still it lacks performance on composite transformation like rotation + scaling. A two-level clustering followed by SIFT feature extraction and matching is proposed by Abdel-Basset et al. in [24] to remove false pixel detections. However, this clustering technique is a time-consuming process. Another variant of SIFT called affine-SIFT (ASIFT) proposed by Morel and Yu [25] is used in image matching application for extracting and matching features between two images. ASIFT simulates different image views by varying the orientation of the camera axis parameters latitude and longitude. Some papers [26, 27] applied ASIFT for copy–paste forgery detection. Though ASIFT is claimed to be fully affine, it is not scaled invariant as that of SIFT. Due to a large number of features extracted, ASIFT is approximately ten times slower than SIFT. A comprehensive study of local descriptors [28] and analysis of affine region descriptors [29] states that SIFT features are a good solution for region matching because of their robust performance and relatively low computational costs. In this work, we propose an improved copy–paste forgery detection system based on the SIFT keypoints by introducing a new clustering algorithm to detect copy–paste forgeries present in the tampered image. The main aim of our work is to reduce false matches resulting from the SIFT-based matching algorithm. We have also identified multiple copy–paste forged regions in the image and localised it. The proposed system also works well even if the pasted region has undergone post-processing geometric attacks such as rotation or rescaling or combination of both. The detection results of the proposed work prove that it removes many of the false matched pixels identified in the previous works. 3 Preliminaries In this section, we briefly describe SIFT, a keypoint-based feature used for detecting copy–paste forgery in images. 3.1 Scale-invariant feature transform SIFT is one of the commonly used and robust features transform for image matching. In SIFT feature extraction, the image is transformed into scale-invariant coordinates relative to local features. Its algorithm has the following four main steps: (i) scale-invariant feature detection, (ii) key localisation (iii) orientation assignment and (iv) descriptor generation. Scale-invariant feature detection searches stable points over different scales and locations of the image. From the input image data, SIFT features are identified at multiple scales using an image pyramid. Key localisation performs detailed stability fit to determine location and scale at each keypoint candidate. Orientation assignment step assigns one or more canonical orientations to each keypoint location based on local image gradient directions to achieve invariance to image rotation. A gradient orientation histogram is formed from the sample points within a region around the keypoint for determining the keypoint orientation. The computed orientation histogram has 36 bins covering the 360° range of orientations. The peaks obtained in the orientation histogram plot correspond to dominant orientations of a keypoint. Descriptor generation generates SIFT descriptors at keypoint locations that are highly distinctive yet is as invariant as possible to remaining variations, such as change in illumination or 3D viewpoint. The descriptor size for each feature descriptor is 4 × 4 × 8 = 128 elements. 3.2 Descriptor matching After obtaining the SIFT keypoints and their corresponding feature descriptors , descriptor matching has to be performed for each keypoint. In our work, we use generalisation 2NN (referred to as g2NN) procedure proposed by Amerini et al. [17], which is able to deal with multiple copies of the same features. The matching procedure of g2NN is based on iterating on until this ratio is greater than . A keypoint is matched only if the following constraint is satisfied: (1)If k is the value in which the procedure terminates, keypoints which correspond to the distances , are considered as matches for the inspected keypoint. By repeating this procedure for all the keypoints in K, we can obtain the final set of matched points. 3.3 Our contribution The existing SIFT-based copy–paste forgery methods lead to many false match detections. This is a serious issue, since the next localisation step may segment original image regions as tampered ones. We observe that the false matched pixel detections in the previous works are due to the clustering algorithms like hierarchical agglomerative, DBSCAN methods. We attempt to reduce the false matched pixels by proposing a new sensitivity-based clustering algorithm in this work. This clustering algorithm outperforms the existing clustering algorithms with visible better results. The detailed description of our new clustering algorithm is given in Section 4. 4 Proposed methodology The proposed work is based on the extraction of robust SIFT features to identify whether a portion of an image is copy–pasted or not. Since copy–paste forgery involves copying an image region and pasting it at another location of the same image, the copied and pasted regions will have the same appearance. The basic idea behind keypoint-based copy–paste forgery detection is that the keypoints extracted in the tampered regions will be more similar to each other. Therefore, the concept of SIFT feature extraction and matching can be applied for copy–paste tampering detection. A simple system diagram that depicts the modules in the proposed system is shown in Fig. 1. Fig. 1Open in figure viewerPowerPoint Overview of the proposed system The first step of the proposed work is to extract SIFT features based on the work of Lowe [13]. These extracted keypoints are used as robust feature vectors representing location, scale and orientation. Keypoint matching within an image is the second step, which is based on the work of Amerini et al. [17] and Soni et al. [18]. g2NN procedure is used for performing matching among the image regions within the image. Each extracted keypoint is matched with remaining keypoints to find a perfect match. This matching is robust to geometric attacks and also detects multiple matched regions within the image. The detailed description of SIFT feature extraction and matching procedure is given in [14]. The third step is devoted to matched keypoint clustering and forgery detection. Unlike other clustering techniques used in the previous work, we have applied sensitivity based clustering technique, which is the novelty of the proposed work. The fourth step locates the tampered regions in the image. 4.1 Sensitivity-based clustering Clustering is the task of partitioning the objects into groups (clusters) so that objects in the same group are much similar to each other than to those in other groups. Given a set of objects, sensitivity-based clustering constructs partitions of the data, where each partition represents a cluster and . The sensitivity parameter is defined as the range of points under consideration for clustering. In this method, sensitivity is used to adjust the 'difference sensitivity' between clusters. The sensitivity should be provided as a normalised value . The higher the value for sensitivity, yields fewer clusters, whereas, with a lower sensitivity, more clusters will be obtained. Also, this partitioning method is exclusive cluster separation, i.e. each object belongs to exactly one group. This clustering technique does not need any prior knowledge of the number of bins, or the distance between bins. This approach relies on the relative difference between sorted elements of the matched keypoints. Also, this technique ensures that the difference between clusters is bigger than the difference between elements within a cluster. SIFT keypoint matching algorithm returns two sets of points and which are matched to each other. Now clustering has to be done to locate the copy–pasted region pairs and to remove falsely identified pixel matches. Let the matched coordinates and be combined together to form a set P, i.e. as in (2). Now P consists of coordinates of the matched keypoints in both x and y directions (2)The geometrically nearer points were grouped together to form a cluster in both x and y directions. Clusters with a very less number of points are discarded as they do not belong to an object in the image. Clustering is done column-wise, i.e. the first in the x dimension and then in the y dimension. Let and be the sorted x and y coordinate vectors from P. The difference between adjacent elements in and is computed as (3)The maximum difference and minimum difference are calculated from D in both x and y directions as follows: (4) (5)where max function in (4) returns the maximum value. The points that are within distance are grouped together as a cluster. Since minimum difference, is sensitivity times maximum difference , sensitivity s is the deciding parameter for number of clusters. Two points in a column vector ( or ) belong to the same cluster if the condition in (6) is satisfied for the corresponding dimension. If D value for a point (in the x direction) is less than , then the points and are separated by a distance less than the minimum distance . Hence both and belong to same cluster (6)Clustering in x and y directions results in two set of clusters and . The elements of and are grouped to one of the clusters in and , respectively. (7) (8)where M and N are the number of clusters obtained after clustering and , respectively. Once column-wise clustering is done, a set S is formed which consists of the cluster numbers to which the elements of P belong to (9)where denotes the cluster number of a point in the direction. The solution space of S consists of a set of all possible combinations between and , hence maximum possible number of tuples in S is . The unique tuples in S forms the final clusters. The set of points in corresponding to cluster such that the x and y coordinates of P belongs to clusters and , respectively, form a unique cluster. Thus, all the matched points in P are linked to a cluster in S. The clusters of small size were discarded as they cannot be a region in an image. If the ratio of cluster size to the total number of matched keypoints is less than the threshold, , then the cluster is small in size and should be discarded (10)If the proposed method detects two (or more) clusters satisfying (10), then the test image is affected by copy–paste forgery attack. The pseudo-code of sensitivity-based clustering is given in Fig. 2. Fig. 2Open in figure viewerPowerPoint Proposed clustering algorithm Line number 6 iterates for two directions – x and y for performing column-wise clustering. Line number 7 computes A after sorting the matched points in P, whereas line number 8 finds the index vector for sorted vector. Line numbers from 10 to 14 find the difference between adjacent elements in A and the maximum difference. Line number 16 finds the minimum distance from the sensitivity parameter. Line numbers from 17 to 27 identify the boundary points of the clusters. If the distance between a point and its adjacent next point in A is greater than the minimum distance, then those points are the boundary points belonging to different clusters. Line numbers 25–27 form a set S, which consists of tuples of cluster numbers as mentioned in (9). Line numbers from 31 to 40 identify unique tuples from S, which forms the final clusters. The clusters returned by this algorithm have to be tested for condition in (10), which ignores small-sized clusters; i.e. an image is said to be tampered by copy–paste forgery attack if the proposed algorithm detects at least two clusters with minimum () pairs of matched points. Since small cluster does not contribute to an object in an image it should be removed as it is a false detection. This clustering algorithm improved the results of copy–paste forgery detection than the existing methods. In the proposed system, multiple forgeries are also handled using the same matching and clustering procedure as handling single forgery. SIFT keypoints are extracted and g2NN matching is performed, which results in more than one matched region. Finally, sensitivity based clustering is performed on the keypoints spatial position to discover and separate the different copied regions. RANdom SAmple Consensus (RANSAC) algorithm [30] is applied following the clustering algorithm to improve the performance further. This method models the best fit for a 2D homograph with more appropriate inliers and excludes the outlier points from a set of matched points. The estimation of inlier points is robust and accurate. This algorithm estimates the homograph by randomly selecting a set of minimum number of points (set to 3 in our experiments) from the set of matched points. The selected points are compared with their corresponding matched ones in terms of distance. Based on this distance metric the selected points are classified as inliers or outliers. 5 Experimental results and discussions In this section, different benchmarking datasets used for our experimental purpose are described. The proposed methodology is evaluated on the benchmarking datasets for robustness tests and performance measures. Also, the comparison of the proposed work with existing approaches is elaborated in this section. 5.1 Dataset We evaluate our work using four standard benchmark datasets MICC-F220, MICC-F2000, MICC-F8multi and MICC-F600. These datasets consist of images mostly from the Columbia photographic image repository collected by Ng et al. [31]. Media Integration and Communication Center (MICC) was established by the Italian Ministry for Education at the University of Florence. MICC-F220 dataset is composed of 220 images, out of which 110 are tampered and 110 are originals. MICC-F2000 dataset consists of a total of 2000 images, out of which 700 are tampered and 1300 are original images. The copy–pasted regions in MICC-F220 and MICC-F2000 dataset images have undergone geometrical attacks such as rotation, scaling and both. The dataset MICC-F8multi contains eight challenging multiple forged images. This dataset is used for analysing the results of multiple forgery detection of the proposed work. MICC-F600 dataset consists of realistic and challenging forged images. This dataset contains 600 high-resolution images out of which 440 are original and 160 are tampered images. The description of these datasets is given in Table 1. Table 1. Description of datasets used for evaluation Dataset Image size No. of images MICC-F220 722 × 480 to 800 × 600 220 MICC-F2000 2048 × 1536 2000 MICC-F8 multi 2048 × 1536 8 MICC-F600 800 × 533 to 3888 × 2592 600 Tables 2 and 3 show the geometric transformations applied in the tampered regions of the MICC-F220 dataset (10 transformations, from a to j in Table 2) and MICC-F2000 dataset (14 transformations, from A to N in Table 3) where rotation and scaling transformations are involved. Thus for each transformation, rotation in degrees and scaling factors applied to the x and y axes of the cloned image part are reported in Tables 2 and 3. Table 2. Details of geometric transformation attacks applied to the original image patches in MICC-F220 dataset Attack Attack a 0 1 1 f 0 1.2 1.2 b 10 1 1 g 0 1.3 1.3 c 20 1 1 h 0 1.4 1.2 d 30 1 1 i 10 1.2 1.2 e 40 1 1 j 20 1.4 1.2 Table 3. Details of geometric transformation attacks applied to the original image patches in MICC-F2000 dataset Attack Attack A 0 1 1 H 0 1.2 1.6 B 0 0.5 0.5 I 5 1 1 C 0 0.7 0.7 J 30 1 1 D 0 1.2 1.2 K 70 1 1 E 0 1.6 1.6 L 90 1 1 F 0 2 2 M 40 1.1 1.6 G 0 1.6 1.2 N 30 0.7 0.9 In Table 2, a denotes no geometric attack, b - e denotes rotation attack, f - h denotes scaling attack and i - j denotes rotation + scaling attack. Similarly, in Table 3, A denotes no geometric attack, B - H denotes scaling attack, I - L denotes rotation attack and M - N denotes rotation + scaling attack. 5.2 Experimental results of proposed work In this section, we evaluate the proposed work on three kinds of datasets and show the results of the detection and localisation of tampered areas. To check the robustness and susceptibility of our method, experiments are performed for the following cases: (i) Copy and move without any transformation. (ii) Copy and move after rotation. (iii) Copy and move after scaling. (iv) Copy and move after scaling and rotation. Figs. 3 and 4 show the results obtained in MICC-F220 dataset. The copied and pasted image regions are correctly identified, even in the presence of different geometrical transformations. Hence the proposed method is robust and has good detection accuracy. The sample experimental results of testing our work in MICC-F2000 dataset are shown in Figs. 5 and 6. As mentioned in Table 1, the size of an image in this dataset is larger than MICC-F220 dataset. Our method shows good performance and robustness for this dataset also. The results of multi-forgery tested in MICC-F8multi are presented in Fig. 7. In this multi-tampering, one or more image regions are copied and pasted in more than one distinct location of the same image. To handle the multiple copy–paste forgery, g2NN, SIFT keypoints matching procedure and sensitivity-based clustering on the matched keypoint spatial positions are used to separate the different copied regions. It is clearly shown in Fig. 7 that the proposed system is able to detect accurately multiple forgeries present in images. In all the three datasets, false points detection is almost nil for the proposed work with our new clustering algorithm. Fig. 3Open in figure viewerPowerPoint Input images from MICC-F220 dataset depicting tampered cemetery region are shown in the top; the proposed forgery detection results of the corresponding images are shown in the bottom (a) Copy–paste attack without any transformation, (b) Copy–paste attack followed by rotation, (c) Copy–paste attack followed by scaling, (d) Copy–paste attack followed by rotation and scaling Fig. 4Open in figure viewerPowerPoint Input images from MICC-F220 dataset depicting tampered car region are shown in the top; the proposed forgery detection results of the corresponding images are shown in the bottom (a) Copy–paste attack without any transformation, (b) Copy–paste attack followed by rotation, (c) Copy–pa