SafePaint: Anti-forensic Image Inpainting with Domain Adaptation

Venue: acmmm_org_ACMMM_2024_Conference
Published: 2024-04-07 11:15:03

Abstract: Existing image inpainting methods have achieved remarkable accomplishments in generating visually appealing results, often accompanied by a trend toward creating more intricate structural textures. However, while these models excel at creating more realistic image content, they often leave noticeable traces of tampering, posing a significant threat to security. In this work, we take the anti-forensic capabilities into consideration, firstly proposing an end-to-end training framework for anti-forensic image inpainting named SafePaint. Specifically, we innovatively formulated image inpainting as two major tasks: semantically plausible content completion and region-wise optimization. The former is similar to current inpainting methods that aim to restore the missing regions of corrupted images. The latter, through domain adaptation, endeavors to reconcile the discrepancies between the inpainted region and the unaltered area to achieve anti-forensic goals. Through comprehensive theoretical analysis, we validate the effectiveness of domain adaptation for anti-forensic performance. Furthermore, we meticulously crafted a region-wise separated attention (RWSA) module, which not only aligns with our objective of anti-forensics but also enhances the performance of the model. Extensive qualitative and quantitative evaluations show our approach achieves comparable results to existing image inpainting methods while offering anti-forensic capabilities not available in other methods.

Main paper: 
 introduction: 
 The purpose of image inpainting is to restore the missing regions of damaged images, enabling them to have semantically plausible content. Recently, image inpainting techniques have matured significantly, achieving results that are often imperceptible to the human eye. However, while inspiring, they still exhibit shortcomings. On the one hand, image inpainting techniques have positive applications, such as removing private objects from images [7] . If privacy information in images is exposed by malicious individuals through forensic methods and reverse engineering, it could be disastrous for Figure 1 : Examples of images inpainted by SOTA method FcF [19] and our SafePaint, which are selected from the Places2 dataset. Each result of the methods includes an inpainted image and the corresponding heatmap on inpainting detector IID-Net [33] .stakeholders. On the other hand, as mentioned in [41] , commonly used image evaluation metrics in this field (such as PSNR, SSIM, etc.) do not adequately represent the performance of image inpainting. An approach that performs better according to these metrics may not necessarily be visually superior to another. The ability of images to resist forensic analysis to some extent aligns with human perceptual judgment and can serve as a basis for measuring the quality of image inpainting. Therefore, the exploration of anti-forensic image inpainting techniques is of considerable significance.The coarse-to-fine framework [31, 35, 36, 38, 39] is a classic strategy in the field of image inpainting. It resembles human deductive reasoning, gradually restoring the image from easy to difficult, from the outer to the inner, and from superficial to deep. However, in the design of this coarse-to-fine inpainting manner, the functionalities of different networks are approximately similar, which does not fully exploit the advantages. Another research focus in image inpainting is the parallel inpainting of structure and texture [13, 19] , allowing models to generate finer structural textures and make the inpainted content more realistic. However, texture structures inconsistent with the background content will attract the attention of forensic detectors, thereby posing a serious threat to the security of the inpainted image.To further improve the performance of image inpainting methods, many approaches have incorporated attention modules into the models. Traditional spatial attention modules can enhance the network's receptive field and extraction of long-range information. However, for specific tasks such as image inpainting, their performance improvement is often limited. Yu et al. drew inspiration from patch-based image inpainting methods and proposed a context attention module [35, 36] tailored to the characteristics of the image inpainting tasks, simultaneously meeting the requirements of enhancing the network's receptive field and leveraging foreground information. However, its core idea of copy-move is precisely one of the key elements that forensic methods pay attention to, severely restricting the anti-forensic capabilities of the inpainting methods.In this paper, we propose a new image inpainting architecture to achieve anti-forensic image inpainting tasks. Unlike the previous coarse-to-fine image inpainting methods that simply stack networks for multiple inpainting processes, we adopt a task-decoupled inpainting mode, using an "inpainting first, adjust later" strategy. After completing the inpainting process, we enhance the semantic alignment between the image foreground and background using the domain adaptation method, thereby improving its anti-forensic performance. Furthermore, we carefully designed a region-wise separated attention (RWSA) module, which not only fits our tasks well but also improves the performance of the model. As shown in Fig. 1 , compared to the state-of-the-art (SOTA) method, our approach possesses strong anti-forensic capabilities.The main contributions are summarized as follows:• To our best knowledge, we are the first to introduce an end-to-end training framework for anti-forensic image inpainting, making image inpainting more secure and reliable. Moreover, we innovatively incorporate the anti-forensic capabilities of image inpainting methods as one of the criteria for evaluating the quality of image inpainting, thus compensating for the shortcomings of traditional image inpainting evaluation metrics. • We have improved the traditional coarse-to-fine image inpainting approaches by fully decoupling the image inpainting and refinement processes. This not only fully utilizes the prior knowledge obtained during the inpainting process, but also significantly enhances the anti-forensic performance of the inpainted images through domain adaptation. • To meet the requirements of anti-forensic image inpainting tasks and fully exploit the potential of the model, we specifically designed the region-wise separated attention (RWSA) module. This module can also be inserted into other image inpainting models to improve their anti-forensic performance. • Comprehensive experiments on three widely-used datasets demonstrate that our proposed method significantly outperforms state-of-the-art (SOTA) methods in terms of antiforensic capabilities, while achieving comparable performance with them in traditional evaluation metrics. 
related work 2.1 traditional inpainting methods: 
 Traditionally, inpainting methods have two main categories, i.e., diffusion-based and patch-based. Diffusion-Based methods. Diffusion-based methods [1, 2, 5] refer to the utilization of mathematical or physical partial differential equations (PDEs) to smoothly propagate known pixel values from the image's existing regions into the missing areas to restore damaged images.Patch-Based methods. Patch-based image inpainting methods [4, [6] [7] [8] [9] 16] involve calculating and searching for samples within the known regions of the damaged images that bear the highest similarity to the missing areas, thereby utilizing them as materials to restore the damaged image. These methods exhibited strong performance in early image inpainting tasks.However, due to significant time costs and limitations imposed by the limited information of a single image, which makes it challenging to reconstruct semantically rich images, these approaches have gradually been abandoned.From the standpoint of design strategy, existing deep-learningbased image inpainting methods can be roughly divided into two types: one-stage and coarse-to-fine methods.One-Stage Methods. Pathak et al. [30] were the first to apply the generative adversarial network (GAN) [11] to image inpainting, laying the foundation for using deep learning to tackle image inpainting problems. Iizuka et al. [17] improved the consistency of image completion by incorporating global and local discriminators into the network. To address the limitation of traditional convolutional methods, Liu et al. [24] proposed partial convolution, which dynamically updates the mask during convolution computation, achieving favorable results for corrupted images with irregularly shaped masks. Zeng et al. [37] enhances the contextual reasoning capabilities of the network through carefully designed AOT blocks. To generate content where structure and texture align more closely, work [13] adequately models both types of information in a coupled manner. Inpainting high-resolution images with large missing areas remains a challenge. Therefore, Suvorov et al. [32] introduced a network structure based on Fast Fourier convolution (FFC), which has a larger receptive field in the initial stage of the network and performs well in complex image inpainting tasks. Inspired by this work, Jain et al. [19] combined the receptive power of Fast Fourier convolution with a co-modulated coarse-to-fine generator, addressing image inpainting problems by simultaneously considering image structures and textures. Due to insufficient prior knowledge, these methods are sometimes plagued by visible artifacts.Coarse-to-Fine Methods. Attention mechanism enables image inpainting models to leverage long-range feature information. Inspired by patch-based methods, Yu et al. [35] proposed contextual attention. This module divides the image into several pixel patches and calculates their cosine similarity to guide the inpainting process. Building upon [35] , Yu et al. [36] introduced gated convolution to address the shortcomings of partial convolution from [24] . Simultaneously, the authors introduced a sample discriminator SN-PatchGAN to generate high-quality inpainting results and accelerate training. Liu et al. [25] extended the attention mechanism to missing pixels. Progressive inpainting strategies have become another major technological innovation in the field of image inpainting. Zhang et al. [40] divided the image inpainting process into four phases and used a long short-term memory (LSTM) architecture [14] to control the information flow of the progressive process. Nazeri et al. [29] drew inspiration from sketching, first generating a hypothetical edge image of the image to be inpainted as prior knowledge, and then filling in the missing region in the next step. Observing that there still exists useful information in the failure cases, Zeng et al. [39] proposed an iterative method with feedback mechanisms for inpainting. Reference [44] improved upon the limitations of the specialized convolution in [24, 36] , allowing for a more flexible implementation of coarse-to-fine inpainting. Although these methods provide valuable insights into image inpainting, they fail to meet the requirements of anti-forensic image inpainting tasks.While image inpainting methods are rapidly developing, the advancement of their counterpart, image inpainting detection methods, is also noteworthy. Conventional methods [23, 34] mostly focus on manual features, e.g., the similarities between image patches. Such methods incur high computational cost and often yield minimal effectiveness when confronted with complex scenarios. Recently, Many methods [22, 33] have shifted their focus toward deep-learning-based inpainting techniques. By automatic feature extraction, these methods are capable of achieving precise detection in even more complex situations. Moreover, many methods [12, 26] are proposed to detect complicated combinations of forgeries (including inpainting).However, these detection methods start from a fundamentally similar point, which is leveraging the inconsistency between the distribution of generated data and that of original data.The objective of domain adaptation is to achieve the transformation from a source domain to a target domain. The pioneering work Pix2Pix [18] demonstrated remarkable results in image-to-image (I2I) translation tasks by leveraging paired images and conditional generative adversarial networks (GAN). To overcome the reliance on paired image data, Zhu et al. introduced CycleGAN [43] . The aforementioned methods require domain labels; in contrast, the methods [3, 20] necessitate no domain labels. In image inpainting, domain labels are typically not explicitly available. Therefore, it is proposed to use unlabeled domain adaptation aimed at enhancing the anti-forensic capabilities of our method. 
method 3.1 model architecture: 
 As shown in Fig. 2 , our SafePaint achieves anti-forensic inpainting through two subtasks, i.e., content completion and region-wise optimization. We will introduce the generators, the discriminator, and the domain pattern extractor of the model below.Generator and Discriminator. We propose an image inpainting network that consists of two stages. It includes a generator in each stage and a discriminator in stage 𝐼𝐼 . The two generators 𝐺 1 , 𝐺 2 follow an architecture similar to the method proposed by Johnson et al. [21] that we use only the image generator as our backbone generator. In the upsampling process of 𝐺 2 , we further integrated the RWSA module to facilitate domain adaptation and enhance performance. For the discriminator, We apply the Patch-GAN [18, 43] instead of vanilla GAN to classify each image patch as pristine or inpainted. To improve the stability of the training, we also introduced Spectral normalization (SN) [28] to both the generators and the discriminator. Domain Pattern Extractor. We design the domain pattern extractor to implement domain adaptation. Considering that the background and foreground regions have irregular shapes, we replaced vanilla convolutional layers with partial convolutional layers [24] , which are specially designed for image inpainting with irregular masks. The output of the network will be represented in vector form, serving to characterize features from different domains. To achieve the goal of anti-forensics, we investigate the commonalities of inpainting methods and the shared properties among their corresponding forensic methods by analyzing massive corrupted image samples. It is drawn that detectors are more inclined to perceive regions where data distribution significantly deviates from that of real image data as tampered areas, a point we will further explore in Section 3.3. Hence, we employ domain adaptation as a means to narrow the gap between the foreground and background of the image, thereby evading the detection of forensic detectors.We refer to the area to be inpainted as the foreground, and the original area as the background. Specifically, we design a domain pattern extractor to characterize the features within the delineated regions. For image inpainting, the background region is generally not allowed to be modified. In that case, the background region in images to be inpainted is considered as unified across different stages.As shown in Fig. 3 , we extracted three different domain pattern vectors 𝑍 𝑏 , 𝑍 𝑔𝑡 , and 𝑍 𝑜𝑢𝑡 by the domain pattern extractor, representing background domain features, foreground domain features of ground-truth, and foreground domain features of the output image, respectively. Since domain pattern vectors can represent the data distributions within specific regions of an image, intuitively, minimizing the disparities between them can be equivalent to reducing the distance between their corresponding domain pattern vectors. To achieve this, we design a special loss function, which will be explained specifically in Section 3.5.In this section, we will explain how our design of domain adaptation can effectively achieve anti-forensics by analyzing the principles of inpainting methods and their respective forensic approaches. We categorize inpainting methods into diffusion-based, patch-based, and deep-learning-based methods, and we will discuss each of them separately below. Diffusion-Based Methods. Taking the work [5] as an example, it leverages partial differential equations (PDEs) for image inpainting. The core idea is to gradually propagate pixels from the known regions of the image to the areas requiring inpainting. The algorithm can be formulated as follows:𝐼 𝑛+1 (𝑖, 𝑗) = 𝐼 𝑛 (𝑖, 𝑗) + Δ𝑣𝐼 𝑛 𝑡 (𝑖, 𝑗), ∀(𝑖, 𝑗) ∈ Ω (1)where 𝑛 denotes the number of iterations for the pixel 𝐼 (𝑖, 𝑗) at the coordinate (𝑖, 𝑗), Δ𝑣 represents the update rates, Ω signifies the region to be inpainted, and 𝐼 𝑛 𝑡 (𝑖, 𝑗) is the updated value, calculated by the following formula:EQUATIONwhere ---→ Δ𝐿 𝑛 (𝑖, 𝑗) measure the change of 𝐿 𝑛 (𝑖, 𝑗), which denotes the propagated information.-→ 𝑁 𝑛 (𝑖, 𝑗) is the propagation direction. Since the diffusion process runs only inside Ω, as the propagation progresses, 𝐼 𝑛 𝑡 (𝑖, 𝑗) will decrease gradually and converge to 0. That means 𝐼 (𝑖, 𝑗) will remain constant in the direction of -→ 𝑁 . However, this phenomenon is extremely rare in the known regions of the image. The forensic method [23] specific to this inpainting mode confirms this, as it locates the inpainting area by comparing the local variances of the inpainted region and the known region.Patch-Based Methods. The design of the work [7] is to find the most suitable patch within the known region to fill the area to be inpainted. It repeats the following three steps: (1) Computing patch priorities. (2) Propagating texture and structure information.(3) Filling region to be inpainted. We define Ψ f as the patch with the highest priority, and then we search in the known region for that patch Ψ b which is most similar to Ψ f , calculated by following rule:EQUATIONwhere Ω represents the region to be filled, the distance 𝑑 (Ψ m , Ψ n ) between two patches Ψ m and Ψ n is simply computed using the Euclidean distance. This rigid computational approach may introduce abnormal similarity between the inpainting region and the rest of the image, as the patch block set of the inpainting region may exhibit a one-to-many or many-to-many relationship with that of the known region. Wu et al. [34] utilizes this characteristic by identifying sets of block pairs exhibiting abnormal similarity as a basis for locating the area that has been inpainted.Deep-Learning-Based Methods. While deep-learning-based inpainting methods mainly rely on data-driven approaches, their basic principles predominantly derive from traditional inpainting methods. For example, the contextual attention mechanism used in [35, 36] draws inspiration from patch-based inpainting methods, while the search and generate process in [25] embodies the properties of diffusion-based inpainting methods. Although these inpainting methods differ, their core ideas are nearly identical, namely, making full use of existing pixel information in images. On the other side of the coin, their corresponding forensic methods also share similarities, primarily utilizing inconsistencies of the statistical distributions between real data and generated data.Based on the analysis above, we eventually validate the effectiveness of the proposed domain adaptation for image inpainting. Let 𝑋 Ω the set of pixel values in the inpainting area, and 𝑋 Φ denote the set of pixel values in the known area. Then we use 𝑃 (𝑋 Ω ) and 𝑃 (𝑋 Φ ) to represent their respective probability distribution. The Kullback-Leibler divergence (KL divergence) is utilized to express the difference between them, formulated as:𝐷 KL (𝑃 (𝑋 Φ ) ∥𝑃 (𝑋 Ω )) = ∫ ∞ -∞ 𝑃 (𝑋 Φ (𝑥)) log 𝑃 (𝑋 Φ (𝑥)) 𝑃 (𝑋 Ω (𝑥)) 𝑑𝑥 (4)We denote the domain adaptation transformation as function 𝑇 . Our objective is to find a 𝑇 that minimizes the KL divergence between these two distributions, thereby achieving the purpose of anti-forensics. As illustrated in Fig. 4 , given an intermediate feature map 𝑥 ∈ R 𝐶 ×𝐻 ×𝑊 , and the binary mask 𝑀 ∈ R 1×𝐻 ×𝑊 of the inpainted region as the inputs (C is the feature channel, H and W is the height and width of the feature map respectively), three 1D channel attention modules 𝐺 𝑏 , 𝐺 𝑓 1 , 𝐺 𝑓 2 are utilized to weight the input features for different purposes, with the help of foreground region mask 𝑀 and background region mask 𝑀 ′ . In detail, 𝐺 𝑏 aims to select the features that are necessary for image rebuild in the background region. 𝐺 𝑓 1 intends to weigh the input features that have differences between the input and target. Moreover, we design the third channel attention module 𝐺 𝑓 2 to weight features that are unnecessary to change in the foreground region. Additionally, for the output features of 𝐺 𝑓 1 , we add a learnable block 𝐿𝐵 to learn the domain adaptation. Therefore, the output features of foreground ℱ 𝑓 and background ℱ 𝑏 can be respectively formalized as:EQUATIONThe core component of proposed RWSA module are channel attention modules (𝐺 𝑏 , 𝐺 𝑓 1 , 𝐺 𝑓 2 ), learnable block 𝐿𝐵, and Feature Fusion Block 𝐹 𝐹 𝐵. Regarding the three channel attention modules, we utilize a modified Squeeze-and-Excitation Network [15] in which we add a MaxPooling layer. The learnable block 𝐿𝐵 consists of 3 × 3 convolutional layers with two CONV-BN-ELU blocks to learn the textural and structural changes in the inpainted region. In order to fully integrate the feature obtained, we have designed the feature fusion block 𝐹 𝐹 𝐵, which concatenates the sum of ℱ 𝑓 and ℱ 𝑏 with the input 𝑥 in channel axis. Subsequently, the channel size will halve to complete the fusion operation. Above all, our region-wise separated attention (RWSA) module between input features 𝑥 and output features ℱ can be finally formalized as:EQUATION)where 𝐶𝑜𝑛𝑐𝑎𝑡 is the implementation of concatenation.The model training contains the ℓ 1 loss, perceptual loss, style loss, adversarial loss, and domain distance loss, to achieve visual plausibility and anti-forensic performance. Formally, we designate the two generators as 𝐺 1 and 𝐺 2 respectively and denote the discriminator in stage 𝐼𝐼 as 𝐷. Let 𝐼 𝑔𝑡 be the ground truth image, 𝑀 be the input binary mask which represents the region to be inpainted, 𝐼 𝑖𝑛 = 𝐼 𝑔𝑡 ⊙ (1 -𝑀) + 𝑀 be the input image, 𝑉 be the domain pattern map. Then the coarse result 𝐼 𝑐 and final output 𝐼 𝑜𝑢𝑡 are formulated as:𝐼 𝑐 = 𝐼 𝑔𝑡 ⊙ (1 -𝑀) + 𝐺 1 (𝐶𝑜𝑛𝑐𝑎𝑡 (𝐼 𝑖𝑛 , 𝑀)) ⊙ 𝑀 𝐼 𝑜𝑢𝑡 = 𝐼 𝑔𝑡 ⊙ (1 -𝑀) + 𝐺 2 (𝐶𝑜𝑛𝑐𝑎𝑡 (𝐼 𝑐 , 𝑀, 𝑉 )) ⊙ 𝑀 (7)ℓ 1 Loss. We adopt the ℓ 1 loss to calculate difference between 𝐼 𝑐 and 𝐼 𝑔𝑡 as well as the difference from 𝐼 𝑜𝑢𝑡 to 𝐼 𝑔𝑡 , formulated as:EQUATION)Perceptual Loss. For image inpainting, the perceptual loss from a pre-trained and fixed VGG-19 is used. The perceptual loss compares the difference between the deep feature map of the generated image and the ground truth. Such a loss function can compensate for the weakness of ℓ 1 Loss that helps the model capture high-level semantics. The perceptual loss is defined as:EQUATION)Style Loss. For the purpose of guaranteeing style coherence, we use style loss to compute the distance between feature maps. The style loss can be illustrated as: (10) where 𝜓 𝑖 (•)= 𝜙 𝑖 (•) 𝑇 𝜙 𝑖 (•) denotes the Gram matrix constructed from the activation map 𝜙 𝑖 . Adversarial Loss. Adversarial Loss is designed to help the model generate plausible images. Given the characteristic of our task, the discriminator 𝐷 takes 𝐼 𝑔𝑡 as real images and 𝐼 𝑜𝑢𝑡 as fake images. The adversarial loss is as follows:ℒ 𝑠𝑡 𝑦 = E ∑︁ 𝑖 ∥𝜓 𝑖 (𝐼 𝑐 ) -𝜓 𝑖 (𝐼 𝑔𝑡 )∥ 1 + ∥𝜓 𝑖 (𝐼 𝑜𝑢𝑡 ) -𝜓 𝑖 (𝐼 𝑔𝑡 ) ∥ 1ℒ 𝑎𝑑𝑣 = E[max(0, 1 -𝐷 (𝐼 𝑔𝑡 ))] + E[max(0, 1 + 𝐷 (𝐼 𝑜𝑢𝑡 ))] (11)Domain Distance Loss. Enhancing the alignment between the foreground and background is a crucial step in our SafePaint, and we accomplish this goal by incorporating the domain distance loss. Firstly, the character 𝑃 is used to denote the domain pattern extractor. As mentioned in section 3.2, the three different domain pattern vectors 𝑍 𝑏 , 𝑍 𝑔𝑡 , 𝑍 𝑜𝑢𝑡 can be computed by:EQUATIONwhere 𝑀 represents the input binary mask and 𝑀 represents its reverse form.After inpainting, we attempt to make the background of the result as close to the foreground as possible while ensuring the quality of inpainted images. Therefore, we bring 𝑍 𝑜𝑢𝑡 and 𝑍 𝑏 closer while draw 𝑍 𝑜𝑢𝑡 and 𝑍 𝑔𝑡 nearer by following domain distance loss:EQUATIONin which 𝑑 (•, •) is Euclidean distance.To summarize, the total loss can be written correspondingly as:EQUATIONwhere 𝜆 1 , 𝜆 𝑝𝑒𝑟 , 𝜆 𝑠𝑡 𝑦 , 𝜆 𝑎𝑑𝑣 , 𝜆 𝑑 are the weights for respective loss terms. 
experiments 4.1 experimental settings: 
 We evaluate the proposed method on the Places2 [42] , Paris-Street-View [10] , CelebA [27] datasets, which are commonly used in the field. NVIDIA Irregular Mask [18] is used for training and testing and we classified the mask data based on their mask ratio relative to the image with an increment of 10%. All the images and corresponding masks are resized to 256 × 256. We train our model in pytorch. Training is launched on a single NVIDIA RTX 4090 GPU with the batch size of 4, optimized by the Adam optimizer with 𝑏𝑒𝑡𝑎𝑠 = (0.5, 0.999). We adjust the learning rate as 2 × 10 -4 for both discriminator and generator. Besides, 𝜆 1 , 𝜆 𝑝𝑒𝑟 , 𝜆 𝑠𝑡 𝑦 , 𝜆 𝑎𝑑𝑣 , 𝜆 𝑑 in Eq.( 14) are set to 1, 0,1, 250, 0.1, 0.01.We compare our SafePaint on the Places2 [42] , Paris-Street-View [10] , CelebA [27] datasets with the state-of-the-art methods introduced previously: EdgeConnect [29] , MADF [44] , CTSDG [13] , AOT-GAN [37] , Lama [32] , FcF [19] . We use their official implementation and the models provided by the authors. As traditional image quality assessment metrics, LPIPS and PSNR are used for quantitative quality evaluation. Since this is a piece focusing on anti-forensics, we take the anti-forensic ability of inpainting methods into account. Therefore, we adopt two pre-trained image forgery detectors PSCC-Net [26] and TruFor [12] , and one inpainting-specific detector [33] Figure 5 : Visual comparison of our SafePaint with EdgeConnect [29] , MADF [44] , CTSDG [13] , AOT-GAN [37] , Lama [32] , FcF [19] on dataset Places2.Figure 6 : Visualization of anti-forensic capabilities comparisons on detector PSCC-Net [26] .for anti-forensics analysis. To quantify the anti-forensic performance, we use pixel-level Area Under Curve (AUC), F1 score on manipulation masks, and the accuracy of detectors as main metrics.Since the inpainted images serve as negative samples, lower AUC, F1, and ACC indicate better anti-forensic performance.Quantitative evaluation on visual quality. Table 1 reports the numerical comparisons of proposed SafePaint with EdgeConnect [29] , MADF [44] , CTSDG [13] , AOT-GAN [37] , Lama [32] , and FcF [19] across the three datasets. The comparisons are based on two image quality metrics: LPIPS and PSNR. It can be observed that although we compromise visual quality to a certain extent to enhance image security in stage 𝐼𝐼 , the proposed network is comparable to state-of-the-art methods in visual comparison evaluation, and the inpainted images remain visually appealing.Qualitative evaluation on visual quality. Fig. 5 shows the visual quality results of the SafePaint compared with existing methods on the dataset Places2. It can be seen, as one of the representative works of parallel structural and textural image inpainting, FcF [19] exhibits powerful detail generation capabilities, which sometimes paradoxically lead to content generation that is discordant with the surrounding areas. For other methods, the inpainted content may deviate from the known regions to some extent. Owing to our domain adaptation design, our SafePaint can generate more harmonious content.Anti-forensics analysis. Anti-forensic performance signifies the ability of image inpainting methods to evade the detection of forensic detectors. We select two representative image forgery detectors and one inpainting detector to evaluate the anti-forensic ability of our SafePaint and the comparison methods, including [26] , TruFor [12] and IID-Net [33] . Table 2 , Table 3 and Table 4 illustrate the comparisons regarding their anti-forensic performance on Places2. The AUC, F1, and ACC with smaller values indicate better anti-forensic ability. We can see that our model always outperforms the state-of-the-art counterparts by a significant margin. The visual results are depicted via the heatmap in Fig. 6 .It is evident that compared to the other methods, our approach leaves very faint tampering traces in the image, showing excellent anti-forensic capabilities.In this section, we will investigate the effectiveness of each component in the SafePaint, including domain distance loss (DDL) and region-wise separated attention (RWSA) module proposed by us.Given their design for anti-forensic purposes, we will only compare the detection resistance performance of each model. Specifically, the ablation experiments are conducted on the dataset Places2 and the mask ratio is set to 30%-40%. The results of ablating each component are reported in Table 5 , representing their detection evasion performance on detectors PSCC-Net, TruFor and IID-Net. We denote our SafePaint without DDL and RWSA module as "Backbone". It can be observed that "Backbone+DDL" outperforms "Backbone", demonstrating the benefit of using domain distance loss. Another observation is that "Backbone+RWSA" performs more favorably than "Backbone", which indicates the advantage of the region-wise separated attention (RWSA) module. Finally, our full method Safe-Paint, i.e., "Backbone+DDL+RWSA", achieves the best results.Based on considerations regarding anti-forensics and the requirements stemming from domain adaptation, we meticulously design the RWSA module. It is natural to elegantly insert the RWSA in any existing networks to enhance their anti-forensic performance. To substantiate this assertion, we will quantitatively compare the original versions and updated versions of some state-of-the-art methods, while keeping the experimental setup consistent with Section 4.3.To be precise, we conduct experiments on EdgeConnect [29] and AOT-GAN [37] . The training strategy also adopts their original In line with our approach, we incorporate RWSA into the downsampling layer of the inpainting network. The corresponding numerical results are shown in Table 6 . It is evident that RWSA can serve as an effective supplement to address insufficient resistance of traditional inpainting methods against detection. 
conclusions: 
 In this work, we propose the first end-to-end learning architecture called SafePaint for anti-forensic inpainting, which is achieved with the help of domain adaptation. The task-decoupled strategy ensures that each stage of the network performs its designated role, while domain adaptation achieves compatibility between the background and the foreground of an image. Moreover, the region-wise separated attention module effectively addresses the shortcomings of the previous attention mechanism module and can be naturally integrated as a plug-in into existing inpainting methods to improve their anti-forensic capabilities. Extensive experiments have demonstrated that our SafePaint has very strong anti-forensic ability while maintaining commendable inpainting performance. 
 
 Appendix:  


Decision: The paper describes an image inpainting method that (i) realistically fills the missing parts of an image and (ii) does it such that traditional forensics detection algorithm do not detect the tampering caused by inpainting.

Most modules in this work are existing, but they are assembled in an original and solid way -- Novelty is limited, yet, inspiring contributions are nicely put together. The proposed system is end-to-end. 

Performance evaluation against adequate benchmarks show the gains compared to recent competitors.

Most questions raised by the reviewers were appropriately addressed in the rebuttal. Authors made several promises for the final version, including something about diffusion models as an alternative. 

Overall, (confident) reviews highlight the merits of this paper, which should be accepted at the conference. 

Anti-forensic inpainting is a nice target as it tackles privacy/security issues. However, it raises ethical concerns as erroneously considering that an image is genuine (i.e. pristine, authentic) may be problematic in some situations. Better motivations for coupling inpainting and forensics would be nice, as well as some higher level discussion about ethics.

Reviews: review: paper_summary: paper_summary: This paper proposes an image inpainting architecture with anti-forensic capabilities.The key idea is to enhance the semantic alignment between the image foreground and background using the domain adaptation method. It is experimentally verified that the anti-forensic capabilities can be significantly improved, while keeping the inpainting performance drop relatively small.
strength_weakness: strength_weakness: -  Introduce an end-to-end training framework for anti-forensic image inpainting, making image inpainting more secure and reliable
-  Adopt a domain adaptation method for solving the inpainting detection problem
- Performance gains are noticeable when compared with SOTA methods.
limitations: limitations: - Novelty: The model architecture almost has nothing new; all components (e.g., Domain Pattern Extractor etc.) are from existing works. Also, the only new module: RWSA module needs to be further clarified why it can improve the anti-forensic capability. The second point of contribution has not been well justified. 
- It would be more interesting to add some experiments on the effect of inpainting+post-processing (low pass for example). When using inpainting to remove objects, some post-processing operations are usually employed too.
- The datasets are a bit outdated. More recent datasets could better demonstrate the superiority of the proposed method. Also, from Table 1, it seems that many latest inpaiting methods are even worse than some earlier ones, which is not quite reasonable.
paper_summary: paper_summary: The paper targets the problem that existing image inpainting methods fail to resist forensic analysis, leading to security vulnerabilities. The theoretical proof reveals that the root cause of poor anti-forensic capability for an image lies in the inconsistency of statistical distributions between the inpainted and original parts. Based on this foundation, the authors introduce the domain adaptation method into image inpainting to achieve the goal of anti-forensics. There are two kinds of experiments, i.e., evaluation on visual quality and anti-forensic analysis, and the results prove the effectiveness of the proposed framework.
strength_weakness: strength_weakness: - The topic holds significant importance in the community from the perspective of security.
- The paper makes a strong contribution and has great guiding significance for the follow-up work.
- This paper explores an interesting and exciting problem in image inpainting, pioneeringly taking the anti-forensic capabilities into consideration.
- Comprehensive theoretical explanation, extensive experimental validation.
limitations: limitations: - Some details are not adequately explained (e.g., the criteria for defining metric ACC).
- The adversarial attack method can also deceive the detector to achieve the goal of anti-forensics. In the author's opinion, what are the main advantages of this proposed method compared with adversarial attack?
- Is the metric ACC in the experiment based on the threshold predefined by the authors or directly calculated by the detectors?
- Based on my understanding, image inpainting algorithms are designed to fill in the missing parts of an image.  Once this step is completed, the task of the inpainting algorithm is considered finished.  I am not at all concerned with whether someone might use detection algorithms to identify inpainted areas, nor do I care about the results of such detection.  As long as the image is restored realistically enough, the inpainting algorithm is deemed satisfactory.  I do not understand the purpose of anti-forensic capabilities, just as with a "denoising algorithm with anti-forensic functions."  This seems absurd, and I fail to see its significance;  the author appears to be pursuing innovation for its own sake.  The author's method does not surpass the current state-of-the-art inpainting algorithms, which makes it unsatisfactory.  Moreover, the author has not compared their method with inpainting algorithms from 2023 and 2024.  I will adjust my score based on the author's rebuttal.
paper_summary: paper_summary: In this paper, the authors explore a new inpainting scheme from the perspective of creating realistic image content and leaving no traces of tampering. Briefly, it first performs inpainting to recover the missing regions of a corrupted image. Subsequently, the difference between internal and unaltered regions is reconciled by domain adaptation to avoid leaving traces. Experimental results demonstrate its high image quality while avoiding detection.
strength_weakness: strength_weakness: The authors' proposal to use the anti-forensic capability as one of the criteria for evaluating the quality of image inpainting is valuable.
This article is well written, easy to understand and sufficiently experimental.
limitations: limitations: 1. The background research is not extensive enough. In the current research, diffusion model for inpainting is effective [1,2], but this paper does not mention the work in this area. So, how does the diffusion model perform for inpainting in the context of anti-forensics? What is the result of this paper comparing with diffusion model in terms of image quality and anti-forensics?
2. Is end-to-end training inpainting necessary and how generalizable is it in the context of diffusion models with strong prior knowledge of image?
3. Lacking an original design, this paper is more akin to a combination of two works. The authors claim to be task-decoupled, however, as shown by the insufficient results in Table 1, the separate tasks seem to be imperfect. There seems to be a loss of image quality in order to achieve anti-forensics.
4. As an architecture for adversarial generative networks, common sense dictates that an adversarial training form of the game should be used, whereas this paper seems to use supervised learning (as in Equation 14). The authors are requested to clarify.
5. Does the domain pattern extractor need to be trained? What are the experimental details?

 Minor issue
1. For easy comparison, authors may underline the suboptimal experimental results.

[1] 10.1109/CVPR52688.2022.01117
[2] 10.1109/PCEMS58491.2023.10136091