# Robust Image Inpainting Forensics by Using an Attention-Based Feature Pyramid Network
###### Abstract  
Deep learning has injected a new sense of vitality into the field of image inpainting, allowing for the creation of more realistic inpainted images that are difficult to distinguish from the original ones. However, this also means that the malicious use of image inpainting technology to tamper with images could lead to more serious consequences. In this paper, we use an attention-based feature pyramid network (AFFN) to locate the inpainting traces left by deep learning. AFPN employs a feature pyramid to extract low- and high-level features of inpainted images. It further utilizes a multi-scale convolution attention (MSCA) module to optimize the high-level feature maps. The optimized high-level feature map is then fused with the low-level feature map to detect inpainted regions. Additionally, we introduce a fusion loss function to improve the training effectiveness. The experimental results show that AFPN exhibits remarkable precision in deep inpainting forensics and effectively resists JPEG compression and additive noise attacks. digital forensics; image inpainting; tampering detection; feature pyramid network; multi-scale convolution attention; deep learning +
Footnote †: journal: Medical  
The conventional inpainting forensics techniques [18, 19] primarily depend on the similarity between image blocks to locate inpainting traces. Unfortunately, these methods exhibit significant limitations, such as high computational costs, low accuracy, and limited generalizability. Currently, research on deep learning-based image inpainting forensics remains in the developmental and exploratory phase [20, 21, 22, 23, 24, 25], while these deep learning-based techniques exhibit superior performance compared to conventional methods, the majority of them pay attention to enhancing the feature extraction block, with suboptimal optimization of the extracted features, resulting in limited accuracy of inpainting forensics. It is imperative to reiterate that the majority of pixels within the inpainted region are derived from known portions of the image. Therefore, optimizing the utilization of local-to-global contextual information of the inpainting image is crucial in improving the results of inpainting forensics. Based on the above considerations, we establish an end-to-end image inpainting forensics network, which uses the attention-based feature pyramid network (AFPN) to locate inpainting traces. AFPN was widely used in object detection [26, 27, 28] and image segmentation [29, 30], but few people use AFPN in the field of image inpainting forensics. By using multi-scale convolution attention [31] (MSCA) after extracting features, the employed AFPN makes the most of low- and high-level features from inpainted images, which are particularly important in inpainting forensics. In the field of image processing, low-level features refer to basic visual characteristics that can be extracted directly from the pixel values of an image, such as color, texture, edges, and corners. The significance of low-level features lies in their ability to capture basic visual cues that are essential for image inpainting detection. High-level features, on the other hand, represent more abstract and semantic aspects of an image, capturing complex visual patterns, structures, and relationships. High-level features can include object shapes, object categories, spatial arrangements, or even more sophisticated attributes, enabling a deeper understanding of the visual content of an image. By combining low-level and high-level features, computer vision systems can achieve a more comprehensive understanding of images. In order to observe the low-level feature, high-level feature, and the key differences between them more intuitively, we show an example of applying AFPN in Figure 1.  
We obtain Figure 1c by adding the low-level feature map to itself and then add spatial attention (SA) to obtain Figure 1d. Figure 1e-g can be obtained in a similar way.