IMDPrompter: Adapting SAM to Image Manipulation Detection by Cross-View Automated Prompt Learning
Authors: Quan Zhang
Yuxin Qi
Xi Tang
Jinwei Fang
Xi Lin
Ke Zhang
Chun Yuan
Venue: ICLR 2025 Poster (2025)
Keywords: Image Manipulation Detection；Segment Anything Model；Prompt learning；Semantic-Agnostic
Primary area: applications to computer vision, audio, language, and other modalities

Abstract: Using extensive training data from SA-1B, the Segment Anything Model (SAM) has demonstrated exceptional generalization and zero-shot capabilities, attracting widespread attention in areas such as medical image segmentation and remote sensing image segmentation. However, its performance in the field of image manipulation detection remains largely unexplored and unconfirmed. There are two main challenges in applying SAM to image manipulation detection: a) reliance on manual prompts, and b) the difficulty of single-view information in supporting cross-dataset generalization. To address these challenges, we develops a cross-view prompt learning paradigm called IMDPrompter based on SAM. Benefiting from the design of automated prompts, IMDPrompter no longer relies on manual guidance, enabling automated detection and localization. Additionally, we propose components such as Cross-view Feature Perception, Optimal Prompt Selection, and Cross-View Prompt Consistency, which facilitate cross-view perceptual learning and guide SAM to generate accurate masks. Extensive experimental results from five datasets (CASIA, Columbia, Coverage, IMD2020, and NIST16) validate the effectiveness of our proposed method.

Published as a conference paper at ICLR 2025
IMDPROMPTER: ADAPTING SAM TO IMAGE MANIP- ULATION DETECTION BY CROSS-VIEW AUTOMATED PROMPT LEARNING
Quan Zhang'; Yuxin Qi Xi Tang', Jinwei Fang", Lin", Ke Zhang"! Chun Yuan!! TTinnguua University Shanghai Jiao Tong University University of Science and Technology of China
(zhangqua22, ke-zhang1 (ezzaagg19eane emails is ts tsinghua. edu. cn, qiyuxin988s jtu. edu, en, yuanc@sz www.ghua...nni...... edu. cn
ABSTRACT
Using extensive training data from SA-1B, the Segment Anything Model (SAM) has demonstrated exceptional generalization and zero-shot capabilities, attracting widespread attention in areas such as medical image segmentation and remote sensing image segmentation. However, its performance in the field of image manipulation detection remains largely unexplored and unconfirmed. There are two main challenges in applying SAM image manipulation detection: a) reliance manual prompts, and b) the difficulty of single-view information in supporting cross-dataset generalization. To address these challenges, we develops cross- view prompt learning paradigm called IMDPrompter based on SAM. Benefiting from the design automated prompts, IMDPrompter no longer relies on manual guidance, enabling automated detection and localization. Additionally, we propose components such as Cross-viee Feature Perception, Optimal Prompt Selection, and Cross View Prompt Consistency, which facilitate cross- voessvview perceptual learning and guide SAM to generate accurate masks. Extensive experimental results from five datasets (CASIA, Columbia, Coverage, IMD2020,, and NIST16) validate the effectiveness of our proposed method.
INTRODUCTION
With the continuous emergence of powerful editing tools Zhang et (2025a), image manipulation has become unprecedentedly simple. These new opportunities have sparked the creativity of both benevolent and malicious users. In the past, organizing multimedia misinformation activities required complex skills, with attackers limited to splicing, copying, or deleting objects. Recently, deep learning has advanced a rapid pace, leading to the development of more user-friendly and powerful image editing tools. As result, users can now swiftly generate images of fictional characters or create highly convincing deepfakes. Generative models can produce realistic image edits based on natural language prompts, perfectly matching inserted elements with the style and lighting of the environment Avrahami et al. (2023); Nichol et al. (2021).
However, the risks of these tools falling into the wrong hands are evident. In recent years, govermments and funding agencies have shown increasing interest developing forensic tools capable of addressing such attacks, particularly focusing local image tampering that alters the semantic content of images Le et al. (2021). response to these challenges, the fields multimedia forensics and related sciences have rapidly expanded, proposing various methods and tools for image manipulation detection (IMD) and localization Guillaro et al. (2023). Despite significant progress in this area, the performance of state-of-the-arr detectors in practical applications remains insufficient, primarily limited by several shortcomings that require further research: a) limited generalization ability; b) limited robustness; c) poor detection performance.
'These authors contributed equally.
1 "orresponniig authors.

---

Published as a conference paper at ICLR 2025
To address the shortcomings of existing IMD methods, turn our attention to foundational models Kirillov et al. (2023); Radford et al. (2021); Jia et al. (2021); Sharif Razavian et al. (2014). Thanks to large-scale pretraining, foundational models like GPT-4 Achiam et al. (2023), Flamingo Alayrac et al. (2022), and SAM Kirillov et al. (2023) have made significant strides and contributed importantly to societal advancements. Among them, SAM, trained on one billion masks, demonstrates exceptional generalization ability, inspiring our research. However, using SAM for iimage manipulation detection tasks presents significant challenges: a) reliance on manual prompts: SAM's interactive framework requires predefined prompts for input images, such as points, boxes, or masks. As antegory-agnostie segmentation method, these limitations hinder SAM's s ability to achieve fully automated understanding in IMD tasks; b) single-view information struggles support cross-dataset generalization: previous studies Zhou et al. (2020) have reported that DeepLabv2 Chen et al. (2017) trained on the CASIAv2 Dong et al. (2013) dataset performed well on the CAISAv1 Dong et al. (2013) dataset (sourced from CASIAv2) but poorly on the non-sourced COVER Wen et al. (2016) dataset. Similar behavior has also been observed with FCN Long et al. (2015) in this research.
To activate SAM's IMD capabilities, we propose view automated prompt learning paradigm called IMDPrompter, aimed at learning how generate prompts to enhance SAM's functionality: a) In- spired by Battle & Gollapudi (2024) using automated prompt optimizers to improve the performance of LLMs, we constructed the automated prompt learning paradigm IMDPrompter, which generates prompt formation automatically; b) To enhance SAM's cross-dataset generalization ability, we introduced three noise views: SRM fil- tering, Bayer Conv, and Noiseprint, aiming to generate enhanced prompt information from multiple perspectives. To integrate information from multiple views, previous methods Zhai et al. (2023) employed strat- egy of weighted averaging of the outputs from multiple views. However, due to the inaccuracy of prompts generated from in- dividual views, simple weighted averaging can smooth out the accurate prompts from the optimal view to suboptimal ones. To preserve the accurate prompts from the optimal view, we propose an Optimal Prompt Selection (OPS) module based minimizing localization loss; meanwhile, we noted that ideally, the prompt information from each view should converge toward the optimal prompt. Based on this motivation, we constructed Cross- View Prompt Consistency (CPC) constraint to achieve alignment enhancement across views. For the mask generation process, we developed an Attention-basel Cross-view Fea- ture Perception (CFP) module and Multi-Layer Perceptron-based Prompt Mixing module (PMM) to achieve the fusion of cross-view information and the integration of multiple types of prompts sequentially.
Prompts
Manuslly gonerated prompix.
RGB
SAM
Prompts Generator
O
Automatically oily generated prompts
SRM
SAP
Dayar
Accurate mask
Noiseprint
Integration of peemanii--ggnostie information
Figure 1: Improvements of proposed IMDPrompter over existing pipelines of baselines: (a). Automated prompt learning without the need for manual input. (b). Flexible integration of semantic-agnostic information crucial for Image Manipulation Detection.
In summary, our main contributions are as follows.
We are the first apply SAM the field of image manipulation detection and propose an automated prompt learning paradigm, IMDPrompter, eliminating the original SAM's reliance on manual prompts.
We propose modules such as Optimal Prompt Selection and Cross- Prompt Consistency Constraint, achieving alignment enhancement across views.
We propose Cross-view Feature Perception and Prompt Mixing modules, achieving the fusion of cross-view information and the integration of multiple types of prompts.
2

---

Published as a conference paper at ICLR 2025
We demonstrate extensive results across five different image manipulation detection datasets, thoroughly validating the strong in-distribution and -ft-o--diittibuttin image manipulation detection and localization capabilities of IMDPrompter.
2
RELATED WORK
Image Manipulation Detection. Currently, methods for image manipulation detection can be broadly categorized into two types, primarily distinguished by their recognition of manipulated artifacts. Some techniques Wu et al. (2019); Chen et al. (2021); Wu et al. (2022); Bi et al. (2019); Hu et al. (2020); Yang et al. (2020); Marra et al. (2020) rely on detecting abnormal features and often use high-pass noise filters Yang al. (2020): Li & Huang (2019) to suppress content information. Other methods Park et al. (2018); Kwon et al. (2022); Mareen et al. (2022) attempt to detect inconsistencies in compression within tampered images, as they assume different compression Quality Factors (QFs) before and after the operation. Additionally, some researchers focus their attention on camera- based artifacts, such model fingerprints Mareen al. (2022); Cozzolino & Verdoliva (2019); Cozzolino et al. (2015).
Foundational Models. In recent years, foundational models have sparked a tremendous transfor- mation in the field of artificial intelligence These models, trained on extensive datasets, have demonstrated impressive generalization capabilities across various scenarios Kirillov et al. (2023); Cai et al. (2024); Liu et al. (2024b); Pan et al. (2024b;a; 2025). Renowned models such as Chat-GPT Ouyang al. (2022), GPT-4 Achiam et al. (2023), and Stable Diffusion Rombach et al. (2022) have further propelled the development of artificial intelligence, making significant contributions to human civilization and exerting considerable influence across various industries. Inspired by the success of foundational models in natural language processing (NLP), researchers have begun exploring their potential applications in computer vision. While most of these models are aimed at extracting acces- sible knowledge from freely available data Alayrac et al. (2022); Radford et al. (2021); Chen et al. (2023), the recent SAM model Kirillov et al. (2023) adopts an innovative approach by constructing a data engine where the model co-develops annotations with environmental datasets. SAM uniquely leverages a vast collection of masks, showcasing robust generalization capabilities. However, it was initially designed as a task-agnostic segmentation model, requiring prompts (i.e., inputs of prior points, bounding boxes, or masks), and therefore does not directly facilitate end-to-end automated segmentation perception. This paper does not delve into the design and training of foundational image manipulation detection models; instead, we explore the potential of utilizing SAM's powerful universal segmentation capabilities for image manipulation detection and localization. Furthermore, the proposed method of learning prompts can be extended to other visual foundational models beyond SAM.
Prompt Learning. In the past, machine learning tasks were primarily focused on fully super- vised learning, where task-spec models were trained only on labeled instances of the target task Krizhevsky et al. (2017). However, over time, there has been significant shift in learning paradigms, transitioning from fully supervised learning towards pretraining and fine-tuning approaches for downstream tasks. This shift allows models to leverage general features acquired during pretraining Russakovsky et al. (2015); Simonyan & Zisserman (2014); He et al. (2016). More recently, with the advent of foundational models, new paradigm has emerged known pretraining and prompting Chen et al. (2023); Lester et al. (2021); Liu et al. (2023); Zhou et al. (2022). In this paradigm, researchers no longer train models specifically for downstream tasks but instead redesign inputs using prompts to reformulate the downstream tasks to align with the original pretraining task Radford et al. (2021): Devlin et al. (2018); Radford et al. (2019), Prompting helps to reduce semantic gaps, bridge the gap between pretraining and fine-tuning, and prevent overfitting of the heads. Since the advent of GPT-3 Brown et al. (2020), prompting has evolved from traditional discrete Liu al. (2023) and continuous prompt constructions Chen et al. (2023); Zhou et al. (2022) to large-scale model-centric contextual learning Alayrac et al. (2022), instruction tuning Liu et al. (2024a); Peng et al. (2023); Gupta et al. (2022), and chaining approaches Wei et al. (2022); Wang et al. (2022a); Zhang et al. (2022). Currently, methods for constructing prompts include manual templates, heuristic- based templates, generation, fine-tuning word embeddings, and pscudo-labeling Liu et al. (2023); Wang et al. (2022b). In this paper, we propose a prompt generator for generating prompts compatible with SAM.
3

---

Published as a conference paper at ICLR 2025
3
PROPOSED METHOD
Image Encoder
Maak Decodar
Image
SAM-image
OOF
Firsc
GFP
For
PMM
SRM
SRM Encoder
FaRM
clessly
Mapt
RGB Encoder
clessiy
GPC
DPS
Prompl Encoder
CPC
Bayar
Beyar Encoder
faayar
clessly
Bapt
Noiseprint
Naiseprint
ciassity
Encoder
Nolsepritt
Bayar:
Bayer
Noiseprint
Noksapeinnee
Figure 2: Overall framework of IMDPrompter. The prompter part consists of four views: RGB, SRM, Bayar and Noiseprint. OPS selects the optimal prediction from the four views to generate the best prompt. CPC enhances cross view consistency. CFP achieves cross-view feature perception fusion. PMM achieves a mixture of multiple prompt information.
As illustrated in Figure 2, we introduce IMDPrompter, an automated prompt learning framework leveraging cross-view view perception to unlock SAM's s detection and localization capabilities. It employs optimal prompt selection module identify the best prompts, enhances alignment via cross-view prompt consistency, and achieves efficient image manipulation detection and localization through feature fusion and prompt mixing.
3.1 MULTI-VIEW FEATURE REPRESENTATION
Current technology Zhou et al. (2020) shows that while RGB views work well for in-domain (IND) manipulation detection, they perform poorly in out-of-domain (OOD) detection. Moreover, using noise views to capture semantic-agnostic information enhances performance. Therefore, relying solely on RGB view data is inadequate for detecting and localizing manipulations. To address this, we incorporate three semantic-agnostic views: SRM, Bayer, and Noiseprint noise views, outlined below:
Fsam 4 saam-iig (I);
(1)
JRGB &s RGB (I), /SRM 4 SRM (#SRM (I) /Rayer Sey. Seg-BBaer Bayer
S (Z))
(Pyyerr
[Noiseprint 4 Seg. (NNiseprin
In the system, I represents the input image, and & &sam-img denotes the SAM image encoder. Fum represents the features encoded by the SAM image encoder. 4SRM- Bayer and Nessepria respectively represent the SRM noise map extractor, the Bayer noise map extractor and the Noiseprint noise map extractor, Sey RGB+ & SRM. & Bayer and Seeg Noiseprint each represent the segmenter for the RGB view, SRM view, Bayer view and Noiseprint view, with all four segmenters having identical structures
Bayer
4

---

Published as a conference paper at ICLR 2025
but non-shared parameters. JRGB, fSRM. /Bayer and /Noisepriut represent the features from the RGB, SRM, Bayer, and Noiseprint views, respectively.
3.2 OPTIMAL PROMPT SELECTION
In the masked prompt generation process, we integrate mask probability distributions from four views: PRGB. PSUM. Playar. and Pooiepprint to form an ensemble distribution Pans- We select the optimal mask probability segmentation Popt by minimizing segmentation loss, and derive the corresponding mask Mopt and bounding box prompts Bopt- These are fed into SAM's prompt encoder to obtain the prompt encoding Popt- The process is outlined follows:
PRGB =
PSRM D CLS(())M)), Paayer = Payer -ayer-CIS (J
Nolsepelut Nolseprint-CLs (f
(5)
PROB + PSRM + PBayer + Noiseprrnt
PERS
(6)
4 Pops argmin L Seg (P.G),P- (PROB, PSRM, Prayer; Poiseprrntt Prins
(7)
PEP
Mopt = 4 mask (Pap)) Bops = Dox Dox (Mopt),
(8)
Pop = p-ennc (Mopt: Bopt):
(9)
where PRGB-CLS, PSRM-CIS CLS. B and NNiseprrnnt-CLS represent classifiers for the RGB, SRM, Bayer, and Noiseprint views, respectively, G represents the one-hot encoded mask labels, Lseg the segmentation loss function, Pmakk represents the mask generation operation, Phox the bounding box generation operation, and L pp-ene represents SAM's prompt encoder.
3.3 CROSS-VIL VIEW PROMPT CONSISTENCY
Ideally, the segmentation masks from the four views should consistent with the optimal segmenta- tion mask. Therefore, we constructed a cross-view prompt consistency enhancement loss to achieve enhanced prompt consistency across views. The CPC loss function is expressed as follows:
Lcrc Liseg PRGB, Pap.) Lseg (PRRM, Popi) LSeg Bayyer Pap)) Liseg Noisepriut, Papt
(10)
where Lseg is the segmentation loss function.
3.4 CROSS-VIEW FEATURE PERCEPTION
To enhance the segmentation accuracy of SAM, we propose Cross-View Feature Perception (CFP) module that integrates features from RGB, SRM, Bayer, and Noiseprint views. This module enriches the feature representations for SAM's de- coding process, as shown in Figure 3. Here, &CFP denotes the CFP module, and FCFP represents the fused cross-view features.
FCFP
Concrtmation Operatioo
S12M00D
Dot
Auagg Peo
The process begins by aligning features FRGB, fSRM. fBayer, and SNoiseprint with F'am via bicubic interpolation. These features are then concate- nated with Fsam after 1 convolution. The concatenated features pass through two stacked convolution blocks (3 x 3 convolution, instance normalization, and GELU activation). Meanwhile, semantic selection attention branch (global average pooling, fully connected layer, GELU, and sigmoid) weights the shortcut branch to pro- duce the integrated feature Forp. The CFP module effectively fuses multi-view features to improve segmentation performance.
Convlation Blak
Ctertil
Condioi
Cancisl
Crerls
FSAM
fsum
FRem
Juwer
JNokeptts
Figure 3: Architecture of the Cross-view Feature Perception (CFP) unit.
FCrP &CrP (Fam; fRGB, JSRM: Bayer, JNoiseprina)
(11)
5

---

Published as a conference paper at ICLR 2025
3.5 PROMPT MIXING MODULE
Since PCFP aggregates semantically related and unrelated features, it aids the guidance of SAM for image manipulation detection. Therefore, constructed the PMM module, which is primarily based on MLP, to integrate multiple types of prompt information. First, the two types of prompt inputs, FCFP and Fopt. are concatenated, followed by an MLP layer perform dimension transformation to align with the original SAM prompt encoder's 's prompt embedding. Then, Fixx and FcFr are input into SAM's mask decoder to achieve the image manipulation localization process. The process is as follows:
Fixx = PMLP ([FOFP; Fopt))
(12)
P sam & dec (FFP,, Fmix);
(13)
where PMLP represents the MLP layer operation, Fmix represents the integrated prompt embed- ding, and & @sam-dde dec represents SAM's mask decoder, with Psam representing the mask probability prediction output of the mask decoder.
3.6 TRAINING AND INFERENCE PIPELINE
The training loss function of IMDPrompter includes the segmentation loss from four prompt views Segg-p the CPC loss LCpc, the SAM decoder segmentation loss Lseg-sam- and the image-level prediction loss Leme-level- The formula is expressed follows:
L = Lsog-sam + Cs + AgCoPC + A3
(14)
where
Seg.ssa Lay Psam: G);
(15)
Lseg-p Lssg(PRRBB G) + Lseg( ( Paayar G) + Lseg (PSRM: G) + L ( Noissepenns G).
(16)
In this paper, we use Focal Loss Lseg.
For image-level detection, following the work of Zhai et al. (2023), we adopt an adaptive pooling based on minimizing intra-class prediction variance. The overall computational process is follows, first using Otsu's method Otsu et al. (1975) to find a threshold wp that minimizes the intra-class prediction variance:
wp arg min (Pr.. Paj < wIll var (17s Pig < w)) +
(e(
(17)
Kits Pi.s 2 will var (ties pi.j 2 w))
where var(-) denotes variance, Pt.j the pixel-level response at position (i.j). Then the image-level prediction is aggregated from pixel-level responses above the threshold and the image-level loss is:
Σ p: Ih = (pr.s Pcg 2 wo).
(18)
Ph
DE
Limg-level = LBCE (v. 9A).
(19)
where ÚA is the image-level prediction, LECE denotes the binary cross-entropy loss.
During the inference process of IMDPrompter, since there are no true labels, our OPS module defaults to selecting PEns for generating masks and bounding box prompts, with other components functioning as during the training process.
EXPERIMENT
Dataset. Our method is trained only on the CASIAv2 dataset Dong et al. (2013). For in-distribution (IND) evaluation, evaluation, we use three datasets: Columbia Hsu & Chang (2006), Coverage Wen et al. (2016)and IMD2020 Novozamsky et al. (2020).
the CASIAv1 dataset Dong et al. (2013). For out-of-distribution (OOD)
Evaluation Metrics. For image-level manipulation detection, we report specificity, sensitivity, and their F1-score (I-F1). The area under the receiver operating characteristic curve (AUC) is also

---

Published as a conference paper at ICLR 2025
reported threshold-indee independent metric for image-level detection. For pixel-level manipulation localization, we follow previous methods Chen et al. 1) Zhou et al. (2018b;a); Salloum et al. (2018) to compute pixel accuracy, recall, and their Fl-score (P-F1) manipulated images. The overall performance of image and pixel-level manipulation detection/localizatii is measured by the harmonic mean of pixel-level and image-level F1 F1-scores Chen et al. (2021), denoted composite Fl (C-FI), and is sensitive to lower values of P-F1 and I-FI. To ensure fair comparison, a default threshold of 0.5 used for Fl computation unless otherwise specified.
Implementation Details. In our experiments, unless otherwise specified, we consistently use the VIT- L backbone SAM and employ FCN (lightweight architecture based on MobileNet Howard (2017)) as the segmentor for the three prompt views. We maintain image size at 1024x1024 consistent with the original input of the SAM model. To augment training samples, we use data augmentation techniques such as horizontal flipping and random cropping. The image encoder remains frozen during the training phase. All experiments are run on NVIDIA A6000 GPUs. For the optimization process, train our model using the AdamW optimizer with an initial learning rate of le-4. We use a batch size of 4 and train for 100 epochs. We implement a linear warm-up strategy with cosine annealing scheduler Loshchilov & Hutter (2016) to decay the learning rate.
4.1 COMPARISON WITH STATE-OF-THE METHOD
Table Pixel-level manipulation detection performance. Performance metric Fl-score (%). The best results for each test set are highlighted in bold and second-best values are underlined.
Method
Best threshold
Fixed threshold (0.5)
CASIA
COVER
Columbia
IMD
NIST
MEAN
CASIA
COVER
Columbia
IMD
NIST
MEAN
MFCN Salloum (2018)
54.1
61.2
RGB-N Zhou (2018b)
40.8
37,9
H-LTTM Bappy (2019)
20.9
21.3
31.00
16.3
13.0
19.5
35.4
19,92
ManTra-Net Wa et al. (2019)
69.2
77.2
70.9
70.5
45.5
66.66
15.5
28.6
36.4
18.7
0.0
19.84
HP-PCN Li & Huung (2019)
21.4
19.9
47.1
16.9
36.0
28.26
15.4
0.3
6.7
11.2
12.1
9.14
CR-CNN Yang
(2020)
60.0
29.1
GSR Net Zhou al. (2018a)
57.4
48.9
62.2
68.7
15.6
56.56
38.7
28.5
61.3
24.3
28.1
36.22
SPAN Hu (2020)
68.8
71.8
77.4
69.6
68.3
71.18
17.2
48.7
22,1
CAT.Net Keoon al. (2021)
57.3
48.5
77.6
51.7
59.9
59.00
$5.5
5.4
17.9
21,06
MVSS-XA Net Chen al (2021)
25.3
82.4
70.3
75.7
73.2
75.18
45,2
15.3
63.8
26.0
29.2
11.90
Trufor Guillaro et all (2023)
83.5
82.1
68.8
73.7
60.0
85.9
30.9
PCN Long al. (2015)
74.2
$7.3
58.6
64.5
50.7
61.06
44.1
22.3
21.0
16.7
24,80
IMDPrompter
85.1
83.5
87.3
76.3
74.6
81.36
76.3
63.6
87.3
30.6
41.1
59.78
+10.9
126.2
11..8
+20.3
165.0
124.4
Table 2: Image-level manipulation detection performance. Decision threshold: 0.5. NIST16 is excluded as it lacks genuine images. The best results for each test set are highlighted in bold and second- best values are underlined.
CASIA
DOIVRR
ColanNaa
IMD
MRAN
ABC
Scn.
Spc.
F1
ALE.
Scc.
Spc
F1
AUL
Scn
Sgc.
F1
ALE
Sca.
Spe:
21
ALC
H-LSTM Eqpar (2019)
0.293
59.7
0.0
0.0
0.500
100.0
0.0
0.0
0.505
105.0
1.3
2.2
0.50]
130.D
0.0
0.0
1.5013
0.35
Manilta-Ne (2019)
0.303
100.0
0.300
100.0
0.0
0.0
0.301
102.0
0.0
0.500
156.0
0.0
0..501
0.00
CR-CNN Ying (2000)
0.119
710
0.783
96.1
39.2
0665
92.9
12.3
116771
24.55
SPAN (2000)
0.0
100.0
0.0
0.5000
CXI-NNi Koun (2501)
0.647
$0.0
0.971
$7.2
91.5
0.58
27.5
$1.5
0.000
$3.03
sivss Ner Cher (2021)
81917
61.5
988
75%
0/731
91.0
14.0
344
0.980
06.9
100.0
30.
0.65
91.5
13.0
B336)
53.88
"Inttr Canllam et (1823)
8.916
0.999
FCN Long at (20155
0.770
72.8
64.5
08.5
0.541
90.0
1000
130
0.702
95.0
32.2
48.
0.532
946
15.5
36.2
B.643
40.15
IMDesonppee
0.578
91.6
66.9
77.3
0.7%6
95.3
55.2
70.3
0.983
92.3
00.2
93.6
8,611
71.6
57.4
63.7
0.8576
76.23
+15.3
-5.3
-0.221
-50
+45.5
+0
-13.0
+11.9
+0.3130
Pixel-level Manipulation Detection Table illustrates the pixel-level detection performance of different models. We evaluate the F1 -core under two settings: best threshold and fixed threshold (0.5). From Table 1, it is evident that IMDPrompter achieves the best performance across nearly all datasets. Under the optimal threshold setting, we achieved an average Fl Score of 81.36%. Under the fixed threshold setting, IMDPrompter achieved average F1 Score of 59.78%. indicating that our proposed IMDPrompter demonstrates better robustness in threshold settings.
Image-level Manipulation Detection. Table 2 presents the image-level performance of different models. For image-level performance, we use a default decision threshold of 0.5. Once again, IMDPrompter emerges as the top performer, leveraging our prompt learning paradigm achieve
7

---

Published as a conference paper at ICLR 2025
Image
GT
ManTra-Net
MVSS-Net
Trufor
IMDPrompter
Figure 4: Visualization Comparison Image Manipulation Detection and Localization Results
Table 3: Evaluation of each component of method in IND dataset.
Table 4: Evaluation of each component of method in OOD dataset.
CASIA
COVER
CFP
CPC
OPS
PMM
CFP
CPC
OPS
PMM
1-F1
P-F1
C-FI
I.F1
P.F1
C-F1
70.6
70.3
70.4
20.4
39.8
27.0
2
V
73.2
73.1
73.1
49.6
54.3
51.8
V
74.3
74.3
74.3
8
V
58.6
57.0
57.8
75.1
75.6
V
V
65.2
61.4
63.2
5
V
77.3
76.3
76.8
10
70.3
63.6
66.8
higher specificity in most test settings, thereby reducing false positives. Additionally, IMDPrompter achieves the best average AUC and average Fl-scores. The average Fl-score of IMDPrompter significantly surpasses that of the second-bes -bet method, MVSS-Net, with an improvement of 22.25%.
Qualitative Results. As shown Figure we visualized the image manipulation localization results of four methods: ManTra-Net, MVSS-Net, Trufor, and IMDPrompter. It is evident that IMDPrompter generates more accurate confidence maps for image manipulation localization, achieving better detection and localization.
4.2 ABLATION STUDIES
To reveal the impact of different components, we evaluated the proposed model under various settings, analyzing distinct effects of each component on in-domain and out-of-domain datasets. As shown in Tables 3 and 4, we conducted ablation experiments on the in-domain dataset CASIA and the out-of-domain dataset COVER, respectively.
Baseline. From Experiment 1, it is known that our baseline method achieved performance scores of 70.6%, 70.3%, and 70.4% on the I-F1, P-F1, and C-FI metrics respectively in the CASIA dataset. In the COVER dataset, it scored 20.4%, 39.8%, and 27.0% on the same metrics. These experiments show that using only RGB visual view information has limited generalization ability in oft-of-dommin datasets.
Impact of CFP. From Experiment 2, By implementing cross-view feature perception fusion through the CFP module, the I-F1, P-F1, and C-F1 metrics on the CASIA dataset improved by 2.6%, 2.8%.
8

---

Published as a conference paper at ICLR 2025
Table 5: Ablation Study of View Combinations
Naisqprine Bogar SRM
14:39
75.90
74.70
62.90
58.70
69.70
56.00
59.90
27.90
28.10
14.30
70.00
12.40
61.30
56.30
58.70
83.40
81.60
82.50
54.00
25.70
34.90
0.896
75.10
11.60
13.30
0.713
50.60
55.60
$7.50
0.505
84.60
79.30
81.90
0.593
$6.30
24.30
33.00
0.941
76.90
15.20
76.00
0.781
66.80
61.90
64.30
0.969
91.70
85. 40
88.40
0.655
61.50
28.80
39.30
0
71.10
75.90
76.50
0
08.70
63. 80
05.
80
0.977
92. 40
86.
89.
60
0.000
62.00
29.50
00.
0.98
77:30
76
76.80
0. 79
70 99
6300
66
0.981
91.00
87.30
90.
90
0.671
63.70
90.00
41.90
Table 6: Ablation Study of SAM
COVER
Columbia
IMD
Method
L-AUC:
[-F1
P-P1
C-F1
L-ALCC
LF1
P-F1
C-F
L-AUC
LPI
P-F1
C-FI
LAUCC
I.F1
P-P1
C-F1
FCN
0.770
68.3
14.1
53.6
0.541
13.0
19.9
18.9
0.762
48.1
22.3
30.3
0.502
26.2
21.0
23.3
FCN+
0.589
IMDPrompter
0.978
77.3
76.3
76.8
0.796
70.3
63.6
66.8
0.983
93.6
87.3
90.3
0.671
63.7
30.6
41.3
and 2.7%, respectively. Experiment VII showed that on the COVER dataset, the I-F1, P-FI, and C-F1 metrics increased by 29.2%, 14.5%, and 24.8%, respectively. This indicates that cross-view feature perception fusion can enhance performance on both in-domain and out-of-domain datasets, especially out-of-domain datasets.
Impact of CPC. From Experiment 3, the inclusion of CPC, which introduced cross-view view consistency enhancement, resulted in performance gains of 3.7%, 4.0%, and 3.9% the I-F1, P-F1, C-F1 metrics respectively in the CASIA dataset. Experiment VIII showed performance gains of 38.2%, 17.2%, and 30.8% respectively on the COVER dataset, confirming that the prompt information between different views is complementary and integrative enhancement can bring objective performance improvements.
Impact of OPS. The inclusion of OPS for heuristic selection of optimal prompts resulted in im- provements of 4.5%, 5.8%. and 5.2% in the I-F1. P-F1, and C-F1 metries on the CASIA dataset, respectively. Experiment IX showed gains of 44.8%. 21.6%, and 36.2% respectively on the COVER dataset, indicating that simply adding together prompts from different views is not enough to generate the best prompts, and a optimal prompts selection strategy can optimize the prompt selection process. From Experiment 4, the inclusion of OPS adaptive selection of the optimal prompts resulted in performance gains of 4.5%, 5.8%, and 5.2% on the I-F1, P-F1, C-F1 metrics respectively in the CASIA dataset.
Impact of PMM. From Experiment 5, with the inclusion PMM which fully integrates various types of prompt information, there an increase in performance of 6.7%. 6.0%, and 6.4% on the I-F1, P-F1, C-Fl metrics respectively in the CASIA datasct. Experiment 10 showed an increase of 49.9%. 23.8%, and 39.8% respectively on the COVER dataset, further demonstrating the effectiveness of PMM.
Impact of View Combinations. As shown in Table 5, the performance of using the RGB view alone does not differ significantly from that of using semantic-agnostic views such as SRM, Bayer, and Noiseprint individually on the in-domain test CASIA. In contrast, for the out-of-domain test sets COVER. Columbia, and IMD, there is significant performance gap between the RGB view and the semantic-agnostic views. This indicates that cemantic-agnostie views play a crucial role in generalization on out-of-domain datasets. Additionally, we conducted ablation studies on various view combinations, and using both the RGB view and the Noiseprint view simultaneously resulted in substantial performance improvements. This demonstrates that the Noiseprint view serves as a critical semantic-agnostic feature. Next, we sequentially introduced the Bayer and SRM views, achieving further steady improvements across all metrics. When we combined the RGB, Noiseprint, SRM, and Bayer views, our method achieved optimal performance, validating the effectiveness of this combination.
Impact of SAM. As shown in Table 6, we constructed an FCN+ model that integrates four views without using SAM, further analyzing the impact of SAM on IMD. Due to the lack of potential priors from a large-scale pre-training dataset, the performance ceiling of FCN+ is relatively low. In contrast, IMDPrompter fully leverages these potential priors, thereby enhancing the performance ceiling
Impact of Quality Degradation. As shown in Figure 5, following Dong et al. (2022), we evaluated the robustness of the models under two common image processing operations encountered during the

---

Published as a conference paper at ICLR 2025
H-LSTM
CR-CNN
SPAN CAT-Net
MVSS-Net
TraNee
GSR-Net
IMDPrompter
80
Poellovel FF (N)
Image- Tage-lovell F1 (15)
60
Com-F1 (99
40
40
20
20
20
100
90
80
70
60
100
90
80
70
60
100
90
80
70
60
00
JPEG Comprassion Quality
JPEG Compression Quality
JPEG Compression Quillty
(8)
(b)
(c)
80
80
Pixel-kvell Fr [98
60
Image-ooel F1 (15]
Con-FT (99]
40
40
40
20
20
20
23
23
23
Gaussian Blur Kernel Size
Gaussian Blur Kerel Size
Gaussian Blur Size
(d)
(e)
(f)
Figure 5: Robustness Analysis Against JPEG Compression and Gaussian Blur Interference
dissemination of images the internet, namely JPEG compression and Gaussian blur. Comparing these two operations, Gaussian blur has a more significant impact on detection performance, especially when using larger 17x17 scale convolution kernels. Compared previous methods, our IMDPrompter exhibits better robustness.
5 CONSLUSION
In this paper. we propose cross-view perceptual prompt learning paradigm. IMDPrompter, which applies SAM the image manipulation detection task for the first time. We sequentially introduce components such optimal prompt selection, cross-view prompt consistency, cross-view feature perception, and prompt mixing modules to achieve efficient and automated image manipulation detection and localization. Our IMDPrompter demonstrates significant image-level and pixel-level manipulation detection performance both in-domain (IND) and oft-of-domain (OOD) scenarios, well across various robustness evaluation settings.
ACKNOWLEDGEMENT
We thank all the anonymous reviewers for their helpful comments. This work was sup- ported by the National Key R&D Program of China 42022YFB44004000477000000 SSTIC 13916002,,GH22224000 13604008), Beijing Key Lab of Networked Multimedia and National Natural Science Foundation of China under Grant 62202302.
10

---

Published as a conference paper at ICLR 2025
REFERENCES
Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, et Gpt-4 technical report. arXiv preprint arXiv:2303.088774 2023.
lean-Baptiste Alayrac, Jeff Donahue, Pauline Luc, Antoine Miech, lain Barr, Yana Hasson, Karel Lene, Arthur Mensch, Katherine Millican, Malcolm Reynolds, et al. Flamingo: a visual language model for few-shot learning, Advances in neural information processing systems, 35:23716-237366 2022.
Omri Avrahami, Ohad Fried, and Dani Lischinski. Blended latent diffusion. ACM transactions graphics 11, 2023.
Jawadul H Bappy, Cody Simons, Lakshmanan Nataraj, BS Manjunath, and Amit K Roy-Chowdhury. Hybrid Istm and decoderr architecture for detection of image forgeries. IEEE transactions on image processing, 28(7):3286-3000 2019.
Rick Battle and Teja Gollapudi. The unreasonable effectiveness of eccentric automatic prompts. arXiv preprint arXiv:2402.10900 2024.
Xiuli Bi, Yang Wei, Bin Xiao, and Weisheng Li. Rru-net: The ringed residual u-net for image splicing forgery detection. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 0-0, 2019.
Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. Language models are few-shot learners. Advances in neural information processing systems, 33:1877-1901, 2020.
Miaomiao Cai, Xiaoyu Liu, Zhiwei Xiong, and Xuejin Chen. Biosam: Generating sam prompts from superpixel graph for biological instance segmentation. IEEE Journal of Biomedical and Health Informatics, 2024.
Keyan Chen, Xiaolong Jiang, Yao Hu, Xu Tang, Yan Gao, Jianqi Chen, and Weidi Xie. Ovarnet: Towards open-vocabulary object attribute recognition. In Proceedings of the IEEF/CVF Conference Computer Vision and Pattern Recognition, pp. 33518-23527, 2023.
Liang-Chieh Chen, George Papandreou, lasonas Kokkinos, Kevin Murphy, and Alan L. Yuille. Deeplab: Semantic image segmentation with deep convolutional nets, atrous convolution, and fully connected crfs. IEEE transactions on pattern analysis and machine intelligence, 40(4))834484) 2017.
Xinru Chen, Chengho Dong, Jiaqi Ji, Juan Cao, and Xirong Li. Image manipulation detection by multi-view multi-scale supervision. In Proceedings of the IEEE/CVF international conference on computer vision, pp. 14185-14193, 2021.
Yirui Chen, Xudong Huang, Quan Zhang, Wei Li, Mingjian Zhu, Qiangyu Yan, Simiao Li, Hant- ing Chen, Hailin Hu, Jie Yang, et al. Gim: A million-scale benchmark for generative image manipulation detection and localization. arXiv preprint arXiv:2406. 16531, 2024.
Davide Cozzolino and Luisa Verdoliva. Noiseprint: A enn- cnn-based camera model fingerprint. IEEE Transactions on Information Forensics and Security, 15:144-159, 2019.
Davide Cozzolino, Giovanni Poggi, and Luisa Verdoliva. Splicebuster: A new blind image splicing detector. In 2015 JEEE International Workshop on Information Forensics and Security (WIFS), pp- 1-6. IEEE, 2015.
Jacob Devlin, Ming-We Wei Chang, Kenton Lee, and Kristina Toutanova. Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint 1rXiv:1810.04880 2018.
Chengbo Dong, Xinru Chen, Ruohan Hu, Juan Cao, and Xirong Li. Mvss-net: Multi-view multi-scale supervised networks for image manipulation detection. IEEE Transactions on Pattern Analysis and Machine Intelligence, 45(3)33539-3553, 2022.

---

Published as a conference paper at ICLR 2025
Jing Dong, Wei Wang, and Tieniu Tan. Casia image tampering detection evaluation database. In 2013 IEEE China summit and international conference on signal and information processing, pp. 422-426. IEEE, 2013.
Shuting Dong, Feng Lu, Zhe Wu, and Chun Yuan. Dfvsr: Directional frequency video super-resolution via asymmetric and enhancement alignment network. In IJCAI, pp. 681-689, 2023a.
Shuting Dong, Feng Lu, and Chun Yuan. Frequency reciprocal action and fusion for single image super-resolution. In ICASSP 2023-2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 1-5. IEEE, 2023b.
Shuting Dong, Zhe Wu, Feng Lu, and Chun Yuan. Enhanced image deblurring: An efficient frequency exploitation and preservation network. In Proceedings of the 31st ACM International Conference on Multimedia, pp. 7184-7193, 2023c.
Hongfan Gao, Wangmeng Shen, Xiangfei Qiu, Ronghui Xu, Jilin Hu, and Bin Yang. Diffimp: Efficient diffusion model for probabilistic time series imputation with bidirectional mamba backbone. arXiv preprint ar ArXiv:2410. 13338, 2024.
Fabrizio Guillaro, Davide Cozzolino, Avneesh Sud, Nicholas Dufour, and Luisa Verdoliva. Trufor: Leveraging all-round clues for trustworthy image forgery detection and localization. In Proceedings of the IEEE/CVF Conference on Computer Vision and Puttern Recognition, pp. 20606-2000 20615, 2023.
Prakhar Gupta, Cathy Jiao, Yi-Ting Yeh, Shikib Mehri, Maxine Eskenazi, and Jeffrey P Bigham. Instructdial: Improving zero and few-shot generalization in dialogue through instruction tuning. arXiv preprint arXiv:2205. 2tt:::2255...7333. 2022.
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 770-778, 2016.
Andrew G Howard. Mobilenets: Efficient convolutional neural networks for mobile vision applica- tions. arXiv preprint arXiv:1704.0486.1 2017.
Yu-Feng Hsu and Shih-Fu Chang. Detecting image splicing using geometry invariants and camera characteristics consistency. In 2006 IFFE International Conference on Multimedia and Expo, pp- 549-552. IEEE, 2006.
Shiyan Hu, Kai Zhao, Xiangfei Qiu, Yang Shu, Jilin Hu, Bin Yang, and Chenjuan Guo. Multire: Joint learning for time series anomaly prediction and detection with multi-scale reconstructive contrast. arXiv preprint S.t1::2100.19999) 2024.
Xuefeng Hu, Zhihan Zhang, Zhenye Jiang, Syomantak Chaudhuri, Zhenheng Yang, and Ram Nevatia. Span: Spatial pyramid attention network for image manipulation localization. In Computer Vision- ECCV 2020: 16th European Conference, Glasgow, UK, August 23-28, 2020, Proceedings. Part XXI 16, pp. 312 328. Springer, 2020.
Chao Jia, Yinfei Yang. Ye Xia, Yi-Ting Chen, Zarana Parekh, Hieu Pham, Quoe Le. Yun-Hsuan Sung. Zhen Li, and Tom Duerig. Scaling up visual and vision- laiguala representation learning with noisy text supervision. In International conference on machine learning, pp. 4904 4916. PMLR, 2021.
Alexander Kirillov, Eric Mintun, Nikhila Ravi, Hanzi Mao, Chloc Rolland, Laura Gustafson, Tete Xiao, Spencer Whitehead, Alexander C Berg, Wan Yen Lo, et al. Segment anything. In Proceedings of the IEEE/CVF International Conference on Computer Vision, 2023.
Vladimir V Kniaz, Vladimir Knyaz, and Fabio Remondino. The point where reality meets fantasy: Mixed adversarial generators for image splice detection. Advances in neural information processing systems, 32, 2019.
Alex Krizhevsky, Ily Sutskever, and Geoffrey E Hinton. Imagenet classification with deep convolu- tional neural networks. Communications of the ACM, 60(6):84-00, 2017.
12

---

Published as a conference paper at ICLR 2025
Myung-Joon Kwon, In-Jae Yu, Seung- Hun Nam, and Heung- Kyu Lee. Cat-net: Compression artifact tracing network for detection and localization of image splicing. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Viston, pp. 375-384, 2021.
Myung-Joon Kwon, Seung- Hun Nam, In-Jae Yu, Heung-Ky Lee, and Changick Kim. Learning jpeg compression artifacts for image manipulation detection and localization. International Journal of Computer Vision, 130(8): (185):1875-1955 2022.
Trung-Nghia Le, Huy H Nguyen, Junichi Yamagishi, and Isao Echizen. Openforensics: Large-scale challenging dataset for multi-face forgery detection and segmentation in-the-wild. In Proceedings of the IEEE/CVF international conference computer vision, pp. 10117-10127, 2021.
Brian Lester, Rami Al-Rfou, and Noah Constant. The power of scale for parameter-efficient prompt tuning. arXiv preprint arXiv:2104.08691, 2021.
Haodong Li and Jiwu Huang. Localization of deep inpainting using high-pass fully convolutional network. In proceedings of the IEEE/CVF international conference on computer vision, pp- 8301-8311 -301- 2019.
Zhe Li, Xiangfei Qiu, Peng Chen, Yihang Wang, Hanyin Cheng, Yang Shu, Jilin Hu, Chenjuan Guo, Aoying Zhou, Qingsong Wen, et al. Foundts: Comprehensive and unified benchmarking of foundation models for time series forecasting. arXiv preprint arXiv:2410.1 1802, 2024.
Haotian Liu, Chunyuan Li, Qingyang Wu, and Yong Jae Lee. Visual instruction tuning. Advances in neural information processing systems, 36, 2024a.
Pengfei Liu, Weizhe Yuan, Jinlan Fu, Zhengbao Jiang, Hiroaki Hayashi, and Graham Neubig. Pre-train, prompt, and predict: A systematic survey of prompting methods in natural language processing. ACM Computing Surveys, 55(9): 2023.
Xiaoyu Liu, Xin Ding, Lei Yu, Yuanyuan Xi, Wei Li, Zhijun Tu, Jie Hu, Hanting Chen, Baoqun Yin, and Zhiwei Xiong. Pq-sam: Post-training quantization for segment anything model. In European Conference on Computer Vision, pp. 420-437. Springer, 2024b.
Jonathan Long, Evan Shelhamer, and Trevor Darrell. Fully convolutional networks for semantic segmentation. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 3431-3440, 2015.
Iya Loshchilov and Frank Hutter. Sgdr: Stochastic gradient descent with warm restarts. arXiv preprint arXiv:1608.0333 2016.
Jun Ma. Yuting He, Feifci Li, Lin Han, Chenyu You, and Bo Wang. Segment anything in medical images. Nature Communications, 15(1)6654, 2024.
Hannes Mareen, Dante Vanden Bussche, Fabrizio Guillaro. Davide Cozzolino, Glenn Van Wallendael, Peter Lambert, and Luisa Verdoliva. Comprint: Image forgery detection and localization using compression fingerprints. In International Conference on Pattern Recognition, pp. 281-299. Springer, 2022.
Francesco Marra, Diego Gragnaniello, Luisa Verdoliva, and Giovanni Poggi. A full-image full- resolution tnd-i-lend-rranaalle framework for image forgery detection. IEEE Access, 8: 133488-133502, 2020.
Alex Nichol, Prafulla Dhariwal, Aditya Ramesh, Pranav Shyam, Pamela Mishkin, Bob McGrew, Ilya Sutskever, and Mark Chen. Glide: Towards photorcalistic image generation and editing with text-guided diffusion models. arXiv preprint arXiv:21 2021.
Adam Novozamsky, Babak Mahdian, and Stanislav Saic. Imd2020: large-scale annotated dataset tailored for detecting manipulated images. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision Workshops, pp. 71-80, 2020.
Nobuyuki Otsu et al. A threshold selection method from geay- histograms. Automatica, 11 (285-296)23327,, 1975.
13

---

Published as a conference paper at ICLR 2025
Long Ouyang, Jeffrey Wu, Xu Jiang. Diogo Almeida, Carroll Wainwright, Pamela Mishkin, Chong Zhang. Sandhini Agarwal, Katarina Slama, Alex Ray, et al. Training language models to follow instructions with human feedback. Advances in neural information processing systems, 35:27730- 27744, 2022.
Zhenyu Pan, Rongyu Cao, Yongchang Cao, Yingwei Ma, Binhua Li, Fei Huang, Han Liu, and Yongbin Li. Codev-bench: How do Ilms understand developer-centric code completion? arXiv preprint arXiv:2410.01355 2024a.
Zhenyu Pan, Haozheng Luo, Manling Li, and Han Liu. Conv-coa: Improving open-domain ques- tion answering in large language models via conversational ciiin-of-ctionn arXiv preprint arXiv:2405.17822, 2024b.
Zhenyu Pan, Xuefeng Song, Yunkun Wang, Rongyu Cao, Binhua Li, Yongbin Li, and Han Liu. Do code llms understand design patterns? arXiv preprint arXiv:2501.0453 2025.
Jinseok Park, Donghyeon Cho, Wonhyuk Ahn, and Heung-Kyu Lee. Double jpeg detection in mixed jpeg quality factors using deep convolutional neural network. In Proceedings of the European conference on computer vision (ECCV), pp. 636-652, 2018.
Baolin Peng, Chunyuan Li, Pengcheng He, Michel Galley, and Jianfeng Gao. Instruction tuning with gpt-4. arXiv preprint arXiv:2304.03277, 2023.
Xiangfei Qiu, Jilin Hu, Lekui Zhou, Xingjian Wu, Junyang Du, Buang Zhang, Chenjuan Guo, Aoying Zhou, Christian S. Jensen, Zhenli Sheng, and Bin Yang. Tfb: Towards comprehensive and fair benchmarking of time series forecasting methods. Proc. VLDB Endow. 170):2363-2377, 2024.
Xiangfei Qiu, Xiuwen Li, Ruiyang Pang, Zhicheng Pan, Xingjian Wu, Liu Yang. Jilin Hu, Yang Shu, Xuesong Lu, Chengcheng Yang. Chenjuan Guo, Aoying Zhou, Christian S. Jensen, and Bin Yang. Easytime: Time series forecasting made easy. In ICDE, 2025a.
Xiangfei Qiu, Xingjian Wu, Yan Lin, Chenjuan Guo, Jilin Hu, and Bin Yang. Duet: Dual clustering enhanced multivariate time series forecasting. In SIGKDD, 2025b.
Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, llya Sutskever, et al. Language models are unsupervised multitask learners. OpenAl blog, 1(8):9, 2019.
Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervision. In International conference on machine learning. PMLR, 2021.
Yuan Rao and Jiangqun Ni. Self-supervised domain adaptation for forgery localization of jpeg compressed images. In Proceedings of the IEEF/CVF international conference on computer vision, pp. 15034-15043 2021.
Nikhila Ravi, Valentin Gabeur, Yuan-Ting Hu, Ronghang Hu, Chaitanya Ryali, Tengyu Ma, Haitham Khedr, Roman Rädle, Chloe Rolland, Laura Gustafson, et al. Sam 2: Segment anything in images and videos. arXiv preprint arXiv:2408.00 00714. 2024.
Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Björn Ommer. High- resolution image synthesis with latent diffusion models. In Proceedings of the IEEE/CVF confer- ence on computer vision and pattern recognition, pp. 10684-10695, 2022.
Olga Russakovsky, Jia Deng, Hao Su, Jonathan Krause, Sanjeev Satheesh, Sean Ma, Zhiheng Huang, Andrej Karpathy, Aditya Khosla, Michael Bernstein, al. Imagenet large scale visual recognition challenge. International journal of computer vision, 115:211-252, 2015.
Ronald Salloum, Yuzhuo Ren, and C-C Kuo. Image splicing localization using a multi-task fully convolutional network (mfen). Journal of Visual Communication and Image Representation, 51: 201-209, 2018
Tal Shaharabany, Aviad Dahan, Raja Giryes, and Lior Wolf. Autosam: Adapting sam to medical images by overloading the prompt encoder. arXiv preprint arXiv:2306.066 06370, 2023.
14

---

Published as a conference paper at ICLR 2025
Ali Sharif Razavian, Hossein Azizpour, Josephine Sullivan, and Stefan Carlsson. Cnn features off-the-shelf: an astounding baseline for recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition workshops, pp. 806- 813, 2014.
Karen Simonyan and Andrew Zisserman. Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1555 2014.
Xuezhi Wang, Jason Wei, Dale Schuurmans, Quoe Le, Ed Chi, Sharan Narang, Aakanksha Chowdh- cry, and Denny Zhou. Self-consistency improves chain of thought reasoning in language models. arXiv preprint ar arXhv:2203.111770 2022a.
Zifeng Wang, Zizhao Zhang, Chen-Ye Yu Lee, Han Zhang, Ruoxi Sun, Xiaoqi Ren, Guolong Su, Vincent Perot, Jennifer Dy, and Tomas Plister. Learning to prompt for continual learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 139-149, 2022b.
Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Fei Xia, Ed Chi, Quoc Le, Denny Zhou, et al. Chain-of-thought prompting elicits reasoning in large language models. Advances in neural information processing systems, 35:24824 24428224-2437,, 2022.
Yongxian Wei, Zixuan Hu, Li Shen, Zhenyi Wang, Yu Li, Chun Yuan, and Dacheng Tao. Task groupings regularization: Data-free meta-learning with heterogeneous pre-trained models. arXiv preprint arXiv:2405.16560 2024a.
Yongxian Wei, Zixuan Hu, Zhenyi Wang, Li Shen, Chun Yuan, and Dacheng Tao. Free: Faster and better data-free meta-learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 23273-23233 2024b.
Yongxian Wei, Anke Tang, Li Shen, Feng Xiong, Chun Yuan, and Xiaochun Cao. Modeling multi-task model merging as adaptive projective gradient descent. arXiv preprint arXiv:2501.012500 2025.
Bihan Wen, Ye Zhu, Ramanathan Subramanian, Tian-Tsong Ng. Xuanjing Shen, and Stefan Win- kler. Coverage a novel database for copy-move forgery detection. In 20/6 IEEE international conference on image processing (ICIP), pp. 161-165. IEEE, 2016.
Haiwei Wu, Jiantao Zhou, Jinyu Tian, Jun Liu, and Yu Qiao. Robust image forgery detection against transmission over online social networks. IEEE Transactions on Information Forensics and Security, 17:443-456, 2022.
Junde Wu, Wei Ji, Yuanpei Liu, Huazhu Fu, Min Xu, Yanwu Xu, and Yueming Jin. Medical sam adapter: Adapting segment anything model for medical image segmentation. arXiv preprint arXiv:2304. /2620, 2023.
Xingjian Wu, Xiangfei Qiu, Zhengyu Li, Yihang Wang, Jilin Hu, Chenjuan Guo, Hui Xiong, and Bin Yang. Catch: Channel-aware multivariate time series anomaly detection via frequency patching. In ICLR, 2025.
Xinle Wu, Xingjian Wu, Bin Yang. Lekui Zhou, Chenjuan Guo, Xiangfei Qiu, Jilin Hu, Zhenli Sheng, and Christian S. Jensen. Autocis++: zero-shot joint neural architecture and hyperparameter search for correlated time series forecasting. VLDB J. 33(5): 2024.
Yue Wu, Wael AbdAlmageed, and Premkumar Natarajan. Mantra-net: Manipulation tracing network for detection and localization of image forgeries with anomalous features. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 9543-9552, 2019.
Enze Xie. Wenhai Wang, Zhiding Yu, Anima Anandkumar, Jose M Alvarez, and Ping Luo. Segformer: Simple and efficient design for semantic segmentation with transformers. Advances in neural information processing systems, 34:1 1477- 2021.
Chao Yang, Huizhou Li, Fangting Lin, Bin Jiang, and Hao Zhao. Constrained r-cnm: A general image manipulation detection model. In 2020 IEEE International conference on multimedia and expo (ICME), pp. 1-6. IEER, 2020.
15

---

Published as a conference paper at ICLR 2025
Chao Yang. Zhiyu Wang, Huawei Shen, Huizhou Li, and Bin Jiang. Multi modai-m image manipula- tion detection. In 2021 IEEE International conference on multimedia and expo (ICME), pp. 1-6. IEEE, 2021.
Yuanhao Zhai, Tianyu Luan, David Doermann, and Junsong Yuan. Towards generic image manipula- tion detection with weakly-supervised self-consistency learning. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 12390-224000 2023.
Kaidong Zhang and Dong Liu. Customized segment anything model for medical image segmentation. arXiv preprint arXiv:2304. 13785, 2023.
Quan Zhang and Yuxin Qi. Can milms guide seakky-supervised temporal action localization tasks? arXiv preprint 2xXv:2411.08866 2024.
Quan Zhang, Xiaoyu Liu, Wei Li. Hanting Chen, Junchao Liu, Jie Hu, Zhiwei Xiong, Chun Yuan, and Yunhe Wang. Distilling semantic priors from sam to efficient image restoration models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 35409-25411 2024.
Quan Zhang, Yuxin Qi, Xi Tang, Jinwei Fang, Xi Lin, Ke Zhang, and Chun Yuan. IMDPrompter: Adapting SAM to image manipulation detection by cross-view automated prompt learning. In The Thirteenth International Conference on Learning Representations, 2025a. URL https:
// //pperevieww... net/
Quan Zhang, Yuxin Qi, Xi Tang, Rui Yuan, Xi Lin, Ke Zhang, and Chun Yuan. Rethinking pseudo- label guided learning for weakly supervised temporal action localization from the perspective of noise correction. arXiv preprint arXiv:2501. 11124, 2025b.
Zhuosheng Zhang, Aston Zhang, Mu Li, and Alex Smola. Automatic chain of thought prompting in large language models. arXiv preprint arXiv:2210.0345: 2022.
Hengshuang Zhao, Jianping Shi, Xiaojuan Qi, Xiaogang Wang, and Jiaya Jia. Pyramid scene parsing network. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp.
2881-2890, 2017.
Sixiao Zheng, Jiachen Lu, Hengshuang Zhao, Xiatian Zhu, Zekun Luo, Yabiao Wang, Yanwei Fu, Jianfeng Feng, Tao Xiang, Philip HS Torr, et al. Rethinking semantic segmentation from -equence-to-sequence perspective with transformers. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 18810 2021.
Kaiyang Zhou, Jingkang Yang, Chen Change Loy, and Ziwei Liu. Learning to prompt for vision- language models. International Journal of Computer Vision, 130(9):2337-233 2022.
Peng Zhou, B Chen, Xintong Han, Mahyar Najibi, and Larry Davis. Generate, segment and replace: Towards generic manipulation segmentation. arXiv preprint 1rXiv:181 2018a.
Peng Zhou, Xintong Han, Vlad I Morariu, and Larry S Davis. Learning rich features for image manipulation detection. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 1053-1061 1061, 2018b.
Peng Zhou, Bor-Chun Chen, Xintong Han, Mahyar Najibi, Abhinav Shrivastava, Ser-Nam Lim, and Larry Davis. Generate, segment, and refine: Towards generic manipulation segmentation. In Proceedings of the AAAI conference on artificial intelligence, volume 34, pp. 13058-13065. 2020.
Xueyan Zou, Jianwei Yang, Hao Zhang, Feng Li, Linjie Li, Jianfeng Wang, Lijuan Wang, Jianfeng Gao, and Yong Jae Lee. Segment everything everywhere all once. Advances in Neural Information Processing Systems, 36, 2024.
16

---

Published as a conference paper at ICLR 2025
7 APPENDIX
7.1 LIMITATIONS AND BROADER IMPACTS
Limitations. We have some limitations our method. First, it cannot detect completely generated images. Second, training IMDrrompter requires complete pixel-level supervision. In future research, we plan to explore more efficient label utilization through weakly supervised and semi-supervised learning approaches and extend these methods to newer foundational models such as SEEM Zou et al. (2024) and SAM2 Ravi al. (2024).
Broader Impacts. In recent years, with the advancement of deep learning technologies Zhang et al. (2024; 2025b); Qiu et al. (2024; 2025b;a); Wu et al. (2025; 2024); Li et al. (2024); Hu et al. (2024); Wei et al. (2024h;a; 2025); Zhang & Qi (2024); Dong et al. (2023a;c:b), the editing and manipulation of digital media have become increasingly accessible and widespread. Advances in image editing software, along with deep generative models such as Generative Adversarial Networks and diffusion models Gao et al. (2024); Chen et al. (2024), have made image manipulations. often imperceptible the human eye, much easier. These technologies are even being widely employed by potential malicious users. The ubiquitous use of smartphones and social networks has further accelerated the dissemination of these manipulated media. As result, when these edited images are employed to support disinformation or to distort news content in order to mislead the public, they can create significant social problems and contribute crisis of trust. this context, IMDPrompter, as an effective image manipulation detection method, can play crucial role in mitigating the negative consequences of intentional image manipulation.
7.2 DATASET DESCRIPTION
In order to directly compare with state-of-the-ar technologies, we trained on CASIAv2 Dong et al. (2013) and conducted extensive testing on COVER Wen et al. (2016), Columbia Hsu & Chang (2006), NIST16 Hsu & Chang (2006), CASIAv1 Dong et (2013), and the recent IMD Novozamsky et al. (2020).
Table 7: Details of the training set and five test sets used in our experiments. The symbol indicates unavailable information. Copy-move, splicing, and inpainting operations are denoted as cpmv, spli, and inpa, respectively. Our model was trained on the CASIAv2 dataset and evaluated across all test sets.
Dataset
Negative
Positive
cpmv
spli
inpa
Training
CASIAv2 Dong et al. (2013)
7491
5063
3235
1828
0
Testing
COVER Wen et al. (2016)
100
100
100
0
Columbia Hsu & Chang (2006)
183
180
0
180
NISTI6 Dong et al. (2013)
0
564
68
288
208
CASIAv Dong et al. (2013)
800
920
459
461
IMD Novozamsky et al. (2020)
414
2010
7.3 BASELINE METHODS
As shown in Table 8, we have organized the technical routes of the baseline methods. To ensure fair and reproducible comparisons, we selected state-of-the-art models that meet any of the following criteria: 1) pretrained models released by the paper authors, 2) publicly available source code, and 3) adherence to common evaluation protocol, where CASIAv2 is used for training and other public datasets for testing. Based on these criteria, we compiled list of nine published baselines, as follows:
H-LSTM Bappy et al. (2019): Pretrained on a custom dataset of 65k processed images and fine-tuned NIST16 and IEBE Forensics Challenge data.
17

---

Published as a conference paper at ICLR 2025
Table 8: Comparison of Technical Routes of Baseline Methods
Method
RGB
Noise
View
Fuxion
Semantic Segrentatton
Backbune
MFCNSalloom et al. (2018)
FCN
RGB-NZhou (2018b)
SRM
Late Fusion(Bilinca) Pooling)
Faster R-CNN
H-LSIMBBappy et al. (2019)
Patcb-LSTM
ManTra-NetWu al. (2019)
SRM-Bayar
Early Fusion(Feature Concatenation)
Wider VGG
HP-CCLI & Huang (2019)
High-pass diteers
PCN
GSR-NeiZhou et (2018a)
DeepLabv2
CR-CNNYang al. (2020)
Bayar
Mask R-CNN
SPANHu al. (2020)
SRM-Bayar
Early Fenture Concatenation)
Wider VGG
MMMNNcYang et al. (2021)
Bay
Middle Fusionl Attention Guidance)
Mask R-CNN
JPEG-ComNetRae
(2021)
SRM
Early Fusion(Feature Concatenation)
Siamese FCN
CAI-NetKwon et (2021)
DCT
Middle Fusion(Frcature Concatenation)
HRNet
MVSS-NeCChen et (2021)
Bayar
Fusion[Dual Atteation)
FCN
Truber Guillaro et al. (2023)
Noiseprint
Late FusiontFraature Concatenation)
Segformer
Fusioni(Qpiina Prompt Selection
IMDPrompter
SRM+Bayyr+Noieprinn
+Cross-view Prompt Consistency)
PCN+SAM
ManTra-Net Wu et al. (2019): Trained on millions of processed images in a private collec-
tion.
HP-FCN Li & Huang (2019): Trained on private set of repaired images.
CR-CNN Yang et al. (2020): Trained on CASIAv2.
SPAN Hu et al. (2020): Trained using the same data as ManTra-Net and fine-tuned on CASIAv2.
CAT--Nt Kwon et al. (2021): Trained on a joint dataset including CASIAv2, IMD, Fantastic Reality Kniaz et al. (2019), and self-spliced COCO.
For models with publicly available code, we trained them using the code provided by the authors, such as GSR-Net Zhou et al. (2018a). When citing their results, we adopt the original data where appropriate and only use our retrained models when necessary. For models that follow the same evaluation protocol, such as MFCN Salloum et al. (2018) and RGB-N N Zhou et al. (2018b), we directly cite results from the same team Yang et al. (2020). For fair comparison, we retrained FCN Long et al. (2015), MVSS- WeeS-NeDDong et al. (2022), and Trufor Guillaro et al. (2023) from scratch on CASIAv2. As previous studies rarely report their image-level performance, these models typically lack an image classification head in their implementations. To obtain a baseline for image-level predictions without modifying their models or code, we adopted the GMP strategy used in MVSS-Net.
7.4 SUPPLEMENTARY EXPERIMENTS
Experimental Setups. To improve reproducibility, we present the used parameters details in Table
Overall Performance in Detection and Localization. Table 10 and Figure shows the overall performance of pixel-level and image-level manipulation detection. We use the harmonic mean of image-level detection Fl and pixel-level localization F1, denoted as C-F1, as our overall performance metric. As shown in Table 3, IMDPrompter achieves the best performance in all settings, with performance improvement of 45.3% over the next best method. Notably, in the experiments on the COVER dataset, we achieved a performance gain of 110.7%.
Ablation study of the segmenters for the four prompt views: We selected several segmenters, including FCN, PSPNet. SETR, and Segformer, for ablation experiments. As shown in the table below, the lightweight FCN based on MobileNet is sufficient to effectively guide SAM's automated prompt learning. The introduction of more powerful segmenters like PSPNet, SETR, and Segformer did not result in significant improvement in detection performance but instead introduced greater computational overhead. Therefore, we chose the lightweight FCN as our segmenter.
Computational Complexity Analysis: As shown in Table 1.5, I-AUC represents Image-level AUC, I-FI represents Image-level F1, P-F1 represents Pixel-level F1, and C-F1 represents the harmonic mean of I-F1 and P-F1. It is worth noting that the IMDPrompter mentioned in the main text is based on SAM-H, which has a high computational complexity. Therefore, we implemented a lightweight version, IMDPrompter* based Mobile SAM. As can be seen from Table 15, IMDPrompter
18

---

Published as a conference paper at ICLR 2025
Table 9: Parameter Settings Details
Parameter
Value
Warm-up Learning Rate Scheduler
Type
LineurLR
Start Factor
1.00F-04
By Epoch
TRUE
Begin
0
End
Convert Iter Based
TRUE
Main Learning Rate Scheduler
Type
CosineAmeallngL.R
T.max
max_epochs
By Epuch
TRUE 1
Begin
End
max epochs
Data Preprocessor
Mean
[123.675, 116.28, 103.53]
Standard Deviation
[58.395, 57.12, 57.125
BGR to RGB
TRUE
Padding Value
0
Segmentation Padding Value
255
Size
-10,241,024
Trainer
Number Things Classes
Train Batch Size per GPU
1
Train Number of Workers 4
Test Batch Size per GPU
1
Test Number of Workers 4
Epochs
100
Table 10: Overall Detection and Localization Performance (Com-F1 Score). The best results for each test are highlighted in bold and second-best values underlined.
Method
CASIA
COVER
Columbia
IMD
MEAN
H-LSTM Bappy et al. (2019)
0.0
3.8
0.0
0.94
MunTra-Net Wu et al. (2019)
0.0
0.0
0.0
0.0
0.00
CR-CNN Yang et al. (2020)
30.3
18.1
41.3
23.7
28.35
GSR-Net Zhou
(2018a)
0.0
4.2
0.0
1.06
SPAN Hu et (2020)
0.0
0.0
0.0
0.0
0.00
CAT-Net Kwon et al. (2021)
20.0
19.7
69.1
9.5
29.59
NVSS-Net Chen et (2021)
56.6
31.7
71.1
30.0
47.36
FCN Long et al. (2015)
53.6
30.5
23.3
31.57
76.8
66.8
90.3
41.3
68.82
IMDPrompter
+23.2
+47.9
+59.8
+18.0
+37.25
achieved the best performance on almost all metrics, and IMDPrompter* achieved performance second only to IMDPrompter in nearly all metrics.
As shown Table 15, We supplement the performance of four SAM-based image segmentation methods: MedSAM, MedSAMMAdaptes AutoSAM, and SAMed. As we can see, these four SAseb methods exhibit limited performance in the iimage manipulation detection task due to their reliance solely semantically relevant information. In particular, there a significant performance gap between them and IMDPrompter, especially on the out-of-domain test sets CVOER, Columbia, and IMD.
Comparison with SAM-based Methods:
19

---

Published as a conference paper at ICLR 2025
Counb
uo
Go Comma
CASIA
DARA
(a) I-F1
(b) I-AUC
(c) C-FI
Columbi
Columbia
CR-CNN
CAT-Net
MVSS-Net
FCN
IMDPrompter
(d) Best threshold P-F1
(c) Fixed threshold P-F1
(f)
Figure 6: Visualization of Metric Data Comparison between IMDPrompter and State-of-the-Arn (SOTA) Methods
Table 11: Impact of JPEG Compression and Gaussian Blur on Pixel-level Fl
JPEG Compression
Gaussian Blurs
Method
100
70
60
50
0
H-LSTM Bappy et al. (2019)
15.4
15.0
14.8
14.7
14.0
13.9
15.4
13.4
12.1
10.8
10.6
10.3
Man MrnTra-Ner Wu al. (2019)
15.5
11.9
8.0
8.7
8.6
15.5
12.0
11.3
12.0
11.6
11.2
CR-CNN Yang et al. (2020)
40.4
33.6
22.4
17.1
11.7
9.2
40.4
23.3
22.1
21.1
20.7
19.9
GSR-Net Zhou et al. (2018a)
37.4
35.9
32.5
32.4
30.8
27.0
37.4
30.5
26.5
20.8
19.0
SPAN Hu et al. (2020)
18.2
16.6
15.7
14.9
18.2
10.7
10.6
10.0
CAT-Net Kwon
(2021)
13.5
0.9
0.9
13.5
9.4
MVSS-Net Chen et al. (2021)
40.2
39.5
45.1
37.4
36.0
IMDPrompter
76.3
73.6
72.0
71.6
70.7
70.5
76.3
71.8
71.1
68.8
56.6
51.6
Figure 7: Hyperparametric analysis of \1, X2, and Ag
Proportion of Four Views Selected as Optimal Prompts. As shown in Table 17, we summarize the proportion of each prompt view being selected the optimal prompt during training. We observe that the RGB view is selected as the optimal prompt in only 13.9% cases, indicating that semantically relevant features alone are insufficient to guide SAM's prompt learning. On the other hand, the Noiseprint view is chosen as the optimal prompt in 46.3% cases, demonstrating that the semantically irrlevant information provided by Noiseprint is the most erucial.
20

---

Published as conference paper at ICLR 2025
Table 12: Impact of JPEG Compression and Gaussian Blur on Image-level F!
JPEG Compression
Gaussian Blur
Method
100
90
80
70
60
50
0
5
11
17
23
29
H-LSTM Bappy al. (2019)
0.0
0.0
0.0
0.0
0.0
0.0
0.0
0.0
0.0
0.0
0.0
ManTraa-Net Wu al. (2019)
0.0
0.0
0.0
0.0
0.0
0.0
0.0
0.0
0.0
0.0
0.0
0.0
CR-CNN Yang et al. (2020)
24.3
25.8
24.0
22.4
20.0
18.4
24.3
14.3
13.9
13.7
12.4
11.4
GSR-Net Zhou et al. (a)
0.0
0.0
0.0
0.0
0.0
0.0
0.0
0.0
0.0
0.0
0.0
0.0
SPAN Hu et al. (2020)
0.0
0.0
0.0
0.0
0.0
0.0
0.0
0.0
0.0
0.0
0.0
0.0
CAT-Net Kwon et al. (2021)
38.0
17.9
17.2
15.0
12.4
11.0
38.0
11.9
11.0
10.5
10.3
10.7
MVSS-Net Chen et al. (2021)
75.9
76.2
75.7
75.3
74.8
73.0
75.9
73.8
72.8
70.7
65.8
63.4
IMDPrompter
77.3
77.8
77.2
76.8
76.1
74.6
77.3
75.9
74.6
73.1
69.3
66.6
Table 13: Impact of JPEG Compression and Gaussian Blur on Com-Fl
JPEG Compression
Gaussian Blur
Method
100
90
80
70
60
50
0
5
11
17
23
29
H-LSTM Bappy et al. (2019)
0.0
0.0
0.0
0.0
0.0
0.0
0.0
0.0
0.0
0.0
0.0
0.0
Tra-ra-Ner Wu et al. (2019)
0.0
0.0
0.0
0.0
0.0
0.0
0.0
0.0
0.0
0.0
0.0
0.0
CR-CNN Yang ct al. (2020)
30.3
29.2
23.2
19.4
14.8
12.3
30.3
17.7
17.0
16.6
15.5
14.5
GSR-Net Zhou et al. (2018a)
0.0
0.0
0.0
0.0
0.0
0.0
0.0
0.0
0.0
0.0
0.0
0.0
SPAN Hu et al. (2020)
0.0
0.0
0.0
0.0
0.0
0.0
0.0
0.0
0.0
0.0
0.0
0.0
CAT-Net Kwon al. (2021)
19.9
3.9
3.6
2.3
1.7
1.7
19.9
10.5
10.1
8.3
7.7
7.7
MVSS-Net Chen et al. (2021)
56.6
54.4
53.2
52.3
51.2
56.6
49.7
48.2
45.0
32.0
27.2
IMDPrompter
74.1
73.3
72.5
76.8
73.8
72.8
70.9
Table 14: Ablation Study the Segmenter
Paramoh
GILON
COVER
Calumbia
DMD
I-AIIC
1-F1
P-F1
C-F1
T-AUC:
1-F1
1-F1
C-FT
1-AUCC
1-91
P-F1
C-PT
TALE
B-F1
P-F1
C-F1
PSPNe1Zhan
41.4
1591.
2131.8
0.801
SegfarmerKie (2821)
5963
1769
0.983
76.5
03111
31.2
65.1
0991
86.7
91.7
0.644
410
FCNLong (205))
347.5
1533.2
0.978
77.4
7644
76.9
0.755
70.2
63.7
66.8
0.944
93.6
87.6
90.5
0.070
63.8
30,6
41.4
Table 15: Computational Complexity Overhead
ParamoM
GFLOPs
CASIA
CONTR
Calamiia
IMD
Man NetWar (2019)
4.0
1009.7
0,500
0.6
15.5
0.0
0.500
0.0
29.5
0.0
0.201
0.0
36.4
0.0
0.500
0.0
15.7
0.0
SPANThu (3128)
15.4
989
11400
0.6
18.4
0.0
0.508
00
12.2
0.0
0:400
0.0
48.7
0.0
08900
0.0
17.0
0.0
MYSS--NiCChen (2021)
1880
96.6
45.3
0.655
36.0
Gulllae (2023)
INDDProspeer
1538.2
76.8
6.7%
IMDiPrmmpiae*
55.
191.3
0.96
16.1
70.3
75.1
0.709
54.5
57.7
60.9
0.980
83.1
81.6
$4.6
0.661
56.9
15.9
36.5
Table 16: Comparison with SAM-based methods.
L-AUCC
L-AUCC
C-F1
MedSAMMla (2023)
0.563
0.791
24.6
0.513
26.1
22.4
24.1
MedSAM- AM-AMMAAdea
0.877
71.0
0.543
20.3
20.9
0.542
22.1
30.0
0.594
26.1
21.6
23.2
AutoSAMSShhhraaamm (2023)
0.845
0.513
18.2
0.772
28.4
36.0
0.523
21.9
22.9
25.2
SAMeZZhung & (2023)
0.845
69.9
0.571
27.6
24.6
25.0
0.301
53.1
31.3
39.7
0.502
23.1
25.2
IMDirroppee
0.7%
0.983
90.3
30.6
41.3
IMDPrompter*
0.980
$4.8
0.667
26,9
36.5
Table 17: The proportion of each view selected as the optimal view
Prompt View
RGB
SRM
Bayer
Noiseprint
Percent
13.9%
18.4%
21.4%
46.3%
Hyperparametric Analysis As shown in Figure 7, we conducted hyperparameter analysis on A1,
and Xg. ultimately selecting the optimal parameter configuration: X1 1.0, X2 0.1, 1.0.
21

---

Published as a conference paper at ICLR 2025
Table 18: Performance of Generative Image Manipulation Detection
CocoGlide
Method
P-FI(best)
P-F1(fixed)
I-AUC
I-Acc
Trn et (2019)
0.673
0.516
0.778
0.500
SPANHu et al. (2020)
0.35
0.298
0.475
0.491
MVSS-NetChen et al. (2021)
0.642
0.486
0.654
0.536
TruforGuillaro et al. (2023)
0.72
0.523
0.752
0.639
IMDPrompter
0.746
0.539
0.781
0.652
Performance of Generative Image Manipulation Detection: Following the work of Trufor, we supplement the performance of IMDPrompter on CocoGlide (image tampering based on diffusion models). As shown in the table below, our IMDPrompter also achieves objective results for AIGC- edited images.
Impact of Quality Degradation. As shown in Table 11,12,13, following Dong et al. (2022), we evaluated the robustness the models under two common image processing operations encountered during the dissemination of images on the internet, namely JPEG compression and Gaussian blur. Comparing these two operations, Gaussian blur has more significant impact on detection perfor- mance, especially when using larger 17x17 scale convolution kernels. Compared to previous methods, our IMDPrompter exhibits better robustness.
More Visualization. As shown in Figure 10, we have included additional visualizations.
22

---

Published as a conference paper at ICLR 2025
Image
GT
ManTra-Net
MVSS-Net
Trufor
IMDPrompter
Figure 8: Some qualitative results, compared with the state-of-the-art
23

---

Published as a conference paper at ICLR 2025
Image
GT
ManTra-Net
MVSS-Net
Trufor
IMDPrompter
Figure 9: Some qualitative results, compared with the state-of-the-art
24

---

Published as a conference paper at ICLR 2025
Image
GT
ManTra-Net
MVSS-Net
Trufor
IMDPrompter
Figure 10: Some qualitative results, compared with the state-of-the-art
25


Reviews:
[{"review_id": "5jyHGJ1tb8", "reviewer": "Reviewer_z9xA", "summary": "The authors propose an automated prompt learning SAM-based method for image manipulation detection. Multiple views, such as RGB, SRM, Bayar, and Noiseprint, are utilized and integrated to generate auxiliary masks and bounding boxes for SAM. Meanwhile, many modules, such as Cross-view Feature Perception and Prompt Mixing modules, are proposed for mixing features for the Mask Decoder. Extensive results demonstrate the effectiveness of the proposed method.", "questions": "Please see the weakness.", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. The authors utilize SAM's zero-shot capabilities for image manipulation detection.\n2. Multiple views, such as RGB, SRM, Bayar, and Noiseprint, are utilized and integrated to generate auxiliary masks and bounding boxes for SAM\n3. Many modules, such as Cross-view Feature Perception and Prompt Mixing modules, are proposed for mixing features for the Mask Decoder. \n4. Extensive results demonstrate the effectiveness of the proposed method.", "weaknesses": "1. Can the authors clarify why SAM is employed in image manipulation detection? The authors utilize four different views to generate masks and then bounding boxes, which serve as prompts for the Mask Decoder. To the best of my knowledge, the output masks of the Mask Decoder are significantly dependent on prompt accuracy. It is essential that the generated masks are sufficiently accurate. Can the authors provide the accuracy metrics for both the generated masks and the output masks from the Mask Decoder? Additionally, is it necessary to employ SAM?  \n2. Are the Mask Decoder and Prompt Encoder kept frozen during the training process?\n3. In the PMM module, the feature F_{SAF}, which is encoded from images, is resized to the same shape as the output of the Prompt Encoder, F_{opt}, which is encoded based on coordinates. Could the authors elaborate on the motivation for this approach? The fusions of image embeddings and coordinate embeddings appears inconsistent.  \n4. The authors claim that the proposed OPS and CPC enhance alignment across views. Ideally, they utilize the CPC loss function to achieve prompt consistency. However, However, this claim lacks convincing evidence. Can the authors provide details on how the two proposed modules contribute to improved alignment?\n5. The proposed method is trained only on the CASIAv2 dataset, while several other studies, such as CAT-Net v2 [1], TruFor [2], and UnionFormer [3], utilize additional datasets for training. In Table 1 and Table 2, the metrics are based on the CASIAv2 dataset without additional datasets. Can you explain why the method is only trained on the CASIAv2 dataset?\n6. Table 2 presents numerous NaN values for Sensitivity, Specificity, and F1 scores related to TruFor. Can the authors provide the corresponding metrics?\n7. Can you provide more recent methods for comparison? such as UnionFormer [3].\n\n[1] Kwon, Myung-Joon, et al. \"Learning jpeg compression artifacts for image manipulation detection and localization.\" International Journal of Computer Vision 130.8 (2022): 1875-1895.\n\n[2] Guillaro, Fabrizio, et al. \"Trufor: Leveraging all-round clues for trustworthy image forgery detection and localization.\" Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 2023.\n\n[3] Li, Shuaibo, et al. \"UnionFormer: Unified-Learning Transformer with Multi-View Representation for Image Manipulation Detection and Localization.\" Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2024.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "DeWCY6uTsg", "reviewer": "Reviewer_t8Py", "summary": "The paper addresses the application of the SAM to the domain of image manipulation detection. The paper proposes IMDPrompter, a cross-view automated prompt learning paradigm that extends SAM's capabilities for IMD tasks. The proposed method is evaluated on five datasets (CASIA, Columbia, COVER, IMD2020, and NIST16), demonstrating significant improvements over existing state-of-the-art methods in both in-distribution and out-of-distribution settings.", "questions": "Please see the weakness part 2, 3, 4, 5, 6, 7.", "limitations": "", "rating": 6, "confidence": 3, "soundness": 2, "presentation": 2, "contribution": 2, "strengths": "The paper is well-motivated. The authors use SAM to address the challenges in the image manipulation detection domain. The introduction of automated prompt learning and integration of semantic-agnostic features is innovative and extends the applicability of SAM to a new domain.\n\nThe design is straightforward and reasonable. Apart from RGB images, using multiple views provides more information for IMD.\n\nFive datasets and ablation studies demonstrate the effectiveness of the proposed IMDPrompter.", "weaknesses": "1. The proposed method introduces several additional modules, encoders, and views, which increase the complexity of the model.\n\n2. Although the combination of four additional modules increases the performance, the authors do not provide the mechanism or reason behind why they chose views of SRM, Bayar, and Noiseprint, and which type of image processing information would help the IMDPrompter generate a more precise mask.\n\n3. There is no CFP in Figure 2, but it is described in the caption. \n\n4. There are no details of the architecture of the SRM/RGB/Bayar/Noiseprint encoder and its computational cost. \n\n5. While the paper acknowledges that relying solely on RGB information is insufficient for cross-dataset generalization, there is limited discussion on scenarios where the proposed semantic-agnostic views might also fail, such as advanced manipulation techniques that bypass noise-based detectors.\n\n6. The number of abbreviations is too many, which may interrupt the experience and hinder understanding for readers not familiar with the notation.\n\n7. It would be better to bold the best results in Table 3,4,5,6.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "LOF5ClQbL4", "reviewer": "Reviewer_tfmM", "summary": "This work introduces IMDPrompter, a novel cross-view prompt learning framework based on the Segment Anything Model (SAM) to automate detection and localization in image manipulation tasks, overcoming SAM’s reliance on manual prompts and enhancing cross-dataset generalization through cross-view perceptual learning techniques.", "questions": "Real-World Deployment: Has IMDPrompter been tested in real-world settings, where variations in lighting, compression, and manipulation styles may further challenge the model’s robustness?", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. Innovative Application: The paper applies SAM  to the underexplored area of image manipulation detection, which extends SAM's use case beyond traditional segmentation tasks. The proposal of a cross-view automated prompt learning paradigm with IMDPrompter is unique and addresses the challenges specific to manipulation detection tasks.\n\n2. Automation of Prompt Generation: One of the standout innovations is the elimination of SAM's reliance on manual prompts. The introduction of modules like Optimal Prompt Selection (OPS) and Cross-View Prompt Consistency (CPC) strengthens SAM’s utility by automating the prompt generation, potentially making SAM more accessible for manipulation detection.\n\n3. Robustness and Generalizability: IMDPrompter's multi-view approach—integrating RGB, SRM, Bayer, and Noiseprint—demonstrates enhanced generalization, particularly on out-of-domain datasets. The ablation studies further substantiate the contributions of each module, which supports the validity of the multi-view and prompt-learning design.\n\n4. Strong Experimental Validation: The model shows significant improvements in image-level and pixel-level metrics across multiple datasets (CASIA, Columbia, IMD2020, etc.), indicating its robustness. The experimental setup includes various metrics (AUC, F1-scores), highlighting the model’s strengths compared to prior approaches.", "weaknesses": "1. Complexity and Computational Cost: IMDPrompter’s architecture includes multiple modules (OPS, CPC, CFP, PMM), each introducing additional computational overhead. The increased complexity may impact the model’s efficiency, potentially limiting real-world deployment, especially for large-scale or time-sensitive tasks. Maybe we can provide computational complexity analysis or runtime comparisons with existing methods.\n\n2. Limited Modality Discussion: While IMDPrompter is tested on a range of datasets for image manipulation, it would be beneficial if the authors discussed the potential application of this approach to other domains or modalities, such as video manipulation, to establish broader applicability. You can discuss particular challenges or modifications that would be needed to apply IMDPrompter to video manipulation detection.\n\n3. The current framework provides limited insight into how each view or prompt contributes to the final decision-making process. Adding visualizations or more detailed explanations on how SAM’s interpretability might translate into manipulation detection could improve the work’s practical value.  For example, you can add the ablation studies showing the impact of each view, or visualizations of the learned prompts for different types of manipulations.\n\n4. Reliance on Specific Views: The model’s reliance on SRM, Bayer, and Noiseprint views may limit its utility across other manipulation detection types that do not exhibit these specific signal properties. Further exploration into the model's adaptability to new types of data without these views might be necessary. I think you can discuss or demonstrate how IMDPrompter might be adapted to work with different types of views or features. Maybe you can also add an experiment with a subset of the current views to assess the model's flexibility.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "IJs9TolH6l", "reviewer": "Reviewer_tXob", "summary": "The paper introduces IMDPrompter, a novel approach for image manipulation detection (IMD) that leverages the Segment Anything Model (SAM). It addresses the challenges of manual prompt reliance and cross-dataset generalization by proposing a cross-view automated prompt learning paradigm. This includes components like Cross-view Feature Perception, Optimal Prompt Selection, and Cross-View Prompt Consistency, which enhance SAM's ability to generate accurate masks for detection and localization without manual guidance. The method is validated across five datasets, demonstrating its effectiveness in both in-distribution and out-of-distribution image manipulation detection and localization.", "questions": "1. Can you provide more detailed technical information about the consistency loss formulation in CPC? It would be helpful to see visual examples of how CPC affects the prompt generation process and its impact on the final detection results.\n2. Why were certain recent methods excluded from the comparison? Have you considered comparing with methods published in 2024 or other SAM-based approaches?\n3. How does IMDPrompter handle images from different domains in practical applications? What mechanisms could be added to make the model more adaptive to distribution shifts?\n4. Can you provide comprehensive ablation studies showing the contribution of each view and how different view combinations affect the model's performance? This should include an analysis of the computational overhead associated with each additional view.\n5. Could you explain the rationale behind choosing SRM, Bayer, and Noiseprint specifically as noise perspectives? Have experiments been conducted with other noise perspectives, and if so, what were the results?", "limitations": "", "rating": 5, "confidence": 5, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "The primary strength of this paper lies in its application of the Segment Anything Model (SAM) to the field of image manipulation detection (IMD), introducing a cross-view automatic prompt learning paradigm. This paradigm significantly enhances the automation and generalization across datasets in IMD tasks through components such as automated prompt generation, optimal prompt selection, and cross-view prompt consistency. Furthermore, the paper demonstrates the effectiveness of the proposed method through extensive experiments across five different datasets, testing not only the model's in-distribution performance but also its robustness in out-of-distribution scenarios. These contributions not only advance the technology of image manipulation detection but also provide a powerful new tool for the field of multimedia forensics.", "weaknesses": "1. The technical details about the Cross-View Consistency Enhancement Module (CPC) implementation are insufficient. The paper does not clearly explain how different consistency loss formulations impact the results, and there is no visualization or analysis demonstrating how cross-view consistency is maintained throughout the detection process.\n2. The comparison baselines are relatively outdated, missing important recent work from 2024 and other foundation model-based approaches. This limits the comprehensiveness of the comparative analysis and makes it difficult to assess the method's standing against the latest advancements in the field.\n3. The paper shows limited discussion of model robustness to distribution shifts and lacks experiments demonstrating how the model adapts to real-world scenarios where data distribution varies. There is no clear mechanism described for handling domain adaptation.\n4. The multi-view fusion analysis is incomplete. The paper lacks detailed ablation studies that quantify the individual contribution of each view, and there is no discussion of the computational trade-offs associated with different view combinations.\n5. The paper lacks justification for selecting SRM, Bayer, and Noiseprint as noise perspectives. There is no analysis of their specific effectiveness for different types of image manipulations, nor any comparison with other potential noise perspectives that could potentially achieve similar or better results.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)