LOKI: A Comprehensive Synthetic Data Detection Benchmark using Large Multimodal Models
Authors: Junyan Ye
Baichuan Zhou
Zilong Huang
Junan Zhang
Tianyi Bai
Hengrui Kang
Jun He
Honglin Lin
Zihao Wang
Tong Wu
Zhizheng Wu
Yiping Chen
Dahua Lin
Conghui He
Weijia Li
Venue: ICLR 2025 Spotlight (2025)
Keywords: LMMs；Deepfake；Multimodality
Primary area: datasets and benchmarks
TL;DR: A Comprehensive Synthetic Data Detection Benchmark using Large Multimodal Models

Abstract: With the rapid development of AI-generated content, the future internet may be inundated with synthetic data, making the discrimination of authentic and credible multimodal data increasingly challenging. Synthetic data detection has thus garnered widespread attention, and the performance of large multimodal models (LMMs) in this task has attracted significant interest. LMMs can provide natural language explanations for their authenticity judgments, enhancing the explainability of synthetic content detection. Simultaneously, the task of distinguishing between real and synthetic data effectively tests the perception, knowledge, and reasoning capabilities of LMMs. In response, we introduce LOKI, a novel benchmark designed to evaluate the ability of LMMs to detect synthetic data across multiple modalities. LOKI encompasses video, image, 3D, text, and audio modalities, comprising 18K carefully curated questions across 26 subcategories with clear difficulty levels. The benchmark includes coarse-grained judgment and multiple-choice questions, as well as fine-grained anomaly selection and explanation tasks, allowing for a comprehensive analysis of LMMs. We evaluated 22 open-source LMMs and 6 closed-source models on LOKI, highlighting their potential as synthetic data detectors and also revealing some limitations in the development of LMM capabilities. More information about LOKI can be found at https://opendatalab.github.io/LOKI/.

Published as a conference paper at ICLR 2025
LOKI: A COMPREHENSIVE SYNTHETIC DATA DE- TECTION BENCHMARK USING LARGE MULTIMODAL MODELS
Junyan Yel.2; Baichuaa Zhou Zilong Huang" Junan Zhang Tianyi Baiz.
Hengrui Kang², Jun He' Honglin Lin Zihao Wang', Tong Wu' Zhizheng Wu Yiping Chen' Dahua Lin Conghui He Weijia Li
Sun Yat-sen University. 2 Shanghai AI Laboratory, 3 SenseTTmme Research,
4 The Chinese University of Hong Kong." The Hong Kong University of Science and Technology.
SDS, SRIBD, The Chinese University of Hong Kong, Shenzhen
ng real on "rraded a the Queen tten f. positi
HER
BO
ual roles
KON
a wish
wit
Diverse Modalities
Heterogeneous Domain
Multi-Jevel Annotations
Synthetic Detection Evaluation Frumework
Vidra
Vileo
Orgaaez dTaMez
ta gron povrrted a
Image
3D
Documarra Saatkki
Chsie sic
Judpemestt Yee
Widppdda E Sooenticaanrr
Elitooppe
Absernsd Derall Salection
Ansodationns
modele
[x
Recal
Audio
Siudis MMuis
Extmemenntiend
<C=
che..
<0>
UILAA800
OFT
Sare
Es Nerfhaudd 12 caters
30
Figure 1: Overview of LOKI benchmark. LOKI possesses four key characteristics: 1) Diverse modalities (video, image, 3D, text and audio); 2) Heterogencous categories (26 detailed subcate- gories); 3) Multi-level annotations; Multimodal synthetic data evaluation framework.
ABSTRACT
With the rapid development of Al-generated content, the future internet may be inundated with synthetic data, making the discrimination of authentic and credi- ble multimodal data increasingly challenging. Synthetic data detection has thus garnered widespread attention, and the performance of large multimodal models (LMMs) in this task has attracted significant interest. LMMs can provide natural language explanations for their authenticity judgments, enhancing the explain- ability of synthetic content detection. Simultaneously, the task of distinguish- ing between real and synthetic data effectiivly tests the perception, knowledge, and reasoning capabilities LMMs. In response, we introduce LOKI, a novel benchmark designed to evaluate the ability of LMMs to detect synthetic data across multiple modalities. LOKI encompasses video, image, 3D, text, and audio modalities. comprising 18K carefully curated questions across 26 subcategories with clear difficulty levels. The benchmark includes coarse-grained judgment and multiple-choice questions, well fine-grained anomaly selection and expla- nation tasks, allowing for a comprehensive analysis of LMMs. We evaluated 22 open-source LMMs and 6 closed-source models on LOKI, highlighting their po- tential synthetic data detectors and also revealing some limitations in the de- velopment of LMM capabilities. More information about LOKI can be found https: /
wwt.hgitth.u..... 1 io/boiio//KK//// LOKI/
"These authors contributed equally to this work. Corresponning author(s). F-mail(s): @wwij29@@aillsyyuuu.
saconghui@pjlah.orgg

---

Published as a conference paper at ICLR 2025
INTRODUCTION
With the rapid development of diffusion models (Rombach et al. 2022 Dhariwal & Nichol b02
and large language models (Abdullah et al., 2022; Brown 2020), Al Algenerrte content (AIGC) technology has increasingly integrated synthetie multimodal data into our daily lives. For instance, tools like SORA (Brooks al.) 2024) can produce highly realistic video, while Suno (Shulman it al. 2022) enables the creation of music at a level comparable to professional artists However, synthetic multimodal data also brings significant risks, including potential misuse and societal dis- ruption (Cooke et al. 2024 Ju et al. 2022). For example, the risks include generating fake news using large language models (LLMs). synthesizing fraudulent faces with diffusion models for scams, and potential contamination of internet training data. Due to the convenience of artificial intelligence synthesis, the future Internet may be saturated with Al-generated content, making the task of dis- cerning the authenticity and trustworthiness of multimodal data increasingly challenging.
To address such threats, the field of synthetic data detection has garnered widespread attention in recent years (Barni et al. 2020; Frank et al.) 2020 Gragnaniello et al. 2021 [Shao et al., 2023 2024). However, most current synthetic data detection methods are primarily focused on authentic- ity evaluation, with certain limitations regarding the human interpretability of the prediction results et al. 2024b). The recent rapid advancement of large multimodal models (LMMs) has sparked curiosity about their performance in detecting synthetic multimodal data (Ku et al., 2023 Wu al.) 2024b). On one hand, for synthetic data detection tasks, LMMs can provide reasoning behind au- thenticity judgments in natural language, paving the way for enhanced explainability. On the other hand, the task of distinguishing between real and synthetic data involves the perception, knowledge, and reasoning abilities of multimodal data, serving as an excellent test of LMM capabilities. There- fore, the focus of this paper is to evaluate the performance of LMMs in synthetic data detection tasks.
However, traditional synthetic data detection benchmarks, such as Fake2M (Lu et al. 2023b) and ASVSpoof 2019 (Wang et al.] (020b), primarily assess conventional detection methods, and eval- uations of LMMs in detecting multimodal synthetic data are still lacking. 'hese benchmarks often miss inn-grrined anomaly annotations represented in natural human language. making it difficult to transparently analyze the explainability capabilities of LMMs. FakeBench (Li et al. 2024a) aligns more closely with our objectives, but it only evaluates the performance of LMMs within a single standard image modality, lacking both breadth and depth. Specifically, FakeBench fails explore other modalities such as audio and 3D data, focusing primarily on general image types and not conducting thorough tests on expert domain images like satellite imagery. To bridge this gap, we introduce LOKI, a comprehensive benchmark for evaluating the performance of LMMs on synthetic data detection. The key highlights of the LOKI benchmark include:
Diverse Modalities. LOKI includes high-quality multimodal data generated by recent popular synthetic models, covering video, image, 3D data, text, and audio.
Heterogeneous Categories. Our collected dataset includes 26 detailed categories across different modalities, such as specialized satellite and medical images; texts like philosophy and ancient Chinese; and audio data like singing voices, environmental sound and music.
Multi-level Annotations. LOKI includes basic "Synthetic or Real" labels, suitable for fundamental question settings like true/false and multiple-choice questions. It also incorporates fine-grained anomalies for inferential explanations, enabling tasks like abnormal detail selection and abnormal explanation, to explore LMMs' capabilities in explainable synthetic data detection.
Multimodal Synthetic Evaluation Framework. We propose a comprchensive evaluation framework that supports inputs of various data formats and over 25 mainstream multimodal models.
On the LOKI benchmark, we evaluated 22 open-source LMMs, 6 advanced proprietary LMMs, and several expert synthetic detection models. Our key findings are summarized as follows:
For synthetic data detection tasks we find: (1) LMMs exhibit moderate capabilities in synthetic data detection tasks, with certain levels of explainability and generalization, but there is still a gap compared to human performance; (2) Compared to expert synthetic detection models. LMMs ex- hibit greater explainability and, compared to humans, can detect features invisible to the naked eye, demonstrating promising developmental prospects.
2

---

Published as a conference paper at ICLR 2025
For LMMs capabilities find: (1) Most LMMs exhibit certain model biases, tending favor syn- thetic or real data in their responses; (2) LMMs lack of expert domain knowledge, performing poorly on specialized image types like satellite and medical images; (3 Current LMMs show unbalanced multimodal capabilities, excelling in image and text tasks but underperforming in 3D and audio tasks; (4) Chain-of-thought prompting enhances LMMs' performance in synthetic data detection, whereas simple few-shot prompting falls short of providing the necessary reasoning support.
These findings highlight the challenging and comprehensive nature of the LOKI task and the promis- ing future of LMMs in synthetic data detection tasks.
2 RELATED WORK
2.1 SYNTHETIC DATA DETECTION
Currently, synthetic data detection has garnered widespread attention to prevent the misuse of mul- timedia synthetic data (Gagnaniello et al. 2021 Hou et al., 2023). The detection of synthetic data
in image and audio has long been a popular aa research (Barni et al. 2020) Frank et al. 2020). while
methods for synthetic video detection have recently emerged, such DuB3D(J et al. (024] and
AIGVDet(Bai et al., 2024a). However, most work primarily focuses on the binary distinction be- tween authentic and synthetic data, resulting in poor interpretability. Some studies aim enhance the interpretability of synthetic detection by providing latent representations! Dong et al. 20022) fea- ture explanations
et al., (020), and artifact localization (Zhang et al.) [203aa] Shao al. 2023] 2024] however, most research remains limited to the interpretability of abstract symbols, leaving a significant gap alignment with human understanding. In practice, current Al Al-generated syn- thetic data still exhibits noticeable flaws, such as discontinuities in synthetic videos and insufficient geometric accuracy in 3D data. These shortcomings can be effectively captured and perceived by hu- man users users(Tariang Tariang et al. 2024), who can provide reasonable explanations. However, existing expert synthetic data detection methods fail to provide human- numan-interpretable bases for their judgments.
2.2 LARGE MULTIMODAL MODELS
Recently, the rapid development of multimodal large models (LMMs) has been notable, with models like GPT-4o (OpenAT, 2024) and Claude 3.5 (Anthropic 2024) excelling in various tasks such as scientific questioning (Lu et al. 2022; Yue et al. 2024) and commonsense reasoning (Talmor et al.) 2018), showcasing exceptional perceptual and reasoning abilities (Bai et al. 2024b). Research has also applied LMMs to evaluate AIGC synthetic results, utilizing GPT to assess the quality of gener- ated images (Ku et al., 2023] Peng et al., 2024) and 3D models (Wu et al. 2024b), providing scores that align with human preferences along with interpretable justifications. Consequently, in synthetic data detection, LMMs can offer reasons for determining authenticity in natural language, paving the way for enhanced interpretability in synthetic detection. Moreover, LMMs can access features invisible to human users, such as deep image and spectral features, demonstrating their potential exceed human detection capabilities. Furthermore, synthetic data detection involves multimodal data perception and complex logical reasoning, making it an excellent task to assess the capabilities of LMMs. This task also provides quantitative evaluation metrics like accuracy, allowing for more direct assessment of model performance compared to more qualitative scoring tasks.
2.3 SYNTHETIC DATA DETECTION BENCHMARK
Currently, there are numerous datasets corresponding to synthetie data detection tasks, including those designed for traditional detection methods and those tailored for LMMs. For instance, tradi- tional synthetic datasets such as Fake2M (Lu et al.) 2023b), HC3 (Guo et al.) 2023). and ASVSpoof 2019 (Wang et al.] 20205) have explored the performance of traditional deepfake detection meth- ods across various modalities, but they lack assessments for LMMs models. VANE (Bharadwaj eet al. (024)) evaluates the capability of LMMs in detecting video anomalies, including the detec- tion of criminal activities in real videos and synthetic video detection, although focuses more on video content understanding. Fakebench (Li et al. (024b) assesses LMM performance in the im- age modality, yet concentrates on a single modality and offers limited subcategories. In contrast, LOKI covers broader range of data modalities, including video, image, 3D, text, and audio, as well as data from specialized fields such as remote sensing, medical imaging. and environmental sounds. In terms of problem design, LOKI encompasses tasks for authenticity judgment, well as more complex challenges like Abnormal Details selection and Abnormal Explanation, which test the LMMs' ability to explain reasons in synthetic data detection.
3

---

Published as a conference paper at ICLR 2025
3 DATASET
3.1 OVERVIEW OF LOKI
We introduce LOKI, a multimodal synthetic data detection benchmark, designed specifically to com- prehensively assess the capabilities of LMMs in detecting synthetic data. As illustrated in Figure LOKI encompasses variety of modalities including video, image, 3D, text, and audio, with 26 specific subcategories of data. The benchmark utilizes fine- ame-grained anomaly annotations to con- struct tiered variety of question types, including judgment questions, multiple-choie questions, abnomal detail selection and abnomal explanation questions, totaling over 18k questions.
Table provides a detailed comparison of LOKI with existing datasets, including traditional syn- thetic detection benchmarks and those tailored for evaluating LMMs. In terms of breadth, LOKI covers wider range of modalities and finer categories. In depth, it goes beyond binary judgment question designs to include questions that require deep understanding and explanation of detailed anomalies. Additionally, LOKI classifies question difficulty based on human evaluation metrics.
Chiire
20000
Abbinie
Spreni
Moderrn
hulgrmmn
2330
Liter sttrattor
Obients
Video
pailosophyy
370
Text
Mutiple
Chaie 1240
Text 3389
Aleurtaal Selection
Video
Sensiin
Rediute
Image 2217
Chinere
268
Audio
Jusgereer 2259
Scaonry
Mulippe
Doruments
Medical
Not bardbao
Composan
1201
Audio 2260
Raria parienan mentrna
Haver
Singinal
Voice
bolemmett
2217
Aburrnat
Image
3D
Aelenson
Eapleattio 377
Absomae
t Otners
Bounen S sclectan
Choice
Mallgpec
Figure 2: Statistical information of LOKI. The left side displays the detailed categories of each modality, while the right side presents the questions across different modalities. The inner circle numbers represent the data volume, and the outer circle numbers indicate the number of questions.
Table 1:
The comparison between LOKI and other benchmarks. Answer types include JD (Judg- ment), MC (Multiple Choice), and OE (Open-ended). "Real paired" indicates whether real data paired within the same domain, while "Difficulty Level" shows if questions are graded by difficulty.
Dataset
Size
Category
Data Modality
Answer
Real Paired
Difficulty Level
Img Vid Txt Aud 3D JD MC OE
FFHQ
70k
X
Fake2M
> IM
types
HC3
80K
types
Mixset
3.6
5 types
/
X
ASVS2019
108K
Codeefake
IM
FakeBench
6K
6 types
/
X
X
VANE
0.9K
/
x
LOKI
18K
26 types
/
/
3.2 DATA COLLECTION AND ANNOTATION
Video:
We collected 593 video clips by utilizing various closed- dosed-source and open- models
such as SORA (OpenAl 2024). Keling, and Open sora (Zheng et al.) (2024), generating high-quality text-to-video synthesis data along with corresponding real domain sample data. For the Al Al-generated
4

---

Published as a conference paper at ICLR 2025
(a) Video
Alneema Frame
Row Vićres
1:00
"s.
transforms the video The chair flants and moves in weid air without anyone hoiding it."
Abuormal Fragment
Absormal Fragment
"In the extracted video segment, piace
detaches peopless seguant,
from this seamingly
into chair. This gradually trarsforws
muld: which contradicts reality of gravity Additionally, the shape of the chair changes over time."
not align with
how things oceur reality"
Image
(b) 3D Object
Overall A bnormal
"The overall seturation of the imaage relatively high, expression insprrect, degres distortion. in
Abnormal Ahoormal
Texture Aboormal
Normal Abnormal
"The of bear aspmmmetrical and the number & incorract."
""The surface sexture the cheir and the book is sightls blurred, and there is some blanding of the edges of the the chair
"The surface normals of the cheir and the baok are relatively smooth, where the shopes the book the margge together.
Images
Figure 3: Examples of Synthetic Data Annotations: (a) Detailed annotations of video anomalies; (b) Detailed annotations of image anomalies; (c) Detailed annotations of 3D anomalies. video clips, ments and their descriptions, anomalous key frames, and global anomaly descriptions. As shown in Figure 3 (a), anomalies in the videos, such as "violating natural physics" and "frame flickering," are also annotated globally. Additionally, the anomalous segment from 02:54 to 06:27 is highlighted, with the corresponding reasons for the anomalies explained by human annotators. Furthermore, each anomalous segment includes an anomalous key frame to facilitate subsequent LMMs in accurately reading the anomalous frames when processing video data.
employed the Labell tool to annotate anomaly details, including anomalous seg-
Image: We have collected over 2,200 images from 7 subcategories through existing dataset ex- traction, internet collection, and new data synthesis. The image synthesis methods include FLUX, Midjourney (Al 2023), Stable Diffusion (Blattmann al. (023). and ten other different methods to ensure high quality and diversity of the data. For the synthesized image data, addition to overall annotations, performed anomaly region bounding and explanations, shown in Figure 3 (b). The region anomaly annotations allow for more fine-gra grained and specific labeling, which can be used for generating subsequent anomaly detail questions.
3D data:
We conducted a comprehensive analysis of OmniObject3D (Wu et al. 2023 selecting scanned instances as ground truth within the same domain. By constructing prompt texts, we syn- thesized three Nerf models (Poole et al., 2022) and three 3D GS models (Tang et al. 2023). and supplemented them with results from the advanced commercial model Clay and some Nert- results from GPTEval3D (Wu et al.) 2024b). We collected total of over 1,200 3D models from different synthesis methods, including both synthesized and real scanned data. Additionally, as shown in Figure 3 (c), we performed texture anomaly description annotations corresponding to the RGB four views of the synthesized 3D data, as well as normal anomaly description annotations. No- tably, besides the multi-view format, the 3D data also supports point clouds and panoramic videos.
We collected various categories of audio, including speech, singing voice, environmen- tal sounds, and music. The speech and singing voice data ensured consistency in speaker timbre, sourced from the Logical Access part ASVSpoof2019 (Wang et al.] (020b) and the CtrSVDD Benchmark, covering four generation paradigms: TTS, VC. SVS. and SVC. Environmental audio data came from DCASE 2023 Task 7, with real audio from the development set and synthetic audio generated using multiple methods from Track A. Music data were sourced from MusicCaps, with synthetic music generated based on descriptions using MusicGien (Copet et al. 2024), AudioLDM22 Music (Liu et al. 2024a), and Sund
Audio:
'LabelU:
Suno: htts://suno.com/
5

---

Published as a conference paper at ICLR 2025
(a) J udgement
(b) Multiple Choice
(d) Abnormal Explanation
<Question>
<Questiou>
<Answer>
<Question>
the provided audio gene rated by
"Whish the followiig
tese
<
provided Al-
<Answer>
Yes
@ No
Real jist
Sb G
genensedd""
SENO
which make Rigeds to of and whith are the warrees wehorsly man Assomillyy Soveraign Power
and indivisiile entity.
"Saversignty, must
"The shape of the ief fooe in the
<Answer>
These rights are not meree privileges aigurnoed bud the fasndattinal
structurs of haman fast, shawing
commentricall and the tranation
part ty
(c) bnormal Details Selection
Questionn
<Answer>
143 coThemmnberaferrtt
Generded SORA
Genertoddyy Midjounny
Figure 4: Example Questions of LOKI. LOKI includes four types of questions:(a) Judgment ques-
tions; (b) Multiple choice questions; (c) Abnormal detail selection; (d) Abnormal explanation.
Text: Based on summarization and regeneration methods, we generated counterfeit texts similar to the original texts using mainstream models such as GPT-4o. Qwen-Max, and Llama 3. 3.1-B05B (Bai jet al.) (022c)). We collected eight categories text data, pairing each sample with real text and model-generated similar text, totaling 3.359 text entries. Our text data were categorized by length and language, including short texts (50-100 characters), medium texts (100-200 characters), and long texts (over 200 characters), with 1:1 ratio of Chinese to English data. More information regarding the collection and statistics each modality can be found in Appendix
3.3 QUESTION GENERATION
Judgment Task: This task requires large language models (LMMs) to determine whether the input data is synthetic or real. As shown in Figure 4 (a). LMMs need to answer the judgment question, "Is the provided audio generated by AI?" To minimize the influence of prompts on model judgments, questions are asked in two forms: whether the data is Al-synthesized or real, and identifying either the real or Al-synthesized data, Furthermore, we categorize the data into different difficulty levels based on human performance. If all tested human users (more than three) answer correctly, the task classified as "easy"; if more than 50% answer incorrectly, it is classified as "hard"; all other cases fall into the "medium" category.
This task requires LMMs to correctly select Al-generated or real data from the provided synthetic and real data. As illustrated in Figure 4 (b). LMMs need to complete the multiple-choice question, "Which of the following texts is generated?" The design of this question benefits from our collection of real paired data within the same domain, effectively assessing LMMs' comparative analysis capabilities.
Multiple Choice Task:
Abnormal Detail Selection: Based fine-grained anomaly annotation data from modalities such as video, images, and 3D, we effectively design prompts and utilize GPT-4o to generate questions for Abnormal Detail Selection. As shown in Figure (c). for video content's detail anomalies, we ask, "What elements can be seen as inconsistent?" By providing clear anomaly annotations, we can effectively reduce the hallucination phenomenon in GPT--4,, ensuring the quality of the questions. More details can be found in the supplementary materials.
Abnormal Explanation: Furthermore, we design open-ended abnormal explanation questions, re- quiring LMMs to independently identify anomalies and explain their reasons. As shown in Figure (d), we ask, "Why is the provided image Al- Al-generated?" It is worth to note that in real anomaly explanation tasks, the input does not include bounding boxes around anomalous areas. Tasks related to Abnormal Detail Selection and Abnormal Explanation can more precisely test whether LMMs genuinely perceive corresponding detail anomalies rather than guessing answers.
To mitigate the impact of hallucinations of GPT-4o during question generation in abnormal detail selection task, all samples in this task undergo manual reviews. Each question that involves GPT must pass through at least two rounds of verification by human users. A total of 20 users participated in the verification process, which took approximately 160 hours to complete.
Quality Control:

---

Published as a conference paper at ICLR 2025
4 EXPERIMENT
In this section, we evaluate various Language Model Multimodalities (LMMs) under our proposed LOKI evaluation framework, which includes both open-source and proprietary models, multimodal LMMs, Audio LMMs, and text-based LLMs. Our evaluations are conducted in a zero-shot setting. In the following subsections, we first introduce our evaluation models and the evaluation protocols. Next, we analyze the performance of existing LMMs in synthetic data detection tasks, comparing them with human users and expert models. We will then discuss the challenges and shortcomings faced by multimodal large models in the current task settings. Additionally, we explore the potential impact of few-shot or chain-of of-thoughtttt prompting on this task.
4.1 BASELINES
LMMs. We evaluate closed-source and 18 open-source LMMs across different model types and sizes, For closed-source models, we consider GPT-4o ( Gemini-1.5-Pp (Team et al, (Anthropic, 2024). Given that modality alignment in multimodal LMMs may lead to decline in LLM performance on text-based tasks (Dai et al.) 2024). we also selected pure text LLMs, such as LLaMA-3. -00B (Team) 2024), Qwen-Max (Chu et. 2023) and Mistral- Large (Mistral 2024), to evaluate the text modality. In the evaluation of Audio LMMs, we selected high-performing open-source models such as Qwen-Audio (Chu et al. 2023) and SALMONN-7B (Sun et al. 2024). For proprietary models, we chose Gemini-Flash
2023),
(Team et al. 2023), which
supports audio input.
Human Users. We invited over 50 human users, including senior university students and regular users, to participate in the judgment and multiple-choice question tests for different modalities of synthetic data. Each question Additionally, we designed an online platform to distribute random questionnaires, and more than 200 users participated in the testing of 15 basic questions.
tested by least 3 users to ensure the robustness of the results.
Expert Models. We selected recently open-sourced expert-level synthetic data detection meth- ods and their corresponding weights for testing, including video detection (AIGVDet (Yang al., 2024), image detection (AIDE Yan et al. (204)). text detection (RADAR-Vicun) Vicuna-7 (Hu et al. 2023 ). and audio detection (AASIST (Jung al. 2022 Due to the limited availability of 3D syn- thetic data detection methods, 3D was not considered. Additionally, there is no overlap between the training sets of these methods and the LOKI test data, reducing the possibility of data contamination. We selected only a small number of expert models for evaluation, primarily to serve as references, similar to the role human references.
For the video modality, we utilize an 8-frame video clip along with corresponding questions as input. For 3D modal data, we employ the commonly used multi- view input method. Results based surround video and point cloud inputs are also included in the supplementary materials. For other modalities, inputs are based on textual prompts combined with corresponding images, audio, and textual materials. During the evaluation, each model inde- pendently generates responses to questions without retaining any dialogue history.
Evaluation Protocol. Data Input:
Evaluation Metric: For judgement, multiple-choice and abnormal detail selection questions, the average accuracy rate as a metric. In addition to accuracy, we also calculate the Normalized Bias Index (NBI) based recall rates assess model bias. For open-ended questions regarding anomalous details, we use the GPT-4 model to assess the score of the responses. Further details on the calculation of evaluation metrics can be found in Appendix C.2
Evaluation Framework: To standardize the evaluation of different LMMs and various input modal- ities for synthetic data detection, we propose comprehensive multimodal evaluation framework. This framework provides support for various input modalities such as 3D point clouds, videos, im- ages, audio, and text, while unifying APIs of over 25 mainstream LMMs, ensuring both model compatibility and consistency throughout the evaluation process.
4.2 SYNTHETIC DATA DETECTION RESULTS
In this section, we provide a comprehensive analysis of the performance various LMMs and L.MSs on synthetic data detection tasks using the LOKI dataset.
7

---

Published as a conference paper at ICLR 2025
Table 2: Results of different models on the LOKI for Judgment and Multiple Choice questions. (a) Multimodal evaluation of LMMs; (b) Text evaluaion of LLMs; (c) Audio evaluation of Audio LMMs; denotes the closed-source models.
(a) Multimodal evaluation of LMMs
Judgment
Multiple Choice
Video
Image
3D
Text
Overall
Video
Image
3D
Text
Overall
Random Choice
51.1
50.5
50.5
49.9
50.3
47.7
49.0
49.7
45.2
46.9
Human (Medium)
83.5
80.1
72.0
68.5
76.0
91.3
84.5
91,2
78.5
86.4
Expert models
53.1
63.1
72.1
62.8
Phi-a.5- Vision
56.8
52.5
50.0
49.4
52.2
58.2
44.0
59.6
42.0
50.9
MiniCPM-V-24.
57.2
44.8
56.4
49.4
52.0
52.8
49.8
50.7
48.9
50.6
IntenL_M-XCoopooee225
58.4
46.4
43.9
52.6
50.3
56.3
$1.0
48.0
40.5
49.0
InPLUG-OW33-78
55.3
45.9
49.9
53.6
51.1
60.3
52.5
49.9
50.0
53.1
LongVA-7B
60.4
46.2
49.9
48.6
51.7
51.6
61.4
48.9
52.6
Qwen2-VL-7B
59.5
47.8
72.3
57.1
64.0
55.5
46.4
$7.7
LLaVA-OV-7B
56.8
49.8
57.0
59.8
51.7
53.8
48.4
53.4
Laans-3-LooggIIAA--88
51.9
49.8
32.2
49.9
46.0
54.0
51.1
50.5
44.3
50.0
Idefics2-8B
54.8
45,0
38.4
47.2
46.3
55.6
51.3
54.2
37.0
49.5
Mantis-8B
55.4
54.6
50.0
$2.0
53.0
47.9
61.5
62.5
48.4
S5.1
InterVVL2-8B
60.8
49.7
49.4
50.3
52.6
54.0
51.4
53.1
46.6
51.3
Intern
55.0
44.3
50.4
$1.1
49.9
62.4
48.5
48.8
50.3
53.2
InternVT.2-408
62.0
49.6
49.9
53.1
52.2
65.7
63.1
59.9
45.2
52.7
VILA115-13B
51.9
49.3
34.0
47.7
45.7
52.1
55.3
53.5
44.0
51.2
VILAI.5-40B
59.2
50.1
64.0
47.9
50.4
53.7
Owen2-V1L-2BB
59.6
53.2
52.8
55.4
68.6
58.7
69.7
65.6
LLaVA-Ov-72B
56.5
46.3
51.3
61.2
56.3
62.9
70.8
61.3
69.2
65.2
Claude-3.5-Sond
61.7
53.6
58.0
61.5
61.6
60.5
65.5
51.9
89.2
74.8
Gemini-1155-P
58.5
43.5
55.4
55.7
53.2
66.1
67.3
60.2
57.3
62.7
GPT-4o"
71.3
63.4
65.2
55.9
63.9
77.3
80.8
70.2
66.6
73.7
(b) Text evaluation of LLMs
(c) Audio evaluation of Audio LMMs
Model
Judgment
Choice
Overall
Model
Judgment
Choice
Overall
Human (Medium)
69.2
71.1
70.1
Human (Medium)
69.2
71.1
70.1
Expert model
69.4
69.4
Expert model
69.4
69.4
1LaMA-3.1-405B
56.8
73.1
64.4
Qwen-Audio
49.8
50.1
49.9
Mistral-Large'
52.2
69.1
$7.8
SALMONN-7B
51,2
51.2
Qwen-Max
48.3
44.4
46.5
AnyGPT
49.8
50.3
50.1
Claude-3.5-
61.5
89.2
70.7
OncLLM
49.9
Gemini-115-Pro
55.7
57.3
56.2
LUT
44.4
44.4
GPT-4"
66.6
59.5
1.5Fiai-.---lashh"
49.4
49.2
Judgment and Multiple Choice. Table 2 illustrates the performance of various models judg- ment and multiple-choice questions in LOKI. For the synthetic data judgment task, the closed-source model GPT-4o achieves the best results, with an overall accuracy (excluding audio) of 63.9%. When real paired data is included for comparison in the multiple-choie questions, accuracy further in- creases to 73.7%. In the text modality, Claude-3.5 outperform other LMMs and LLMs, achieving accuracies exceeding 70%. In the Audio LMMs category, both open-source and closed-source mod- els show performances comparable to random selection, which is not satisfactory.
Abnormal Detail Selection and Explanation. We compared the performance of different models on the tasks of abnormal detail selection and abnormal reason explanation, as shown Table 3. GPT-4o achieved an accuracy exceeding 75% in abnormal detail selection and score 70% in abnormal reason explanation. This indicates that advanced LMMs like GPT-4o has demonstrated strong detail understanding capabilities, effectively analyzing and interpreting "synthetic traces." Notably, we observe that Claude-3.5 3.5te-3.5--oeettt (Anthropic) 2024) tends to misclassify synthetic images real, despite the primary goal of tasks being explain abnormalities in synthetic images. More examples of abnormal explanations can be found in Appendix
Comparing Humans and Expert Models. Humans exhibit average performance of 76% in judg- ment tasks and 86.4% in multipe-chooce questions, both 10% higher than the LMM method. No-
8

---

Published as a conference paper at ICLR 2025
Table 3: Results of different models on the LOKI for Abormal Details Selection and Abnormal explanation questions. denotes the closed-source models.
Abnormal Details Selection
Abnormal Explanation
Video
Image
Overall
Video
Image
Overall
LLaVA-OV-7B
76.9
18.8
43.1
46.7
68.9
71.0
62.0
Qwen2-VL-7B VL--BB
79.4
31.5
51.5
48.4
63.8
73.4
61.9
Intern VL2-8B
66.8
70.2
68.8
46.5
72.2
71.3
63.0
Gemini-1.5-PHt
58.7
40.0
47.8
57.6
77.1
70.8
68.1
Claude-3.5-Somme
50.9
19.8
32.8
50.1
1.7
78.2
45.8
GPT-4o
74.0
76.2
75.3
67.6
72.9
77.0
72.6
Geeal VL-T
GPI-0
Diflerenee (Hlmmn GPT-40)
Vidce
Vilar (1)
Tirr())
Vidte-
Difference from tummn (%g
Image-
Image
Test
Audio (S)
10
Aodoo (n
Audio
1La0 Ma
Dewet
Over bodo
Cenaai 5.h
Qnzn-Adde
Image Types
Oemes Plcc
Figure 5: The multimodal large model capability assessment analysis results. (a) Model bias assessment, where the closer the color is to red, the more the model is biased towards classifying the data as real; the closer to blue, the more it leans towards synthetic data. The size of the square also represents the degree of bias. (b) The performance of GPT-4o across different image types and its difference from human users. (c) A relative radar chart of the model's performance across various modalities, with Human benchmarks for comparison.
tably, if LMM tools are to be applied in production, their decision making performance in judgment tasks must exceed 90% to be convineing. As synthesis technologies advance, the distinct "traces" of synthetic data are becoming increasingly subtle. However, LMMs capture minute details, such as image features imperceptible to the human eye, demonstrating their potential surpass human.
LMMs demonstrate superior performance in most tasks compared to expert models. This is primar- ily due to the rich and diverse sources synthetic data collected by LOKI, which significantly differ from existing data domains, resulting in suboptimal generalization performance of expert models. The accuracy synthetic detection by expert models trained on similar data should significantly improve. Currently, LMMs perform moderate level in synthetic data detection but surpass pert models in generalization ability. Unlike traditional expert models, LMMs possess the capability explain the reasons behind anomalies, highlighting their unique advantage synthetic detectors.
4.3 LARGE MULTIMODAL MODELS CAPABILITIES
Model Bias. The heatmap of the Normalized Bias Index calculated based on recall rates, as shown in Figure [5] (a), is utilized for analyzing model biases. The results indicate that most models exhibit significant biases in synthetic data detection tasks, with tendency incorrectly categorize data as either real or synthetic. For instance, GPT-4o tends to classify textual data as real, whereas it is bi- ased towards judging 3D data as Al-generated. Despite diverse questioning techniques implemented to minimize cucing effects, pronounced bias is still evident across most models.
Lack Expert Domain Knowledge. In Figure (b), we present the varying performance GPT- 4o across different image subcategories. The experimental results clearly indicate that GPT exhibits strong recognition abilities on common image types such as objects and landscapes, even surpassing human users. However, GPT-4o's performance significantly deteriorates in specialized fields such as satellite and medical imaging, and in less- trained image types like documents. This suggests that current LMMs still lack certain expert domain knowledge.

---

Published as a conference paper at ICLR 2025
Table 4: Result decomposition across ques- tions difficulty levels.
Table 5: LMMs' performances under different
prompting strategies for judgement tasks.
Difficulty Levels (Video & Image 3D & Text)
Prompting Strategies Performances (Image
Easy (2470)
Medium (1104)
Hard (3938)
Overall (7512)
Baseline
FS
CoT
Few-shot
Chain-of- Thought
L.aVA-OV:7B
60.4
47.6
39.1
47.3
ILaVA-OV-78
56.6
46.4
18.8
IntemVL2-8B
64.5
47,8
33.5
45.7
Intern VL2-8B
49.6
46.1
50,4
Qwen2-V1. VL.-BB
67.7
45.6
35.2
47.4
Qwen2: V1.7B
56.8
52.6
59.5
Gemini-1 1.5
70.8
42.4
32.4
46.4
Gemini-1 5-pro
47.9
41.2
51.0
Claude-3.5-Some
76.0
44.7
29.8
47.1
Claude-3.5 S.5none
55.2
53.7
56.4
GPT-40
78.8
52.3
44.4
56.8
GPT-4o
64.1
75.1
74.2
Unbalanced Multimodal Capabilities. In Figure 5 (c), we compare the performance of various
LMMs across different modalities. Results indicate that current models excel in frequently trained modalities such as images and text, even surpassing human performance in some tests. However, their performance declines significantly on audio tasks, with most open-source models lacking cor- responding capabilities. For future AGI to develop into versatile assistant, it needs to possess more balanced multimodal abilities.
Model Performance across Different Levels. Based human performance, we categorized the difficulty levels of the questions, as shown in Table[4] which presents the performance of selected models across different difficulty levels. As the difficulty increases, the performance of LMMs gradually declines, consistent with human user performance. Under challenging conditions, GPT- 4o's accuracy drops only 44.4%, which is lower than that achieved by random selection. This indicates that LMMs have certain limitations in handling complex synthetic data detection tasks.
Prompting Strategies Impact LMMs Capabilities.
In Table 5 we demonstrate the effects of different prompting strategies in LOKI's image and 3D judgement tasks, where CoT refers to the Chain of Thought prompting (Wei et al. 20226) and FS refers to the few-shot prompting (Alayrac et al. 2022). During inference, models are prompted with two random examples that are in the same domain as the questions by different strategies. In CoT prompting, we manually craft "thought chains" with our human annotations to elicit reasoning steps out of LMMs, while in FS prompting, we simply prepend examples with answers to the questions. Interestingly, GPT-4o shows strong reasoning ability without chain-of-thought prompting, while other models rely on it for improved performance. Few-shot learning fails to support the necessary step-by-step reasoning for synthetic data detection, but GPT-4o performs well regardless, suggesting its inherent ability to reason effectively without additional reasoning guidance. However, LLaVA- OV-7B experienced significant performance drop when prompted with COT. We conjecture that this degradation may result from a decline in its ability to understand long contexts after fine-tuning (Zhai et al. 2023). More CoT experimental results available in Appendix E.3
5 CONCLUSION
In this paper, we introduced LOKI, a multimodal benchmark designed to evaluate the performance of large multimodal models in detecting synthetic data across various modalities. We conducted comprehensive study of LMMs' performance on video, image, 3D, audio, text, and specialized sub- domains, and we also analyzed LMMs' ability to explain detailed anomalies in synthetic data. The experimental results indicate that LMMs have a certain level of competence in detecting synthetic data and a preliminary ability to explain anomalies. Synthetic data detection tasks also effectively evaluate the various capabilities of LMMs during their development. These findings highlight the challenging and comprehensive nature of the LOKI task, as well as the potential of LMMs in fu- ture synthetic data detection tasks. We aim to inspire more powerful and interpretable synthetic data detection methods through LOKI to address the potential risks posed by rapidly advancing AI synthesis technologies. Furthermore, while the relationship between synthesis and detection is ad- versarial, they are mutually beneficial; better and more explainable synthetic detectors will further advance Al synthesis technologies.
10

---

Published as a conference paper at ICLR 2025
ACKNOWLEDGMENTS
This project was funded by National Natural Science Foundation of China (Grant No. 42201358) and Shanghai Al Laboratory. Additionally, this work partially supported by the NSFC under Grant 62376237, Shenzhen Science and Technology Program ZDSYS2023062609 2220200620091000006 332006, and Internal Project Fund from Shenzhen Research Institute Big Data (Grant No. 2023000))
REFERENCES
Ai safety summit, 2023. URL UK.
gov.uk/ Hosted by the
Malak Abdullah, Alia Madain, and Yaser Jararweh. Chatgpt: Fundamentals, applications and social impacts, In 2022 Ninth International Conference on Social Networks Analysis, Management and Security (SNAMS), pp. 1-8. leee, 2022.
Andrea Agostinelli, Timo I. Denk, Zalán Borsos, Jesse Engel, Mauro Verzetti, Antoine Caillon, Qingqing Huang, Aren Jansen, Adam Roberts, Marco Tagliasacchi, Matt Sharifi, Neil Zeghidour, and Christian Frank. Musiclm: Generating music from text, 2023. URL https: //ttps://axiv.
orq bxq/abs/230 2301. 11325
M. AL Midjourny: Text image with ai art generator, 2023. URL https::/www
mi mid midjourneyai. ai/en Accessed: 2024-09-21.
Jean-Bas CannBBppistte Alayrac, Jeff Donahue, Pauline Luc, Antoine Miech, lain Barr, Yana Hasson, Karel Lenc, Arthur Mensch, Katherine Millican, Malcolm Reynolds, et al. Flamingo: visual language model for few-shot learning. Advances in neural information processing systems, 35:23716- 23736, 2022.
Sam Altman. Openai now generates about 100 billion words per day. 2024. URL https://x. Accessed: 2024-09-24.
com/ ham//saa/staaiee httpa//meeetttaaa./.333333333110
Anthropic. The claude 3 model family: Opus, sonnet, haiku, 2024. URL hittps:/wwat
anthropic.co com) Accessed: 2024-09-23.
Jianfa Bai, Man Lin, Gang Cao, and Zijie Lou. Al-generated video detection via spatial-temporal anomaly learning. The 7th Chinese Conference on Pattern Recognition and Computer Vision (PRCV), 2024a.
Jinze Bai, Shuai Bai, Shusheng Yang, Shijie Wang, Sinan Tan, Peng Wang, Junyang Lin, Chang Zhou, and Jingren Zhou Qwen-vl: A versatile vasion-language model for understanding, local- ization, text reading, and beyond, 2023. URL https: wtwpiv.ivvv..///ii///..///aaatva org abaaabas//3550 12966
Tianyi Bai, Hao Liang, Binwang Wan, Yanran Xu, Xi Li, Shiyu Li, Ling Yang, Bozhou Li, Yifan Wang, Bin Cui, Ping Huang, Jiulong Shan, Conghui He, Binhang Yuan, and Wentao Zhang. A survey of multimodal large language model from a data-centric perspective, 2024b. URL httts:://rxxiii
wtgg/abs///ae al///////2455 /2055 16640
Tianyi Bai. Ling Yang, Zhen Hao Wong, Jiahui Peng, Xinlin Zhuang, Chi Zhang, Lijun Wu, Jiantao Qiu, Wentao Zhang, Binhang Yuan, and Conghui He. Multi-agent collaborative data selection for efficient Ilm pretraining, 2024c. URL http:ttst//x....crgttte. hwt.:/orggabb.a... /2400 08102
Omer Bar-Tal, Hila Chefer, Omer Tov, Charles Herrmann, Roni Paiss, Shiran Zada, Ariel Ephrat, Junhwa Hur, Yuanzhen Li, Tomer Michaeli, Oliver Wang, Deqing Sun, Tali Dekel, and Inbar Mosseri. Lumiere: A space-time diffusion model for video generation. 2024. URL https:
//api. wwweeanttcccclllll :or.oolaarooroooooosIt-::---- 267099111
Mauro Barni, Kassem Kallas, Ehsan Nowroozi, and Benedetta Tondi. Cnn detection of gan- generated face images based on cross- band co-oceurrences analysis. In 2020 IEEE international workshop information forensics and security (WIFS), pp. 1-6. IEEF, 2020.
11

---

Published as a conference paper at ICLR 2025
Quentin Bertrand, Joey Bose, Alexandre Duplessis, Marco Jiralerspong, and Gauthier Gidel. On the stability of iterative retraining of generative models on their own data. In The Twelfth International Conference on Learning Representations, 2024.
Rohit Bharadwaj, Hanan Gani, Muzammal Naseer, Fahad Shahbaz Khan, and Salman Khan. Vane- bench: Video anomaly evaluation benchmark for conversational Imms, 2024.
Joseph R Biden. Executive order on the safe, secure, and trustworthy development and use of artificial intelligence. 2023.
Andreas Blattmann, Tim Dockhorn, Sumith Kulal, Daniel Mendelevitch, Maciej Kilian, Dominik Lorenz, Yam Levi, Zion English, Vikram Voleti, Adam Letts, et al. Stable video diffusion: Scaling latent video diffusion models to large datasets. arXiv preprint arXiv:2311.15127 2023.
Matyas Bohacek and Hany Farid. preprint arXiv:231 1. 12202, 2023.
Nepotistically trained generative-ai ai models collapse. arXiv
Matyas Bohacek and Hany Farid. The making of an ai news anchor-and its implications. Proceed- ings of the National Academy of Sciences, 121(1)=2315678112) 2024.
Tim Brooks, Bill Peebles, Connor Holmes, Will DePue, Yufei Guo, Li Jing, David Schnurr, Joe Taylor, Troy Luhman, Eric Luhman, Clarence Ng. Ricky Wang, and Aditya Ramesh. Video generation models as world simulators. 2024. URL pttttps://openai... wwwpppaa...o/eeeerreeeeeeeeeeeee wwt.//////eeerreeeeeeeeeeeeeeeee wilde--generation-noon
ws- www.oooddssmmllsaee... simulators
Tom B Brown. Language models are few-shot learners. arXiv preprint arXiv: 14165, 2020.
Lucy Chai, David Bau, Ser-Nam Lim, and Phillip Isola. What makes fake images detectable? under- standing properties that generalize. In Computer Vision-ECCV 2020: 16th European Conference, Glasgow, UK. August 23-28. 2020, Proceedings, Part XXVI 16, pp. 103-120. Springer. 2020.
Rui Chen, Yongwei Chen, Ningxin Jiao, and Kui Jia. Fantasia3d: Disentangling geometry and appearance for high-quality text-to-3d content creation. In Proceedings of the IEEE/CVF inter- national conference on computer vision, pp. 22246-22256, 2023a.
Sanyuan Chen, Yu Wu, Chengyi Wang, Shujie Liu, Daniel Tompkins, Zhuo Chen, Wanxiang Che, Xiangzhan Yu, and Furu Wei. Beats: audio pre-training with acoustic tokenizers. In I Proceedings of the 40th International Conference on Machine Learning, pp. 5178-5199 2023b.
Zhe Chen, Weiyun Wang, Hao Tian, Shenglong Ye, Zhangwei Gao, Erfei Cui, Wenwen Tong, Kongzhi Hu, Jiapeng Luo, Zheng Ma. et al. How far are we gpt-4v? closing the gap to commercial multimodal models with open-source suites. arXiv preprint 2rXiv:2404.168..1 2024.
Hiuyi Cheng, Peirong Zhang, Sihang Wu, Jiaxin Zhang. Qiyuan Zhu, Zecheng Xie, Jing Li, Kai Ding, and Lianwen Jin. M6doc: A large-scale multi-format, multi-type, multi-layout, multi- language, multi-annotation category dataset for modern document layout analysis. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 15138- 15147, June 2023.
Keunwoo Choi, Jaekwon Im, Laurie Heller, Brian McFee, Keisuke Imoto, Yuki Okamoto, Mathieu Lagrange, and Shinosuke Takamichi. Foley sound synthesis at the dcase 2023 challenge. arXiv preprint arXiv:2304. 125211, 2023.
Yunfei Chu, Jin Xu, Xiaohuan Zhou, Qian Yang, Shiliang Zhang, Zhijie Yan, Chang Zhou, and Jingren Zhou. Qwen-audio: Advancing universal audio understanding via unified large-scale audio-language models. arXiv preprint arXiv:2311.07991 2023.
Di Cooke, Abigail Edwards, Sophia Barkoff, and Kathryn Kelly. As good as a coin toss hu- man detection of ti-generated images, videos, audio, and audiovisual stimuli. arXiv preprint 17X6::2403.16760, 2024.
Jade Copet, Felix Kreuk, Itai Gat. Tal Remez, David Kant, Gabriel Synnaeve, Yossi Adi, and Alexan- dre Défossez. Simple and controllable music generation. Advances in Neural Information Pro- cessing Systems, 36, 2024.
12

---

Published as a conference paper at ICLR 2025
Wenliang Dai, Nayeon Lee, Boxin Wang, Zhuoling Yang, Zihan Liu, Jon Barker, Tuomas Rinta- maki, Mohammad Shoeybi, Bryan Catanzaro, and Wei Ping. Nvlm: Open frontier-class multi- modal lims. arXiv preprint arXlv:2409.11402, 2024.
Prafulla Dhariwal and Alex Nichol. Diffusion models beat gans on image synthesis, 2021a. URL https:// org/ absaaaass///0 05233
Prafulla Dhariwal and Alexander Nichol. Diffusion models beat gans image synthesis. Advances in neural information processing systems, 34:8780-8794, 8794, 2021b.
Elvis Dohmatob, Yunzhen Feng, and Julia Kempe. Model collapse demystified: The of regres- sion. arXiv preprint arXiv:2402.07771 2024a.
Elvis Dohmatob, Yunzhen Feng, Pu Yang, Francois Charton, and Julia Kempe. A tale of tails: Model collapse as change of scaling laws. In Forty-first International Conference on Machine Learning, 2024b.
Shichao Dong, Jin Wang, Jiajun Liang, Haoqiang Fan, and Renhe Ji. Explaining deepfake detection by analysing image matching. In European conference on computer vision, pp. 18-35. Springer, 2022.
Yunzhen Feng, Elvis Dohmatob, Pu Yang, Francois Charton, and Julia Kempe. Beyond model col- lapse: Scaling up with synthesized data requires reinforcement, arXiv preprint urXiv:2406.07515, 2024.
Joel Frank, Thorsten Eisenhofer, Lea Schönherr, Asja Fischer, Dorothea Kolossa, and Thorsten Holz. Leveraging frequency analysis for deep fake image recognition. In International conference on machine learning, pp. 3247-3258. PMLR, 2020.
Liang Yu Gong and Xue Jun Li. A contemporary survey on deepfake detection; datasets, algorithms, and challenges. Electronics, 13(3):585, 2024.
Yuan Gong, Yu-An Chung, and James Glass. Ast: Audio spectrogram transformer. In Interspeech 2021, pp. 571-575, 2021.
Diego Gragnaniello, Davide Cozzolino, Francesco Marra, Giovanni Poggi, and Luisa Verdoliva. Are gan generated images easy to detect? critical analysis of the state-of-the-art. In 2021 IEEE international conference on multimedia and expo (ICME), pp. 1 1-6. IEEE, 2021.
Suriya Gunasekar, Yi Zhang, Jyoti Aneja, Caio César Teodoro Mendes, Allie Del Giorno, Sivakanth Gopi, Mojan Javaheripi, Piero Kauffmann, Gustavo de Rosa, Olli Saarikivi, Adil Salim, Shital Shah, Harkirat Singh Behl, Xin Wang. Sébastien Bubeck, Ronen Eldan, Adam Tauman Kalai, Yin Tat Lee, and Yuanzhi Li. Textbooks are all you need, 2023.
abs/ abs/2306. 2206 11644
Biyang Guo, Xin Zhang, Ziyuan Wang, Minqi Jiang, Jinran Nie, Yuxuan Ding, Jianwei Yue, and Yu- peng Wu. How close is chatgpt to human experts? comparison corpus, evaluation, and detection. arXiv preprint orxiv:2301.07755 2023.
Agrim Gupta, Lijun Yu, Kihyuk Sohn, Xiuye Gu, Meera Hahn, Li Fei-Fei, Irfan Essa, Lu Jiang, and José Lezama. Photorealistic video generation with diffusion models. arXiv preprint arXiv:2312.0666) 2023.
David Gutman, Noel C. F. Codella, Emre Celebi, Brian Helba, Michael Marchetti, Nabin Mishra, and Allan Halper. Skin lesion analysis toward melanoma detection: challenge at the inter- national symposium on biomedical imaging (isbi) 2016, hosted by the international skin imaging collaboration (isic), 2016. (tRpsttps: /aaxiviii// orr.or///aaoo /abs////////// 200...//166000000
605. 01397
Ryuichiro Hataya, Han Bao, and Hiromi Arai. Will large-scale generative models corrupt future datasets? In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp- 20555-20565 2023.
13

---

Published as a conference paper at ICLR 2025
Ruifei He, Shuyang Sun, Xin Yu, Chuhui Xue. Wenqing Zhang, Philip Torr, Song Bai, and XIAO- JUAN QL Is synthetic data from generative models ready for image recognition? The Eleventh International Conference Learning Representations, 2023.
Yang Hou, Qing Guo, Yihao Huang, Xiaofei Xie, Lei Ma, and Jianjun Zhao. Evading deepfake detectors via adversarial statistical consistency. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. (2271-12280) 2023.
Shengding Hu, Yuge Tu, Xu Han, Chaoqun He, Ganqu Cui, Xiang Long, Zhi Zheng, Yewei Fang, Yuxiang Huang, Weilin Zhao, et al. Minicpm: Unveiling the potential of small language models with scalable training strategies. arXiv preprint orXiv2240.066394 2024.
Xiaomeng Hu, Pin-Y Yu Chen, and Tsung-Y- Yi Ho. Radar: Robust ai-text detection via adversarial learning. Advances in Neural Information Processing Systems, 36:15077-15095, 2023.
Phillip Isola, Jun- Yan Zhu, Tinghui Zhou, and Alexei A Efros. Image-to-image translation with conditional adversarial networks. In Proceedings of the IEEE conference computer vision and pattem recognition, pp. 1125-1134, 1134, 2017.
Lichuan Ji, Yingqi Lin, Zhenhua Huang, Yan Han, Xiaogang Xu, Jiafei Wu, Chong Wang, and Zhe Liu. Distinguish any fake videos: Unleashing the power of large-scale data and motion features. arXiv preprint arXiv:2405.15343 2024.
Albert Q Jiang, Alexandre Sablayrolles, Arthur Mensch, Chris Bamford, Devendra Singh Chaplot, Diego de las Casas, Florian Bressand, Gianna Lengyel, Guillaume Lample, Lucile Saulnier, al. Mistral 7b. arXiv preprint arXiv:2310.06825 2023a.
Harry H Jiang, Lauren Brown, Jessica Cheng, Mehtab Khan, Abhishek Gupta, Deja Workman, Alex Hanna, Johnathan Flowers, and Timnit Gebru. Ai art and its impact on artists. In Proceedings of the 2023 AAAI/ACM Conference on AI, Ethics, and Society, pp. 363-374, 2023b.
Yan Ju, Shan Jia, Lipeng Ke, Hongfei Xue, Koki Nagano, and Siwei Lyu. Fusing global and local features for generalized ai-synthesized image detection. In 2022 IEEE International Conference on Image Processing (ICIP), pp. 3465-3469, IEEE, 2022.
Jee-weon Jung, Hee-Soo Heo, Hemlata Tak, Hye-jin Shim, Joon Son Chung, Bong-Jin Lee, Ha-Jin Yu, and Nicholas Evans. Aasist: Audio anti-spoofing using integrated spectro-temporal graph attention networks. In ICASSP 2022-2022 IEEE international conference on acoustics, speech and signal processing (ICASSP), pp. 6367-6371. IEEE, 2022.
Nal Kalchbrenner, Erich Elsen, Karen Simonyan, Seb Noury, Norman Casagrande, Edward Lock- hart, Florian Stimberg, Aaron Oord, Sander Dieleman, and Koray Kavukcuoglu. Efficient neural audio synthesis. In International Conference on Machine Learning, pp. 2410-2419, PMLR, 2018.
Tero Karras, Samuli Laine, and Timo Aila. A style-based generator architecture for generative adversarial networks. In Proceedings the IEEE/CVF conference on computer vision and pattern recognition, pp. 4401-4410, 2019a.
Tero Karras, Samuli Laine, and Timo Aila. A style-based generator architecture for generative http:/::///xxxx.../a..bbbs/
adversarial networks, 2019b.
04948
Tero Karras, Miika Aittala, Samuli Laine, Erik Härkönen, Janne Hellsten, Jaakko Lehtinen, and Timo Aila. Alias-free generative adversarial networks. In Proc. NeurIPS, 2021.
Sohail Ahmed Khan and Duc-Tien Dang-Nguyen. Clipping the deception: Adapting vision- language models for universal deepfake detection. In Proceedings of the 2024 International Conference on Mullimedia Retrieval, pp. 1006-1015, 2024.
Jachyeon Kim, Jungil Kong, and Juhee Son. Conditional variational autoencoder with adversarial learning for end-to-end text-to-speech. In International Conference on Machine Learning, pp- 5530-5540. PMLR, 2021.
14

---

Published as a conference paper at ICLR 2025
Takeshi Kojima, Shixiang Shane Gu, Machel Reid, Yutaka Matsuo, and Yusuke Iwasawa. Large language models are zero-shot reasoners. Advances in neural information processing systems, 35:22199-2223 22219,, 2022.
Jungil Kong, Jachyeon Kim, and Jackyoung Bac. Hifi-gan: Generative adversarial networks for efficient and high fidelity speech synthesis. Advances in neural information processing systems, 33:17022-17000 17033, 2020.
Max Ku, Dongfu Jiang, Cong Wei, Xiang Yue, and Wenhu Chen. Viescore: Towards explainable metrics for conditional image synthesis evaluation. arXiv preprini arXiv:2312. 14867 2023.
Hugo Laurençon, Léo Tronchon, Matthieu Cord, and Victor Sanh. What matters when building vision-language models?, 2024.
Yixuan Li, Xuelin Liu, Xiaoyang Wang. Bu Sung Lee, Shiqi Wang, Anderson Rocha, and Weisi Lin. Fakebench: Probing explainable fake image detection via large multimodal models, 2024a.
URL https://arxiv org/ at////b//2244 13306
Yixuan Li, Xuelin Liu, Xiaoyang Wang, Shiqi Wang, and Weisi Lin. Fakebench: Uncover the achilles' heels of fake images with large multimodal models. arXiv preprint arXiv:2404.13306, 2024b.
Chen-Hsuan Lin, Jun Gao, Luming Tang, Towaki Takikawa, Xiaohui Zeng, Xun Huang, Karsten Kreis, Sanja Fidler, Ming- Yu Liu, and Tsung- Yi Lin. Magic3d: High-resolution text-to-3d con- tent creation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 300-309, 2023.
Haohe Liu, Zehua Chen, Yi Yuan, Xinhao Mei. Xubo Liu, Danilo Mandic, Wenwu Wang, and Mark D Plumbley. Audioldm: Text-to-audio generation with latent diffusion models. arXiv preprint ar arXiv:2301.12503, 2023a.
Haohe Liu, Yi Yuan, Xubo Liu, Xinhao Mei, Qiuqiang Kong, Qiao Tian, Yuping Wang, Wenwu Wang, Yuxuan Wang, and Mark D Plumbley. Audioldm 2: Learning holistic audio generation with self supervisdd pretraining. /EEE/ACM Transactions on Audio, Speech, and Language Pro- cessing, 2024a.
Haotian Liu, Chunyuan Li, Qingyang Wu, and Yong Jae Lee. Visual instruction tuning, 2023b.
Tianchi Liu, Lin Zhang, Rohan Kumar Das, Yi Ma, Ruijie Tao, and Haizhou Li. How do ral spoofing countermeasures detect partially spoofed audio? arXiv preprint arXiv:2406.0244 2024b.
Yuan Liu, Cheng Lin, Zijiao Zeng. Xiaoxiao Long, Lingjie Liu, Taku Komura, and Wenping Wang. Syncdreamer: Generating multiview-consister images from a single-view image. arXiv preprint N.XV:22309.03453. 2023c.
Xiaoxiao Long, Yuan-Chen Guo, Cheng Lin, Yuan Liu, Zhiyang Dou, Lingjie Liu, Yuexin Ma, Song-Hai Zhang, Mare Habermann, Christian Theobalt, et al. Wonder3d: Single image to 3d using cross-domain diffusion. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 9970-9980, 2024.
Cheng Lu, Yuhao Zhou, Fan Bao, Jianfei Chen, Chongxuan Li, and Jun Zhu. Dpm-solver++: Fast solver for guided sampling of diffusion probabilistic models, 2023a. URL https://arrx
org/abs/233 20bb/221111 01095
Pan Lu, Swaroop Mishra, Tanglin Xia, Liang Qiu, Kai- Wei Chang, Song-Chun Zhu, Oyvind Tafjord, Peter Clark, and Ashwin Kalyan. Learn to explain: Multimodal reasoning via thought chains for science question answering. Advances in Neural Information Processing Systems, 35:2507-2521,
2022.
15

---

Published as a conference paper at ICLR 2025
Zeyu Lu, Di Huang, LEI BAI, Jingjing Qu, Chengyue Wu, Xihui Liu, and Wanli Ouyang. Seeing is not always believing: Benchmarking human and model perception of ai ai-generated images. In A. Oh, Т. Naumann, A. Globerson, K. Saenko, M. Hardt, and S. Levine (eds.), Advances in Neural Information Processing Systems, volume 36, pp. 25435-25447 Curran Associates, Inc., 2023b. URL hcctps://proceeednggg neur paa cc/ paper_ firerr__lllee files /
OOC11L1400SS10000001111117222000 PatperaaaaaaPparaaaaaaaaaaeeeee Datasets and
Benchmarks pdf
Zeyu Lu, Di Huang, Chunli Zhang, Chengyue Wu, Xihui Liu, Lei Bai, and Wanli Ouyang. Sentry- Sentry 2023c.
image leaderboard. hnttp::/ com/ Inf imaginnee / Seeggginn//ettyyy
Gonzalo Martínez, Lauren Watson, Pedro Reviriego, José Alberto Hernández, Mare Juarez, and Rik Sarkar. Towards understanding the interplay of generative artificial intelligence and the inter- net. In International Workshop on Epistemic Uncertainty in Artificial Intelligence, pp. 59-73. Springer, 2023.
Nichols Michelle. Un security council meets for the first time on ai risks. Reuters. Last accessed, 15, 2023.
Mistral. Large
enough, 2024.
URL
mistral- 1arge-24077 Accessed: 2024-09-13.
Mekhail Mustak, Joni Salminen, Matti Mäntymäki, Arafat Rahman, and Yogesh K Dwivedi. Deep- fakes: Deceptions, mitigations, and opportunities. Journal of Business Research, 154:113368, 2023.
Eliya Nachmani and Lior Wolf. Unsupervised singing voice conversion. arXiv preprint arXiv:1904.06590, 2019.
Utkarsh Ojha, Yuheng Li, and Yong Jae Lee. Towards universal fake image detectors that generalize across generative models. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition, pp. 24480-24489, 2023.
OpenAl. Hello gpt-4o.
http://paaa..ccm/neeeee///llloe/ 2024.
OpenAI. Sora: Creating video from text, 2024. URL URL 2024-09-21.
Accessed:
Eleftheria Papageorgiou, Christos Chronis, Iraklis Varlamis, and Yassine Himeur. A survey on the use of large language models (Ilms) in fake news. Future Internet, 16(8):298, 2024.
Daniel S Park, William Chan, Zhang, Chung-Cheng Chiu, Barret Zoph, Ekin D Cubuk, and Quoc V Le. Specaugment: A simple data augmentation method for automatic speech recognition. arXiv preprint 190:.iv::9040000 08779, 2019.
Taesung Park, Alexei A Efros, Richard Zhang, and Jun-Yan Zhu. Contrastive learning for unpaired image-to-image translation. In Computer Vision-ECCV 2020: 16th European Conference, Glas- gow; UK, August 23-28. 2020, Proceedings, Part IX 16, pp. 319-345. Springer, 2020.
Maria Pawelec. Deepfakes and democracy (theory): How synthetic audio-visual media for disinfor- mation and hate speech threaten democratic functions. Digital society, 1(2):19, 2022.
Yuang Peng, Yuxin Cui, Haomiao Tang, Zekun Qi, Runpei Dong, Jing Bai, Chunrui Han, Zheng Ge, Xiangyu Zhang, and Shu-Tar Xia. Dreambench++: A human-aligned benchmark for personalized image generation. arXiv preprint arXiv:2406. 16855, 2024.
Ben Poole, Ajay Jain, Jonathan T Barron, and Ben Mildenhall. Dreamfusion: Text-to-3d using 2d diffusion. arXiv preprint arXiv:2209. 14988, 2022.
Alec Radford, Jong Wook Kim, Tao Xu, Greg Brockman, Christine McLeavey, and Ilya Sutskever. Robust speech recognition via large-scale weak supervision. In International conference on ma- chine learning, pp. 18492-28518 PMLR, 2023.
16

---

Published as a conference paper at ICLR 2025
Suman Ravuri and Oriol Vinyals. Classification aceuracy score for conditional generative models. Advances in neural information processing systems, 32, 2019.
Yi Ren, Yangjun Ruan, Xu Tan, Tao Qin, Sheng Zhao, Zhou Zhao, and Tie- Yan Liu. Fastspeech: Fast, robust and controllable text to speech. Advances in neural information processing systems, 32, 2019.
Dario Rethage, Jordi Pons, and Xavier Serra. A wavenet for speech denoising. In 2018 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 5069-5073. IEEE, 2018.
Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Björ Ommer. High- resolution image synthesis with latent diffusion models. In Proceedings of the IEEE/CVF confer- ence on computer vision and pattern recognition, pp. 10684-10695, 2022.
Christoph Schuhmann, Romain Beaumont, Richard Vencu, Cade Gordon, Ross Wightman, Mehdi Cherti, Theo Coombes, Aarush Katta, Clayton Mullis, Mitchell Wortsman, al. Laion-5b: An open large-scale dataset for training next generation image-text models. Advances in Neural Information Processing Systems, 35:25278-25298 2022.
Rui Shao, Tianxing Wu, and Ziwei Liu. Detecting and grounding multi-modal media manipulation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp- 6904-6913, 2023.
Rui Shao, Tianxing Wu, Jianlong Wu, Liqiang Nie, and Ziwei Liu. Detecting and grounding multi- modal media manipulation and beyond. IEEE Transactions on Puttern Analysis and Machine Intelligence, 2024.
Jonathan Shen, Ruoming Pang, Ron Weiss, Mike Schuster, Navdeep Jaitly, Zongheng Yang, Zhifeng Chen, Yu Zhang, Yuxuan Wang, Rj Skerrv-Ryan, et al. Natural tts synthesis by con- ditioning wavenet on mel spectrogram predictions. In 2018 IEEE international conference on acoustics, speech and signal processing (ICASSP), pp. 4779-4783. IEEE, 2018.
Michael Shulman, Georg Kuesko, Martin Camacho, and Keenan Freyberg. Suno, 2022. URL
https://suno.. com
Ilia Shumailov, Zakhar Shumaylov, Yiren Zhao, Nicolas Papernot, Ross Anderson, and Yarin Gal. Ai models collapse when trained on recursively generated data. Nature, 631(8022)7555-55, 2024.
Berrak Sisman, Junichi Yamagishi, Simon King, and Haizhou Li. An overview of voice conversion and its challenges: From statistical modeling to deep learning. /EEE/ACM Transactions on Audio, Speech, and Language Processing, 29:132-157, 2020.
Gowthami Somepalli, Vasu Singla, Micah Goldblum, Jonas Geiping, and Tom Goldstein. Diffusion art digital forgery? investigating data replication in diffusion models. In Proceedings of the JEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 6048-6058, 2023.
Haixu Song, Shiyu Huang, Yinpeng Dong, and Wei- Tu. Robustness and generalizability of deepfake detection: A study with diffusion models, 2023.
Jiaming Song, Chenlin Meng, and Stefano Ermon. Denoising diffusion implicit models. arXiv preprint arXiv:2010.02502 2020a.
Xingcheng Song, Zhiyong Wu, Yiheng Huang, Dan Su, and Helen Meng. Specswap: A simple data augmentation method for end-to-end speech recognition. In Interspeech, pp. 58 1855,, 2020b.
Guangzhi Sun, Wenyi Yu, Changli Tang, Xianzhao Chen, Tian Tan, Wei Li. Lu Lu. Zejun Ma, Yuxuan Wang, and Chao Zhang, video-salmonn: Speech-enhanced audio-visual large language models. arXiv preprint arXiv:2406.1 /5704, 2024.
svc-develop team. so-vits-sve: A github project for voice conversion. 2024. Accessed: 2024-10-40 10-01.
com/
17

---

Published as a conference paper at ICLR 2025
Alon Talmor, Jonathan Herzig, Nicholas Lourie, and Jonathan Berant. Commonsenseqa: A question answering challenge targeting commonsense knowledge. arXiv preprint arXiv: arXiv:1811.00937, 2018.
Jiaxiang Tang, Jiawei Ren, Hang Zhou, Ziwei Liu, and Gang Zeng. Dreamgaussian: Generative gaussian splatting for efficient 3d content creation. arXiv preprint arXiv:2309.16633 2023.
Diangarti Tariang, Riccardo Corvi, Davide Cozzolino, Giovanni Poggi, Koki Nagano, and Luisa Verdoliva. Synthetic image verification in the era of generative artificial intelligence: What works and what isn't there yet. IEEE Security & Privacy, 2024.
Gemini Team, Rohan Anil, Sebastian Borgeaud, Yonghui Wu, Jean-Baptiste Alayrac, Jiahui Yu, Radu Soricut, Johan Schalkwyk, Andrew M Dai, Anja Hauth, et al. Gemini: family of highly capable multimodal models. arXiv preprint arXiv:2312.11805, 2023.
Llama Team. The Llama 3 Herd of Models, 2024. URL /ttp//////e///.aaaaaaaaaao////ea htt.://a..mttaccmmeeeeeeeeerr publications the 1lama 3 herd of models/ of-deelss
ThsDataBeast. Ted talk transcripts - 2001) 2021. URL httpp:://www.aagglll....
Timedomain. Ace studio, 2023. URL htt:ps:
Accessed: 2024-09-21.
Trapoom Ukarapol and Kevin Pruvost. Gradeadreamer: Enhanced text-to-3d generation using gaus- sian splatting and multi-view diffusion. arXiv preprint h0t::2406.09850, 2024.
Aaron Van Den Oord, Oriol Vinyals, al. Neural discrete representation learning. Advances in neural information processing systems, 30, 2017.
Vivek Verma, Eve Fleisig, Nicholas Tomlin, and Dan Klein. Ghostbuster: Detecting text ghostwrit- ten by large language models, 2024. BtLLtttp::// http:////wwwwwwrg//aaaaea hbt.///aaa//./....ee 15047.
Peng Wang. Shuai Bai, Sinan Tan, Shijie Wang. Zhihao Fan, Jinze Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, Yang Fan, Kai Dang, Mengfei Du, Xuancheng Ren, Rui Men, Dayiheng Liu, Chang Zhou, Jingren Zhou, and Junyang Lin. Qwen2-vl: Enhancing lisionnlanguage model's perception of the world at any resolution. arXiv preprint arXhv:2409.121999 2024.
Sheng- Yu Wang, Oliver Wang, Richard Zhang, Andrew Owens, and Alexei A Efros. Con-generated images are surprisingly easy to spot... for now. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 8695-8704, 2020a.
Xin Wang, Junichi Yamagishi, Massimiliano Todisco, Hector Delgado, Andreas Nautsch, Nicholas Evans, Md Sahidullah, Ville Vestman, Tomi Kinnunen, Kong Aik Lee, Lauri Juvela, Paavo Alku, Yu-Huai Peng, Hsin-Te Hwang, Yu Tsao, Hsin-Min Wang, Sebastien Le Maguer, Markus Becker, Fergus Henderson, Rob Clark, Yu Zhang, Quan Wang. Ye Jia, Kai Onuma, Koji Mushika, Takashi Kaneda. Yuan Jiang, Li-Juan Liu, Yi-Chiao Wu, Wen-Chin Huang, Tomoki Toda, Kou Tanaka, Hirokazu Kameoka, Ingmar Steiner, Driss Matrouf, Jean-Francois Bonastre, Avashna Govender, Srikanth Ronanki, Jing-Xuan Zhang, and Zhen-Hua Ling. Asvspoof 2019: A large-scale public database of synthesized, converted and replayed speech, 2020b. URL htRpstts:: htt.tpsss//aii....tttt.tt http///axxvv.rr////// abs / 1911.1 01601
Jason Wei, Yi Tay, Rishi Bommasani, Colin Raffel, Barret Zoph, Sebastian Borgeaud, Dani Yo- gatama, Maarten Bosma, Denny Zhou, Donald Metzler, al. Emergent abilities of large language models. arXiv preprint arXiv:2206.074 2022a.
Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Fei Xia, Ed Chi, Quoe V Le, Denny Zhou, et al. ofaiin-f-ttouughh prompting elicits reasoning in large language models. Advances in neural information processing systems, 0124824-248377 2022b.
Scott Workman, Richard Souvenir, and Nathan Jacobs. Wide-area image geolocalization with aerial reference imagery. In IEEE International Conference on Computer Vision (ICCV), pp. 1-9, 2015. doi: 10.1 109/CCC.20155.55 Acceptance rate: 30.3%.
18

---

Published as a conference paper at ICLR 2025
Tong Wu, Jiarui Zhang, Xiao Fu, Yuxin Wang. Jiawei Ren, Liang Pan, Wayne Wu, Lei Yang, Jiaqi 3d object dataset for realistic percep-
Wang, Chen Qian, et al. Omniobject3d: tion, reconstruction and generation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 803-814, 2023.
Tong Wu, Guandao Yang, Zhibing Li, Kai Zhang, Ziwei Liu, Leonidas Guibas, Dahua Lin, and Gor- don Wetzstein. Gpt-4v(ision) is a human-ald human-aligned evaluator for text-to-3d generation. In CVPR, 2024a.
Tong Wu, Guandao Yang, Zhibing Li, Kai Zhang. Ziwei Liu, Leonidas Guibas, Dahua Lin, and Gordon Wetzstein. Gpt-4y (ision) is human-aligned evaluator for text-to-3d generation. In Pro- ceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 22227- 22238, 2024b.
Bright Xu. NIp chinese corpus: Large scale chinese corpus for nlp. September 2019. URL https: // //doi. org/ 10. 5281/ zenodo... /ww1//enodo.....oooooeeeeeaoeoom
3402023
Danni Xu, Shaojing Fan, and Mohan Kankanhalli. Combating misinformation in the era of gener- ative ai models. In Proceedings of the 31st ACM International Conference on Multimedia, pp. 9291-9298, 2023.
Shilin Yan, Ouxiang Li, Jiayin Cai, Yanbin Hao, Xiaolong Jiang, Yao Hu, and Weidi Xie. A sanity check for ai-generated image detection. arXiv preprint arXiv:2406. /9435, 2024.
Ziqin Yang, Fuxin Xie, Jian Zhou, Yuan Yao, Cheng Hu, and Baoding Zhou. Aigdet: Altitude- information guided vehicle target detection in uav-based images. IEEE Sensors Journal, 2024.
Junyan Ye, Jun He, Weijia Li, Zhutao Lv, Jinhua Yu, Haote Yang, and Conghui He. Skydiffu- sion: Street-to-satellice image synthesis with diffusion models and bev paradigm. arXiv preprint arXiv:2408. 01812, 2024.
Taoran Yi, Jiemin Fang, Guanjun Wu, Lingxi Xie, Xiaopeng Zhang, Wenyu Liu, Qi Tian, and Xinggang Wang. Gaussiandreamer: Fast generation from text to 3d gaussian splatting with point cloud priors. arXiv preprint arXiv:23 10.02222 2023.
Jiquan Yuan, Xinyan Cao, Changjin Li, Fanyi Yang, Jinlong Lin, and Xixin Cao. Pku-i2iqa: An image-to-image quality assessment database for generated images, 2023.
Xiang Yue, Yuansheng Ni, Kai Zhang, Tianyu Zheng, Ruoqi Liu, Ge Zhang, Samuel Stevens, Dongfu Jiang, Weiming Ren, Yuxuan Sun, et al. Mmmu: A massive multi-discipline multi- modal understanding and reasoning benchmark for expert agi. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 9556-9567, 2024.
Yongyi Zang, Jiatong Shi, You Zhang, Ryuichi Yamamoto, Jionghao Han, Yuxun Tang. Shengyuan Xu, Wenxiao Zhao, Jing Guo, Tomoki Toda, et al. Ctrsvdd: A benchmark dataset and baseline analysis for controlled singing voice deepfake detection. arXiv preprint arXiv:2406.02 02438, 2024.
Yuexiang Zhai, Shengbang Tong, Xiao Li, Mu Cai, Qing Qu, Yong Jae Lee, and Yi Ma. In- vestigating the catastrophic forgetting in multimodal large language models. arXiv preprint 1oXiv:3309.1033. 2023.
Lingzhi Zhang, Zhengjie Xu, Connelly Barnes, Yuqian Zhou, Qing Liu, He Zhang, Sohrab Amirgh- odsi, Zhe Lin, Eli Shechtman, and Jianbo Shi. Perceptual artifacts localization for image synthesis tasks. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 7579- 7590, 2023a.
Longwen Zhang, Ziyu Wang, Qixuan Zhang, Qiwei Qiu, Anqi Pang, Haoran Jiang, Wei Yang. Lan Xu, and Jingyi Yu. Clay: A controllable large-scale generative model for creating high-qualit 3d assets. ACM Transactions on Graphics (TOG), 43(4):1-20. 2024a.
Pan Zhang, Xiaoyi Dong, Bin Wang, Yuhang Cao, Chao Xu, Linke Ouyang, Zhiyuan Zhao, Shuan- grui Ding, Songyang Zhang, Haodong Duan, Wenwei Zhang, Hang Yan. Xinyue Zhang, Wei Li, Jingwen Li, Kai Chen, Conghui He, Xingcheng Zhang, Yu Qiao, Dahua Lin, and Jiaqi Wang. InternIm Internlm-xcomposeer A lision-language large model for advanced text- text-image comprehension and composition. arXiv preprint arXiv:2309. 15/ 12, 2023b.
19

---

Published as a conference paper at ICLR 2025
Qihui Zhang, Chujie Gao, Dongping Chen, Yue Huang, Yixin Huang, Zhenyang Sun, Shilin Zhang, Weiye Li, Zhengyan Fu, Yao Wan, and Lichao Sun. LIm-as-a-coauthor Can mixed human- written and machine-generated text be detected?, 2024b.
Zangwei Zheng, Xiangyu Peng, and Yang You. Open-sora: Democratizing efficient video pro- duction for all, 2024. URL
-oppra Accessed:
2024-09-21.
Mingjian Zhu, Hanting Chen, Qiangyu Yan, Xudong Huang, Guanyu Lin, Wei Li, Zhijun Tu, Hailin Hu, Jie Hu, and Yunhe Wang. Genimage: A million-scale benchmark for detecting ai-generated image, 2023.
Sijie Zhu, Taojiannan Yang, and Chen Chen. Vigor: Cross-view image geo-localization beyond one- to-one retrieval. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition, pp. 3640-3649, 2021.
Giada Zingarini, Davide Cozzolino, Riccardo Corvi, Giovanni Poggi, and Luisa Verdoliva. M3dsynth: A dataset of medical 3d images with ai-generated local manipulations, 2024. URL https: /taps:///axivv org/ abs 2tt.///3...227. 07973
20

---

Published as a conference paper at ICLR 2025
LOKI: A Comprehensive Synthetic Data Detection Benchmark using Large Multimodal Models
Supplementary Material
Table of Contents in Appendix
Synthetic Data Detection
22
A.1 Social Impact of Synthetic Data
22
A.2 Synthetic Data Contamination
22
Increasing Attention on Synthetic Data Detection
23
B
Dataset Description
24
Data Collection
24
B.2
Dataset Annotation
26
................................
B.2.1
Annotation Guidelines
26
B.2.2
Annotator Informed Consent
27
B.3
Quality Control and Validation
27
B.4
Special data description
28
Evaluation
29
Evaluation Model
29
Evaluation Metrie
30
Breakdown Results Different Modalities
31
D. Video
31
D.2 Image
................................
D.3 3D
31
D.4 Audio
31
D.5 Text
32
More experimental results and discussions
38
E.1
Compression artifact tests
38
Deepfake detection
38
E.3
More CoT experiments results
39
Performance across Different Levels and Modalities
40
Case Study
41
21

---

Published as a conference paper at ICLR 2025
A SYNTHETIC DATA DETECTION
In this appendix, we introduce and discuss the social impacts of synthetic data, such as deepfakes, as well as data contamination introduced by synthetic data. Finally we discuss the increasing attention on synthetic data detection.
A.1 SOCIAL IMPACT OF SYNTHETIC DATA
While synthetic data generated by AIGC technology has offered numerous benefits to various as- pects of society, it has also introduced significant challenges and risks. One of the most concerning risks the potential to use synthetic data to create deepfakes. All forms synthetic data can be leveraged to generate deepfakes, which can then be used to deceive, manipulate, defraud individ- uals or organizations (see Fig (6. For instance, synthetic text data can be exploited to create fake al. (2024). phishing emails, manipulative advertisements. Similarly, syn- thetic image and 3D data can be used to generate realistic fake face<Xu et al. (0023) scenes, or even content that leads to copyright violations Diang al. (2023b); [Somepalli video data poses threat by enabling the production of fake videos or fake news (c.g.. political propaganda| Pawelec (2022)), well deepfake video fraud calls Mustak al. (2023). Likewise, synthetic audio data can be used for fake calls, voice cand even fake broadcasts. Furthermore, the advancements in synthetic data technologies are also impacting employment in creative industries, exemplified by the months-long strikes in the film industry BBohacek & Farid (2024).
news Papageorgiou
al. (023). Synthetic
model 0PF-3 (dio/efonms better than humans
Misoalina
Fale
Maniudaiiee
Al will make scam emails look genuine,
Now Tecadle Das Mac Boarr Latendt
Pale News
Video
Thect
Plisting Eraads
UK cybersseecurly agency
Fraad Cills
Palen News
MCSC poetrattee su vace ersncr beacc whw.siadaaighheeeeeo
Desplik
Imagee
Emaga/ /3D
Audio
Pake Souncs
Deepfaite Audio Of Philippino President Urging Millitary Action Against China Sparks Concerns
Copyright Viabiio
Pake Invoccart
Figure 6: Soical impact of synthetic data across different modalities
A.2 SYNTHETIC DATA CONTAMINATION
Original
Gan
Gan
Emarglend tus ne of T- Thee swild afonnd Hold rediige dagel
produre
Trgeas dunte fatle 180 typee wo sodiepell wore
Wilgt atter agjat onggent
fnttalld an bailing
perbll
corty
Outpess
werking meegle Peopendi Pevamallett Bannall kentee the
One enc aeneraa trow perpert
81
thadio fome 'Wore the nims of Pope bavent bes it pos tayy rair heve boca butt
of seneee en Pheer sue
There vndeee
tnggis thoel Comman batles wilk Peroggeee Dhrrt, Seeclee
Gen bentert Ehachea bnteriee dide Basniea Airals ltrope
Thams Wright
L'see it jurt fooe wost
aided dhe lane you
Cins desice
wkis d-0 salle leguns pepolation jatosiiaee sudee jolredd vello
Figure 7: Model Performance Collapse Trained On Synthetic Data (Image from Martínez et al. (2023), Text from Shumailov et al. (2024)
In today's LLM era, the internet is flooded with substantial amount of synthetic data, existing web-scale datasets are known to contain synthetic content ISchuhmann et al.] (2022). According openai Altman (2024), they now generate about 100 billion words per day, while all people
22

---

Published as a conference paper at ICLR 2025
earth generate about 100 trillion words per day. All of this points the fact that synthetic data will dominate the internet data side.
The of synthetic data has been shown to significantly degrade the performance of deep learning models (see Fig 7) both for generation tasks and classification tasks Hataya et al. (2023); Ravuri
& Vinyals 2019)) Martínez et al. (2023); Shumailov et al. (024);; Bohacek & Farid (2023). Ad- dressing the impact of synthetic data is crucial for the development of the next generation of models. There are two primary approaches mitigating the negative effects of synthetic data. The first is exploring ways to better utilize synthetic data, proposing strategies for optimizing the integration of synthetic data into training pipelines Dohmatob al. (2024a b): Feng al. (2024); [Bertrand et al. (224);; He et al. (2023). The second approach involves developing methods to accurately detect synthetic data, allowing models to distinguish between real and synthetic inputs.
A.3 INCREASING ATTENTION ON SYNTHETIC DATA DETECTION
3030
2738
Publbatton Caurt Dreepase Detection
2520
Df
2020
1655
1500
747
2018
2000
2022
2023
2013 2019
2002
2002
2023
(a) The number BBC official news reports on deepfake topic over the years.
(b) number of publications on deepfake tection over the years. (From Gong Li (2024)
Figure 8: The rising concern of deepfakes in both media and academic research.
The growing prevalence of synthetic data has garnered increasing attention from society, including news reports, academic research, and government policies. The number of papers deepfake detection has been steadily increasing, with the BBC reporting on deepfakes more frequently each year (see Fig { In response the rise of synthetic data, several governments and global conferences have also introduced policies aimed at regulating the use of deepfakes and synthetic data AIS 2003); Michelle 2023): Biden (2023).
23

---

Published as a conference paper at ICLR 2025
B DATASET DESCRIPTION
B. DATA COLLECTION
Our data primarily originates from online internet collections, reused from public datasets, and self- synthesized into composite data, detailed in Table 6 To ensure diversity in synthetic data, each modality incorporates more than five different synthesis methods (Figure[)) TO). To guarantee the quality of synthetic data, also collected samples synthesized by mature proprietary models such as Sora, Midjourney, CLAY, Suno, and GPT-4. The far-right column of the table displays the public datasets that underpin our collected synthetic authentically paired data.
Table Synthetic Methods and Public Datasets Across Modalities
Modality
Public
Video
Open-cara (Zoing 24 31 2024 Runway, W.A.L.T CCupla Sapiee 2025
OpennA
Kelino-
Lumiere Rac-al
Image
DPM+
Stable Diffusiur
FLUX, DDIM Song al
M023a
ADM Dhurrwal
2000a]
(Bliitman
Midioumey Al 2023
Stylegun3 Karra Milammm
Genlmand Zin 2023
121QA Yoom et al 2023
CVL
FFIQ
Sentry Lart 2023g
2015
Stylegan Narpe 2019a
Bolla kir 2017
CUT (Park
2000
M3DSynth Zingrini et aaal NFO MeDoe Frin
Deepfakerng Ming el MIAS VIGOK V160K77mmm (al 422
CLAY Zhang al. 2024z
SyncDreamer
[Liu et
2023g
Fantosia:
Mapic!D
DreamFusion
DreamCGnnseii
OmniObjeztiD Wa
2023
Long
Ckaapol Pravosi
2029
Sunn, WineNet Rethane ot al
2018
WaveRNN
Kalchnemer
2011
Tacotron2
Piray
ASVSSpol200 bi2th
Audio
DiffSSnnger
DDilwwr Kachal
(Vi-dleriipp
20210
DCCAS2003 Tae
Zanger 2025
VQ-VAE
Die Untu
201
Music ago Caw.aaaaoAAgoaannn Agostidell
Audiol DM 202-
[Ja
3021
Coperer
[ama3, 1-05 [Tom 2024
GPT-4u
OpenAl
N24
TheDataBuut [TheDataeass) 2021 Mixert Mizeet/hnng et 2024
Text
Qwen-MMxx
Bai ar X15
Naterlll arge
Miiig pi mi
M23
(jlmmsseoomn crri 2hy
Clande-3 3.5 3.5-Somme
Amiling 2144
Gemini-1 S-run
2025
ghosibusterddnn Verma 3044
Animal
Object
Person
Animal
Object
Person
Scenery
Scenery
Satellite
Satellite
Medical
Document
NeRF-based
221
GS based
Others
Medical
Document
Real
F-bsed
Gaussian-based
Others
Figure 9: Examples of some 3D and image datasets, with the bar chart showing the quantity of data in different categories.
24

---

Published as a conference paper at ICLR 2025
Authentic pairing data: We have collected significant amount of authentic paired data from the internet, including sources such as arXiv, Wikipedia, Gutenberg, YouTube, TikTok, and Civitai. For data sourced from the internet, text authored by human users, rather than content synthesized using Al technology. It is important note that current research primarily focuses multimedia data directly synthesized by AI, with limited consideration of methods like deepfake involving manual editing; we will continue to update our approach in future studies.
will rigorously verify that it consists of authentic recordings or
Data Availability and Social Impact: In collecting data, we strictly adhere to copyright and li- censing regulations of the source websites, avoiding data acquisition from resources that prohibit copying or redistribution. For the LOKI dataset, which is open-sourced, users must submit down- load request to the authors prevent misuse of the data.
SORA
Open-SORA
Runway
Aon Flov
Lumiere
CogVideo
Keling
WALT
Figure 10: Examples of video data. We used 7 video generation models to obtain corresponding data for LMMs evaluation.
25

---

Published as a conference paper at ICLR 2025
B.2 DATASET ANNOTATION
B.2.1 ANNOTATION GUIDELINES
During the annotation of synthetic videos, we categorize the identified anomalies into two types: global anomalies and segment anomalies. Global anomalies refer errors that persist for more than 80% of the video's s duration, while segment anomalies are issues that occur for limited portion of the video. For example, as shown in Fig. 11 (a), the anomaly of "flickering textures and distorted geometries of fences and utility poles." which is present throughout the video, is labeled as a global anomaly. In contrast, the "abnormal flames" and "basketball penetrating the hoop" in the video are classified segment anomalies. Additionally, each identified anomaly, including both global and segment anomalies, is associated with a key frame that represents the anomaly, facilitating subsequent processing of video data by large multimodal models (LMMs).
Video:
Image: For the synthetic image data, we provide global anomaly annotations for overall image is- sues, as well bounding box selections and textual descriptions for abnormal regions. The bound- ing boxes indicate the location and extent of the abnormal areas within the image, while the textual descriptions detail the specific anomalies present those regions. As shown in Fig. (b), the "tex- ture quality issues" and "color distortion" in the image are annotated as global anomalies, whereas area errors such as the "texture errors" of the duck and "reflection anomalies" on the water surface are classified as region anomalies. Annotators mark these areas by drawing bounding boxes and provide textual explanations for the reasons behind the anomalies.
Unlike video and image annotations, the annotation of 3D data involves a global-scale analysis of textures and normals. In terms texture anomalies, we focus on assessing the authen- ticity, smoothness, and edge clarity of the textures. For normals, analyze whether the model's geometric fluidity, surface smoothness, physical stability, and topological coherence are accurately represented. For instance, shown in Fig. (c), conduct a detailed analysis of the "multiview discrepancies" and "texture blurriness" in the model's 's textures, while labeling issues such as "abnor- mal protrusions" and "asymmetrical structures" related normals, accompanied by appropriate textual explanations.
3D Data:
(a)
Overalll 1- L-Anoorsal
Eugunt
texture geometry the tackground
Atauzal
appear
Als.rmul
010
0.77-80
0/71
1800
8/22
Abnormal Fragment
Abnornal Fragrent
passed through Tames viciiity the basket.
of fraket intcc and undamaged, the biskatbal passed drecty through
(b)
(c)
Overall Abnornal
The surfice texture of the duck tas poor carlly and finenss, the zucu and are large difetences cowrsl satuwiion to high, the coloris distited. and docs fot motch actall Fitetorr.
pictuee
Texture Abnormal
Norsal Abnoreal
Area Abnoreal
The sedure chersc white the soxl vey fom disrent anple The gue the front The edges off les sonorall tastune the logs the stecl
Thee abnomal and bugee en the sizels georaatic surfece sides the sttel asyreratiinl slaxlin of
The duck's feature abnurmal and does not makch the tortue. The nose stuctire wrong
Aree
sizzi lecuae
dans
In caln water wass, the rsf oction symmetrial with the cucc abese the water surfern end there cesrmmsmattt betweenn the rellecion the dack heel.
water sufface is not
Figure 11: Examples of the synthetic data annotation process under different modalities, including (a) Video, (b) Image, (c) 3D Data.
26

---

Published as a conference paper at ICLR 2025
B.2.2 ANNOTATOR INFORMED CONSENT
Before commencing the annotation process, we ensure that all participating annotators are fully informed and provide their explicit agreement to the following terms and conditions. This compre- hensive informed consent is designed to promote transparency, respect their autonomy, and align with ethical standards in research. It is imperative that each annotator has a thorough understanding of the nature, purpose, and potential implications of their contributions to the labeling process. The terms outlined as follows:
Data Usage. Annotators acknowledge and consent to the possibility that the labeled data they generate may be used various academic and scientific contexts, including the development of re- search papers, presentations at conferences, and other related scholarly activities. They understand that their work may significantly contribute to advancements in research fields such as natural lan- guage processing, machine learning, and artificial intelligence. Furthermore, annotators recognize that their contributions may be referenced or cited in scientific publications, thereby playing a role in shaping future research directions and applications.
Al-Generated Content. Annotators are informed that some of the content they will be labeling may have been produced by artificial intelligence models. This includes text, images, or other data types generated by algorithms designed simulate human-like outputs. Annotators understand that this knowledge is crucial, as it may influence their perception, judgment, and approach to the labeling task. They agree to remain mindful of the potential biases or preconceived notions that may arise from this awareness and commit to maintaining objectivity and accuracy in their work.
Potential Implications. Annotators are aware of the broader implications of their labeling ac- tivities, which extend beyond the immediate scope of data annotation. They recognize the ethical considerations inherent in AI research, particularly concerning issues such as bias, faimess, and the societal impact of deploying AI technologies. Annotators agree to reflect on these ethical dimensions and to engage in the labeling process with a conscientious approach, acknowledging that their work may contribute to both the positive advancements and challenges associated with Al development and implementation.
Commitment to Ethical Standards. By agreeing to these terms, annotators affirm their commit- ment to upholding high ethical standards throughout the annotation process. They understand that their participation is voluntary and that they have the right to withdraw from the project any time, without penalty, Annotators also acknowledge their responsibility to report any concerns or issues that may arise during the labeling process, ensuring the integrity and reliability of the data they provide.
This informed consent process ensures that all annotators equipped with a comprehensive un- derstanding of their role and its significance. aims to foster an environment of mutual respect and collaboration, where the contributions of annotators in research are protected. By clarifying the expectations and responsibilities involved, we seek to create a foundation for ethical and impactful research that benefits both the scientific community and society at large.
valued and their rights as participants
B.3 QUALITY CONTROL AND VALIDATION
In annotating videos, images, and 3D synthetic data for anomaly details, we maintain high standards and accuracy. All annotators possess at least university degree and demonstrate strong decision- making and judgment skills. Before annotation, human annotators receive extensive training with numerous examples of common errors to ensure a comprehensive understanding of the synthetic data detection task. Each data instance is annotated for detailed anomalies by at least two human annotators to ensure quality. Ambiguous or unclear instances are marked for further study and annotation during team meetings. Furthermore, avoid the impact of hallucinations from Large Language Models (LMMs) on tasks involving LMMs, all anomaly detail explanation tasks undergo manual review based on LabelLLM
27

---

Published as a conference paper at ICLR 2025
B.4 SPECIAL DATA DESCRIPTION
In the field of document images, we have collected synthesized images in four categories: news- papers, academic papers, magazines, and reconstructed documents. The corresponding real data for these categories come from the M6Doc dataset. Currently, document synthesis primarily follows layout-first, content-renderinglaan approach. For the first three document types, we design specific empirical rules for layout generation during the layout phase; during the content rendering phase, elements are selected from constructed corpus fill the structure. For the reconstructed type, we employ restructuring algorithm on the M6Doc dataset to rearrange the content of the documents.
In the field of remote sensing imagery, synthetic data is primarily generated based on street-to- satellite datasets such as CVUSA and VIGOR, utilizing methods like CUT and Skydiffusion. The synthetic imagery encompasses major natural scenes such as urban and suburban environments, with the satellite remote sensing images being of high resolution.
In the field of medical imaging, we primarily collected two types of data: the ISIC 2016 skin dataset and the M3DSynth CT dataset. For the ISIC 2016 dataset, we utilized GAN methods for direct data synthesis. The M3DSynth dataset comprises synthetic images generated from the real-world LIDC dataset using Diffusion and GAN models. The images are categorized into two types: those with real tumors removed and those with synthetic tumors artificially inserted by the model. Each synthetic image is paired with its corresponding original image, complete with precise annotations of the tumor insertion or removal locations. Considering that most users are not medically trained, we deliberately selected more evident abnormal images reduce the need for specialized knowledge in making decisions about synthetic data.
28

---

Published as a conference paper at ICLR 2025
C EVALUATION
C.I EVALUATION MODEL
We compared various models on the LOKI benchmark to understand their capabilities across mul- tiple tasks. We support over ten open-source models, including InternVL2 (Chen al. 2024),
LLaVA (Liu et al. (023b), Phi (Gunasekar et al. 2023). XComposer (Zhang et al. [023b).
Qwen2-
VL (Wang al., 2024), MiniCPM (Hu al. 2024), and Idefics2 (Laurençon et al. 2024), well
as proprietary models such as GPT-4 (OpenAl 2024), Gemini (Team et al.) 2023). Qwen-VI--Ma VL.-Max
Bai et al. 2023). and Claude (Anthropic 2024). The following list details these models.
Family
Version
Parumeters
Clove-ourred, API
GPT4
GPT-4o GPT-4
NIA
het For. www...oe..........o.....o....... openal .com model a /apt- 4o
St past /7pl Foom. pppenaa coom/ model ss?
gpt 4- Lurbo and gpc
Gemini
Gemini- Somini---.5PPp Pro-Pro
NAA
hptp://gnnnnnnn/nosss//eee /pl/doc///oocle/ /nodele/ geminitgenin-- 5-p20
bttpe://al.
N/A
https://a sttpa///...gggg.. ww..ool....ee..o...ooooo dwwee////annnn. ni-apl /docs /node a//
gomini
Claude
Clande335-Soonet
NIA
bttes://dosa anthboop..nttrrpico.... con/en/ docs/ abous: oll sude/model.
Mistral
Mistral-Large
NIA
https://docs mi stral ai ott.rroooi/epeettttrooooooooooo
Qwen
Qwen-Max
N/A
htt / //wwe al tbabae loud wow.too//oeelppppeeeeoom en/nodel 1- at oo daveloper waappeee--eeeeeeeeeeeeee zelerence/
Jee see-gwen-byy Ing-opt
Open-soucce
ULMMA
1LAMA-311-05B
405B
https:/ httpsss//hugngggaace comneta-- 1lans/hlame-3 4058
VL2-8B
SB
Sctee: /tc.g//cgggg face wttp://aaccceaaaaaeeeeeeeee http://aaaaaaaa/anaetarra 88
IntemVLL2
26B
Stt Sttgs://hc http://hhgggg.ccccoo//o
226
InternV1.2-46 12-40B
40B
stt /hogg inglace co/ CpenoV
Inlmm-L2-Lamm5-76B
76B
sttps: hhtp:///hhggggggggeeee co/openivian
Intervvi
L. VLaVA-OneViiooon
LLVAA-OteVision-18
Llava www.iolooeoooo.oooooooooooo..o.e www.oo--ooaaaaa.e qwenz- Tb-ov
/tlpgs///hhggggg fabe .o/ snne
LLaVA-QDeVlsson-728
72B
aLt //hug////uuggu ing fabe Imna Jab/
2vb-ov-22t
VILA-115-13B
13B
Sttps:/ http:://hgggttaatc...oo//tttoooa
VILA
VILA. -5-4b
VII.A-1.5-0RR
40B
Stt ott
VILA..5-105
Phi
Ph-vsonVission
3.5B
bttes: http://htttggcccccccccco http://cwa....ooooocooooottaaaa. vivision instruct
Melics
ideliox2-8b
8B
Sttpa:/ httsss//huugingfeee http://ee......oggogggoooooo htt.://ggggoggiggcceee.... idcfics2-90 9b
Qwen2-VL.
Qwen2-VL--7e
7B
bttea: / /hugging face .co//wen/
Uvee2
Qwen2: VL-72B
72B
bttee: / /hogg face co/Cven/
Qvenz 125
IntenLM XComposser
IntemL MXCalMMMMommmu222255555
bttpe:/ http://httggggg..cc// wtt.:/eecccccccocoooeeeeeee co/interni//
Intern Interrrln-xxoopeee www.pooe 22d5
mPL.UG-OWE
mpplag-oo13
78
het httpss//hhggg //uug mng faw.. co//et/p//
mFtin oftl0--0013 78 240728
MiniCPM
MiniCPM V2.6
8.1B
bttps:/ httpss//hhugggtaccc.oo// http:///ooooobb/tteeeeceeee
LongVILA
LoogVILA
SB
Satpa://huqgine wtt.://oooooooo./eaeeeee www.erreeeeeettioooo Large Model/
L. 1-mo----oonVV1 12mm
LongVA
LongVA-7B
7B
78-3P0
Qwen-Audio
Quen-Audio-Chal
78
SALMONN
SALMONN-7B
7B
bttpas//hugginne wtw.://aww.....//annnaaaaaaaaaaa ##/SAINON 78
AnyGPT
AnyGPT-CCai
7B
OneLLIM
OneL.LM-7B
78
httpss//hugginee
LTU
LTU-AS-7B
pretrasiaed-ndde1
29

---

Published as a conference paper at ICLR 2025
C.2 EVALUATION METRIC
Average accuracy: For judgment, multiple-choice, and detailed selection questions, we use the average accuracy as the primary metric. The accuracy rate calculated using the following formula:
Neurrect
Accuracy =
X 100%
Noota
In this context, Neereett is the number of correctly answered questions, and Nioil is the total number of questions. To minimize the influence of prompts model judgments, each question is presented two forms: one asks whether the data Al-synthesized or real, and the other asks the model to identify either the real Al-synthesized data. By averaging the accuracy rates across different forms of the questions, we aim to reduce the potential bias introduced by the phrasing prompts and ensure fair evaluation of the model's performance.
Normalized Bias Index (NBI): To evaluate whether there is potential bias existing models when determining authenticity on the LOKI benchmark, introduce metric termed the Normalized Bias Index (NBI) to quantify the performance differences the model on natural and Al-generated data across different modalities, which is defined follows:
Raatural
R Rgenerated
NBI
E [-
Raatural + Rgencrated
In this context, Rnatuaal and Reeneruted represent the recall rates for natural and Al-generated samples, respectively, under the corresponding modality. By normalizing the difference between the two, the model's 's unexpected preference making predictions can be quantified. Specifically, a positive and larger NBI indicates that the model is more biased toward predicting samples natural, whereas negative and smaller NBI suggests a bias toward predicting samples as Al-generated.
GPT-Score:
For open-ended questions regarding anomalous details, assess the score of the responses. We adopted rating scheme, establishing a 5-level rating system with scores ranging from (poor) to 5 (excellent). The final scores are normalized scale of 0 to 100. We adhere to the following scoring criteria:
the GPT-4 model to
1) Identification: Accurately detect the globally annotated anomalies and their corresponding de- tailed anomalous regions specified by human annotators.
Explanation: Provide accurate explanations for the causes of the anomalies, ensuring consistency with the reasons outlined in the human annotations.
3) Plausibility: Avoid misclassifying authentic regions anomalous while encouraging other rea- sonable explanations for anomalies.
While the scoring criteria similar across different modalities, they slightly adjusted according to their content characteristics; for example, the image modality is subdivided into global score and regional score, whereas 3D data subdivided into texture score and normal score.
Task Description:
We request assistance
Instructions
evaluating response of Al essistent to the question...
Evaluation Dimensions:
Global Score:
GPT-4o
Score
Reference Image
Synihetic Imagee
Completeness:
Evaluator
Final Scoring:
Global Score (4 Good) The ascistar edlectively expured
Corectness:
(4--4-55 Total:
inconciooacies,
Scoring Range:
1- P-Puur 2 Fair 3 Average
Plausibility:
Data
Answer
unnstical posce that nro othen chacecturieti of Al- generated images
inntthertie primurily due slight
Overall imagee
Reason:
mand of le rapion makked the humas sunotatirss, incloding the athioee', poso. nocket hand
noe the bleading iasass with the and sock
Carrectaese (4-Goodd) The seseantl explaration was mosly
Campleteness (4-Gond)
annotations. coneedl sterntte with Fomewerr
iduntifiedd
Good 5 Excellent
Plunsibility (5-R (5-eacellont) The sevenue dil not incoensctly idket suthertic natiors those merked.
Example:...
avuided
Figure 12: The overall process for automated evaluation of Abnormal explanation questions using GPT-40.
30

---

Published as a conference paper at ICLR 2025
D BREAKDOWN RESULTS ON DIFFERENT MODALITIES
In this appendix, we present detailed results of various synthesis methods or classification themes across different modalities of data.
D.1 VIDEO
Tables 7 display the evaluation results of LMMs in video modality for judgment and multiple- choice questions, respectively. In the evaluation of video modality, the proprietary model GPT-4o achieved an accuracy of 71.3% for judgment questions and 77.3% for multiple-choice questions. The open-source model Intern InternVL.2-40 VL2-40BB recorded accuracies of 62.0% for judgment questions and 65.7% for multiple-choice questions. Table presents the evaluation results of different models in the video modality for open-ended questions.
Additionally, using GPT-4's detection outcomes as a benchmark, we assessed the quality of videos synthesized by different models. It was found that videos generated by Sora had a judgment accuracy of 67.2% and a multiple-choice accuracy of 64.5%, indicating a high quality of video synthesis. Conversely, videos synthesized by Runway and W.A.L.T exhibited more noticeable synthetic traces.
D.2 IMAGE
Tables 10 and present the evaluation results of LMMs in the image modality for judgment and multiple-choiee questions, broken down by specific categories. The proprietary model GPT-4o scored judgment accuracy of 63.4%, primarily impacted by specialized image types such Satel- lite, Doc, and Medicine, while achieving a multiple-choice accuracy of 80.8%. Among open-source models, Qwen2-VL-72B performed exceptionally well, even surpassing GPT-4o in specialized im- age categories. Table [2 presents the evaluation results of different models in the Image modality for open-ended questions.
D.3
3D
Table 13 presents the evaluation results for different 3D synthesis methods. Tests on judgment ques- tions for 3D models reveal that most open-source models perform close to 50% random chance. In contrast, the proprietary model GPT-4o demonstrates better decision-making and judgment ca- pabilities, particularly in identifying low-quality Nerf models, where it achieves exceeding 75Assuming the performance of GPT-4o as benchmark, we can assess the quality of 3D data synthesized using different methods. Among three synthesis methods, Gaussian synthesis exhibits higher quality compared to Nerf synthesis, while commercial models like Clay offer the best quality. Table 14 presents the evaluation results of different models in the Image modality for open-ended questions.
accuracy rate
As shown in Table [5] extracted 200 3D objects, with equal ratio of real synthetic data, and evaluated the performance of different 3D input formats, including point clouds, surround videos with eight input frames, and multi-view inputs. In comparison, for current LMMs, the performance between surround video and multi-view inputs is similar. PointLLM, which partially supports point cloud inputs, performs particularly well, indicating that the point cloud format still holds certain ad- vantages. Additionally, our results indicate that compared to the 7B PointLLM, the 13B PointLLM more inclined respond with cannot determine whether this is a real object or one generated by AI" rather than a straightforward "Yes" or "No." We speculate that as the size of PointLLM in- creases, incorporates more comprehensive measures to correct illusions, making it less likely to provide definitive answers when faced with uncertainties (Wei et al. 2022a).
D.4 AUDIO
Table 16 presents the evaluation results of various audio LMMs the LOKI dataset. Currently, there are few open-source and proprietary LMMs that support the audio modality, and most models perform poorly synthetic data detection tasks, showing little difference from random selection. However, the AASIST method, trained specifically for speech tasks, demonstrates relatively strong performance, although it still underperforms in other modalities. Overall, human performance in
31

---

Published as a conference paper at ICLR 2025
the audio modality is comparatively weak relative to other modalities, yet still exceeds the best- performing LMMs by 20%.
It has been shown that acoustic features, such pauses between words and silent segments within audio, play a more critical role in detecting deepfake audio than the actual content of the audio et al. 2024b). However, current audio language models primarily focused content compre- hension rather than acoustic characteristics. From the model architecture perspective, audio lan- guage models typically initialize their audio encoders with pre-trained audio models such Whis- per(Radford et al., 2023), AST(Gong et al. 2021). or BEATss(Chen et al. 2023b). These pre-trained models are trained with various data augmentation techniques. including SpecAugment(Patt 2019) and SpecSwap(Song et al., 2020b). The target of the augmented data remains constant, of- ten focused on tasks like transeribing textual content, which shifts the model's ' attention towards understanding content rather than being sensitive to acoustic features. On the training data side, most audio-text datasets date emphasize content comprehension in tasks like speech recognition, emotion/age recognition, audio/music captioning, sound event classification and music genre recog- nition. As a result, these models lack the inherent capability to distinguish between real and fake audio based acoustic cues.
al.,
D.5 TEXT
Tables 17 and 18 present the test results across different genres in the text modality. Most models underperform this modality, primarily due the low distinctiveness of text data itself, coupled with the maturity of current text synthesis technologies. Notably, genres such as modern literature and philosophy may benefit from the extensive training and memory capabilities of GPT series models, showing relatively better performance compared to other categories.
In Table 19
we evaluated the detection results for texts of varying lengths. The findings indicate that longer texts are detected more frequently and are more likely reveal flaws. Additionally, we assessed the performance across different languages and found that, except for GPT-4o, most proprietary models perform significantly better in English than in Chinese. This indicates that the multilingual capabilities of large multimodal models require further development.
Table 7: Judgement questions results of different models on the LOKI Video modality. closed-source models.
denotes
Overall
Sora
Keling
Cog VideoX
Lumiere
Open-sora
Runway
W.A.L.T
Expert (AIGVDet)
39.5
65.0
54.2
37.1
MiniCPM-V-2.6
57.2
57.9
56.5
54.0
66.4
55.2
45.8
62.6
Phi-3.5-Visime
56.8
52.7
52.0
53.0
57.1
57.5
55.1
66.5
VLaVA-OneVision-7B
56.8
57.6
53.7
61.0
57.1
56.6
53.7
59.6
InternL.-XX 2Comporre222
58.4
55.2
61.1
63.0
60.0
57.1
49.5
62.6
mPLUGG-O/3-77B
55.3
56.7
54.3
54.0
56.4
54.2
45.4
61.3
Qwen2-VL-7B
59.5
60.1
58.5
54.0
59.3
51.4
59.7
Long LongWA-7B
66.2
Mantis.8R
Ideficx2-8B
54.8
56.1
$4.0
51.0
57.9
51.4
41.2
64.3
InternVL.2-8B
60.8
61.9
56.3
69.0
59.3
61.3
56.5
64.8
Llama-3-LongVILA--83
51.9
54.0
49.1
53.0
52.1
48.1
52.8
54.1
VILA1.5-13B
$1.9
$1.2
50.9
51.0
54.3
52.8
52,3
$1.9
InternVL.2.266 26B
55.0
55.2
53.4
61.0
57.9
$2.4
50.0
$8.2
VILA115-40B
59.2
54,9
57.4
66.0
61.4
59.9
54.2
64.8
VLterVVL2-40B
62.0
61.3
$7.7
68.0
60.7
61.8
62.0
63.9
Qwen2-V1--228
59.6
58.2
61.4
59.0
58.6
54.7
58.8
63.2
56.5
$7.0
62.0
46.8
Gemini- 1.5 Pro"
58.5
53.4
60.5
58.0
63.6
62.3
45.4
64.8
Claudo-3.56 5-Sonnttt-Sonett
61.7
60.1
57.4
59.0
68.6
64.2
49,5
71.2
GPT-4o
71.3
66.8
67.9
70.0
68.6
72.6
76,9
75.8
32

---

Published as a conference paper at ICLR 2025
Table 8: Multiple Choice questions results of different models on the LOKI Video modality. denotes the closed-source models.
Overall
Sora
Keling
Cog VideoX
Lumiere
Open-sora
Runway
W.A.L.T
MiniCPM-V-2.6
52.8
39.5
62.1
55.6
58.8
54.0
48.0
53.7
Phi-3.5.
55.3
InternL XComposer2.5
56.3
43.4
54.5
66.7
67.6
54.0
53.7
mPLL'G-Ow13-7
60.3
53,9
53.0
61.1
66.2
62.0
Qwen2-VL-7B
64.0
63.2
62.1
52.8
69.1
66.0
70.0
62.2
LongVA-78
57.5
59.2
50.0
58.3
63.2
52.0
58.0
59.8
Mantis-88 8B
47,9
56.6
43.9
50.0
52,9
$2.0
42,0
39.0
Ideficx2-BB
55.6
56,6
54.5
55.6
57.4
58.0
52.0
54.9
InterVVL--8B
54.0
53.9
57.6
55.6
61.8
48.0
52.0
48.8
Llama-3-LongVILA--BB
54.0
50.0
47.0
55.6
61.8
54.0
52.0
57.3
VILAI15-13B
52.1
51.3
53.0
55.6
60.3
54.0
50.0
43,9
VL2
61.8
VILA11.540B
ILtemWL2-40B
65.7
53.9
77.8
76.5
66.0
70.7
Owen2-VL-722
65.7
57.9
72.2
72.1
60.0
80.0
62.2
VLaVA-OneVission722
62.9
52.6
48.5
66.7
69.1
74.0
78.0
61.0
Gemini- 1,5 Pro"
66.1
46.1
60.6
66.7
75.0
64.0
72.0
79.3
3.5 Sonnet"
60.5
53,9
54.5
58.3
72.1
72.0
58.0
57.3
GPT-4o
77.3
61.8
71.2
75.0
82.0
Table 9: Abnormal Explanation questions results of different models the LOKI Video modality. denotes the closed-source models.
Model
Overall
Correctness
Explanation
Plausibility
LLaVA-OV-7B
46.7
41.0
48.4
50.8
Intern VL2-8B
46.5
38.0
48.4
53.2
Owen22-VL-7B-Instruuc
48.4
41.2
50.4
53.6
Claude-3-5-sonner
50.1
44.2
48.8
57.4
GPT-4o
67.6
60.6
69.2
73.0
33

---

Published as a conference paper at ICLR 2025
Table 10: Judgment questions results of different models on the LOKI Image modality. the closed-source models.
denotes
Overall
Seene
Animal
Person
Object
Medicine
Doc
Satellite
Random Choice
18.0
21.6
18.6
22.1
27.3
24.0
22.1
Expert (AIDE)
MiniCPM-V.2
44.8
52.0
34.4
53.1
31.5
53.8
51.5
38.3
Phi-3.5-Vision
52.5
50.8
41.7
71.5
34.1
57.3
54.3
60.5
LLaVA-OneVisionn-7
49.8
59.2
41.9
58.1
37.3
52.3
53.0
50.1
Intern. 5.
46.4
52.7
40.0
56.7
32.5
56.1
49.8
38.2
mPLUG-OWi3-7B
45.9
52.1
37.3
31.4
55.3
53.8
38.1
Owen2-VL-7B
LongVA-7B
46.2
57.6
49.8
Mantis-8B
54.6
54.9
53.1
51.9
Idefics2. 8B
45.0
51.8
35.3
$2.3
29.2
52.3
53.9
40.6
IntemV112-88
49.7
58.8
37.8
53,9
60.2
44.2
Llama-3-LongVILAA VILA-8B
49.8
49.8
50.5
50.6
47.2
50.0
49.9
50.0
VILAI15-13B
49.3
52.0
38.6
$4.2
31.0
50.1
56.6
62.4
IntemVLL2-26B
44.3
51.6
35.4
50.8
28.2
51.3
54.4
37.6
VILAI15-40B
48.8
53.7
39.3
50.0
33,4
52.5
59.9
50.6
Intern 40B
49.6
55.7
37.3
59.2
34.8
55,5
64.8
40.8
Owen2-VL-72B
53.2
55,9
73.7
LLaVA-OneVisiore
46.3
54.7
67.9
CIande-3.5-Somee
53.6
51.6
$1.6
55.2
51.4
51.9
59.1
50.9
Gemini-1.5-Pro
43.5
53.7
35.7
$1.5
30.3
50.0
47.2
38.1
GPT 46"
63.4
70.1
69.7
84.4
70.3
54.3
60.1
45.0
Table 11: Multiple Choice questions results of different models the LOKI Image modality. denotes the closed-source models.
Overall
Scene
Animal
Person
Object
Medicine
Doc
Satellite
MiniCPM-V-25
49.8
49.2
52.3
49.2
45.1
Phi-3.5 Vision
44.0
31.3
22.9
67.9
20.7
58.3
52.1
LLaVa-One Vision-7B
51.7
55.9
51.8
63.3
46.9
52.8
51.4
46.3
InterL MMemLLM-CCompeer2.55
51.0
48.4
49.5
46.3
50.5
57.0
52.0
$1.2
I.PLUG-OW!3.
52.5
53.1
53.5
53.8
49.5
55.5
49.4
$2.3
Qwen2- VL-7B
65.1
60.9
71.3
70.4
60.5
68.0
63.1
59.6
VA-7V
51.6
58.2
54.9
42.9
48.2
55,3
46.6
$1.6
Mantis-8B
61.5
66.0
58.1
62.3
68.3
Idefics2-8B
51.3
47.3
55.3
46.5
48.0
V1lemVV1..8B
1lama.34LangVll... V...888
VILAI.5-13B
55.3
57.4
$4.9
47.7
InterVLL2-26B
48.5
50.4
49.1
50.8
40.3
50.8
53.1
46.7
VILAI.5-40B
64.0
59.8
71.0
$8.8
53,8
71.5
62.9
63.8
Intern V1.2m/11.-400
63.1
60.9
72.0
62.5
62.5
62.3
61.6
56.8
Qwen2 VL-72B
68.6
60,9
71.1
60.8
68.9
74.3
77.7
61.0
Viaii-OneVisoon-222
70.8
65.6
62.3
76.7
67.1
73.0
77.6
72.4
Clmsde-3.5-Somee
65.5
68.6
Gemini-1.5-Pro
72.7
74.2
GPT-46"
80.8
79.7
92.3
91.3
94.9
65.3
81.7
34

---

Published as a conference paper at ICLR 2025
Table 12:
Abnormal Explanation questions results of different models on the LOKI Image modal- ity. denotes the closed-source models.
Overall
Global Score
Completeness
Correctness
Plausibility
63.8
66.6
60.8
53.2
74.4
Llava-OV-7B
68.9
72.4
63.0
60.6
79.8
InternVL2-83 VL2-8B
72.2
70.0
67.6
61.2
85.2
Claude-3-5"
1.6
Gemini-1.5-pro GPT-4o
77.1
77.6
70.8
90.2
72.9
69.8
68.6
63.4
Table 13: Judgment & Multiple Choice questions results of different models on the LOKI 3D modality. denotes the closed-source models.
Judgment
Multiple Choice
Nerf
Gaussian
Other
Overall
Nerf
Gaussian
Other
Overall
MiniCPM-V...
56.7
56.1
56.7
56.4
50.6
50.0
52.9
50.7
Phi-3.5-Vision
50.0
50.0
50.0
50.0
62.0
56.1
62.9
59.6
LLaVA Visionn2
69.3
67.4
69.0
68.4
52.5
54.6
54.6
53.8
IntemLM XCumposer2.5
42.8
44.6
44.4
43.9
41.2
51.0
55.4
48.0
mPLUG-OW(3-7B
49.1
50.5
49.8
46.6
52.5
49,9
Qwen2-VL-7B
72.1
73.2
70.4
72.3
53.2
55.0
55.5
VAn7VA-7B
Mantis-BB
50.0
50.0
50.0
50.0
51.3
73.0
60.4
62.5
Mefice2-8BB
38.5
38.1
39.0
38.4
51.7
54,2
54.2
IntemVLL-88B
48.8
49.8
49.6
49.4
52.7
54.3
51.3
53.1
Llama-S-LogVVIL--S
32.3
32.1
32,5
32.2
51.5
49.8
50.0
50.5
VIIAI1.5
32.3
35.5
33.8
34.0
49.0
57.3
53.8
53.5
IntemV1.2-266
50.3
50.4
50.4
50.4
44.8
51.2
51.7
48.8
VILAI.5-40B
50.0
50.0
50.0
50.0
44.1
50.2
50.4
47.9
IntemVL.2-40B
49,8
50.0
50.0
49.9
$4.0
63.6
63.8
59.9
Qwen2-VL-72B
60.9
60.3
59.0
60.3
56.5
57.5
58.7
LLaVA Vision
Claude 3.5-oonee
59.5
57.1
57.1
$8.0
52.3
52.0
50.8
$1.9
Gemini-1.5-Pro
60.5
50.7
56.0
55.4
64.3
56.0
61.7
60.2
GPT-4o
65.9
64.4
65.4
65.2
78.8
66.4
60.8
70.2
Human
87.3
78.4
71.8
82.0
94.5
90.2
78.6
91.3
Table 14: Abnormal Explanation questions results of different models the LOKI 3D modality. denotes the closed-source models.
Overall
Texture Correctuess
Normal Correctness
Texture Plausibility
Normal Plausibility
Llava-Ov-7B
71.0
70.8
73.8
76.2
IntemnVL2-8B
71.3
62.4
69.4
76.4
76.8
Qwen2-VL-7B.
73.4
64.8
72.2
77.8
78.6
Claude-3-S-somee
78.2
70.0
80.0
78.8
83.8
1.5pmn-1..5-mm
70.8
62.4
67.2
77.2
76.4
GPT-46"
77.0
64.2
82.4
81.4
Table 15: Judgment & Multiple Choice questions results of different models the LOKI 3D modality. S- V-Video denotes Surround Video, MV-Image denotes Multi-Views Image.
Judgment
Multiple Choice
Judgment PointCloud
V-Video
MV-Image
S-Video
MV-Image
LLa LLaVA-OneVisioon77
67.4
PointLLM-7B
61.9
Idefics2-8B
38.1
39.0
51.7
56.5
PointLLM-
16.9
LaVA-OOneisiion-P
49.6
51.9
63.0
58.3
OneLLM
55.6
35

---

Published as a conference paper at ICLR 2025
Table 16: Overall results of different models on the LOKI Audio modality.
Judyment
Multiple Choice
Speech (800)
Singing (800)
Audio (280)
Music
Overall (2280)
Speech (400)
Singing (400)
Andlo (140)
Music (300)
Overall (1240)
(400)
AASIST Jung
2022
96.4
54.4
57.1
54.3
69.4
Owen-Audio
49.8
49.8
48.0
58.9
48.0
50.1
SALMONN-77B
52.8
52.4
49.3
46.8
51.2
AnyGPT
48.5
52.4
51.4
45.9
49.8
50.8
48.0
58.2
49.0
50.3
OneLLM
50.3
50.2
50.9
47.9
49.9
LTU
40.2
45.3
50.5
46.9
44.4
Qwen-Audio2
49.4
40.0
49.6
27.5
42.3
49.3
49.3
48.6
51.3
49.7
Gemini- Flash Team al. 2023
50.1
48.3
47.3
49.8
49.4
48.6
49.5
48.6
49.7
49.2
Table 17: Judgment questions results of different models on the LOKI text modality. denotes the closed-source models.
Scientific Papers
Modern Literature
News
Essay
Wikipedia
Speech
Phiosophy
Ancient Chinese
Expert (RADAR Vicuna-7B8
76.2
81.4
90.0
67.2
73.9
63.3
79.45
1.7
MiniCPM-V-2.6
51.4
47.7
49.8
50.8
49.4
50.5
49.4
43.5
Phi-3.5-Vision
48.1
50.3
47.9
47.5
48.2
53.1
50.7
50.0
V.VA-Onneissoon-78
54.5
50.3
55.1
55.8
52.7
57.0
51.2
42.1
IncenLM-XCoomooer25
$2.5
55.8
48.2
49.7
55.7
54.9
52.2
mPLUG-OW(3-7B
50.5
53.4
53.2
60.6
59.5
41.9
Qwen2-VL--B
49,9
48.3
49.6
51.4
48.1
50.5
48.0
42.1
LongVA-7B
51.0
48.8
48.0
49.8
47.2
50,2
44,8
49,0
Mantis-8B
51.0
51.8
51.5
52.8
53.5
54.9
52.7
43.2
Idefics2-8b
45.5
44.0
$2.4
49,4
43.1
InternVL2-8B
51.6
49.4
45.6
49.1
47.1
52.7
54.7
54.4
Llama-A-LogVILA--BB
49.4
50.6
49.1
48.9
51.2
50.0
49,8
50.0
VILAI15-13B
50.5
48.0
44.0
47.2
42.6
55.1
45,8
48.5
VntemnV1.2:26B
51.6
49.7
50.4
48.4
47.2
56.2
53.3
$3.6
VILAA15540B
50.7
Qwen2-VL-72B
49.9
50.6
50.6
52.0
53.1
59,8
52.5
LLaVA-OneVision-222
56.0
6L.7
50.7
$7.0
54.7
73.4
71.5
68.6
Llama-3. 1-05BB
54.5
53.4
49.2
57.8
56.7
57.7
67.2
58.8
Qwen-mux
48.0
51.5
47.6
48.3
50.1
49.1
47.4
41.1
Mistral Large'
49,4
48.9
45.1
49.4
50.9
60.6
61.3
50.8
Claude-3.5-Soneee
53,2
56.0
60.2
58.9
66.1
68.3
68.2
60.1
Gemini-1 P.5-Pro
55.0
$2.2
49.9
52.9
$6.5
62.5
59.0
58.6
GPTT-4""
64.2
61.5
36

---

Published as a conference paper at ICLR 2025
Table 18: Multiple Choice questions results of different models the LOKI text modality. denotes the closed-source models.
Scientific Papers
Modern Literature
News
Essay
Wikipedia
Speech
Phiosophy
Ancient Chinese
MiniCPM-V-2.6
48.6
42.6
47.8
48.2
48.5
50.1
56.3
50.0
Phi-3.5-Vision
45.8
39.9
41.0
36.3
42.6
44.3
47.5
LLaVA-Oue LLaVA-OneVision: Vnsionnn-
51.5
48.2
48.6
44,9
44.9
49.6
MieemLM-XCompoer255
44.0
42.8
33.9
41.0
34.3
46.3
40.8
41.9
mPLUG-OW(3-7B
51.8
49.2
45.1
48.3
44.9
54.0
62.1
38.6
Qwen2-VI. V1-7B
49.6
46.3
42.6
44,2
45.8
45.4
50.7
46.1
Lnn7VA-7B
Mantis-8B
58.9
54.4
49.7
59.6
34.7
Idefica2-8B
50.1
40.6
31,5
40.1
35.8
35.0
33.6
20.8
IntemVL2--8
53.9
52.8
38.9
47.2
42.5
43.8
50.7
40.0
Uama-3-LLogVVILA-BB
46.4
43.5
42,2
37.4
51,4
46.7
49.2
VILA11.-13b
39.9
InternVL2-26B
52.6
54.9
43.2
49.3
47.6
48.9
VILAI.5-40B
49.9
52.5
39.4
48.9
46.3
57.6
58.2
50.8
Qwen2-VL-72B
67.2
79.2
53,1
72.9
62.5
76.7
76.1
69.7
LLaVA-OnzVihion-222
65.1
79.9
50.6
63.1
82.6
76.9
3amma-3.1-4055
Qwen-max
48.3
48.3
47.4
33.8
43.3
43.8
48.3
48.1
Mistral Large"
55.7
75.1
55.4
62.8
61.8
86.7
86.1
69.4
Claude-3.5
Gemini-1.5-PP 1.5-Pro"
51.3
56.7
70.4
71.5
77.2
GPT-4o
62.2
62.9
49.4
59.0
66.9
81.5
79.3
75.6
Table 19: Performance Comparison of Models across Text Lengths and Languages. closed-source models.
denotes the
Short text
Medium text
Long text
English
Chinese
MiniCPM-V-2.6
46.6
49.7
48.9
Phi-3.5-Vision
63.2
59.2
56.5
48.6
45.5
JLaVA-OmeVision-78
45.5
45.7
46.8
53.1
50.1
IutemLLM-XConppseed
46.4
46.7
47.2
48.2
49,0
mPLUG-O0 JPLUG-Ow13-7B
48.8
47.6
49.0
56.3
48.9
Qwen2-VL-77
43.8
42.4
47.6
48.4
LongVA-7B
37.7
37.0
37.5
49.5
48.0
Mantis-SB
54.7
55.5
55.6
53.7
48.2
Idefiex2-8B 8B
46.9
41.8
33.9
46.7
41.2
45.6
44.4
48.6
Llama-3-LongWILA-SB
47.0
47.7
48.3
VILAI.5-13B
49.4
48.0
44.4
47.0
46.0
V1temnV122-26
34.9
33,9
35.4
51.0
50.7
VII.A1.5-0BB
50.5
50.9
50.7
49.8
50.5
Qwen2-VL-722
56.9
TLaVA-OneVision-22
56.6
64.0
Qwen-max
35.8
34.0
32.8
48.3
45.9
Llama-3. 3-1-B05B
49.0
49.6
56.5
63.4
61.3
Mistral-Large'
44.6
55.5
Saude-3,5-Sonned
56.1
60.3
67.6
73.5
68.3
Gemini-1.5 Pro-Pro
39.7
40.3
47.0
58.4
54.3
GPT-46"
46.9
49.4
58.2
60.0
59.0
37

---

Published as a conference paper at ICLR 2025
E MORE EXPERIMENTAL RESULTS AND DISCUSSIONS
In this appendix, we provide a detailed discussion of additional experimental results, including com- pression artifacts tests, analyses of Deepfake category experiments, and further results from CoT experiments.
E.1 COMPRESSION ARTIFACT TESTS
To better evaluate the applicability of large multimodal models (LMMss) in real-world scenarios, we conducted additional compression artifact tests. Following previous studies Wang et al. [2020a) Ojha 2023]. we conducted JPEG compression tests on judgment tasks under the image modality, based on three representative LMMs and an expert model AIDE, as previously introduced in the main text.
Table 20: Compression artifact test. The test results different models under various compression ratios.
100%
95%
90%
80%
60%
Intern V1.2-8B
56.6%
56.1%
56. %
56.4%
56.6%
Qwen2-VL-7B
54.5%
54.0%
54.2%
54.6%
54.1 %
GPT-4o
63.4%
63.2%
63.1%
62.8%
63.1% %
AIDE
62.8%
46.4%
45.0%
44.2%
45.2%
From these results, we observe that expert models AIDE(Yan et al. 2024) are more significantly impacted by compression artifacts. In contrast, LMMs demonstrate remarkable robustness, with compression levels having little to no effect on their detection capabilities. This distinction high- lights the inherent advantages of LMMs over traditional expert models. There are several possible factors that contribute to the robustness of LMMs to compressed images.
Extensive Pretraining Diverse Data: The vast amount of data available the internet includes images with varying levels of compression. During pretraining, LMMs inherently learn features that are robust such artifacts by including the internet-scale data, demonstrated in works like FaaveesalFFkeeDeec(((iiii et al.] (2023) and CLIPping(Khan & Dang-Nguyen, 2024). Model Scale Advantage: Unlike traditional expert modules, which typically operate with parameter counts in the millions, LMMs are equipped with parameters in the billions. This scale significantly enhances their generalization and robustness, consistent with the emergent properties observed in large-scale models. This finding further underscores the potential of LMMs for real-word applications, partic- ularly in scenarios involving compression artifacts. We will include the above experimental results and analysis of compression robustness in the supplementary material to provide comprehensive view of LMM reliability and applicability. If you have additional questions, we welcome further discussion.
E.2 DEEPFAKE DETECTION
Deepfake detection one of the most prominent synthetic data detection categories today. LOKT already includes partial Deepfake datasets from audio and image modalities. The existing data for the Deepfake category includes:
In audio tasks, some samples of speech and singing are sourced from their re- spective audio deepfake datasets ASVSpoof2019 and DCASE2023 Track 7, as mentioned in Table These datasets are built on state-of of-the-ar voice cloning techniques, such as Text-to- SSSettchppec eet al. 2019), Voice Conversion(Sisman et al.] 2020). and Singing Voice Conversion Nachmani & Conversion/Nachmann Wolf 2019). These technologies address major challenges in deepfake detection, including vulner- abilities in voice authentication systems and the misuse of synthetic voice technology to generate counterfeit content, such as fake artists.
Audio Modality:
Image Modality: As shown in Table our image data includes portion of facial deepfake images, comprising partial datasets from FFHQ and Deepfakeface. These data are directly aligned with the
38

---

Published as a conference paper at ICLR 2025
core challenges in image-basee based deepfake detection, ensuring that our benchmark comprehensively evaluates models on this critical aspect of synthetic data detection.
Table 21: The accuracy of different models on the deepfake dataset and their overall accuracy.
Claul--2--Somet
ImcnVVL2-48
VILAI.5-0B
Llava OV:78
Manti- KB
Qwen2- Qwan2-VL-738 VL--1L-
Llamu-3-8B
GPT-40
Model
Deepfake Accuracy
0.14
0.13
0.29
0.13
0,53
0.34
0.47
0.59
Overall Accuracy
0.54
0.50
0.49
0.50
0.55
0.53
0.50
0.63
We have independently evaluated the deepfake-related components of our current dataset and pre- sented the results below. From the results, it evident that most models perform significantly worse on deepfake data compared to their overall accuracy. This discrepancy likely arises because the overall evaluation task includes simpler synthetic samples (c.g.. basic synthetic data), which are easier for the models to distinguish, thereby inflating their overall accuracy. In contrast, deepfake data often involves more nuanced and fine-grained forgery features that are particularly challeng- ing for caneral-purposs multimodal pretrained models to capture effectively. Interestingly, GPT-4o achieves the best performance on deepfake data among all evaluated models, demonstrating rel- atively stronger ability to handle such complex forgeries.
E.3 MORE COT EXPERIMENTS RESULTS
To further validate the effectiveness of CoT across various question types and modalities, we con- ducted additional experiments. These experiments included testing multiple question types in the image modality (e.g., multiple-choice and anomaly detection) and extending the evaluation to video and text modalities (e.g., binary classification tasks). The results are shown in Table
From the results, we observe that for most models, CoT significantly improves performance across different question types and modalities. Models such as InterVV2--8B Qwen2-VL and GPT-4o consistently benefit from CoT prompting, with noticeable performance gains across both tasks (c.g., anomaly detection, binary classification) and modalities (e.g., image, video, text), which is consis- tent with findings in other CoT prompting methods(Wet) et al. (202b) Kojima et al. [2022). However, not all models benefit from COT.. For example, LLaVA-OV-7B shows a significant performance drop (56.6 18.8). Upon closer examination, we found that LLaVA-OV-- struggles with long-context reasoning, as evidenced by irregular outputs such as incomplete sentences, irrelevant responses, or prematurely ending sentences with token. This suggests limitations in its training for handling extended contexts, which may hinder its ability to utilize CoT effectively.
In conjunction with the results in Table prompting on LMMs may be attributed to the inherent difficulty of synthetic detection tasks, which requires fine-grained perception and complex reasoning recognize artifacts, which might be out of the training scope of current LMMs. Therefore, in FS prompting, simply providing in-context exam- ples without reasoning steps might be insufficient for most models learn how detect artifacts. Furthermore, adding them to their context window might interfere with their original reasoning paths, resulting degraded performances. Therefore, the difficult setup of synthetic detection, the effects of FS prompting on model performance are limited. We note that the strong performance of GPT-4o models after FS prompting might stem from its inherently strong reasoning abilities.
observe that the negative impact of FS (Few-shot)
Table 22: More Chain of Thought prompting experiments results.
Model
Image Multi-chlice w/ CoT
Image Abnormal Selection w/ CoT
Video Judgement w/ CoTl
Text Judgement
wio CoT
win CoT
wio Coll
wio CoT
w/ CoT
11aVA-ON-7R
$1.7
33.7
18.8
14.4
65.0
50.5
$1.6
43.1
IntervLL2-88
31.4
33.2
70.2
75.3
51.2
55.1
41.6
52.3
Qwen2-VL
GPT-4o
30.8
88.9
76.2
84.3
71.2
77.8
53.5
39.6
39

---

Published as a conference paper at ICLR 2025
E.4 PERFORMANCE ACROSS DIFFERENT LEVELS AND MODALITIES
2 the performance of different models across various modalities and difficulty levels. Based on human performance, categorized the difficulty levels of the questions. As the difficulty of the questions increases, most models show gradual decline in performance. Notably, GPT-4o maintains relatively strong performance in both the Video and Image modalities. Given the diverse tasks and modalities involved in LOKI, considering the scores across different question difficulties is beneficial for assessing the overall performance of the models.
Table 23: The performance of different models across various modalities and task difficulty levels.
Vidco
Image
Text
3D
Model
Fasy
Medium
Hand
Fasy
Mediam
Hard
Easy
Medium
Hard
Fasy
Medium
Hard
LLaVA-OV-78
0.58
0.55
0.52
0.55
0.59
0.56
0.62
0.46
0.41
0.61
0,58
0.55
Inter V1.2-B
0.64
0.56
0.49
0.54
0.58
0.53
0.56
0.50
0.39
0.42
0.32
0,25
Qwen2-VL-7B
0.52
0.52
0.55
0.55
0.54
0.62
0.46
0.37
0.67
0.55
Gomini-1.5 ---pro-
0.35
0.52
0.51
0.64
0.14
0.17
0.64
Claude-3.5
0.53
GPT-46*
40

---

Published as a conference paper at ICLR 2025
F CASE STUDY
In this section, we delve into an analysis of GPT-4o's performance on synthetic data detection tasks, with a particular focus on the questions answered incorrectly. This examination crucial for understanding the model's operational capabilities and limitations, The analysis not only identifies the model's current shorteomings but also provides guidance for future design and training improve- ments. We randomly selected 50 erroneous instances from GPT-4o's predictions across various modalities and difficulty levels for detailed inspection. These instances included multiple-choice, true/false, and open-ended responses, and even for the structured questions, GPT-4o was required to provide the reasoning behind its decisions. These instances were analyzed by specialized annota- tors who identified the fundamental causes of the erroneous predictions based on their expertise and annotations.
Image: Animals [Opened-Ended]
Question:
This is fake image generated by edvenced Al model.
Your Tesk: P lease performa detailed analysis aining where this image seems inauthentic from following perspectives: mage seems inauthentic. specific
overall explanation why explanation. eference any general visual cues that contributz to inauthentciyy.
Global pective:
your
2. Regional erspecttivs:: Idently specific regions the image that appear nauthentic. For each region: Describe the location or provide reference to region "topleft corner," "the person's righthandd nd etc.). Explain In detail the region seems inauthentic. Cite anomalles incons stencies
thatreglon.
(GPT-4o) Answer:
visile strings attached do not have strings or any extamal attachments, Material and Texture: The traturn the hird's body and wings appears to made of fabric or synthelic materiel, lacking the natural feather detail and varition seen real birds. Pase and Movement: The bird's pose rigid and lacks the dynamic movement typical of real bird fight. wings expected living creature.
Explanation: Strings
indication
do not show the natural flexbilly motion
Regional Perspectives: Region 1: Head and Beak Location: The head and beak area of the bird. Inauthenticity Explanation The heed and beak are overly symmebrical and lack the sublie asymmetries found in real birds. The eyes appear sainted or printed, lacking depth and reflection seen living Region 2: Wings Location: The
the bird, Inauthenticity Explanations The wings pattems. edges of wings too smooth and lack the individuall feather detail, Region 3: Talors Location: The talons of bird. Inauthenticity Explanation: The talons are unnaturally lly positioned and appear to be made af sold material rather than the articulated structure of real bird ta ons. They lack the sharpness and detail expected in predator bird. Region 4: Strings Location: Attached rlus parts of the bird, including and
color
that typicaly
Inauthenticity Explanation: The presence strings laut and connected to the bird, which not possibble in real bird.
this kite. strings visibly
Ground Truth:
Overall Explanations The texture detail the eagle's feathers is with blured pattems, the material does not match that feathers, resulting
authentictyy
Regional Perspectives: Region 1: Claws Inauthenticity Explanation: The shape the eagle's claws everly enlarged, differing discrepanddcccccc discrepancies the lengths of different claws, and the curveture of the claw tpp varies noticeblyy lacking symmetry and consistency. Acdiiioally the number claws each side inconsistent, Region Rope Inauthenticity rope the eagle does not align with reality. Region 3: Feather Inauthenticity Explenation: The feathers on the head do not exhibit a clear structural or edge bounda ly relationhhip with the feathers on the body, which is inconsistent with the characteristie ristcs of real eagle.
from reality There are significant
Explanation:
obvious breakage,
clear connection
surrounding ropes,
Figure 13: Abnormal detail explanation example of category animals image modality, with GPT- 40's s answers
41

---

Published as a conference paper at ICLR 2025
Image: Object [Opened-Ended]
Question:
This is fake image generatnd by an advanced model.
Your Task: P lease performa detailed analysis lining where this image seems inauthentic from the following perspectives:
1. Globel P pective: Provice overall explanation why entre image seems inauthentic Be specific and detailed explanation.
any general
contribute
inauthenticiy.
2. Regional erspectives: idenäfy specifi regions the image that sppear neuthentic. For each region: Describe the location or provide reference to the region (e g., "top-lett corner" "the person's right nd," " ett.). Explain in detail why the region seems inauthentic. Cite specific visual anomalles or incons stencles observed thatregion.
(GPT-4o) Answer:
Overall Explanation: Labels: The characters keys not nlign
known koyboard layou M them appear overly
symbol, indicaling ieigeneree image, Lighting: The lighting uniform, lacking the natural varistion experted from world lighing Wear and Tear: There noticable absence any wear or teat, which would typically be visible frequendly used keys real keyboard.
nonsersicale distorted
Regional Perspectives: Region 1: Top-Left C
partiullarly Location: Top corner of keyboard. Inauthenticity Explanation: The some agpearing
characters keys
shapes rather recognizable
symbols, highlightin lack of attentin to detail, Region Center of the Keyboard Location: Center of the keyboard. where the aiphanumeric keys are located Insuthenticity Explanation: The characters do not correspond any known language or keyboard layout, and the spacing between the keys inconsister Stent. which unusual for manufactured product. Region 3: Bottom Row keyboard, cantai ning space and mudifier keys. Inauthentiiity Explanation: The space appears everly smooth, without the testure or slight curvature found in real keyboards. Additinally, the symbols on the modifier keys are not standard. Region 4: Key Edges and Shadows Location: The edges of the keys and the shadows they cast Inauthenticily not reflect subdl variations real-wordd lighting. The key edges are too objects.
Explanation: The shadows ovenly uniform smooth
slight imperfecios
Ground Truth:
Overall Explanation: The shapes of the keyboard keys are ireqular, distorted, and the textures are unclear. Regional Perspectives: Region I: Keyboard Cap Inauthenticity Explanation: The shapes of the keys are distorted. and the characters keycaps
exhibiting refeectoos.
Figure 14: Abnormal detail explanation example of category object in image modality, with GPT- 4o's s answers
42

---

Published as a conference paper at ICLR 2025
X
Image: Document [Multi-Choice]
Question:
You have been shown different images. ne the images generated by Al. Which of the above images is most likely generated by Al?
A. The first one
The second one
the fag?
dight
(GPT-4o) Answer:
(B)
The second image features a cartoon-style illustration of person and robot playing chess, which is style more commonly generated by Al or digitally created compared to the first image, which consists mainly of text and straightfonward graphic about stars and stripes on flag.
Ground Truth: (A)
Analysis:
Cartoon-style illustrations cannot be used as a standard to determine whether a document image is Al-generated, humans can also create cartoon-style images through painting. The reasoning behind this analysis flawed.
Figure 15: Multi-choice example of category document in image modality, with GPT-4o's answers
43

---

Published as a conference paper at ICLR 2025
Image: Document [Judgement]
Question:
You have been shown one image, possibly generated by Al. P Please identify whether this image generated by Al or not.
PGPT-4 Answer: NO
languages ngliish Chinese), dagrams, aag, graphs, and hand drawn skeiches-al all seamlessly integrated Al-generatnd images often striggle with producing this level of diverse, yet conteatually appropriate, viuaal content while maintaining distinct styles across different sections Clear and Detailed Text The texts the mathematical formula,,
Consistent Styles: image contains variely of
appear consistent legible. with different font styles sizes matching their context capson
generating lengthy, detailed passages with proper spacing and accurate alignment across various section. Scientific Diagrams and Charts: There are scientiic diegrams and performance graph, including proper annelations and label. Such precie scientific repres entation is challenging for Al generate convincingly due the need accurate placement of elements, meaningful labe and clear line work that conveys information. Natural Layout locically arranged a. text aligned with diagrams, newspaper-lkke column for reading). Al-gererated images can struggle create complex layouts that appear natural and purposeful. Small, Detailed Annotations: There are small details like hander iten notes and tny labels in the scientiiic diagrams, which have a level human touch and Imperfection thatAl-generate Images offen miss overcomplicete.
passage
graph).
sometmes
Organization:
layout image appears very
with
Ground Truth: NO
Figure 16: Judgement example of category document image modality, with GPT-4o's answers
44

---

Published as a conference paper at ICLR 2025
Image: Medical [Multi-Choice]
Question:
You have been shown two different images. On of the images generated by Al. Which of the above images is most likely generated by AI?
The first
The second one
(GPT-40) Answer:
(A)
The first image has more pronounced color artifacts, uneven blending, and unusual texture patterns, which are typical indicators of Al-generated imagery. The second image displays more natural appearance with smoother transitions, realistic textures, and consistent colors, characteristic of an actual skin lesion photograph.
Ground Truth: (A)
Figure 17: Multi-choice example of category medical in image modality, with GPT-4o's answers
45

---

Published as a conference paper at ICLR 2025
X
Image: Medical [Judgement]
Question:
You have been shown one image, possibly generated by Al. P Please identify whether this image generated by Al not.
(GPT-4o) Answer: NO
1.Natural Variation Texture and Color: The image displays netural veriatins skin texture and pigmentation, which are often difficult for Al models to repiiate accurately. G Generated images might show unnaturaly smooth or uniformly textured areas. 2.bbsence Artifacts Anomalies: Al-generated pattem
sometimes exhibit visaa artifacts like distortions, unusual color blending, common Al-generetion artfacts.
3.Realistic Lighting and Shading: The way light reflects chellenging Al models replicate, especially complex textures like human skin.
skin the depth of the pigmentation appear natural, which often
4.0rganic Irregllaittie:: The has small, inregular deta such the shape the central lesion and the surrounding soots, models sometines struggle reproduce organic ineguls accurately, defaulting symmetrical shapes.
Ground Truth: YES
Analysis:
The image's sibn teature exhibits abnormal color artifacts, with uneven color blending an the surface. This skin texture neither smooth uniform. The skin surface the shows unusual blush green tones, and the pigmentation appears unnatural.
Figure 18: Judgement example of category medical in image modality, with GPT-4o's answers
46

---

Published as a conference paper at ICLR 2025
Image: Animals [VQA]
Question:
What seems inauthentic in this image?
Option:
(A) Nothing in this image seems inauthentic.
The lion's eyes are purple, which is inconsistent with the normal eye color of lions. The shape of the teeth inside the lion's mouth is asymmetrical, and the texture details of the tongue and oral cavity insufficient, with distorted colors.
The lion's mane is unevenly colored, with blue stripes appearing. (E) There distinct star-shaped mark on the lion's nose.
(GPT-4o) Answer: (C)
Upon inspection, the teeth appear be asymmetrical, and the details within the mouth, such as the tongue and oral cavity, have unnatural colors and insufficient texturing. This suggests inauthenticity or manipulation in the image.
Ground Truth: (C)
Figure 19: Abnormal selection example of category animals in image modality, with GPT-4o's answers
47

---

Published as a conference paper at ICLR 2025
Image: Object [VQA]
Question:
What seems inauthentic in this image?
Option:
(A) The text on the product packaging paper blurry and hard to read.
(B) pattern on the label upside down.
(C) Nothing in this image seems inauthentic.
(heThere unnatural light and shadow effects on the rock.
The color of the bottle cap completely different from that the bottle body.
CARH (ouueers)
AD1A)
EDUAP
0000
(GPT-4o) Answer: (A)
The image shows that the text the labels of the bottles is distorted, blurry, and difficult to read, which often a sign of inauthenticity or an Al-generated image, as Al models sometimes struggle with replicating clear and consistent text.
Ground Truth: (A)
Figure 20: Abnormal selection example of category object in image modality, with GPT-4o's swers
48

---

Published as a conference paper at ICLR 2025
Image: Object [Opened-Ended]
Question:
This is fake image generatnd by an advanced model.
Your Task: P lease performa detailed analysis lining where this image seems inauthentic from the following perspectives:
1. Globel P pective: Provice overall explanation why entre image seems inauthentic Be specific and detailed explanation.
any general
contribute
inauthentiiiy.
2. Regional erspectives: idenäfy specifi regions the image that sppear neuthentic. For each region: Describe the location or provide reference to the region (e g., "top-lett corner" "the person's right nd," " etc.). Explain in detail why the region seems inauthentic. Cite specific visual anomalles or incons stencles observed thatregion.
(GPT-4o) Answer:
Overall Explanation: Labels: The characters keys not nlign
known koyboard layou M them appear overly
symbol, indicaling ieigeneree image, Lighting: The lighting uniform, lacking the natural varistion experted from world lighing Wear and Tear: There noticable absence any wear or teat, which would typically be visible frequendly used keys real keyboard.
nonsersicale distorted
Regional Perspectives: Region 1: Top-Left C
particllarly Location: Top corner of keyboard. Inauthenticity Explanation: The some agpearing
characters keys
shapes rather recognizable
symbols, highlighting lack of attentin to detail, Region Center of the Keyboard Location: Center of the keyboard. where the aiphanumeric keys are located Insuthenticity Explanation: The characters do not correspond any known language or keyboard layout, and the spacing between the keys inconsiste Stent. which unusual for manufactured product. Region 3: Bottom Row keyboard, cantai ning space and mudifier keys. Inauthentiiity Explanation: The space appears everly smooth, without the testure or slight curvature found in real keyboards. Additinally, the symbols on the modifier keys are not standard. Region 4: Key Edges and Shadows Location: The edges of the keys and the shadows they cast Inauthenticily not reflect subdl variations real-wordd lighting. The key edges are too objects.
Explanation: The shadows are ovenly uniform smooth
slight imperfecios
Ground Truth:
Overall Explanation: The shapes of the keyboard keys are ireqular, distorted, and the textures are unclear. Regional Perspectives: Region I: Keyboard Cap Inauthenticity Explanation: The shapes of the keys are distorted. and the characters keycaps
exhibiting refeectoos.
Figure 21: Abnormal detail explanation example of category object in image modality, with GPT- 4o's s answers
49

---

your Published as a conference paper at ICLR 2025
Image: Person [Opened-Ended]
Question:
This is fake image generated by advenced Al model.
Your Task: lease perform detailed analysis
inauthentic from following perspedives: explanation. Aeference any general visual rues that contributz to inauthenticty.
where this mage seems
Global pective:
everall explanation why
specific
2. R Regional erspectives: Identffy specific regions the image that appear nauthentic. For each region: Describe the location or provide "the right
referance region (n.g.
Explain detail the region inauthentic. incons observed region.
specific
SeUIT
(GPT-4o) Answer:
Overall Explanatione Texture and Detall: The skin texture appears overyy smooth and lacks the natural voristion found real human skin. This uniformity common trait gl-genertted images. Lighting and Shadows: The lighting seems inconsistent, with shadows aigning naturally
appearance.
facial festures may agpear
slightly exaggerated or disproportionate which is common issue Al-generated portra
The eyes and mouth may appear slightly off in terms alignmenton proportion. The might the depth and reflection typical real photographs. and the facial
Regional Parspactives: Region 1: Face Location: Center of the image. Inauthenticitt Explanation:
perfectly
Region Hands Location: Bottom
Image. Inauthenticity
Explanation: hands might signs distortion or unnatural posinning Fingers
unusual number of joints, which is common anomaly Al Al-gennrdted images. Region T-shirt Logo Location: Center of the torso. Inauthenticity the T-shirt appear distorted or lack clarity. The text or des might not tollow the natural tolds and the fabric Indicating lack of realstic interaction with clathing. Reglon Background Location: Enntree background. walls might appear too
appear
Explanation: The
Inauthenticity Explanation: background may depth or have inconsster lightng, which can detract from the realism of scene.
Ground Truth:
Overall Explaration: The image exhib certain degree of color distartion, with noticeable color differences on both sides the
Image.
Regional Parspectives: Region 1: Inauthenticity Explanation:
texture the eyes, with the visual strucue The eyes are uneven size asymmetrical
There
resulting conform to the characteristics of eyes. Region Hands Ineuthentkity Explanation: The texture of the fingers chaotic, with textures of different fingers blending into each other, making them difficul to disbnguish. The edges between the fingers and the clothing with real-world
direction
eyes
blurred, with unnatural trans between two, which does unnatural transtton Inauthenticity
Region 3: Face
area, being uneven size un both sides. Region Arms Insuthenticity Explanation:: The edges between the s sieeve and the arm are blurred, and there are incorrect in the arm jiitt area, as well as in the distibution of the muscles.
asymmetrical.
Figure 22: Abnormal detail explanation example of category person in image modality, with GPT- 40's s answers
50

---

Published as a conference paper at ICLR 2025
Image: Person [VQA]
Question:
What seems inauthentic in this image? Option:
There non-existent pattern the shirt.
The person's ears are asymmetrical.
The colors of the lights in the background are inconsistent.
Nothing in this image seems inauthentic
(E) The cuff area shows an unnatural yellow texture transition, with the texture being discontinuous
(GPT-4o) Answer: (E)
Upon close inspection, the texture of the shirt near the cuff area transitions unnaturally, with a visible inconsistency in the pattern and color that discontinuous, which might indicate that the image has been artificially generated or manipulated.
Ground Truth:
Figure 23: Abnormal selection example of category person in image modality, with GPT-4o's an- swers
51

---

Published as a conference paper at ICLR 2025
X
Image: Satellite [Multi-Choice]
Question:
You have been shown two different images. One of the images is generated by Al. Which the above images is most likely not generated by Al?
A. The first one
The second one
(GPT-4o) Answer:
(B)
The second image contains consistent and natural textures, road details, and shadows characteristic of real satellite imagery. The first image has some visual irregularities and unusual patterns that are more typical of Al-geneerteed content.
Ground Truth: (A)
Figure 24: Multi-choice example of category satellite in image modality, with GPT-4o's answers
52

---

Published as a conference paper at ICLR 2025
X
PP
Image: Satellite [Judgement]
Question:
You have been shown one image, possibly generated by Al. Plaase identify whether this image generated by Al or not.
(GPT-40) Answer: YES
Textures and Patterns: In Al-generated images. textures can sometimes appear "olf subily Incons stant In your Image. while most suildins)
the boundaries vegetation photo, These transitoons might have unusual blending or sight bluring, which can an indicator of Al synthesis, Sharpnass and Edge Details: Real-world mages often have varistion sharpness due to factors like camera tocus and lighting. Al Aligenerated images, however, might have unnatural unnaturalunfomm unioomty sharpness or blurnng. this image. edges certain appearance, acking typical photographs. Anomalles Vegetation: models can sometimes struggle with complex textures vegetation, producing repetitive or unnatiral pattomss Looking closaly at the trees and grass in your imagn, there seems to be siigtty repelitive and overly uniform structure, which could be artifact of Al generation. Lighting and Shadows: Natural images have coherent lighting sources, and Al-generates mages,
look natural, there smoothly as they might a real photo.
(especially
transttonn
slightly
shadows consisaatyy perfectly In your image, feels slightly mismatchedd with the expected light source.
the
directions. intensities, sofnnes may shadows generally align, shading on same onjects (ikee
structure near mad)
Ground Truth: NO
Analysis:
ttrms texture, some edges of the textures exhibi bluriness, which related the resolution the remote sensing images. The blurred edge transitions are caused by the mage resolution. Due to the greater observation distance, certain lighttreuency Information observed may missing, resulting repetitive uniform sens ing images.
ground. unique characteristic
Figure 25; Judgement example of category satellite in image modality, with GPT-4o's answers
53

---

Published as a conference paper at ICLR 2025
Image: Scenery [Opened-Ended]
Question:
This fake generated by advanced model.
Your Task: lease perform detailed analysis exp aininngg this Image seems inauthentic from following perspectves:
1. Global ers pertive: Provide an overall explanation of why the entre mage seems Inauthentic. Be specific and detailed in your explanation. eference general visual cues that contributz to the irauthenticty.
that sppear insuthentic. each Describe the location provide reference region p.g. lop-lett corner," person's right hand," etr.). Explain why the reiion seems inauthentic specilic visual anomalies or incons stencies observed that region.
2. Regional erspectives; idenafy specific
the
(G PT-4o) Answer:
Overall Explanation: Texture and Detall: Color Saturation: The colors, particularly the greens and oranges, are excessively saturated, which is nottypical of natural enviromments the natural imgulaatiies found mal-worid across mage inconsistent. with shadows not aligning properly with a single light source, making scene appear unre alstic. Regional Perspectives: Reglon 1: Cliff Face Side) texture is overly smooth, lacking the mughness and erosion marks typical of real ciffs. The transition between the vegetation and rack and does not blend naturally, which detracts from realis. Region Water Surface (Bottom
water, consistent envirnllevvoomentt Texture Uniformity: The textures across different surfaces. such the citts
Lighting Shadows: lighting
Side) Location: Right side of the image. Inauthenticity Explanation: The cilt's
Location: Battom water appears overly regular tems wave and toam patterns, which lacks natural randamness found in real water bodies. Additionlly. the reflectins and highlights on the water do not correspond well with the light source, further bontribting the artificial look. Region 3: Cliff Edge (Top Center) Location:
comer Image. Inauthenticity Explanation:
Top center of the image.
Inauthenticity Explanation: The edge betwren Vegetation (Right Side) Location: R Right side of the riff. Inauthenticity Explanation: The vegetation appears unnaturally uniform, with consistent co or and texture, does not show the diversity and natural distribution of real plant life, and the way the plants cling to the realisti,, they follow natural contours
natural clff adges. sky abrupt, without atmospheriz blending usually seen natural landscapes. Region
sharply defined
irregulatties
Ground Truth:
reality The boundary between the waves and shore bluree,, lacking sense realism. lighting the upper part differs
Explanation:
excessively high
waves
lower part.
Regional Perspectives: Region 1: Texture Inauthenticity Explanation: The color of the green covered mountains overly saturated.
the tion between colors neighboring
unnatural.
Figure 26: Abnormal detail explanation example of category scenery image modality, with GPT- 40 s answers
54

---

Published as a conference paper at ICLR 2025
Image: Scenery [VQA]
Question:
What seems inauthentic in this image?
Option:
(A) color of the leaves the trees does not match the seasonal changes.
(B) are unnatural cloud formations in the sky.
The boundary between the rocks and the river is unclear, and the texture of the rocks does match the characteristics of years of erosion.
[herhere are unnatural light and shadow effects on the rock.
(E) Nothing in this image seems inauthentic
(GPT-40) Answer:
(C)
The image shows unclear separation between the rock formations and the water, and the rock texture lacks the natural, smooth erosion typically seen in such landscapes. These inconsistencies suggest inauthentic features in the image.
Ground Truth:
(C)
Figure 27: Abnormal selection example of category scenery in image modality, with GPT-4o's answers
55

---

Published as a conference paper at ICLR 2025
Video: Abiotic [Multi-Choice]
Question:
You have been shown different videos. On the videos is taken in the real world, while the other is generated by an advanced Al model. Which of the above videos is most likely generated by Al?
The first one.
B. The second one.
(GPT-40) Answer: (A)
This due to the highly polished visuals, the realistic lighting effects, and the precision water ripples and reflections, which are characteristic of high-quality Al-generated images videos. contrast, the second video has more natural appearance with environmental elements like the palm trees and the slight imperfections typical in real- world recordings
Ground Truth: (A)
Figure 28: Multi-choice example of category Abiotic in video modality, with GPT-4o's answers
56

---

Published as a conference paper at ICLR 2025
Video: Animals [Multi-Choice]
Question:
You have been shown two different videos. One of the videos is taken in the real world, while the other is generated by an advanced Al model. Which of the above videos is most likely generated by Al?
A. The first one.
The second one.
(GPT-4o) Answer:
(A)
This based on the highly polished and uniform look, as well the subtle but noticeable visual effects, such as perfect lighting and highly consistent textures in the environment. The second video has more natural and imperfect feel, which is often characteristic of real-world footage.
Ground Truth: (A)
Analysis:
Al-generated videos typically exhibit refined details and appearances for the main subject, while the background and surrounding environment are often rendered in rougher manner. the first video, the distortion of the windmill in the background indicates that it Al-generated. In contrast, the second video maintains overall consistency, with both the subject and the background showing the same level of detail.
Figure 29: Multi-choice example of category Animals in video modality, with GPT-4o's answers
57

---

Published as a conference paper at ICLR 2025
Video: Animals [VQA]
Question:
What abnormal event occurs to the tentacles on the far right side of jellyfish in the video?
Option:
(A) Two tentacles on the right side merge and then separate again.
(B) Two tentacles on the far right side disappear completely.
Two tentacles on the far right side change color.
Two tentacles on the far right side grow longer.
(GPT-4o) Answer: (A)
Tentacles merging and then separating again is indicative of how Al models sometimes struggle with maintaining fine details over time, especially in dynamic objects like jellyfish tentacles. The merging and separation can occur because the Al may not fully capture or consistently render the individual tentacles' paths. This behavior is less likely to happen in real-world footage but more characteristic of Al-generated visual content, where the model might briefly combine two objects due limitations in its rendering algorithms.
Ground Truth: (A)
Figure 30: Abnormal selection example of category Animals in video modality, with GPT-4o's answers
58

---

Published as a conference paper at ICLR 2025
Video: Human [Opened-Ended]
Question:
These frames Al-generated video. You
these frames references conduct detailed analysis. The video scenes and actions generated artificaa antficialintllliennee
Your Task: Analyze and explain where this video appears inauthentic from following perspectives:
Globaa Issues: Identify any overal problems that affect the entire video. P Provide deta led explanations for why these aspects seem Inauthentic Reference specific characteristics that yopport elssevations.
Keyframee Anomalies:
Explain detail any video is artifiiallly generated U times tamss or frame numbers to reference specific parts of video.
specific
incorsistencies
suggest
3. Segment ssues: Identily any problematic segments within the video where inconsistencies or artifatts are present. rovide detailed explanations for these segments seem inauthentic pec the time ranges of these segments nts support analysis.
PT-4)) Answer:
Global Issues: Lighting and Shadows: Inconsistancy: The lighting on the hands and the glass appears uniform and lacks natural variation. In setting, there would be sublee o efferences shedow and light due the movement of the liquid and the hand. dynamic lighting which
giveaway of gengeeteeed content. The surfaces, including counteriop and glass, appear overly smooth and lack the fine detals and imperfections found in real life. Explanation: Al Al-generated videos sometimes fail capture the tert-tess that are present in real-world object..
Explanation:
unffrrmtyy suggests
Smoothness:
Keyframe Anomalies: Frame
Liquid Dynamics: mil's into Explanation: Real iquids have complex splash ripple
consistent with fludd dynamics.
liquid appears pour in straight interactions with surfaces, which are difficul ta replicate accurately in Al-generated content. Frame Frame 8: Hand Movement: The hand's position and grip on the glass remain almost stati, which is unusual during pouring actio. Normally, there would be slight and
adjustments Segment Explanation: This lock of natural
suggests liitation the abilty simulste istic 1-3): Inconsistency
Initial Pouring Segment does not change as expected between frames. shoould rise more noticeably with each frame. Explanation: This suggests lack trame-b-frame continulty common issue in Al-generated sequences. Mid-Pour Segment (F [Frames 4-6): L Liquid Texture: The texture of the milk struggles with replica The retlections on the glass and the milk do not change dynamica with pouring action, which would be expected real scenario. Explanation: This indicates lack of advanced rendering techn quees that simulate real world physics.
the gless
uniform and lacks the subte variatons opecity color that occur Segment
mik. Explanation: often Reflection
complex
properties
Ground Truth:
1. The milk the cup has already begun to increase even before being poured out, which is inconsistent with reality.
2. The hand structure behind the cup is incorrect the milk difters the material
transparency matching reality
Figure 31: Abnormal detail explanation example of category Human in video modality, with GPT- 4o's s answers
59

---

Published as a conference paper at ICLR 2025
X
Video: Human [Judgement]
Question:
You have been shown video, which might be taken from real world generated advanced Al model. this video taken in the real world? Answer yes you think it is taken in the real world, and answer no otherwise.) Answer with yes or no.
(GPT-40) Answer: YES
texture details as pores, slight variations color. and imperfections around eye araa seem very lieikke. Al- geneeeeeeed images often smooth out these details make them unnaturally perfect, but these images exhibit the subde impe rreeetttion typical of real skin. -Reflections looking reflections
.Natural Skin Texture:
Eyes: reflections
consistent real-world lighting environment. There complex surmundings light sources, which are
mppiiate Al-generated models.
-E EEyelashes and Fine Details: The eyelashes are irregulrr and detailed, which more challenging Al to recreste perfectly. many Allgenerated images, small details like hair lashes can appear too unitorm clumped together, but here, appear naturally varied.
images seem highly symmetric glance, inspection, are slight variations shape and angle the eyes, which is typical of real human features.
Asymmetry:
Ground Truth: NO
Analysis:
Regarding the reflections the eyes, the refecciooss the image are not realistic A certain point, the reflections in the eyes disappoar allng with changes externl lighting, while the overall scene shows significant changes the external light source. Addhionally, of structures, the eyelashes the video overly eyelashes human eye.
notalign natural structure
Figure 32: Judgement example of category Human in video modality, with GPT-4o's 's answers
60

---

Published as a conference paper at ICLR 2025
X
Video: Human [Opened-Ended]
Question:
These are frames of Al generated videa. You need to use these frames as references conduct detailed analysis. The video and actions generated artriccal intellgnne,
Illstratess
Your Analyze and explain where video appears inauthenti from following perspectives:
1. L.Ginbal Issues: iderafy any everal problems that affect the entire video, P rovide deta explanations for why these aspects seom inauthentic R eference specific characteristics that support caservations.
Keyfram Anomalies: xamine spacific frames
vidao inconsistencles is artifcially generated Use timestapps or numbers to reference specific parts video.
Expain detail any anomales suggest
3. Segment issues: identify any emblemmtic segments within the video where inconiistencies or artifacts are present Provide detailed explanations for why these sepments seem inauthentic 5 pecify the time ranges of these segments support your analysis.
PT-T- Answer:
Global Issues: Unnatural Lighting and Shadows: The lighting in the fra mes appears overyy unifom and lacks the natural variation typically seen woaldwardd settings. Shadows either too soft or inconsistenlly placed, which can make the scenes look flat and artificial,
video, especiall crisp, is common sign generation Texture Uniformity: The textures. especially on
Motion
there
some
clothing, appear too smooth and lack the imperfactions and vartations found in real life. Explanations: Lighting: Real-word lighting varies with the environment and time of day, creating dynamic shadows and highights. The uniformity here suggests artificial generation. Motion Blur in
absence
shots, Indicatas lack real-wordd physics simulation. Texture: Al often struggies with human
replicating natural imperfections
leading an averl palished appearance.
Keyframe Anomalies: Frame and 2: Hair Details: The hair appears averly detaied and perfectly styled, lacking the natural movement and rando mness expected in real setting. Frame Background Consistancy: The background buildings and street lacking the natural inguarttiss real architecture. Frame imperfections real
appear perfect
Pose Expression: movement.
subjects pose express on are too stalic
idealized, lacking
Frame 6: Crowd Details: The people the background lack disticc features and appear as generic figures, which is a common issue in All generated scenes. Explanations: Hair and Pose: Al otten generates hair and poses that are too perfect, as it lacks the abilit fully replicate the rendomness real Bac Background and Crowd: Al generated backgrounds
crowds often lack the complexit streeiscene
envirenments people. Segment remains too consistent across frames. with lite variaton lightng perspective. which unus val for real video. Segment: Frames 4-5 Dance Movement The dance movements appear too smooth and lack the natural dynamics and imperfecions of real human motion. Segment Frames 7-8 Walking Away Scene: The transitin
detail
Segment
Consistency:
the detailed streetscene the simpler alleyway lacks depth changes lighting
perspective, making appeea perspective, which are missing here. Dance Movement: Real human mobian includes slight veriations and imperfections that are difficult for Al to replicate. Walking Scene: The lack of depth and perspective in the transitin suggests artificial generation.
Explanations:
capture
Ground Truth:
Global Issues: 1. The video style leains towards painting, the image tlickers 2. The facial features and hands of the character undergo severe distorion Fragment Issues: 1.1. The facial features the character are distorted. 1.2. The person's hands experience severe distortion, with the righthand merging with the street the left hand merging with the shop. 2.1. main character's facial undergo distortion. backgrroudd incorrect.
image passerby completely incorrec. The body
the passerby
Figure 33: Abuormal detail explanation example of category Human in video modality, with GPT- 40's 's answers
61

---

Published as a conference paper at ICLR 2025
X
Video: Human [Judgement]
Question:
You have been shown one video, which might be taken from real world or generated by advanced Al model. this video taken in the real world? Answer yes you think it is taken in the real world, and answer no otherwise.) Answer with yes or no.
(GPT-4o) Answer: YES
erviromentt, particularly sunlght streaming through the window and reflecting on the subject, looks very natural. genereted mode's often struggle capture this level of nuanced lightnn, especially in dynamic settings like the one depicted here.
-Lighting
complex
light
-Natural Details: The texture of skin, the details the
the intricacy the tatloos al very detailed and res istic, These perticularly the randomness .Tattoo Realism: m: The tatioo on the woman's amm show subte inconsistencies and natural
variabiltty interactio (such as siight warping with the skin), Al-generated tattoos may sometimes appear overly clean or not integrate as smoothly with skin movements.
replicate
-Depth and Focus: The depth of field and focus tre bensitions handled very well. The video shows the subiect clearly challenging
foreground. background slightly
accurately,
cameras naturally create this effect to their optical properties.
Ground Truth: NO
Analysis:
terms lighting. at certain momentin the video, unnatural light spot appears the face. Moreaver, the taltoos on the person change throughout the video, and the elothing structure on the waist does motmatzh reality. Additonall,, the environment around the person becomes distorted during camera transitions the video.
Figure 34: Judgement example of category Human in video modality, with GPT-4o's answers
62

---

Published as a conference paper at ICLR 2025
Video: Scenery [VQA]
Question:
What abnormality is present in the video footage of the snow-capped mountain?
Option:
The colors the video footage are distorted.
(B) The video footage is continuously flickering. The video footage blurry and out focus. The video footage playing at an inconsistent frame rate.
(GPT-4o) Answer:
In Al-generated videos, flickering is common issue, particularly in scenes with complex textures like and mountains. This occurs because the Al model may struggle to maintain consistent rendering across frames, leading to slight variations in color, brightness, or details, which result in the flickering effect. This abnommaity is typical of Al- generated content, where maintaining consistent visual quality over time can be challenging, especially in dynamic lighting conditions.
Ground Truth: (B)
Figure 35: Abnormal selection of category Scenery in video modality, with GPT-4o's answers
63

---

Published as a conference paper at ICLR 2025
3D: 30D-bbaaee((Opeeed--neeeee
Question:
This is an genersted 3D object rendering. Displays four view RGB renders on the left and four view normal maps on the right Your TourTask: Analyze and explain the Inauthenti aspects the 30 object
1.RGB Rendering: Identify issues related
Provide detailed explonations why these images support observations.
calors, averall
seem nauthentic.
specific parts armaraal R Randerig: Analye the nornal maps for inconsiseeenne or errors suriace geometry. Explain any anomalles that suggest the osject artifcaall generated. Use specific visual evidence from the normal maps the explanations.
(GPT-4o) Answer:
not typical and lack the fine detais expected in realsti obrect Uniformity: The and lack subtle varatons due lghting and material propertes, making object appear less lreike. Surface Imperfections: Visible holes and ineuulariies in the structure are particularly notccable in the tpp left and bottom left images, suggesting lack of realstic material simulation.Ld Lack of Realistic Shading:The he Shading
RGB Rendering: Inconsistent Texture QualityThhe textures
smooth
not convincingly ingrennightt source. contributing
how light interacts with the object
Shadows highights
with
Normal Rendering: Unnatural Surface Variations: The normal maps show ebrupt changes surface angles, seen in top right and bottom right mages, where the surface appears jagged and uneven. Color Anomalies: The normal maps use colors to represent surface nomais, the transitons too sharp and lack the gradial blending typical of real objects. Indicating lack smooth transitons..
flat sharply angled, suggesting the object was generated without high preciionn. Inconsistent Lighting Simulation: The normal maps not accurately y simulate how lghtt interacts with complex surface, exhibiting inconsistent lightng. which indicates an artifically generated model.
Curvature: The object'
well-defined.
many
appeaning
Ground Truth:
The texture the hat shows color mixing on the sides, with gray and white interspersed with other colors. The color transitin the side
RGB Rendering:
smooth, Normal Rendering: The geometric surace of uneven, with numerous holes, weak topological smoothness, and poor surface smoothress. The geameticc shape of the hat asymmetrical and does not match the rell shape.
unclear.
Figure 36: Abnormal detail explanation example generated by Gaussian Splatting based methods in 3D modality, with GPT-4o's answers
64

---

Published as a conference paper at ICLR 2025
X
3D: 3DGS-based [Judgement]
Question:
This is rendering of 3D object. The left side of the rendering shows 4-view RGB rendering, and the right side shows 4-view normal map, showing the surface texture and lighting interaction of the 3D model. The object may be generated by an Al model or may be scan of real object. Please determine whether the object generated by Al model. Answer with yes or no.
(GPT-4 Answer: NO
Surface Texture Detail: The object, which appears to plush with textured surface. exhibits highly detailed and organic woaldworld
typically imperfections unless the specificall generates rea isti imperfections Normal Maps: The maps shown on the side the image reveal surface detail ad texture interactions with light, which could be from real-world also sinulate such details, making it difficult to differentiate based on nomai ma alone. Color and Lighting: The lighting and co especially gonerative
Al-generated
worlddss scan. However, advanced models can
consistent with world object captures, abstract textures, though modem models can highly reallstic.
often create samewhat smoother
Ground Truth: YES
Analysis:
terms surface texture detais, the texture detai is relatively good, but the distribution of the texture is irreguler and lacks symmetry. Additionilly there are notceable differences in texture fineness acress different with the texture on the doll' head being finer. Regarding the normal the image shows nomal detail, significant differences surface material represented the normal, exhibiing varying degrees protrusions depressions.
Figure 37: Judgement example generated by Gaussian Splatting based methods in 3D modality,
with GPT-4o's answers
65

---

Published as a conference paper at ICLR 2025
3D: 3DGS-based [Multi-Choice]
Question:
You have been shown two different images. These images are rendering displays four-view RGB renders on the left and four-view normal maps the right, illustrating the surface texture and lighting interactions of the 3D model. One the objects generated by Which of the above objects most likely generated by Al?
A. The first one
The second one
(GPT-4o) Answer: (B)
The exaggerated and vibrant color contrasts in the normal maps (on the right) suggest that it was produced by Al, as these types of extreme variations in lighting and surface texture often characteristics of Al-gennerteed 3D models. Additionally, the slightly smoother, less detailed texture in the RGB renders further points to Al generation.On the other hand, the second image (the orange and blue camoufiaged aircraft) appears more realistic with textures and details that align with traditional 3D modeling or real-world designs.
Ground Truth:
(B)
Figure 38: Multi-choice example generated by Gaussian Splatting based methods 3D modality, with GPT-4o's answers
66

---

Published as a conference paper at ICLR 2025
3D: 3DGS-based [Multi-Choice]
Question:
You have been shown two different images. These images are rendering displays four-view RGB renders on the left and four-view normal maps on the right, illustrating the surface texture and lighting interactions of the 3D model. One the objects generated by Which of the above objects most likely generated by Al?
A. The first one The second one
(GPT-4o) Answer:
(B)
The second set of images seems display more stylized color distribution in the normal maps (right side of the images), with unrealistic lighting reflections and a variety of vibrant colors that aren't typical in real-world renders. The left side of the second image also exhibits more abstract, less natural shadows and forms that indicate a computer- generated or stylized -lgeeraed models often focus on abstract or highly stylized visuals, whereas the first image seems have more natural texture and surface patterns, which might be captured through traditional 3D rendering techniques.
Ground Truth:
(B)
Figure 39: Multi-choice example generated by Gaussian Splatting based methods 3D modality, with GPT-4o's answers
67

---

Published as a conference paper at ICLR 2025
3D: NeRF-baaee(OpeeeddEndelll
Question:
This is al generrated 3D renderinnn Displays four-view RGB renders on the left and four vier normal maps on the right Your Task Analyze explain inauthentk aspects the 3D object
Rendering: Identify issues seem neuthertic. Reference specifkc parts the images to support observations.
testures, colors,
apearance.
explanations why aspects
2oNoraal Rendering: Analyze the nomaa maps inconsssiseese es or errors surface geornery. Explain any anomalies that suggest object attifiially generated. Use specific visual evidence from
the explanations.
(GPT-4o) Answer:
appear blurry common issue Al-generared mages. This is evident the unifority of the surface, which lacks the natural Imperfections and variations found in real-world materials. Unnatural Color Consistancy: The colar of the object uniformly dark green, lacking the subdie variations typically caused lighting and materal pmoprties. Real objects otten exhibit slight colar shifts due to shadows, reflections,
Rendering:
Shadows:The ighting can observed the absence depth dimension, making object appear more than a 3D model. Specific Observations: The top lfft botto eft images show a lack of shadowing that would typicall occur on real object, particularty around the edges and comers. Rendering: Uniform
Lighting
shadows highights.
image
geometry. The objects. lacks natural bumps ridges The normal maps use range of colors represent surface angles, but the transitions between these colors are abrupt and do not refeec the gradual changes seen real-word surfaces. This suggests lack complexity the surface geometry. indicating
Normals: The nomall mags display lack variation nomaal maps too consistent, indicating
overly
Color Anomalies:
Specific Observations:
right and bottom right images, the color transitions representation light interacts object.
approach to
modeling
Ground Truth:
RGB Rendering:
The texture the sofa appear merge checticlly, disorganized
enough, with noticeable blurriness backrest. The cushion and quite blury, edges
edges
appearing
Normal Rendering:The geometric surface of the sofa has low precision, with chaolic geometric representation of the backrest. The cushion backrest conform to reality and exhibit georettic asymmety..
Figure 40: Abnormal detail explanation example generated by Nerf based in modality, with GPT-4o's answers
68

---

Published as a conference paper at ICLR 2025
3D: Others [Opened-Ended]
Question:
This is an alngerratted 30 rendertg Displays four-xtew RGB renders on the left and four-view normal maps on the right. inauthentic aspects 1RRG8 Rendering loentily issues related to textures, colors, and overall apearance. Provide detailid explanations why these aspects snem nauthertic. Reference spectic parts the images to support observatios.
Task: Analyze
2.Naraal R kendering: Analyze the nomaa maps for inconsistencies or errors surface geometry. Explain any anomalies that suggest the artific generated. specific visual
maps explanstions.
(GPT-4o) Answer:
teddy bear appears everly smooth found in real materials, The does not exhibit the expected randormness and depth, making look flat and artificial. Color Uniformity: The colors are too uniform and saturated, especially on the bears body and limbs. Real objects tyically have subtle variations color lighting matarial properties, which missing
Rendering:
inconsistencies
single source. Shadows and Highlights: the object appear realstic, liahting would create more coherent shadow pattems. Ears and Feet The ears and feet have uniform color and texture do not reflect the typical wear and tear variation seen real plush toys. Seams and E Edges: The seams and edges of the bear are too clean stuffed animal.
inconsistent, with
highigghts aligning
inconsisterc, stency
lack the fis fing irequlariles that would be maps would present in real object. This smoothress can make object less Ireike, Lighting Interactions
Rendering: Smoothness:
overyy
details
way light interacts with the surface in the normal maps too uniform. Real objects have varied surface geometry that causes light scatter complex interolay light shadow. Limbs and Body: The limbs and body the bear the appear uniform material. Faclal Features: The facial features lack subtte deptn contouring that present real teddy bear, making them appear and less defined.
differett directions, creating expected
Ground Truth:
The texture resolution of the teddy bear average, with abnupt tewture transitions on the surface that da not match limbs inconsistent from different Normal Renderlng:Thh geometric surface of teddy bear has poor precision, with rough surface normals abnormal protrusions. The geometric shapes of limss are incorrect, and theme is lack of geometric symmetty.
RGB Rendering:
surrounding
Figure 41: Abnormal detail explanation example in 3D modality, with GPT-4o's answers
69

---

Published as a conference paper at ICLR 2025
3D: Others [Judgement]
Question:
This is rendering of a 3D object. The left side of the rendering shows 4-view RGB rendering, and the right side shows 4-view normal map, showing the surface texture and lighting interaction of the 3D model. The object may be generated by an Al model or may be scan of a real object. Please determine whether the object was generated by Al model. Answer with yes no.
PT-4)) Answer: YES
The object has simpified, styized design with smooth surfaces and exaggerated features (such as the eyes and penci in is tentacles). These characteristcs are common digital Al-generatted models, especialyy for animations games. Consistent Surface Smoothness: real-world objects physical scans. Al-generated models often exhbitscch moothhnsss because they are optimized clean, stylized appearances. Normal Maps: The nomal maps on the rights de also sugcest simple, comouter-genereet generated surface with minimal texture, inclictinn the lightin and surface detail are computed rather than captured from object. Fantasy Design The holding pencil whimsical, playful style strongly suggest creative, artifiiial design typical modeling a scan of real-world object.
Cartoon-like Appearance:
highly smooth
impertections typically
Ground Truth: YES
Analysis:
Although cartzon appearances are very common in animation and games, there are still items in real world, such as dalls and tays, that a cartoon appearance. These items can be scanned to obtain corresponding models with a cartoon look.
Figure 42: Judgement example 3D modality, with GPT-4o's answers
70

---

Published as a conference paper at ICLR 2025
X
3D: Real [Judgement]
Question:
This rendering a object. The side of the rendering shows 4-view RGB rendering, and the right side shows 4-view normal map. showing the surface texture and lighting interaction of the 3D model. The object may be generated by Al model may be scan a real object. Please determine whether the object was scanned by real- world camera. Answer with yes or no.
(GPT-40) Answer: NO
maps, typically generated during modeling in computer graphics. These maps display surface orlentation and how interats with the object, often used simulate detalled textures without increasing the polygon count. in scans of rest-world objects you usually don't tsee raw normal maps this form-thy processed into final render. Smooth and Clean Geometry: object appears to have smooth surfaces and uniform lighiing suggests nolse, or iregularites due scanning limitations, lightin variatios, or texture distortions. Lighting and Shading: The cons stent lighting across the different viows implies the object was It in rendering enwronment where lighting can be perfectly controlled. Rt Real-world scans often have subde lighting reflections that difficult avoid. Absence Surface Noise: Real-world scans tend to surface nuise small texture distortions due the scanning from discrepanciesscch imperfactions indicating highly contralled digital environment. Rendering Style: The RGB rendoring and normal maps presented this specific format (side by sidei are common in digital workflows 3D madel visualization especially when developing raviewing models the context gaming, simulations. or animation.
Maps: right
image
processes
ing
Realworld
imperfections,
factors. The
object image
Ground Truth: YES
Analysis:
Regarding the normal maps, the results shown the image can be visualize by collecting data through 30 scanning, rather than being generated solely through 3D modeling texture processing, terms of geometric shape, the smooth and clean surfaces result scanning envtromennt, consistent lighting achieved from multiple angles
generated by exhibtt scanning resutts.
By controling the stabiity scanning
Figure 43: Judgement example in 3D modality, with GPT-4o's answers
71

---

Published as a conference paper at ICLR 2025
Audio: Audio [Judgement]
Question:
this audio fake, generated by an Al model, is from real recordings? Answer with single word.
LTU Answer: NO
This audio clip likely to be from real recordings of people coughing and respiratory sounds, as opposed to being artificially generated or edited.
Ground Truth: YES
Analysis:
This audio is synthesized cough sound. The model has understood the content of the audio, but lacks the ability to determine whether the audio is synthesized.
Figure 44: Judgement example of category Audio in audio modality, with LTU's answers
X
Audio: Music [Judgement]
Question:
Is this audio fake, generated by an Al model, or is it from real recordings? Answer with single word.
LTU Answer:
NO
Real. The acoustic features the sound consistent with those produced by electric guitar and amplifier.
Ground Truth: YES
Analysis:
This music is a generated by AudioLDM. The model correctly identified the core elements of the music, but failed to determine whether it is real or fake.
Figure 45: Multi-choice example of category Music audio modality, with LTU's answers
72

---

Published as a conference paper at ICLR 2025
Audio: Singing [Multi-Choice]
Question:
You are provided with two audio clips. One of them real recording, and the other one fake, generated by an model. Identify which audio fake by selecting correct second audio Audio Answer by
Audio number. The first audio Audio
lyping the number of the real audio.
(A)
(B)
Qwen-Audio Answer: (A)
The first audio is real.
Ground Truth: (B)
Figure 46: Multi-choice example of category Singing in Audio modality, with Qwen-Audio's an- swers
Audio: Speech [Multi-Choice]
Question:
You are provided with two audio clips. One of them real recording, and the other one fake, generated
Identify which audio fake by selecting correct Audio number. The first sudio Audio 1, and the second audio Audio 2. Answer by typing the number of the fake audio.
(A)
(B)
AnyGPT Answer:
The second audio is fake, generated by Al model. <sosp>
691><
691>
60>
868>
868>
Ground Truth: (A)
Analysis:
The model did not provide the correct answer; instead, It output irrelevant content from another non-text modality.
Figure 47: Multi-choice example of category Speech in audio modality, with AnyGPT's answers
73

---

Published as a conference paper at ICLR 2025
Text: Abstract [Multi-Choice]
Question:
pafferentaassages You have been shown two different passages, One of the passages generated by LLM, while the other written by real people. Which the above passages most likely written by real people?
B. The second one
A. TAMMFFOOEE - WAR.AAAREEERRRRRRRRRRRRRR 0330UU 00A 2 SALmInA,, 9097712
PEE BL DU SA BWW FRROA
FREE MBS 2FAL V96G09 TE BAN MUM EN
(GPT-4o) Answer:
LiPlished and Concise Structure: assagge One & ebly corces and woll a arganizes, presenting hh prodien and ls sigeiknnccir
strughhereent riaityy
sanoy
2.0 d. and Lack of Nuance: The prossing in Passage Oas is broud and generalzess he bases faced by bunis, facas ng impartince
ooctness out diens doeph
speciiis.
ovrrdia upp comminii Al el-goneratez
conivet the foom summortrnn
3.Matural Flawe and Detal in Passage Twe: Paxcage Tax haw more raturs fove and aekides suenced wa arattee, eart as prodi
eeforuncing the 188
cimilar
damesik small nod
passige
the
aLLess Formel and Mone Contertual C Connections: The senteine emuntinss in Parsage Then are longer and more centedually diccussss lenges by banis contact,
connectas, ahcwing moro roloood cecirraary y
compored
Ground Truth:
Figure 48: Multi-choice example of category Scientific Papers Abstract in text modality, with GPT-4o's s answers
Text: Abstract [Judgement]
Question:
have been shown passage. passage may generated LLM written by real people. Is this passage written by real people?
in AURHAAANGAEEEEEE Mlee de www.--tttowssh.... Yull 908409:94 2998. *6AHHHALL MAAAAUH d6Me6244 RRoEEt TA AA---------- 303te wtw.eettttttettteeeeeeee ae
(GPT-4o) Answer: YES
(cloctommgg ritic uncel "eg ahr" jelccooragnen compotiil by. and "240r tra BP (o,day source corbo lot. These are uses accuriicty and cat erent contart which inples storng indersandinng the tokk. Humron whtee famillars wit subeect
2.Problem 5 elution Soucture: Tis passage love log structuree typica of tochnicca decumentater nosearch summany starts with the prod teleccoonggccle starfornnc and computt do Ry Bases in electoric dos cool. dertiiies the cause (2. souisa proposee
step-by
resekiiinn weteeett:: hh crmoneertte tectrical analyees.
5. Cencise and Oijeccivve Tone. The lons of passage is concis. Earnal and otjectino, whiih is sommro on taalt human wilten beused
tochi a Conteectull Use of tedustry Practice:: The pareace desothes practiaa spproch to solung elecho rocpuedc inereeeeee in and docussing
such ony extoonous
skpisk ikc emazilismmnss This tpoc then
werting wine
interaatvo
specialiedd
The trant sppeee thousing metorall,
losams) "Bige wwwodty--dweoodd methods indiratue cntrycess poinsi. bedkeetee
jeerilstin about erginwedng desigt processe and not genersting prnaee bed
S.Pocu Complaance and Standards: passage emphas aes loding coma somallance
a
(meating
meeinn spectic bectin soandarss liy mnmeenattte contect complecee with mey pnttons is pocthnal cusner for figheeee and prndiot designess trei
anthrr hightyes
concatbillyy it exczooii devices. Bialy orginces beternnaal inoi. The la contast and doaall adsume fam lo techninal leme wnt concepts. whin sugpeon at passage was cootted by homan portic suctence mn...
Ground Truth: YES
Figure 49: Judgement example of category Scientific Papers Abstract in text modality, with GPT- 4o's 's answers
74

---

Published as a conference paper at ICLR 2025
X
Text: Essay(Multi-Choice]]]
Question:
You have been shown two different passages. One of the passages is generated by LLM, while the
written by people. Which the passages most likely written by real people?
A. The first one
The second one
BOTI 10/R MES HAR WAN
Trtt, REVOROO MUSOA in BARROOO 7ha Pe Ao
RERMA oi WEBARNNL,, 20RI M0 MwM MAN PEREE FFEM RESSEAAA
SNRSS MT. 6-ESRONETT BE 2ER 7 ra-hhh
RAMEET PEOPURE 2 RAA
WIR RNT
Rie 257 7 MEE
B34), 7
VILE INLE HE *. Win & MASEGEEE "RRIIEETT" INRERNI 28 RA420877N A PIEAAR
R 10 WEDTTAL ThE $5546 mit
WMB. 28 R.-.2
edti ). 20t1a01s
32 F
57
SFt. T BOW (995 W M.S.. 733037774
(GPT-4o) Answer:
Parsage 'LE 2U2 and L.L. These references traditonalC paetry iterature indicate ot ciltral contect and wrintt for whic ls more typcet ofhuman suthersip.
3.50ylictic
poetic using
motonnaa Flow and Authenticity Passage Tas has more nanual emstional flow mouing from crtical view technology app
experences
metop language
expression
creativty inanspector.
3dvanncd Perspective on Time Passage Two selves a ph lssoltical pesseettve time and Rt. connectingggggg omoorott ceporic 1e2s and ces on reture itaalanees cricksm technoloog log with admnratioo for the beauty of lifs in way that tooks zuthemtic adva2rs
thought provoliing onher
while
mointa
structure balanced tone.
arssending on montir more drsightoonood manner tosusss on the downdites of tochnolagy and the imooria nce of baking srakk, which dolleeies in soruttirdd rod way withoutthe deeger leflcctve syye soon in Passage Two. The use paralla structuess clarity
serphoe bey
sighs Algeeeeeee
Ground Truth: (A)
Figure 50: Multi-choice example of category Essay in text modality, with GPT-4o's answers
Text: Essay [Judgement]
Question:
You have been shown passage. This passage may be generated LLM or written by real people. Is this passage written by real people?
G
#UL RDP: Mi Wwd Reaattd2 AL siakedtt 4/3:70 BE5). 7M055/aa//FFiitaaaaaaa REFRRC22,, SEMIVS
MMLLLMMA4 (C
(GPT-40) Answer: YES
1.Cultural and Liberayy Refernnces: en: The passage stats with gustation furm the Southem Bynasiees "8 statin 3-b 15-4-h esttine 1' 1r
lineness can endere stammig hundred ligtt slite lor ticusanc miss). human willer bpoccall usiss provde desth and authoris hei willing
2.8 legant and Poetic Language: The pessage uses highly poeti and exp esciue inngaaee with metachors and siidd imagery such
"A R. upe the rundi MMA Rariintt the hummnn hears, and "B Siii & Bousard zrigtt iins Tanging light.
eiggent
tytca expressions
cowate impressiin.
3.Comples Sentences and Thoughtful Structure The struttue the pansage is sepristieaee sterd with long, comp senterces, fhat
canwey mull pie ideas themes, from
fistress pack Tewth glmbe unlty procides thoughtld progremkin from organbraten
conterpooay glozel
sheoing
thrduhh
A.Stonn Emotional and Monel Appeal: The peocage enphesines Bhamn of al am, sulderiyy and global cooperation the laow
CoVID-11 pansemnk tning eriiliee lerguage unry and celecci cel-being knd strong manal appeal day water pirkowoptio
5ithe of Rhetorical Techniques:
The passage wilec uass mettrial devices lis repaliion contrelt and para ce to shergghen messaage woampl, phrasse the te MEE (degal harm tamer human heaet and "hh t8 Hacol IF er iclane
comman fealaww luman wring partic
spaeches hes mentt
a.Conteabull Awerranen and Real-World Relevence: The paosage whnters es the gibu CoVID parsemic and Chinala role in providigg cubli healh supoont vaccines.
aniite parpective Liena's sntematonall commbatere traminn
AAd0 geopolikal want moral inparativs. hiss levil contantial messensss and connectio real apicaaly found bumain-sshhaed contare.
Ground Truth: YES
Figure 51: Judgement example of category Essay in text modality, with GPT-4o's answers
75

---

Published as a conference paper at ICLR 2025
X
Text: News [Multi-Choice
Question:
You have been shown two different passages. One of the passages generated a LLM, while the other written by real people. Whic Which of the above passages most likely written
by real people?
A. The first one
B.
second one
Werld Economic Forum (W undenesy Daves, Switzertadd nd. drawing over 2.000 promine it business and political leaders from amo ind the globe. key sttendess include UK Prime Mi misterr Tony Blar and South Affican President Thato Nbskl, who will engage disuusions on pressing global lis offrag
World leaders business and tcal leadess from around the alobo antiing in the swiss mountain resort caves for the ennual word economic forum (wef). For five days, tley will diccass loues ranging from chins's sconomic power to irad's
uncerteinty
2,000
after sunday's
growing economic influence
elections prime
tony and south atrican Uni
following
minimal with ant: globa laation activiss having canci lid planned semonstattin. in co-silel event the Braci an city of Porto legre is hosting the World Social orum, whichh seeks to promate allamaatve soludons to glotel smobles. The wil sackle word's most pressing challer challonges, shaplig the global agends the year ahozd.
againstihe
anicepated
and prodests apainss the wel are expectad to be nuteo, And- grobellstton campa ers have called of demonstritin
caders sttending
planned for the we eelkiend, nd. The brazillan city of parts aleare will host the roai world social forum, timed to run pansllel with the wet's event davos.
(GPT-4o) Answer: (A)
I.Clrity and Structure: P Passage One is well soructured with clear sentences and logical progress on idess. it offacthuely intrddcess the crdi conomic Frrmm (WFF), merrions the bey aftendees, and prosices connect for both the WFF and the pama in World Secial Forum event in Brazi. The shresing is polishes, which & typical proteess lonal human writig.
Fassage One uses now pre se language contectullyy apprepriate vocabulary. Phrrses "pressing Z.anguage "recent dlections" "Shaping global agandal suggesta nusnced understanding of glotal ssues."
3.Basanced and Objectivity The
maintsins
informative and tral
overview avoids every dtamati or ortinoornl language, which oftan be sesn in terated cortent.
4.5tyitte Consistency ansage Two, while inio methee, lacks some stylotic consent etency and foo for esample, "wo leaders gather face uncartainty starts with dramatic and sompwhat amblquous spaning, which contrasts with the more facuual style of rest the passage. Additonally. theee are small g prammatinal mennsstent ies, lice prine ministertory Mallc are less common professional writing by humans but occur generazed texts.
capils iwiton hoaks mouneen resert dowess
Ground Truth: (B)
Figure 52: Multi-choice example of category News in text modality, with GPT-4o's answers
Text: News [Judgement]
Question:
You have been shown one passage. This passage may be generated by LLM or written by real people. Is this passage written by real people?
Worl leaders gather to free uncertainty more than 2 020 business and poliiccll leaders from around the globe are artidin in the swiss mouniai divets annual werld acanomic forum (weli flw days. they will distuss kosses ranging from econtme thebo
powerto lna's future after the sunday's eiectioss prime aepecaad be muted, Arti-glaba kaatinn campaigners have called off demanstralinn planned forthe weekand. The braz lann parto a egre will host the rhra world social forum, tieed to parailel with the wet's rizier eventin davze.
tony
south
smong the
PT-40)) Answer: YES
jourblictic style, covzring real world such World Econemii: Forum (WFF) Davis, See treland, and key allnnees, including Tony Blai and Thaso Mbaki manionss speciic details like the cores, Issues discu ssed (Ch na's coonomic power lrag o Mutur and concument Word Socia Forum Brizil This idnd of conteci-icc resording apically crafled by hunal witsr to
oumalistic and Reat-World Context
readers of cumnnt everts.
2.Proper Struchuring and ogicaal Flow ideas The passage fows logicaly modducing the main event (WEF Davesi, outining the tcipans
and poliicall
salt) and broviing lngically
context (dscussiers about global issues typicl
previus
coherent compreherskv neutial and belenced, commen tralt news repartngg as aims present facts rather than opin ons. There no everty blased anguage or exapgeratinn and the focus is on prov ding information This balances presertation a mark human pumall kmm where objectidly & generally a goal.
3.Balanced and Neutral Tone The lowe the
Alnciusioon of Global and Specific Debiiss: The passage nc udess both brord tockes igooca leaders decussing world issues! and specifc cocals role Eaata procest. concu foruns Ports Alegra
include spacif::
complete
while Al-generated
may grammatical
of
missing captalicatinn laaders heads state laadors attnndingt These inforcctions in grammar and style are more characterists of human hutraw which can prone io acnasional emors or nahhed adlingg especiall ii reai-tmre reporting or drifts..
Use. While passage (e "vaiss mountuin resoe devee" instwed Thoies mountein
coherent Devrad)) sight swicowedness
Ground Truth: YES
Figure 53: Judgement example of category News in text modality, with GPT-4o's answers
76

---

Published as a conference paper at ICLR 2025
T
Text: Novel [Multi-Choice]
Question:
You have been shown two different passages. One of the passages generated by a LLM, while the other is written by real people. Which of the above passages is most likely written
by real people?
A. The first one
The second one
unbellevable monstrosity. tertiely maginative mina and portrayed
eyes widening loomed figure torn straight from nightmares. grotesque impossiblle made real. was close, too close, its menacing presence bearing down on her, choking with tear breathed retreat, but body remained grim fascination her own private horror.
the TDs by actor suitably defar by the gentus make up. this was TD mage- monster was real and he was feet away, her, challenging hurting her, candemning
very
pounded ensnared by
(GPT-4 Answer:
(A)
1.5 tylisticc Complexity and Unconventional Language: P Passage 0 ne uses unique phrasing and complex, theariall descriptinn like 'an unbellevable monstrosity, born in tertlely maginative mind and portrayed the TDs This creativity and suggests
typical Al-generated content. 2.Narrative Depth and Tone Shifls: The passage has subte shifts tone and a layering of emotons that are intricatelly woven together, such as awe, diegust, and condemnation. This kind of muilifaceleed depidion emolions often reflects human author's nuanced grasp of storytailing, whereas Al generattd text sometimes lean toward more prdictable straightforward emotional arts.
exploring language
3.R A.ealistic of feelings Sar experiences awe, hurt, condemnate This flow feels more natural storytelling style, an urgency and intensity that doasn't overly structure each sentence for maximum clartly Human urtting aften embraces these imporfectons for stylistic effect.
Imperfections: There slighty fragmented
Passage
DravttDraatiiztioon Passage
quite
impossiblity
"choking and heightened style, typical of generated content attemping convey strong emotions vividy. 5.More Fluid perception the figure. This indirect apprnach feels nuanced and indicative of human writing, which often details imagination.
paipable poisoned very breathed."
paints picture,
more conventional
Passage One: Passage One uses less expieit descriptinn terror, focusing instead Sar's reaction and
Ground Truth: (A)
Figure 54: Multi-choice example of category Modern Literature text modality, with GPT-4o's answers
77

---

Published as a conference paper at ICLR 2025
X
T
Text: Novel [Judgement]
Question:
You have been shown one passage. This passage may be generated by LLM or written by real people. this passage written by real people?
the dimiyy It drawing rom of kzoottrel's grand estate, the tension was palpable. Merecraft, with his sly smile and obtuscated Intentions, articulated his grand scheme involving the fabled dukndom Drownedland. Fiddottrel, gullbbe and avaricinus, listened with attention, inchpin
deception lurking in every word Merecraft uttered. Merecrafts plot, though groundwork for the follies soon to ensue. The real drama began when F tzdottrel, a fitof misguited bravado, decided to leverage the affice of the Master of Dependencies, pasition ostentatious it was functionally ambiguous, to ncits quarrel with W ittipal. W itipol. through charade but played along for amusement provided. The quarrel, trival appearance led
unaware
unfolding events,
astute and perceptive, unexpected turn of events transfer Fitzdttrel's estate Manly, Maniy, the estato to Mrs. Fitrdattrel, woman undeservedly caught the web her hushand's absurdites. The fourth acts curtain rose to reveal nanish lady, swathed in sils and lace, embodying the very essence of foreign fashion that was both spared facade: skewered absurdity of societal follies miror
sense justie and perhaps pity,
restore
Wittipol in an elaborate disquine admired ridiculed by London society. The satire billng; Jonson's things
impol's disquise reflecting their own pretensi ons and affectations. As the play approached hed it conclusion in, the fithh act delved into a salire an demoniats, adding another layer to anson's criqque of contemporat vary ebuses. The devil-plot with undertones of dark humor, served as a filiin appendix the unfolding madness. Merecraft, the architect of decet, found himself entangled his own schemes, while Fitzdottrel none the wiser, continued hapless journey through for is rich historical context and scathing crilque of societal ils. Through a masterful fusion of character-drivee drama and intricate plet development. onson weaved narrative that was as entertaining as was thought-provoking Elements a comedy of manners intermingled with semi-political satire, creating tapestry of ethics and human folly As the tinal drew close, the audience was left eternal dance
adulation
more
audience.
political
social abyrinths onson deftly constructed.
lauded play
ponder enduring relevance onson's themes backdrop of society that, despite the centuries, all too famillar.
deception truth, felly and wisdom, set against the
PGPT-4 Answer: YES
sophisticated language describe the play's events Terms like "palpable tension," "obtuscated intention, 'nnd "ostentatious was functionally ambiguous reflect nuanced grasp vncabulary and styln. hie model can produce complex sentences, the depth and procision in the choice of words show level of Intentionality that typical of human wrting. 2.Deep Analysis Plotand and and themes within
1.Complex and Contextual Language: The passage
and
the play's characters, motivations, and moral molications the plat offers insightful connections between different elements the play. This lind of thematic dissection and commentary reflects a human's capackly to draw meaning and signific ance from ittrary works, something that models may not execute with the same depth of interpretation.
intepprtt passage provides detalled, analytical
3.Clear, Structured Flow and Transitions: The passage carefuly structured, introducing Merecrafts scheme and Filddotrel's topic the next, and
play. narrative builds toward an insightful conclusion. Such structured analysis, with strang sanse narrative progression and logit, Is characteristic of welli crafted human wriling.
gullibiity before delving
eath
naturaly, smooth lions
4.Historical and Literary Contextualizatin: The passage contextuelize lieeze the play's satire within bruader cultural and histerical
context, as London socielys attitudes loward foreign feshion the criique societal pretensions. situeltss ons on's within its time, whice also hinting at contemporary relevance. This context aware discussion indicates depth of historical and literary knowledge that alinn more closely with human write's perspective. 5.8 alanced ed and Nuanced Critique The passage themes, significance. refiects the ssathhnn criique societal "fusion character-driver drama development" and the "comedy of manners intermingeee with semi politicl satire This kind of balanced criique acknowledging both the entertainn tetertaamenntallll value and the ceeper commentary of the work- demonstraaes human-like criical thinking. 6.5ophis tic ated Use of Metaphor and Imagery: The writer uses metaphor and caliing
summarizes but also critcally engages with the play's structure,
effectively such as comoaring iippols "biling" Such metaphorical language and
dieguise as Sparish lady "mirro use of imagery add depth and color to the analysis, which is often halmmark of skilled human writers.
up to audienoe,
7.Reffection on R Relevance and Themes: The passage ends with reflective statement on the "enduring relevance of onson's themes," contemplating how the dance between tdeception and truth, tolly and wisdom" remains pertinent. The capacity relate historical modem-day
sign
analysis connects past and present meaningful way.
Ground Truth: NO
Figure 55: Judgement example of category Modern Literature in text modality, with GPT-4o's answers
78

---

Published as a conference paper at ICLR 2025
Text: Ancient Literature
T
[Judgemen]]
Question:
You have been shown one passage. This passage may be generated by a LLM or written by real people. Is this passage written by real people?
Wiliam Barker, a devated servent of the late J ohn Fastalf, along with Margret Wyssetyr testified about the unique reletionship between Sirj shn and W viliam Paston, respected gentleman from Norfolk, P Paston was more than just casual arquaintance; he was trusted kiraman ahn, bund that led him being sppuinted lightly, despite not receiving any form payment or reward during legal responsibiltées driven purely by goodwil and the strength of their re resship Following Sir onn's passing, the executors his estate turned to P Pasion for assislance. W thout hesitation, P Paston continud shoulder the considerable burden of managing extensive affais, embarking numerous journeys, and covering substank act loya
feofize, tustee, for ohn's properties. This rals
Paston lifetime, willingly undertock substantial
and
to protect and defend Sir ohn's estate. This
expectation personal gain: commitrent nnot overstated The legal matters dealt with were complex and tme-consuming. requiingg deep understanding the law and unwavering dedication The journeys he undertook were not merely cesual trips but ardupus expeditions often fraught with the challenges Inconveniences travel in that Moreover, managing defending substantial duty and affecton for S oonn He endured great personal incorveneence and expense, demonstratin an unwavering commitment upholding Sir] ohn's egaacy and ensuring that his estate was ma hanaged with the same care and ollience Sir] ohn himself would have shown essence, willam Paston's role in Sirjohn Fastolfss life and posthumous affairs was not merely that trusted kinsman but of ally guardian. actions were evidently recognized and valued by those who knew him, including W William Barker and Margret Wyssetyr Their testimony serves as poigrant reminder of the power of loyalty and goodwill, even in the face of considerable challenges and personal sacritics.
took
respect
kinsman.The magnitude
he incureed were significant, reflecting the high stakes involved self-interest but profound
testament
depth
relationship
strength his
quallttes
PGTT- Answer: NO
1.Complex Structure and Detailed Context The passage provides well organized historical account, describing specific individuals (Wiliam P ustun. Sir Fastoll, W iliem Barker, and M Wyseety) their relatiorships detail The narretive unfalds logical sequence, starting background information reflection on the significance his actions. Such intricate tnnytelling and detailed context are characterssic of human writing. 2.E motional Depth and Character Analysis: The passage delves into the motivations and emotions of Wiliiam Paston, partraying him character of loyalty. duty, and respect Sir Fastolf emphasizes ha sellessnness, sense of respons billy, the emotinal type of emotional depth, character analysis, numam-authrre biographical historical text, as they seek to explore underlying mativetions and tharacter viruus.
characters,
description Paston's
ending
with
understanding human relatiossips are typical
3.5ophis ticated Use of Vocabulary and Style: The passage uses a rich and sophaticated vocabulary, with phrases such as "feoffee trustee," "substa tasks and legal respons billiies "magntude of Paston's commitment; and "a profound sense duty." The formal language and Al can generate sophisicae ated vocabulary, the consistent use of complex phrasing throughout the pessage suggests intentionality and coherence are halmarks is of human-crafted fted text.
sentence structures
butte to elegant writing style aligns
humen euthoree prose.
4.8 Balanced ed and Nuanced Tone The tone the passage ba balanced and respectful, presenting an account Paston's life with reflective dignified approach.
refrains challenges and virtues Paston's actions fair and compreherse when a aiming to provide an unblased and thoughth hituul depiction n of real galliste historical figures.
nuanced perspective, presenting manner. Human writers often adopt balanced approach
S.Focus on Historical Context and Legacy The passage emphasites not just the events themselves, also their significance and impact Paston's legacy. places Paston's actions within
context loyalty, duty, seffessness. making societal va lues of time. focus on interppetin historical events and contemplating their meaning a trait typical human-writhen history and hingraphy, as requires a deeper understanding of zontext and implication.
6.Detalled and Reallstic Depictions of Historiall Challenges: The passage describes the complexties of legal matters, the challenges traveling eratera, and highlights the arduaus nature his historical context, is a strong indicator that the passage is human writtnn
personal sacritces involved managing human touch relating such realistic challenges, paired an understanding
recognizes difficulties Paston faced
Ground Truth: NO
Figure 56: Judgement example of category Ancient Literature text modality, with GPT-4o's answers
79

---

Published as a conference paper at ICLR 2025
Text: Ancient Literature [Multi-Choice]
Question:
You have been shown two different passages. One of the passages generated by a LLM, while the other written by people. Which the above passages most likely written by real people?
A. The first
B. The second one
X2818n9
JOHNIB SEREEREEERR SERE SKT7A, 1E9--91522. #JEJMSLL ME, 2EE
22 SHEE.
FRIKEES
(GPT-4o) Answer:
1.Polished Balanced
solshed
descrliing
language chssical resembles how Al alngenerzzd oflan attemots to enu Nistaiical lanquage the ruanced stfisic variatoos found autherti cissical teat.
clear
formal
The sassage employs classicl Cl Chinase lanquape, uses more standartized style whthot some the deeper comp ecties specite atamatic eegressions oftn in genuire sests, descrbes events is opiiall writing
2.5 2,5impiiffed Classical Language:
coneey cassical themes
manner.
liee @FA E and 15.." which more natural and citalled decree style, showiing the and sik tasture sophicication that indicative of hittarial wringg
adterence classicl Chinese stylcii convention The language whechs
4,5ubdie bifferences in Tone and Emphasis: Passape Two uses tone that feels no rogal and formal with intiiata wording impensl decree. caneful construction sentences convey both command
enoiess that cepture authontative more typical humann wittton hatorical
respectior
Ground Truth: (A)
Figure 57: Multi-choice example of category Ancient Literature text modality, with GPT-4o's answers
Text: Philosophical [Multi-Choice]]
Question:
You have been shown two different passages. One of the passages generated LLM, while the other written by real people. Which of the above passages most likely written
by real people?
A. The first one
B. second ane
the tumble recesses thoughts, keenly aware fractures and conorains that marmarihh the andscape of soul. neach out with heart inden with humilly, to the divene, yearning for touch panccennd these earthy contiiccntiines Almighty,
Narrow mayest enter it's ruihzus recain Thou has that within whih must oflend Thine eyes:l condess and know i. fut who shall cleanse or whom should cry, save Thee? Lord, secred power ohthe enemy.
soul
Thou
contines. My trst
in boundless menry
spare servint do speak.
may expand
ng, mend that
neer ran solce, through seek not ust redemption, but also the furtifiation rostst futura varsgression, and iiss lint. am purged and preserved.
(GPT-40) Answer: (A)
1,Poilshed and Modern L Language Passage One employs mosern, nelehed language that in style and formaliy mality The ptrasing is smooth and re refacove, focusing concect ke the fracures and constrants titt mar cand my soul atrerhew chaneceraate of olAleeenerdedd cortent sim for disity and balanced expression witoou nuances ofhen historical
diene Rellections
redematon wthhut The language
personal historially grounded language.
ihates
wilkly aspiiatle and themaiially generalize, rather than fed to specific contaut insividual style.
3.Classical and Personal Vaice in Passage Twe: Pa Passage Two uses older, more classical style of nggieh. with direct eale the sivie, such as Lord dearse ma from my secret faults. The strucure & intirate and dirctt showing a personal plea for chiine interwertion unguage less polished bur more deep' fell, which is oten che theracterisic of human authorship hetericall relglous Emphatic
passage expression. This ungentand emotive style human-arttin devioonll confecsional works, emphasiinng personal alfonship beltwen the indiiiuul and diaiee
Passage directuse
Thou
Ground Truth: (A)
Figure 58: Multi-choice example of category Philosophy in text modality, with GPT-4o's answers
80

---

Published as a conference paper at ICLR 2025
X
Text: Philosophical [Judgement]
Question:
You have been shown one passage. This passage may be generated by a LLM or written by real people. Is this passage written by real people?
1E A2
RE9A M7. S Se 18 HA MER.. BE KA
#1 HUTten SE May I. E son BeR 5 SEREE Ste.. #ESA SE (BS Scrii
STOUES HEX BIL
15 BE mikay, EEMEEN
RE*RIENN UF PREE A RER Se J RAREER I F99, MERTS CHPLIER H Be..... Hha aott te T
8. A3U ( WOR
PGTT- Answer: YES
1.Logical Structure and Thematic Development The passage organized in clear and logical manner, parens-chid and employer-emppee relations hips. The argument builds gradually, examining they
from general
concepts specific examples, different interpersonal relationships development typ of human written essays or articees, aliming to provide comprehensive perspective.
dependencies. Such
2.Nuanced Exploration of Complex Ideas: The passege delves into the complexilees of interpersonal relationships, exploring not just it highlights both the sote ntaa for dynamics,
spiritual aspects human interests. the terial out also emotional, psychological. understanding harmony
conflict
requires
depth reflection and consideration
more characteristic of human writing.
3.Use of Concrete Examples to illustrate Abstract Concepts: The passage uses relatable examples to explain abstract ideas about relationships. moves from familla des rents children to professional interactions lemoloyers and employees), making the diacussion more
common technique by
philosophical
sociological content more engaging and relevant
a.Balanced Language and Tone: The tone of the passage is reflective and balanced, providing neutral perspective on the dynamics interpersonal re platonships. The benefiti, 'ifft 10 (ational communication and nuanced, striking balance between emotionll rational elements, which typical human-writen content aiming for depth. s.Sophisticated Use of Concepts and Analogles: The passage effechively discussas the role of interests and contictss relatonshiis, using appropriate analogies to illustrate how hamony discard arises. The distinction between materiai and non material lemotional, prychological understanding these concepts, something that requires nuanced thinking is less likely be produced by model.
terms like "H #5 iconfict interests), "# (cooperate achieve mutual
thoughtful
words. The language
spirtuall slaborated clearty insightfully, suggesting
wrtter
R.Reflections on Solutions and Harmony: The canclusion the passage does not just present problems but reflects on finding solutions for balancing confiiting interests, suggesting the use 'S ways achieve harmnny This characteristic human writers
proposing thoughtful 10 10 Bay (rational communization and compramise) balance harmany
ending provide a complete perspective on issue.
Ground Truth: NO
Figure 59: Judgement example of category Philosophy in text modality, with GPT-4o's answers
81

---

Published as a conference paper at ICLR 2025
X
T
Text: Speech [Multi-Choice]
Question:
You have been shown two different passages. One of the passages is generated by a LLM, while the other is written by real people. Which of the above passages is most likely written
by real people?
A. The first one
second one
scientiic whinsical Spiaking
much the
right 1968,
impectil moments, 1988, Hansen testfied about seriousnen smess human-cuuse geeal warming- plottal moment envirommnnaal awareness, And ligter notz. for trose pganning aedding. don't miss the laest cal oction of stunning wodding dresses, falorrd to make your special day unfo gettable.
testilee the United states Congress
werming
vends coused human actiity, and will posa an incressing threatto humanity the fusure Well, wel. That's unusually prescient a human. ooking for we wedding drees? Try new takee en timeless cisssi...
(GPT-40) Answer: (A)
1.Narrative Coherance and Tona: The first passage fows rahrally and maintairs a cohative tive tone throughout connects the seweingly ngly unsebeted lopica of apple pie. Jamso Hansen's herimary, and wwating sheoses with a paylid whimatae touch. This kind narrative creatiity and smooth bensitccing oten tharactbrste of human wrttigg
2.Topic J uxttposiltion White both passages menton simter tapkce tapple plo, James Hansen's testimory, and wedding dreopees the sophiticated style drass paralieis makes transiions, use "whnniccal idea' "rmpactful decusse,, suggests styles
firs: passage
approach storyte together characteristi ewhat sarcastic which
tyliiiii "Well, That's passage
degoired preseient fora
an Al myying mimi humor commentary.
4.Commercial and Promotiooal Language: The passage talks about weoding diesses in seems genuinely eithusisstiz and subily promosonsl, while the second passage feels more overt and directin its promotion with statements like "Baking apple pie? Discountore warehouze has you covered! which may come off as more mechanical.
Ground Truth: (B)
Figure 60: Multi-choice example of category Speech in text modality, with GPT-4o's answers
x
T
Text: Speech [Judgement]
Question:
You have been shown one passage. This passage may be generated by a LLM or written by real people. Is this passage written n by real people?
In times of strees, economic wonriss personal confonal or oriloss, many of us naturally seak refuge in our favorite spots nature. These sanchariess offer us not comfoot
comfort and milel but also space to play rellct, and fine purpose. Immeniing aurselves in the beauly of nature restores our perspcctive and heals our spirt, remindingg ging us of the simple pleasures and profound connections that groundd us humanity Let us cherch and protect these natural havens.
(GPT-4o) Answer: YES
L.Guneric Theme Flowery Language: passage discusses nature's restorative qualities in generic focusing broad stoss rellet,
shiasing somewhat dea spint',
iimerring oursetes profound
perscective specife, ttiltinn Personalizatio Unique Insight the indivitial perspective distrctive steryallnng human with include specife anecdotes, observatioss tone to make the message stand our. The Allgenerate contant ottn uses dilyy caple statements that apply to many scenarios offennolleng uniuee angle.
style
gengeerated
famillar human
that personalized
3.5th Yet Predictable Transitions: The flw between ideas is smooth, moving from stees relief to play and purcose. butin ctacl mannar. Human witers night use a varfety trnsitnn techniques, maing complex and simple sentancess, or even resrrting cortrdicsory steas create more syname namative. Al-generated coneent lencs to maintain a consistent and somewhat repetitive
4.0venuee of Posltive and Encouragement The passage uses posilvve language consistendy, amphasizing cherishing. excloration sallnncd and Non- Spectfic call Action: The fioll action that models lendenc more specific acout actiond to take, how to protectiattue, even explore the challenges dalng so.
sign
any contrast
salt Al-generztad motivationl inspicational
cherish protect these netural havens," paregrachs with troad orpositive takeswsy, human might
call
Ground Truth: NO
Figure 61: Multi-choice example of category Speech in text modality, with GPT-4o's answers
82

---

Published as a conference paper at ICLR 2025
X
T
Text: Wikipedia [Multi-Choice]
Question:
You have been shown two different passages. One of the passages is generated by a LLM, while other written by real people. Which the above passages is most likely written
by real people?
A. The first
B. The second
Mtl *L X
HEB: ENE
7MP7,, x xP 20u Bass yta KHEKK 80 A Sumu HEa 10.2 wieenk. EG 2 aan sogs 11--1 1. #1 B geleeeetttta ti riCAs, #HOM
18 1E ACTEIEE HA BU WI BEX. + 28 DAAEE
W 7 985611 MESA
Sare, Co BERE P SUP,
981 XIRUIX
#1S Mo
Risdillin Vilts. SE
TEANE PE
(GPT-40) Answer: (B)
1.Chhrent 5 Brrucau Bure and omprehensive Overviau Passage Two prosents strcturd and comprihensse sive evenicw the ghi chtppialicall ideas Wilesian schoo, focus ng on the concapt: the four elements and the contributons of anacmenes The passage naturally integrates background context expla ning school's signlitance and the role their cosmolgyy. this type clear intagiated staytelling bptcal of human
where emphasá buildig complete
lons Explanatory Style
cimmetss
7 found
developmental
odis death and reflection
broader
content.
3.Passage One's Fragmented and Descriptions:
suicky
concept
another thour extansive contect or detai led explanation. The phrasing more abrust, such as "Bint WiS m 1135" (eemed the
earlest form of the four elements), which lacls the riceer context and connective language that human wrtter might include to explair corcepts more thoroughly
41ess Polished and More Literal Language in Passage One: The language in Pazsage One soems more ittraa and somewhot
dijpinted, focusing primanty on listing the formation and intrrction of the four elements, The passape feels like direct tonsittion or summary iless withou alborate that willers include to make content more accessible engaging:
Ground Truth: (A)
Figure 62: Multi-choice example of category Wikipedia in text modality, with GPT-4o's answers
T
Text: Wikipedia [Judgement]
Question:
You have been shown one passage. This passage may be generated by a LLM or written by real people. Is this passage written by real people?
(GPT-40) Answer: YES
Terninoooy passage understading the moter which These
Imelodic
"RIj fyer thamonic
within context exploining lizd taoks.
2.Clear Structure and Dafnntiin: The passaga is saucturd in tpikal bestback educational content, starting with broad defin nn" interveli and then treaking down into 12 the"" (melodic intervali and "E se" (harmonic interval.. This clear, logical spproscs definin a tern and providing subcatigonies educationa co taxds.
natural may that humans presentiifommmioon especially
N.taturll Flow C Coherent E Explanations: The passage maintairs coherent fow expleerr the distrction beaween the two types interak stats by defiiing cveraching concept
A 1930 29 ower conceptortenal will then moves on in prov exampe definitin pregressioo layerd
higher pith
crown. This
typical maintains
explanations
informative which approoriate belanced tone hander Alugenerated consistently maintain, especialty specialzed
educational
intraducin
domalss.
S.ackk of Redundancy or inrelevance Al generated content sometmes exhibls reduncarcy or intrduces slighdy off topkc details contrast, this passage is concise and stays strrily on tosk, locasing onl on defining intenviss mus whoout veeiiig off inelesant information or repeating Iself unnecssaafly.
6.Domain- Specific Knowledge The passage assumes some level familarity with music theory focusing directly on the definitoss ps between netes withou elaboration laysoople. This implies that some priori knowledee, supgests intenton educate carlly spectfic technical tams, common human
calaa lo on ausience
Ground Truth: NO
Figure 63: Judgement example of category Wikipedia in text modality, with GPT-4o's answers
83


Reviews:
[{"review_id": "9Mxjv4E9Oq", "reviewer": "Reviewer_xU39", "summary": "The paper introduces a novel benchmark designed to evaluate the capability of LMMs in detecting synthetic data across multiple modalities, including video, image, 3D, text, and audio. LOKI is structured to provide diverse modalities, cover 26 detailed categories, and offer multi-level annotations, enabling tasks that range from basic authenticity judgments to fine-grained anomaly selection and explanation. The benchmark consists of 18,000 questions across various levels of difficulty, allowing for a comprehensive analysis of LMMs in detecting synthetic content. Additionally, the paper includes evaluations of 22 open-source and 6 closed-source LMMs, revealing both the potential and current limitations of these models in terms of detection accuracy and interpretability.", "questions": "No follow-up questions.", "limitations": "", "rating": 8, "confidence": 5, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "+ LOKI is a novel, multimodal dataset.\n\n+ The paper is easy to read and well-organised.\n\n+ Comprehensive Evaluation and Validation.\n\n+ Curates a diverse dataset with 18,000 questions across five modalities and 26 categories, providing a solid foundation for synthetic data detection evaluation.\n\n+ Detailed Annotations.\n\n+ Directly addresses the challenges of synthetic data proliferation, impacting security, misinformation, and content authenticity.\n\n+ LOKI’s findings on LMM strengths and weaknesses have the potential to drive advancements in synthetic data detection and multimodal model development.", "weaknesses": "-  The benchmark lacks a robustness test against common real-world conditions like compression artifacts. To enhance real-world applicability, the authors could include performance evaluations on compressed data.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "YROGUhTbJw", "reviewer": "Reviewer_rrG5", "summary": "The authors introduce  LOKI, a novel benchmark designed to evaluate the ability of LMMs to detect synthetic data across multiple modalities. With the concurrent preponderance of synthetic data and the rise of powerful LLMs, the authors aim to address the emerging research topic of LMM evaluation for synthetic data detection. LLMs can provide reasoning behind authenticity judgments, benefitting explainability. The focus of LOKI is to evaluate the performance of LLMs on synthetic data detection tasks. \n\nIn particular, LOKI is aimed at addressing several shortcomings present in extant synthetic data evaluation datasets, including emphasizing human interpretability and multimodality. Overall, LOKI provides key improvements to synthetic data detection, including: diverse modalities, heterogenous categories, multi-level annotations and multimodal synthetic evaluation. framework.", "questions": "Do you have a sense as to why the prompting strategies (FS, CoT) in general had a poor/neutral impact for most of the model/data pairs (Table 5)?", "limitations": "", "rating": 8, "confidence": 4, "soundness": 3, "presentation": 4, "contribution": 3, "strengths": "The paper is well-written and easy to follow.\n\nThis work addresses a glaring and emergent need in a topic field: synthetic data detection for LLMs; I agree with the authors that there doesn't currently exist a comprehensive, multi-modal, nuanced dataset including explainability assessment for this domain area. \n\nExtensive examples and case studies provided in appendices. \n\nMany data domains are covered in this benchmark, including several categories and characteristics that are often underrepresented in synthetic data detection (e.g., satellite images, \"abnormal details\").", "weaknesses": "While the differentiated modalities, categories and annotation levels are beneficial, the overall size of the dataset actually seems relatively small vis-a-vis related datasets (Table 1). \n\nIt is unclear to me, how a user can methodically compare scores for different models across tasks/categories (e.g., in Table 2); perhaps the authors can address this, given the heterogenous and imbalanced nature of the data modalities and tasks, as well as the problem/domain \"difficulty\". \n\nAs deepfake detection is one of the most prominent synthetic data detection categories today, I believe the benchmark would benefit from its inclusion.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "AJNhLuNGYb", "reviewer": "Reviewer_AhiU", "summary": "This paper introduces LOKI, a novel benchmark for evaluating large multimodal models (LMMs) in detecting synthetic data across multiple modalities, such as video, image, text, and audio.This benchmark features broad-level assessments, including judgment and multiple-choice tasks, alongside detailed anomaly selection and explanation exercises, providing an in-depth evaluation of large multimodal models (LMMs).", "questions": "1.\tIn the fine-grained anomaly selection task, all samples were manually reviewed to ensure question quality. However, how can the robustness of the task design be ensured in future large-scale applications?\n2.\tCould you provide additional insights into why Claude-3.5-Sonnet tends to misclassify synthetic images as real? For instance, does this result from limitations in the model's ability to recognize fine-grained abnormalities, or are there certain types of synthetic image features that are particularly challenging for the model?", "limitations": "", "rating": 8, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1.\tThe paper is well written and easy to follow. The authors provide sufficient technical details for readers to understand their work.\n2.\tThe benchmark designed by the authors encompasses a rich variety of modalities and diverse question types, enabling a comprehensive evaluation of LMM performance.\n3.\tThe authors introduce a metric called the Normalized Bias Index (NBI) to quantify the performance differences of the model on natural and AI-generated data across different modalities, which is an innovative way to assess model bias.", "weaknesses": "1.\tThe current evaluation mainly relies on accuracy and NBI; however, at low recall rates, NBI may not adequately reflect model bias. Additionally, the design of NBI may be insufficient to comprehensively capture various types of bias exhibited by the model.\n2.\tThe paper mentions that the model exhibits \"bias\" across different modalities. However, the specific causes of this bias are not thoroughly explored through experiments or comparative analysis. This conclusion may be based on surface-level observations without further investigation into whether the bias arises from data, model architecture, or task design.\n3.\tThe paper mentions that the Chain-of-Thought (CoT) approach can impact model performance in image and 3D reasoning tasks.  However, it does not provide sufficient experimental details to clarify whether CoT significantly enhances performance across all types of tasks or if it is only effective for the specific tasks currently evaluated.\n4.\tIt is suggested to discuss and compare more related works such as [1,2] in this paper.\n\n[1] Detecting and Grounding Multi-Modal Media Manipulation and Beyond. TPAMI 2024.\n\n[2] Detecting and Grounding Multi-Modal Media Manipulation. CVPR 2023.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "gYeXOBVqm7", "reviewer": "Reviewer_qo6R", "summary": "The paper introduces LOKI, a comprehensive benchmark designed to evaluate the capabilities of Large Multimodal Models in detecting synthetic data across multiple modalities. Recognizing the rapid advancement of AI-generated content and the associated risks of synthetic media proliferation, the authors aim to assess how well LMMs can discern real data from AI-generated counterparts.\n\nLOKI encompasses a diverse set of data modalities, including video, image, 3D models, text, and audio, covering 26 detailed subcategories such as satellite images, medical images, philosophical texts, and various audio types like music and environmental sounds. The benchmark includes over 18,000 carefully curated questions with varying levels of difficulty.\n\nThe tasks within LOKI are multi-faceted:\n\nJudgment Tasks: Binary classification to determine if a piece of data is real or AI-generated.\nMultiple-Choice Questions: Selecting the AI-generated item from a set of options.\nAbnormal Detail Selection: Identifying specific anomalies in synthetic data.\nAbnormal Explanation Tasks: Providing explanations for why data is identified as synthetic.\nThe authors evaluated 22 open-source LMMs and 6 closed-source models (including GPT-4 and Gemini) using LOKI. Their findings highlight that while LMMs show promise in synthetic data detection and offer interpretability advantages over traditional expert models, they also exhibit significant limitations. Models tend to have biases, lack domain-specific knowledge, and display unbalanced performance across different modalities.", "questions": "- Quality and Realism of Synthetic Data: How does the synthetic data in LOKI reflect the latest advancements in generative models? Are there measures taken to ensure that the synthetic data poses a realistic challenge for detection models?\n- Prompting Strategies Implementation: Could you elaborate on how chain-of-thought and few-shot prompting were implemented across different models? Specifically, how did these strategies impact models that are less capable of handling long contexts or reasoning tasks?", "limitations": "", "rating": 8, "confidence": 5, "soundness": 3, "presentation": 3, "contribution": 4, "strengths": "- Comprehensive Multimodal Benchmark: LOKI covers an extensive range of data modalities and subcategories.\n- Inclusion of specialized domains like satellite imagery, medical images, and philosophical texts pushes the boundaries of traditional datasets and tests models in less-explored areas.\n- Multi-Level Task Design: The benchmark doesn't just focus on binary classification but also includes tasks that assess models' abilities to explain their reasoning, promoting the development of interpretable AI systems.\n- Highlighting the Importance of Explainability\n- By testing perception, knowledge, and reasoning across modalities, LOKI contributes to the broader goal of advancing towards AGI.", "weaknesses": "- Limited Performance in Certain Modalities: The benchmark reveals that LMMs perform poorly in modalities like 3D and audio, which may be due to the lack of available models or training data in these areas.\n- Insufficient Details on Data Generation Methods: The paper could provide more in-depth information on how synthetic data was generated for each modality, which is crucial for reproducibility and understanding potential biases in the dataset.\n- Evaluation of Few-Shot and Chain-of-Thought Prompting: The analysis of prompting strategies is somewhat limited.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Spotlight)