LOKI: A Comprehensive Synthetic Data Detection Benchmark using Large Multimodal Models

## Abstract

Abstract With the rapid development of AI-generated content, the future internet may be inundated with synthetic data, making the discrimination of authentic and credible multimodal data increasingly challenging. Synthetic data detection has thus garnered widespread attention, and the performance of large multimodal models (LMMs) in this task has attracted significant interest. LMMs can provide natural language explanations for their authenticity judgments, enhancing the explainability of synthetic content detection. Simultaneously, the task of distinguishing between real and synthetic data effectively tests the perception, knowledge, and reasoning capabilities of LMMs. In response, we introduce LOKI, a novel benchmark designed to evaluate the ability of LMMs to detect synthetic data across multiple modalities. LOKI encompasses video, image, 3D, text, and audio modalities, comprising 18K carefully curated questions across 26 subcategories with clear difficulty levels. The benchmark includes coarse-grained judgment and multiple-choice questions, as well as fine-grained anomaly selection and explanation tasks, allowing for a comprehensive analysis of LMMs. We evaluated 22 open-source LMMs and 6 closed-source models on LOKI, highlighting their potential as synthetic data detectors and also revealing some limitations in the development of LMM capabilities. More information about LOKI can be found at https://opendatalab.github.io/LOKI/ .

## 1 Introduction

With the rapid development of diffusion models (Rombach et al., ; Dhariwal & Nichol, ) and large language models (Abdullah et al., ; Brown, ), AI-generated content (AIGC) technology has increasingly integrated synthetic multimodal data into our daily lives. For instance, tools like SORA (Brooks et al., ) can produce highly realistic video, while Suno (Shulman et al., ) enables the creation of music at a level comparable to professional artists. However, synthetic multimodal data also brings significant risks, including potential misuse and societal disruption (Cooke et al., ; Ju et al., ). For example, the risks include generating fake news using large language models (LLMs), synthesizing fraudulent faces with diffusion models for scams, and potential contamination of internet training data. Due to the convenience of artificial intelligence synthesis, the future Internet may be saturated with AI-generated content, making the task of discerning the authenticity and trustworthiness of multimodal data increasingly challenging.

To address such threats, the field of synthetic data detection has garnered widespread attention in recent years (Barni et al., ; Frank et al., ; Gragnaniello et al., ; Hou et al., ). However, most current synthetic data detection methods are primarily focused on authenticity evaluation, with certain limitations regarding the human interpretability of the prediction results (Li et al., ).
The recent rapid advancement of large multimodal models (LMMs) has sparked curiosity about their performance in detecting synthetic multimodal data (Ku et al., ; Wu et al., ). On one hand, for synthetic data detection tasks, LMMs can provide reasoning behind authenticity judgments in natural language, paving the way for enhanced explainability. On the other hand, the task of distinguishing between real and synthetic data involves the perception, knowledge, and reasoning abilities of multimodal data, serving as an excellent test of LMM capabilities. Therefore, the focus of this paper is to evaluate the performance of LMMs in synthetic data detection tasks.

However, traditional synthetic data detection benchmarks, such as Fake2M (Lu et al., ) and ASVSpoof 2019 (Wang et al., ), primarily assess conventional detection methods, and evaluations of LMMs in detecting multimodal synthetic data are still lacking. These benchmarks often miss fine-grained anomaly annotations represented in natural human language, making it difficult to transparently analyze the explainability capabilities of LMMs. FakeBench (Li et al., ) aligns more closely with our objectives, but it only evaluates the performance of LMMs within a single standard image modality, lacking both breadth and depth. Specifically, FakeBench fails to explore other modalities such as audio and 3D data, focusing primarily on general image types and not conducting thorough tests on expert domain images like satellite imagery.
To bridge this gap, we introduce LOKI, a comprehensive benchmark for evaluating the performance of LMMs on synthetic data detection. The key highlights of the LOKI benchmark include:

- •
Diverse Modalities. LOKI includes high-quality multimodal data generated by recent popular synthetic models, covering video, image, 3D data, text, and audio.
- •
Heterogeneous Categories. Our collected dataset includes 26 detailed categories across different modalities, such as specialized satellite and medical images; texts like philosophy and ancient Chinese; and audio data like singing voices, environmental sound and music.
- •
Multi-level Annotations. LOKI includes basic ”Synthetic or Real” labels, suitable for fundamental question settings like true/false and multiple-choice questions. It also incorporates fine-grained anomalies for inferential explanations, , enabling tasks like abnormal detail selection and abnormal explanation, to explore LMMs’ capabilities in explainable synthetic data detection.
- •
Multimodal Synthetic Evaluation Framework. We propose a comprehensive evaluation framework that supports inputs of various data formats and over 25 mainstream multimodal models.

On the LOKI benchmark, we evaluated 22 open-source LMMs, 6 advanced proprietary LMMs, and several expert synthetic detection models. Our key findings are summarized as follows:

For synthetic data detection tasks we find:
(1) LMMs exhibit moderate capabilities in synthetic data detection tasks, with certain levels of explainability and generalization, but there is still a gap compared to human performance;
(2) Compared to expert synthetic detection models, LMMs exhibit greater explainability and, compared to humans, can detect features invisible to the naked eye, demonstrating promising developmental prospects.

For LMMs capabilities we find:
(1) Most LMMs exhibit certain model biases, tending to favor synthetic or real data in their responses;
(2) LMMs lack of expert domain knowledge, performing poorly on specialized image types like satellite and medical images;
(3) Current LMMs show unbalanced multimodal capabilities, excelling in image and text tasks but underperforming in 3D and audio tasks;
(4) Chain-of-thought prompting enhances LMMs’ performance in synthetic data detection, whereas simple few-shot prompting falls short of providing the necessary reasoning support.

These findings highlight the challenging and comprehensive nature of the LOKI task and the promising future of LMMs in synthetic data detection tasks.

## 2 Related Work

### 2.1 Synthetic Data Detection

Currently, synthetic data detection has garnered widespread attention to prevent the misuse of multimedia synthetic data (Gragnaniello et al., ; Hou et al., ). The detection of synthetic data in image and audio has long been a popular research (Barni et al., ; Frank et al., ), while methods for synthetic video detection have recently emerged, such as DuB3D(Ji et al., ) and AIGVDet(Bai et al., ). However, most work primarily focuses on the binary distinction between authentic and synthetic data, resulting in poor interpretability. Some studies aim to enhance the interpretability of synthetic detection by providing latent representations(Dong et al., ), feature explanations(Chai et al., ), and artifact localization(Zhang et al., ); however, most research remains limited to the interpretability of abstract symbols, leaving a significant gap in alignment with human understanding. In practice, current AI-generated synthetic data still exhibits noticeable flaws, such as discontinuities in synthetic videos and insufficient geometric accuracy in 3D data. These shortcomings can be effectively captured and perceived by human users(Tariang et al., ), who can provide reasonable explanations. However, existing expert synthetic data detection methods fail to provide human-interpretable bases for their judgments.

### 2.2 Large Multimodal Models

Recently, the rapid development of multimodal large models (LMMs) has been notable, with models like GPT-4o (OpenAI, ) and Claude 3.5 (Anthropic, ) excelling in various tasks such as scientific questioning (Lu et al., ; Yue et al., ) and commonsense reasoning (Talmor et al., ), showcasing exceptional perceptual and reasoning abilities (Bai et al., ). Research has also applied LMMs to evaluate AIGC synthetic results, utilizing GPT to assess the quality of generated images (Ku et al., ; Peng et al., ) and 3D models (Wu et al., ), providing scores that align with human preferences along with interpretable justifications. Consequently, in synthetic data detection, LMMs can offer reasons for determining authenticity in natural language, paving the way for enhanced interpretability in synthetic detection. Moreover, LMMs can access features invisible to human users, such as deep image and spectral features, demonstrating their potential to exceed human detection capabilities. Furthermore, synthetic data detection involves multimodal data perception and complex logical reasoning, making it an excellent task to assess the capabilities of LMMs. This task also provides quantitative evaluation metrics like accuracy, allowing for a more direct assessment of model performance compared to more qualitative scoring tasks.

### 2.3 Synthetic Data Detection Benchmark

Currently, there are numerous datasets corresponding to synthetic data detection tasks, including those designed for traditional detection methods and those tailored for LMMs. For instance, traditional synthetic datasets such as Fake2M (Lu et al., ), HC3 (Guo et al., ), and ASVSpoof 2019 (Wang et al., ) have explored the performance of traditional deepfake detection methods across various modalities, but they lack assessments for LMMs models. VANE (Bharadwaj et al., ) evaluates the capability of LMMs in detecting video anomalies, including the detection of criminal activities in real videos and synthetic video detection, although it focuses more on video content understanding. Fakebench (Li et al., ) assesses LMM performance in the image modality, yet it concentrates on a single modality and offers limited subcategories. In contrast, LOKI covers a broader range of data modalities, including video, image, 3D, text, and audio, as well as data from specialized fields such as remote sensing, medical imaging, and environmental sounds. In terms of problem design, LOKI encompasses tasks for authenticity judgment, as well as more complex challenges like Abnormal Details selection and Abnormal Explanation, which test the LMMs’ ability to explain reasons in synthetic data detection.

## 3 Dataset

### 3.1 Overview of LOKI

We introduce LOKI, a multimodal synthetic data detection benchmark, designed specifically to comprehensively assess the capabilities of LMMs in detecting synthetic data. As illustrated in Figure [2](#S3.F2), LOKI encompasses a variety of modalities including video, image, 3D, text, and audio, with over 26 specific subcategories of data. The benchmark utilizes fine-grained anomaly annotations to construct a tiered variety of question types, including judgment questions, multiple-choice questions, abnomal detail selection and abnomal explanation questions, totaling over 18k questions.

Table [1](#S3.T1) provides a detailed comparison of LOKI with existing datasets, including traditional synthetic detection benchmarks and those tailored for evaluating LMMs. In terms of breadth, LOKI covers a wider range of modalities and finer categories. In depth, it goes beyond binary judgment question designs to include questions that require a deep understanding and explanation of detailed anomalies. Additionally, LOKI classifies question difficulty based on human evaluation metrics.

Figure: Figure 2: Statistical information of LOKI. The left side displays the detailed categories of each modality, while the right side presents the questions across different modalities. The inner circle numbers represent the data volume, and the outer circle numbers indicate the number of questions.
Refer to caption: /html/2410.09732/assets/x2.png

Table: Table 1: The comparison between LOKI and other benchmarks. Answer types include JD (Judgment), MC (Multiple Choice), and OE (Open-ended). ”Real paired” indicates whether real data is paired within the same domain, while ”Difficulty Level” shows if questions are graded by difficulty.

Figure: Figure 3: Examples of Synthetic Data Annotations: (a) Detailed annotations of video anomalies; (b) Detailed annotations of image anomalies; (c) Detailed annotations of 3D anomalies.
Refer to caption: /html/2410.09732/assets/x4.png

### 3.2 Data collection and annotation

Video: We collected 593 video clips by utilizing various closed-source and open-source models such as SORA (OpenAI, ), Keling, and Open_sora (Zheng et al., ), generating high-quality text-to-video synthesis data along with corresponding real domain sample data. For the AI-generated video clips, we employed the LabelU(^1^11LabelU: https://github.com/opendatalab/labelU) tool to annotate anomaly details, including anomalous segments and their descriptions, anomalous key frames, and global anomaly descriptions. As shown in Figure [3](#S3.F3) (a), anomalies in the videos, such as ”violating natural physics” and ”frame flickering,” are also annotated globally. Additionally, the anomalous segment from 02:54 to 06:27 is highlighted, with the corresponding reasons for the anomalies explained by human annotators. Furthermore, each anomalous segment includes an anomalous key frame to facilitate subsequent LMMs in accurately reading the anomalous frames when processing video data.

Image: We have collected over 2,217 images from 7 subcategories through existing dataset extraction, internet collection, and new data synthesis. The image synthesis methods include FLUX, Midjourney (AI, ), Stable Diffusion (Blattmann et al., ), and ten other different methods to ensure high quality and diversity of the data. For the synthesized image data, in addition to overall annotations, we performed anomaly region bounding and explanations, as shown in Figure [3](#S3.F3) (b). The region anomaly annotations allow for more fine-grained and specific labeling, which can be used for generating subsequent anomaly detail questions.

3D data: We conducted a comprehensive analysis of OmniObject3D (Wu et al., ), selecting scanned instances as ground truth within the same domain. By constructing prompt texts, we synthesized three Nerf models (Poole et al., ) and three 3D GS models (Tang et al., ), and supplemented them with results from the advanced commercial model Clay and some Nerf-based results from GPTEval3D (Wu et al., ). We collected a total of over 1,200 3D models from ten different synthesis methods, including both synthesized and real scanned data. Additionally, as shown in Figure [3](#S3.F3) (c), we performed texture anomaly description annotations corresponding to the RGB four views of the synthesized 3D data, as well as normal anomaly description annotations. Notably, besides the multi-view format, the 3D data also supports point clouds and panoramic videos.

Audio: We collected various categories of audio, including speech, singing voice, environmental sounds, and music. The speech and singing voice data ensured consistency in speaker timbre, sourced from the Logical Access part of ASVSpoof2019 (Wang et al., ) and the CtrSVDD Benchmark, covering four generation paradigms: TTS, VC, SVS, and SVC. Environmental audio data came from DCASE 2023 Task 7, with real audio from the development set and synthetic audio generated using multiple methods from Track A. Music data were sourced from MusicCaps, with synthetic music generated based on descriptions using MusicGen (Copet et al., ), AudioLDM2-Music (Liu et al., ), and Suno(^2^22Suno: https://suno.com/).

Text: Based on summarization and regeneration methods, we generated counterfeit texts similar to the original texts using mainstream models such as GPT-4o, Qwen-Max, and Llama 3.1-405B. We collected eight categories of text data, pairing each sample with a real text and a model-generated similar text, totaling 3,359 text entries. Our text data were categorized by length and language, including short texts (50-100 characters), medium texts (100-200 characters), and long texts (over 200 characters), with a 1:1 ratio of Chinese to English data. More information regarding the collection and statistics of each modality can be found in Appendix [B](#A2).

Figure: Figure 4: Example Questions of LOKI. LOKI includes four types of questions:(a) Judgment questions; (b) Multiple choice questions; (c) Abnormal detail selection; (d) Abnormal explanation.
Refer to caption: /html/2410.09732/assets/x5.png

### 3.3 Question Generation

Judgment Task: This task requires large language models (LMMs) to determine whether the input data is synthetic or real. As shown in Figure [4](#S3.F4) (a), LMMs need to answer the judgment question, “Is the provided audio generated by AI?” To minimize the influence of prompts on model judgments, questions are asked in two forms: whether the data is AI-synthesized or real, and identifying either the real or AI-synthesized data. Furthermore, we categorize the data into different difficulty levels based on human performance. If all tested human users (more than three) answer correctly, the task is classified as “easy”; if more than 50% answer incorrectly, it is classified as “hard”; all other cases fall into the “medium” category.

Multiple Choice Task: This task requires LMMs to correctly select AI-generated or real data from the provided synthetic and real data. As illustrated in Figure [4](#S3.F4) (b), LMMs need to complete the multiple-choice question, “Which of the following texts is generated?” The design of this question benefits from our collection of real paired data within the same domain, effectively assessing LMMs’ comparative analysis capabilities.

Abnormal Detail Selection: Based on fine-grained anomaly annotation data from modalities such as video, images, and 3D, we effectively design prompts and utilize GPT-4o to generate questions for Abnormal Detail Selection. As shown in Figure [4](#S3.F4) (c), for video content’s detail anomalies, we ask, “What elements can be seen as inconsistent?” By providing clear anomaly annotations, we can effectively reduce the hallucination phenomenon in GPT-4o, ensuring the quality of the questions. More details can be found in the supplementary materials.

Abnormal Explanation: Furthermore, we design open-ended abnormal explanation questions, requiring LMMs to independently identify anomalies and explain their reasons. As shown in Figure [4](#S3.F4) (d), we ask, “Why is the provided image AI-generated?” It is worth to note that in real anomaly explanation tasks, the input does not include bounding boxes around anomalous areas. Tasks related to Abnormal Detail Selection and Abnormal Explanation can more precisely test whether LMMs genuinely perceive corresponding detail anomalies rather than guessing answers.

Quality Control: To mitigate the impact of hallucinations of GPT-4o during question generation in abnormal detail selection task, all samples in this task undergo manual reviews. Each question that involves GPT must pass through at least two rounds of verification by human users. A total of 20 users participated in the verification process, which took approximately 160 hours to complete.

## 4 Experiment

In this section, we evaluate various Language Model Multimodalities (LMMs) under our proposed LOKI evaluation framework, which includes both open-source and proprietary models, multimodal LMMs, Audio LMMs, and text-based LLMs. Our evaluations are conducted in a zero-shot setting. In the following subsections, we first introduce our evaluation models and the evaluation protocols. Next, we analyze the performance of existing LMMs in synthetic data detection tasks, comparing them with human users and expert models. We will then discuss the challenges and shortcomings faced by multimodal large models in the current task settings. Additionally, we explore the potential impact of few-shot or chain-of-thought prompting on this task.

### 4.1 Baselines

LMMs. We evaluate 3 closed-source and 18 open-source LMMs across different model types and sizes. For closed-source models, we consider GPT-4o (OpenAI, ), Gemini-1.5-Pro (Team et al., ), Claude-3.5-Sonnet (Anthropic, ).
Given that modality alignment in multimodal LMMs may lead to a decline in LLM performance on text-based tasks (Dai et al., ), we also selected pure text LLMs, such as LLaMA-3.1-405B (Team, ), Qwen-Max (Chu et al., ) and Mistral-Large (Mistral, ), to evaluate the text modality.
In the evaluation of Audio LMMs, we selected high-performing open-source models such as Qwen-Audio (Chu et al., ) and SALMONN-7B (Sun et al., ). For proprietary models, we chose Gemini-Flash  (Team et al., ), which supports audio input.
Human Users. We invited over 50 human users, including senior university students and regular users, to participate in the judgment and multiple-choice question tests for different modalities of synthetic data. Each question was tested by at least 3 users to ensure the robustness of the results. Additionally, we designed an online platform to distribute random questionnaires, and more than 200 users participated in the testing of 15 basic questions.

Expert Models. We selected recently open-sourced expert-level synthetic data detection methods and their corresponding weights for testing, including video detection (AIGVDet (Yang et al., )), image detection (AIDE (Yan et al., )), text detection (RADAR-Vicuna-7B (Hu et al., )), and audio detection (AASIST (Jung et al., )). Due to the limited availability of 3D synthetic data detection methods, 3D was not considered. Additionally, there is no overlap between the training sets of these methods and the LOKI test data, reducing the possibility of data contamination. We selected only a small number of expert models for evaluation, primarily to serve as references, similar to the role of human references.

Evaluation Protocol.
Data Input: For the video modality, we utilize an 8-frame video clip along with corresponding questions as input. For 3D modal data, we employ the commonly used multi-view input method. Results based on surround video and point cloud inputs are also included in the supplementary materials. For other modalities, inputs are based on textual prompts combined with corresponding images, audio, and textual materials. During the evaluation, each model independently generates responses to questions without retaining any dialogue history.

Evaluation Metric: For judgement, multiple-choice and abnormal detail selection questions, we use the average accuracy rate as a metric. In addition to accuracy, we also calculate the Normalized Bias Index (NBI) based on recall rates to assess model bias. For open-ended questions regarding anomalous details, we use the GPT-4 model to assess the score of the responses. Further details on the calculation of evaluation metrics can be found in Appendix [C.2](#A3.SS2).

Evaluation Framework: To standardize the evaluation of different LMMs and various input modalities for synthetic data detection, we propose a comprehensive multimodal evaluation framework. This framework provides support for various input modalities such as 3D point clouds, videos, images, audio, and text, while unifying APIs of over 25 mainstream LMMs, ensuring both model compatibility and consistency throughout the evaluation process.

**Table 2: Results of different models on the LOKI for Judgment and Multiple Choice questions. (a) Multimodal evaluation of LMMs; (b) Text evaluaion of LLMs; (c) Audio evaluation of Audio LMMs; ^* denotes the closed-source models.**
| Model | Judgment | Choice | Overall |
| --- | --- | --- | --- |
| Human (Medium) | 69.2 | 71.1 | 70.1 |
| Expert model | 69.4 | - | 69.4 |
| LLaMA-3.1-405B | 56.8 | 73.1 | 64.4 |
| Mistral-Large^* | 52.2 | 69.1 | 57.8 |
| Qwen-Max^* | 48.3 | 44.4 | 46.5 |
| Claude-3.5-Sonnet^* | 61.5 | 89.2 | 70.7 |
| Gemini-1.5-Pro^* | 55.7 | 57.3 | 56.2 |
| GPT-4^* | 55.9 | 66.6 | 59.5 |

### 4.2 Synthetic Data Detection Results

In this section, we provide a comprehensive analysis of the performance of various LMMs and LLMs on synthetic data detection tasks using the LOKI dataset.

Judgment and Multiple Choice. Table 2 illustrates the performance of various models on judgment and multiple-choice questions in LOKI. For the synthetic data judgment task, the closed-source model GPT-4o achieves the best results, with an overall accuracy (excluding audio) of 63.9%. When real paired data is included for comparison in the multiple-choice questions, accuracy further increases to 73.7%. In the text modality, Claude-3.5 outperform other LMMs and LLMs, achieving accuracies exceeding 70%. In the Audio LMMs category, both open-source and closed-source models show performances comparable to random selection, which is not satisfactory.

Abnormal Detail Selection and Explanation. We compared the performance of different models on the tasks of abnormal detail selection and abnormal reason explanation, as shown in Table 3. GPT-4o achieved an accuracy exceeding 75% in abnormal detail selection and a score over 70% in abnormal reason explanation. This indicates that advanced LMMs like GPT-4o has demonstrated strong detail understanding capabilities, effectively analyzing and interpreting ”synthetic traces.” Notably, we observe that Claude-3.5-Sonnet (Anthropic, ) tends to misclassify synthetic images as real, despite the primary goal of our tasks being to explain abnormalities in synthetic images. More examples of abnormal explanations can be found in Appendix [E](#A5).

Comparing Humans and Expert Models. Humans exhibit an average performance of 76% in judgment tasks and 86.4% in multiple-choice questions, both 10% higher than the LMM method. Notably, if LMM tools are to be applied in production, their decision-making performance in judgment tasks must exceed 90% to be convincing. As synthesis technologies advance, the distinct ”traces” of synthetic data are becoming increasingly subtle. However, LMMs capture minute details, such as image features imperceptible to the human eye, demonstrating their potential to surpass human.

LMMs demonstrate superior performance in most tasks compared to expert models. This is primarily due to the rich and diverse sources of synthetic data collected by LOKI, which significantly differ from existing data domains, resulting in suboptimal generalization performance of expert models. The accuracy of synthetic detection by expert models trained on similar data should significantly improve. Currently, LMMs perform at a moderate level in synthetic data detection but surpass expert models in generalization ability. Unlike traditional expert models, LMMs possess the capability to explain the reasons behind anomalies, highlighting their unique advantage as synthetic detectors.

Table: Table 3: Results of different models on the LOKI for Abormal Details Selection and Abnormal explanation questions.^* denotes the closed-source models.

Figure: Figure 5: The multimodal large model capability assessment analysis results. (a) Model bias assessment, where the closer the color is to red, the more the model is biased towards classifying the data as real; the closer to blue, the more it leans towards synthetic data. The size of the square also represents the degree of bias. (b) The performance of GPT-4o across different image types and its difference from human users. (c) A relative radar chart of the model’s performance across various modalities, with Human benchmarks for comparison.
Refer to caption: /html/2410.09732/assets/x6.png

### 4.3 Large Multimodal Models Capabilities

Model Bias. The heatmap of the Normalized Bias Index calculated based on recall rates, as shown in Figure [5](#S4.F5) (a), is utilized for analyzing model biases. The results indicate that most models exhibit significant biases in synthetic data detection tasks, with a tendency to incorrectly categorize data as either real or synthetic. For instance, GPT-4o tends to classify textual data as real, whereas it is biased towards judging 3D data as AI-generated. Despite diverse questioning techniques implemented to minimize cueing effects, a pronounced bias is still evident across most models.

Lack of Expert Domain Knowledge. In Figure [5](#S4.F5) (b), we present the varying performance of GPT-4o across different image subcategories. The experimental results clearly indicate that GPT exhibits strong recognition abilities on common image types such as objects and landscapes, even surpassing human users. However, GPT-4o’s performance significantly deteriorates in specialized fields such as satellite and medical imaging, and in less-trained image types like documents. This suggests that current LMMs still lack certain expert domain knowledge.

Unbalanced Multimodal Capabilities. In Figure [5](#S4.F5) (c), we compare the performance of various LMMs across different modalities. Results indicate that current models excel in frequently trained modalities such as images and text, even surpassing human performance in some tests. However, their performance declines significantly on 3D and audio tasks, with most open-source models lacking corresponding capabilities. For future AGI to develop into a versatile assistant, it needs to possess more balanced multimodal abilities.

Model Performance across Different Levels. Based on human user performance, we categorized the difficulty levels of the questions, as shown in Table [5](#S4.T5), which presents the performance of selected models across different difficulty levels. As the difficulty increases, the performance of LMMs gradually declines, consistent with human user performance. Under challenging conditions, GPT-4o’s accuracy drops to only 44.4%, which is lower than that achieved by random selection. This indicates that LMMs have certain limitations in handling complex synthetic data detection tasks.

Prompting Strategies Improve LMMs Capabilities.

In Table [5](#S4.T5), we demonstrate the effects of different prompting strategies in LOKI’s image and 3D judgement tasks, where CoT refers to the Chain of Thought prompting (Wei et al., ) and FS refers to the few-shot prompting (Alayrac et al., ). During inference, models are prompted with two random examples that are in the same domain as the questions by different strategies. In CoT prompting, we manually craft ”thought chains” with our human annotations to elicit reasoning steps out of LMMs, while in FS prompting, we simply prepend examples with answers to the questions. Interestingly, GPT-4o shows strong reasoning ability without chain-of-thought prompting, while other models rely on it for improved performance. Few-shot learning fails to support the necessary step-by-step reasoning for synthetic data detection, but GPT-4o performs well regardless, suggesting its inherent ability to reason effectively without additional reasoning guidance. However, LLaVA-OV-7B experienced significant performance drop when prompted with CoT. We conjecture that this degradation may result from a decline in its ability to understand long contexts after fine-tuning (Zhai et al., ).

**Table 4: Result decomposition across questions difficulty levels.**
| Difficulty Levels (Video & Image & 3D & Text) |  |  |  |  |
| --- | --- | --- | --- | --- |
|  | Easy | Medium | Hard | Overall |
| (2470) | (1104) | (3938) | (7512) |  |
| LLaVA-OV-7B | 60.4 | 47.6 | 39.1 | 47.3 |
| InternVL2-8B | 64.5 | 47.8 | 33.5 | 45.7 |
| Qwen2-VL-7B | 67.7 | 45.6 | 35.2 | 47.4 |
| Gemini-1.5-pro | 70.8 | 42.4 | 32.4 | 46.4 |
| Claude-3.5-Sonnet | 76.0 | 44.7 | 29.8 | 47.1 |
| GPT-4o | 78.8 | 52.3 | 44.4 | 56.8 |

## 5 Conclusion

In this paper, we introduced LOKI, a multimodal benchmark designed to evaluate the performance of large multimodal models in detecting synthetic data across various modalities. We conducted a comprehensive study of LMMs’ performance on video, image, 3D, audio, text, and specialized subdomains, and we also analyzed LMMs’ ability to explain detailed anomalies in synthetic data. The experimental results indicate that LMMs have a certain level of competence in detecting synthetic data and a preliminary ability to explain anomalies. Synthetic data detection tasks also effectively evaluate the various capabilities of LMMs during their development. These findings highlight the challenging and comprehensive nature of the LOKI task, as well as the potential of LMMs in future synthetic data detection tasks. We aim to inspire more powerful and interpretable synthetic data detection methods through LOKI to address the potential risks posed by rapidly advancing AI synthesis technologies. Furthermore, while the relationship between synthesis and detection is adversarial, they are mutually beneficial; better and more explainable synthetic detectors will further advance AI synthesis technologies.

## Appendix A Synthetic Data Detection

In this appendix, we introduce and discuss the social impacts of synthetic data, such as deepfakes, as well as data contamination introduced by synthetic data. Finally we discuss the increasing attention on synthetic data detection.

### A.1 Social Impact of Synthetic Data

While synthetic data generated by AIGC technology has offered numerous benefits to various aspects of society, it has also introduced significant challenges and risks. One of the most concerning risks is the potential to use synthetic data to create deepfakes. All forms of synthetic data can be leveraged to generate deepfakes, which can then be used to deceive, manipulate, or defraud individuals or organizations (see Fig.[6](#A1.F6.1)). For instance, synthetic text data can be exploited to create fake news Papageorgiou et al. (), phishing emails, or manipulative advertisements. Similarly, synthetic image and 3D data can be used to generate realistic fake facesXu et al. (), scenes, or even content that leads to copyright violations Jiang et al. (); Somepalli et al. (). Synthetic video data poses a threat by enabling the production of fake videos or fake news (e.g., political propaganda Pawelec ()), as well as deepfake video fraud calls Mustak et al. (). Likewise, synthetic audio data can be used for fake calls, voice cand even fake broadcasts. Furthermore, the advancements in synthetic data technologies are also impacting employment in creative industries, exemplified by the months-long strikes in the film industry Bohacek & Farid ().

Figure: Figure 6: Soical impact of synthetic data across different modalities
Refer to caption: /html/2410.09732/assets/figure/fig-deepfake-news.png

### A.2 Synthetic Data Contamination

Figure: Figure 7: Model Performance Collapse Trained On Synthetic Data (Image from Martínez et al. (), Text from Shumailov et al. () )
Refer to caption: /html/2410.09732/assets/figure/fig-synthetic-data.png

In today’s LLM era, the internet is flooded with a substantial amount of synthetic data, even existing web-scale datasets are known to contain synthetic content Schuhmann et al. (). According to openai Altman (), they now generate about 100 billion words per day, while all people on earth generate about 100 trillion words per day. All of this points to the fact that synthetic data will dominate the internet data side.

The use of synthetic data has been shown to significantly degrade the performance of deep learning models (see Fig.[7](#A1.F7.1)), both for generation tasks and classification tasks Hataya et al. (); Ravuri & Vinyals (); Martínez et al. (); Shumailov et al. (); Bohacek & Farid (). Addressing the impact of synthetic data is crucial for the development of the next generation of models. There are two primary approaches to mitigating the negative effects of synthetic data. The first is exploring ways to better utilize synthetic data, proposing strategies for optimizing the integration of synthetic data into training pipelines Dohmatob et al. (; ); Feng et al. (); Bertrand et al. (); He et al. (). The second approach involves developing methods to accurately detect synthetic data, allowing models to distinguish between real and synthetic inputs.

### A.3 Increasing Attention on Synthetic Data Detection

Figure: (a) The number of BBC official news reports on deepfake topic over the years.
Refer to caption: /html/2410.09732/assets/figure/supp_deepfake_news_count.png

The growing prevalence of synthetic data has garnered increasing attention from society, including news reports, academic research, and government policies. The number of papers on deepfake detection has been steadily increasing, with the BBC reporting on deepfakes more frequently each year (see Fig.[8](#A1.F8)). In response to the rise of synthetic data, several governments and global conferences have also introduced policies aimed at regulating the use of deepfakes and synthetic data AIS (); Michelle (); Biden ().

## Appendix B Dataset Description

### B.1 Data Collection

Our data primarily originates from online internet collections, reused from public datasets, and self-synthesized into new composite data, as detailed in Table [6](#A2.T6). To ensure diversity in synthetic data, each modality incorporates more than five different synthesis methods (Figure [9](#A2.F9) & [10](#A2.F10)). To guarantee the quality of synthetic data, we also collected samples synthesized by mature proprietary models such as Sora, Midjourney, CLAY, Sunoco, and GPT-4. The far-right column of the table displays the public datasets that underpin our collected synthetic or authentically paired data.

Table: Table 6: Synthetic Methods and Public Datasets Across Modalities

Figure: Figure 9: Examples of some 3D and image datasets, with the bar chart showing the quantity of data in different categories.
Refer to caption: /html/2410.09732/assets/x7.png

Authentic pairing data: We have collected a significant amount of authentic paired data from the internet, including sources such as arXiv, Wikipedia, Gutenberg, YouTube, TikTok, and Civitai. For data sourced from the internet, we will rigorously verify that it consists of authentic recordings or text authored by human users, rather than content synthesized using AI technology. It is important to note that our current research primarily focuses on multimedia data directly synthesized by AI, with limited consideration of methods like deepfake involving manual editing; we will continue to update our approach in future studies.

Data Availability and Social Impact: In collecting data, we strictly adhere to copyright and licensing regulations of the source websites, avoiding data acquisition from resources that prohibit copying or redistribution. For the LOKI dataset, which is open-sourced, users must submit a download request to the authors to prevent misuse of the data.

Figure: Figure 10: Examples of video data. We used 7 video generation models to obtain corresponding data for LMMs evaluation.
Refer to caption: /html/2410.09732/assets/x8.png

### B.2 Dataset Annotation

#### B.2.1 Annotation Guidelines

Video: During the annotation of synthetic videos, we categorize the identified anomalies into two types: global anomalies and segment anomalies. Global anomalies refer to errors that persist for more than 80% of the video’s duration, while segment anomalies are issues that occur for a limited portion of the video. For example, as shown in Fig. [11](#A2.F11) (a), the anomaly of ”flickering textures and distorted geometries of fences and utility poles,” which is present throughout the video, is labeled as a global anomaly. In contrast, the ”abnormal flames” and ”basketball penetrating the hoop” in the video are classified as segment anomalies. Additionally, each identified anomaly, including both global and segment anomalies, is associated with a key frame that represents the anomaly, facilitating subsequent processing of video data by large multimodal models (LMMs).

Image: For the synthetic image data, we provide global anomaly annotations for overall image issues, as well as bounding box selections and textual descriptions for abnormal regions. The bounding boxes indicate the location and extent of the abnormal areas within the image, while the textual descriptions detail the specific anomalies present in those regions. As shown in Fig. [11](#A2.F11) (b), the ”texture quality issues” and ”color distortion” in the image are annotated as global anomalies, whereas area errors such as the ”texture errors” of the duck and ”reflection anomalies” on the water surface are classified as region anomalies. Annotators mark these areas by drawing bounding boxes and provide textual explanations for the reasons behind the anomalies.

3D Data: Unlike video and image annotations, the annotation of 3D data involves a global-scale analysis of textures and normals. In terms of texture anomalies, we focus on assessing the authenticity, smoothness, and edge clarity of the textures. For normals, we analyze whether the model’s geometric fluidity, surface smoothness, physical stability, and topological coherence are accurately represented. For instance, as shown in Fig. [11](#A2.F11) (c), we conduct a detailed analysis of the ”multiview discrepancies” and ”texture blurriness” in the model’s textures, while labeling issues such as ”abnormal protrusions” and ”asymmetrical structures” as related to normals, accompanied by appropriate textual explanations.

Figure: Figure 11: Examples of the synthetic data annotation process under different modalities, including (a) Video, (b) Image, (c) 3D Data.
Refer to caption: /html/2410.09732/assets/x9.png

#### B.2.2 Annotator Informed Consent

Before commencing the annotation process, we ensure that all participating annotators are fully informed and provide their explicit agreement to the following terms and conditions. This comprehensive informed consent is designed to promote transparency, respect their autonomy, and align with ethical standards in research. It is imperative that each annotator has a thorough understanding of the nature, purpose, and potential implications of their contributions to the labeling process. The terms are outlined as follows:

##### Data Usage.

Annotators acknowledge and consent to the possibility that the labeled data they generate may be used in various academic and scientific contexts, including the development of research papers, presentations at conferences, and other related scholarly activities. They understand that their work may significantly contribute to advancements in research fields such as natural language processing, machine learning, and artificial intelligence. Furthermore, annotators recognize that their contributions may be referenced or cited in scientific publications, thereby playing a role in shaping future research directions and applications.

##### AI-Generated Content.

Annotators are informed that some of the content they will be labeling may have been produced by artificial intelligence models. This includes text, images, or other data types generated by algorithms designed to simulate human-like outputs. Annotators understand that this knowledge is crucial, as it may influence their perception, judgment, and approach to the labeling task. They agree to remain mindful of the potential biases or preconceived notions that may arise from this awareness and commit to maintaining objectivity and accuracy in their work.

##### Potential Implications.

Annotators are aware of the broader implications of their labeling activities, which extend beyond the immediate scope of data annotation. They recognize the ethical considerations inherent in AI research, particularly concerning issues such as bias, fairness, and the societal impact of deploying AI technologies. Annotators agree to reflect on these ethical dimensions and to engage in the labeling process with a conscientious approach, acknowledging that their work may contribute to both the positive advancements and challenges associated with AI development and implementation.

##### Commitment to Ethical Standards.

By agreeing to these terms, annotators affirm their commitment to upholding high ethical standards throughout the annotation process. They understand that their participation is voluntary and that they have the right to withdraw from the project at any time, without penalty. Annotators also acknowledge their responsibility to report any concerns or issues that may arise during the labeling process, ensuring the integrity and reliability of the data they provide.

This informed consent process ensures that all annotators are equipped with a comprehensive understanding of their role and its significance. It aims to foster an environment of mutual respect and collaboration, where the contributions of annotators are valued and their rights as participants in research are protected. By clarifying the expectations and responsibilities involved, we seek to create a foundation for ethical and impactful research that benefits both the scientific community and society at large.

### B.3 Quality Control and Validation

In annotating videos, images, and 3D synthetic data for anomaly details, we maintain high standards and accuracy. All annotators possess at least a university degree and demonstrate strong decision-making and judgment skills. Before annotation, human annotators receive extensive training with numerous examples of common errors to ensure a comprehensive understanding of the synthetic data detection task. Each data instance is annotated for detailed anomalies by at least two human annotators to ensure quality. Ambiguous or unclear instances are marked for further study and annotation during team meetings. Furthermore, to avoid the impact of hallucinations from Large Language Models (LMMs) on tasks involving LMMs, all anomaly detail explanation tasks undergo manual review based on LabelLLM (^3^33https://github.com/opendatalab/LabelLLM).

### B.4 Special data description

In the field of document images, we have collected synthesized images in four categories: newspapers, academic papers, magazines, and reconstructed documents. The corresponding real data for these categories come from the M6Doc dataset. Currently, document synthesis primarily follows a layout-first, content-rendering-later approach. For the first three document types, we design specific empirical rules for layout generation during the layout phase; during the content rendering phase, elements are selected from a constructed corpus to fill the structure. For the reconstructed type, we employ a restructuring algorithm on the M6Doc dataset to rearrange the content of the documents.

In the field of remote sensing imagery, synthetic data is primarily generated based on street-to-satellite datasets such as CVUSA and VIGOR, utilizing methods like CUT and Skydiffusion. The synthetic imagery encompasses major natural scenes such as urban and suburban environments, with the satellite remote sensing images being of high resolution.

In the field of medical imaging, we primarily collected two types of data: the ISIC 2016 skin dataset and the M3DSynth CT dataset. For the ISIC 2016 dataset, we utilized GAN methods for direct data synthesis. The M3DSynth dataset comprises synthetic images generated from the real-world LIDC dataset using Diffusion and GAN models. The images are categorized into two types: those with real tumors removed and those with synthetic tumors artificially inserted by the model. Each synthetic image is paired with its corresponding original image, complete with precise annotations of the tumor insertion or removal locations. Considering that most users are not medically trained, we deliberately selected more evident abnormal images to reduce the need for specialized knowledge in making decisions about synthetic data.

## Appendix C Evaluation

### C.1 Evaluation Model

We compared various models on the LOKI benchmark to understand their capabilities across multiple tasks. We support over ten open-source models, including InternVL2 (Chen et al., ), LLaVA (Liu et al., ), Phi (Gunasekar et al., ), XComposer (Zhang et al., ), Qwen2-VL (Wang et al., ), MiniCPM (Hu et al., ), TinyLLaVA (Zhou et al., ) and Idefics2 (Laurençon et al., ), as well as proprietary models such as GPT-4 (OpenAI, ), Gemini (Team et al., ) and Claude (Anthropic, ). The following list details these models.

| Model Family | Model Version | Parameters | Links |
| --- | --- | --- | --- |
| Close-sourced, API |  |  |  |
| GPT4 | GPT-4o | N/A | [https://platform.openai.com/docs/models/gpt-4o](https://platform.openai.com/docs/models/gpt-4o) |
| GPT-4 | N/A | [https://platform.openai.com/docs/models/gpt-4-turbo-and-gpt-4](https://platform.openai.com/docs/models/gpt-4-turbo-and-gpt-4) |  |
| Gemini | Gemini-1.5-Pro | N/A | [https://ai.google.dev/gemini-api/docs/models/gemini#gemini-1.5-pro](https://ai.google.dev/gemini-api/docs/models/gemini#gemini-1.5-pro) |
| Gemini-1.5-Flash | N/A | [https://ai.google.dev/gemini-api/docs/models/gemini#gemini-1.5-flash](https://ai.google.dev/gemini-api/docs/models/gemini#gemini-1.5-flash) |  |
| Claude | Claude-3.5-Sonnet | N/A | [https://docs.anthropic.com/en/docs/about-claude/models](https://docs.anthropic.com/en/docs/about-claude/models) |
| Mistral | Mistral-Large | N/A | [https://docs.mistral.ai/getting-started/models/](https://docs.mistral.ai/getting-started/models/) |
| Qwen | Qwen-Max | N/A | [https://www.alibabacloud.com/help/en/model-studio/developer-reference/use-qwen-by-calling-api](https://www.alibabacloud.com/help/en/model-studio/developer-reference/use-qwen-by-calling-api) |
| Open-sourced |  |  |  |
| LLaMA | LLaMA-3.1-405B | 405B | [https://huggingface.co/meta-llama/Llama-3.1-405B](https://huggingface.co/meta-llama/Llama-3.1-405B) |
| InternVL2 | InternVL2-8B | 8B | [https://huggingface.co/OpenGVLab/InternVL2-8B](https://huggingface.co/OpenGVLab/InternVL2-8B) |
| InternVL2-26B | 26B | [https://huggingface.co/OpenGVLab/InternVL2-26B](https://huggingface.co/OpenGVLab/InternVL2-26B) |  |
| InternVL2-40B | 40B | [https://huggingface.co/OpenGVLab/InternVL2-40B](https://huggingface.co/OpenGVLab/InternVL2-40B) |  |
| InternVL2-Llama3-76B | 76B | [https://huggingface.co/OpenGVLab/InternVL2-Llama3-76B](https://huggingface.co/OpenGVLab/InternVL2-Llama3-76B) |  |
| LLaVA-OneVision | LLaVA-OneVision-7B | 7B | [https://huggingface.co/lmms-lab/llava-onevision-qwen2-7b-ov](https://huggingface.co/lmms-lab/llava-onevision-qwen2-7b-ov) |
| LLaVA-OneVision-72B | 72B | [https://huggingface.co/lmms-lab/llava-onevision-qwen2-72b-ov-sft](https://huggingface.co/lmms-lab/llava-onevision-qwen2-72b-ov-sft) |  |
| VILA | VILA-1.5-13B | 13B | [https://huggingface.co/Efficient-Large-Model/VILA1.5-13b](https://huggingface.co/Efficient-Large-Model/VILA1.5-13b) |
| VILA-1.5-40B | 40B | [https://huggingface.co/Efficient-Large-Model/VILA1.5-40b](https://huggingface.co/Efficient-Large-Model/VILA1.5-40b) |  |
| Phi | Phi-3.5-Vision | 3.5B | [https://huggingface.co/microsoft/Phi-3.5-vision-instruct](https://huggingface.co/microsoft/Phi-3.5-vision-instruct) |
| Idefics2 | idefics2-8b | 8B | [https://huggingface.co/HuggingFaceM4/idefics2-8b](https://huggingface.co/HuggingFaceM4/idefics2-8b) |
| Qwen2-VL | Qwen2-VL-7B | 7B | [https://huggingface.co/Qwen/Qwen2-VL-7B-Instruct](https://huggingface.co/Qwen/Qwen2-VL-7B-Instruct) |
| Qwen2-VL-72B | 72B | [https://huggingface.co/Qwen/Qwen2-VL-72B-Instruct](https://huggingface.co/Qwen/Qwen2-VL-72B-Instruct) |  |
| InternLM-XComposer | InternLM-XComposer-2d5 | 7B | [https://huggingface.co/internlm/internlm-xcomposer2d5-7b](https://huggingface.co/internlm/internlm-xcomposer2d5-7b) |
| mPLUG-Owl3 | MPlug-Owl3 | 7B | [https://huggingface.co/mPLUG/mPLUG-Owl3-7B-240728](https://huggingface.co/mPLUG/mPLUG-Owl3-7B-240728) |
| MiniCPM | MiniCPM-V2.6 | 8.1B | [https://huggingface.co/openbmb/MiniCPM-V-2_6](https://huggingface.co/openbmb/MiniCPM-V-2_6) |
| LongVILA | Llama-3-LongVILA-8B-128Frames | 8B | [https://huggingface.co/Efficient-Large-Model/Llama-3-LongVILA-8B-128Frames](https://huggingface.co/Efficient-Large-Model/Llama-3-LongVILA-8B-128Frames) |
| LongVA | LongVA-7B | 7B | [https://huggingface.co/lmms-lab/LongVA-7B-DPO](https://huggingface.co/lmms-lab/LongVA-7B-DPO) |
| Qwen-Audio | Qwen-Audio-Chat | 7B | [https://huggingface.co/Qwen/Qwen-Audio-Chat](https://huggingface.co/Qwen/Qwen-Audio-Chat) |
| SALMONN | SALMONN-7B | 7B | [https://huggingface.co/tsinghua-ee/SALMONN-7B](https://huggingface.co/tsinghua-ee/SALMONN-7B) |
| AnyGPT | AnyGPT-Chat | 7B | [https://huggingface.co/fnlp/AnyGPT-chat](https://huggingface.co/fnlp/AnyGPT-chat) |
| OneLLM | OneLLM-7B | 7B | [https://huggingface.co/csuhan/OneLLM-7B](https://huggingface.co/csuhan/OneLLM-7B) |
| LTU | LTU-AS-7B | 7B | [https://github.com/YuanGongND/ltu#pretrained-models](https://github.com/YuanGongND/ltu#pretrained-models) |

### C.2 Evaluation Metric

Average accuracy: For judgment, multiple-choice, and detailed selection questions, we use the *average accuracy* as the primary metric. The accuracy rate is calculated using the following formula:

$$ $\text{Accuracy}=\frac{N_{\text{correct}}}{N_{\text{total}}}\times 100\%$ $$

In this context, $N_{\text{correct}}$ is the number of correctly answered questions, and $N_{\text{total}}$ is the total number of questions. To minimize the influence of prompts on model judgments, each question is presented in two forms: one asks whether the data is AI-synthesized or real, and the other asks the model to identify either the real or AI-synthesized data. By averaging the accuracy rates across different forms of the questions, we aim to reduce the potential bias introduced by the phrasing of prompts and ensure a fair evaluation of the model’s performance.

Normalized Bias Index (NBI): To evaluate whether there is potential bias in existing models when determining authenticity on the LOKI benchmark, we introduce a metric termed the Normalized Bias Index (NBI) to quantify the performance differences of the model on natural and AI-generated data across different modalities, which is defined as follows:

$$ $\text{NBI}=\frac{R_{\text{natural}}-R_{\text{generated}}}{R_{\text{natural}}+R_{\text{generated}}}\in[-1,1]$ $$

In this context, $R_{\text{natural}}$ and $R_{\text{generated}}$ represent the recall rates for natural and AI-generated samples, respectively, under the corresponding modality. By normalizing the difference between the two, the model’s unexpected preference in making predictions can be quantified. Specifically, a positive and larger NBI indicates that the model is more biased toward predicting samples as natural, whereas a negative and smaller NBI suggests a bias toward predicting samples as AI-generated.

GPT-Score: For open-ended questions regarding anomalous details, we use the GPT-4 model to assess the score of the responses. We adopted a rating scheme, establishing a 5-level rating system with scores ranging from 1 (poor) to 5 (excellent). The final scores are normalized to a scale of 0 to 100. We adhere to the following scoring criteria:

1) Identification: Accurately detect the globally annotated anomalies and their corresponding detailed anomalous regions specified by human annotators.

2) Explanation: Provide accurate explanations for the causes of the anomalies, ensuring consistency with the reasons outlined in the human annotations.

3) Plausibility: Avoid misclassifying authentic regions as anomalous while encouraging other reasonable explanations for anomalies.

While the scoring criteria are similar across different modalities, they are slightly adjusted according to their content characteristics; for example, the image modality is subdivided into global score and regional score, whereas 3D data is subdivided into texture score and normal score.

Figure: Figure 12: The overall process for automated evaluation of Abnormal explanation questions using GPT-4o.
Refer to caption: /html/2410.09732/assets/figure/GPT-Scoring.jpg

## Appendix D Breakdown Results on Different Modalities

In this appendix, we present detailed results of various synthesis methods or classification themes across different modalities of data.

### D.1 Video

Tables [7](#A4.T7) and [8](#A4.T8) display the evaluation results of LMMs in video modality for judgment and multiple-choice questions, respectively. In the evaluation of video modality, the proprietary model GPT-4o achieved an accuracy of 71.3% for judgment questions and 77.3% for multiple-choice questions. The open-source model InternVL2-40B recorded accuracies of 62.0% for judgment questions and 65.7% for multiple-choice questions. Table [9](#A4.T9) presents the evaluation results of different models in the video modality for open-ended questions.

Additionally, using GPT-4’s detection outcomes as a benchmark, we assessed the quality of videos synthesized by different models. It was found that videos generated by Sora had a judgment accuracy of 67.2% and a multiple-choice accuracy of 64.5%, indicating a high quality of video synthesis. Conversely, videos synthesized by Runway and W.A.L.T exhibited more noticeable synthetic traces.

### D.2 Image

Tables [10](#A4.T10) and [11](#A4.T11) present the evaluation results of LMMs in the image modality for judgment and multiple-choice questions, broken down by specific categories. The proprietary model GPT-4o scored a judgment accuracy of 63.4%, primarily impacted by specialized image types such as Satellite, Doc, and Medicine, while achieving a multiple-choice accuracy of 80.8%. Among open-source models, Qwen2-VL-72B performed exceptionally well, even surpassing GPT-4o in specialized image categories. Table [12](#A4.T12) presents the evaluation results of different models in the Image modality for open-ended questions.

### D.3 3D

Table [13](#A4.T13) presents the evaluation results for different 3D synthesis methods. Tests on judgment questions for 3D models reveal that most open-source models perform close to a 50% random chance. In contrast, the proprietary model GPT-4o demonstrates better decision-making and judgment capabilities, particularly in identifying low-quality Nerf models, where it achieves an accuracy rate exceeding 75Assuming the performance of GPT-4o as a benchmark, we can assess the quality of 3D data synthesized using different methods. Among three synthesis methods, Gaussian synthesis exhibits higher quality compared to Nerf synthesis, while commercial models like Clay offer the best quality. Table [14](#A4.T14) presents the evaluation results of different models in the Image modality for open-ended questions.

As shown in Table [15](#A4.T15.tab2), we extracted 200 3D objects, with an equal ratio of real to synthetic data, and evaluated the performance of different 3D input formats, including point clouds, surround videos with eight input frames, and multi-view inputs. In comparison, for current LMMs, the performance between surround video and multi-view inputs is similar. PointLLM, which partially supports point cloud inputs, performs particularly well, indicating that the point cloud format still holds certain advantages. Additionally, our results indicate that compared to the 7B PointLLM, the 13B PointLLM is more inclined to respond with ”I cannot determine whether this is a real object or one generated by AI” rather than a straightforward ”Yes” or ”No.” We speculate that as the size of PointLLM increases, it incorporates more comprehensive measures to correct illusions, making it less likely to provide definitive answers when faced with uncertainties (Wei et al., ).

### D.4 Audio

Table [16](#A4.T16) presents the evaluation results of various audio LMMs on the LOKI dataset. Currently, there are few open-source and proprietary LMMs that support the audio modality, and most models perform poorly on synthetic data detection tasks, showing little difference from random selection. However, the AASIST method, trained specifically for speech tasks, demonstrates relatively strong performance, although it still underperforms in other modalities. Overall, human performance in the audio modality is comparatively weak relative to other modalities, yet it still exceeds the best-performing LMMs by 20%.

It has been shown that acoustic features, such as pauses between words and silent segments within audio, play a more critical role in detecting deepfake audio than the actual content of the audio (Liu et al., ). However, current audio language models are primarily focused on content comprehension rather than acoustic characteristics. From the model architecture perspective, audio language models typically initialize their audio encoders with pre-trained audio models such as Whisper (Radford et al., ), AST (Gong et al., ), or BEATs (Chen et al., ). These pre-trained models are trained with various data augmentation techniques, including SpecAugmentPark et al. () and SpecSwap (Song et al., ). The target of the augmented data remains constant, often focused on tasks like transcribing textual content, which shifts the model’s attention towards understanding content rather than being sensitive to acoustic features. On the training data side, most audio-text datasets to date emphasize content comprehension in tasks like speech recognition, emotion/age recognition, audio/music captioning, sound event classification and music genre recognition. As a result, these models lack the inherent capability to distinguish between real and fake audio based on acoustic cues.

### D.5 Text

Tables [17](#A4.T17) and [18](#A4.T18) present the test results across different genres in the text modality. Most models underperform in this modality, primarily due to the low distinctiveness of text data itself, coupled with the maturity of current text synthesis technologies. Notably, genres such as modern literature and philosophy may benefit from the extensive training and memory capabilities of GPT series models, showing relatively better performance compared to other categories.

In Table [19](#A4.T19), we evaluated the detection results for texts of varying lengths. The findings indicate that longer texts are detected more frequently and are more likely to reveal flaws. Additionally, we assessed the performance across different languages and found that, except for GPT-4o, most proprietary models perform significantly better in English than in Chinese. This indicates that the multilingual capabilities of large multimodal models require further development.

Table: Table 7: Judgement questions results of different models on the LOKI Video modality. ^* denotes the closed-source models.

Table: Table 8: Multiple Choice questions results of different models on the LOKI Video modality. ^* denotes the closed-source models.

**Table 9: Abnormal Explanation questions results of different models on the LOKI Video modality. ^* denotes the closed-source models.**
| Model | Overall | Correctness | Explanation | Plausibility |
| --- | --- | --- | --- | --- |
| LLaVA-OV-7B | 46.7 | 41.0 | 48.4 | 50.8 |
| InternVL2-8B | 46.5 | 38.0 | 48.4 | 53.2 |
| Qwen2-VL-7B-Instruct | 48.4 | 41.2 | 50.4 | 53.6 |
| Claude-3-5-sonnet^* | 50.1 | 44.2 | 48.8 | 57.4 |
| GPT-4o^* | 67.6 | 60.6 | 69.2 | 73.0 |

Table: Table 10: Judgment questions results of different models on the LOKI Image modality. ^* denotes the closed-source models.

Table: Table 11: Multiple Choice questions results of different models on the LOKI Image modality. ^* denotes the closed-source models.

**Table 12: Abnormal Explanation questions results of different models on the LOKI Image modality. ^* denotes the closed-source models.**
|  | Overall | Global Score | Completeness | Correctness | Plausibility |
| --- | --- | --- | --- | --- | --- |
| Qwen2-VL-7B-Instruct | 63.8 | 66.6 | 60.8 | 53.2 | 74.4 |
| Llava-OV-7B | 68.9 | 72.4 | 63.0 | 60.6 | 79.8 |
| InternVL2-8B | 72.2 | 70.0 | 67.6 | 61.2 | 85.2 |
| Claude-3-5^* | 1.7 | 2.0 | 1.6 | 1.4 | 1.8 |
| Gemini-1.5-pro^* | 77.1 | 77.6 | 70.8 | 70.0 | 90.2 |
| GPT-4o^* | 72.9 | 69.8 | 68.6 | 63.4 | 90.0 |

Table: Table 13: Judgment & Multiple Choice questions results of different models on the LOKI 3D modality. ^* denotes the closed-source models.

**Table 14: Abnormal Explanation questions results of different models on the LOKI 3D modality. ^* denotes the closed-source models.**
|  | Overall | Texture Correctness | Normal Correctness | Texture Plausibility | Normal Plausibility |
| --- | --- | --- | --- | --- | --- |
| Llava-OV-7B | 71.0 | 63.2 | 70.8 | 73.8 | 76.2 |
| InternVL2-8B | 71.3 | 62.4 | 69.4 | 76.4 | 76.8 |
| Qwen2-VL-7B-Instruct | 73.4 | 64.8 | 72.2 | 77.8 | 78.6 |
| Claude-3-5-sonnet^* | 78.2 | 70.0 | 80.0 | 78.8 | 83.8 |
| Gemini-1.5-pro^* | 70.8 | 62.4 | 67.2 | 77.2 | 76.4 |
| GPT-4o^* | 77.0 | 69.8 | 64.2 | 82.4 | 81.4 |

Table: Table 15: Judgment & Multiple Choice questions results of different models on the LOKI 3D modality. S-Video denotes Surround Video, MV-Image denotes Multi-Views Image.

Table: Table 16: Overall results of different models on the LOKI Audio modality.

Table: Table 17: Judgment questions results of different models on the LOKI text modality. ^* denotes the closed-source models.

Table: Table 18: Multiple Choice questions results of different models on the LOKI text modality. ^* denotes the closed-source models.

Table: Table 19: Performance Comparison of Models across Text Lengths and Languages. ^* denotes the closed-source models.

## Appendix E Case Study

In this section, we delve into an analysis of GPT-4o’s performance on synthetic data detection tasks, with a particular focus on the questions it answered incorrectly. This examination is crucial for understanding the model’s operational capabilities and limitations. The analysis not only identifies the model’s current shortcomings but also provides guidance for future design and training improvements. We randomly selected 50 erroneous instances from GPT-4o’s predictions across various modalities and difficulty levels for detailed inspection. These instances included multiple-choice, true/false, and open-ended responses, and even for the structured questions, GPT-4o was required to provide the reasoning behind its decisions. These instances were analyzed by specialized annotators who identified the fundamental causes of the erroneous predictions based on their expertise and annotations.

### E.1 Image

Figure: Figure 13: Abnormal detail explanation example of category animals in image modality, with GPT-4o’s answers
Refer to caption: /html/2410.09732/assets/x10.png

Figure: Figure 14: Abnormal detail explanation example of category object in image modality, with GPT-4o’s answers
Refer to caption: /html/2410.09732/assets/x11.png

Figure: Figure 15: Multi-choice example of category document in image modality, with GPT-4o’s answers
Refer to caption: /html/2410.09732/assets/x12.png

Figure: Figure 16: Judgement example of category document in image modality, with GPT-4o’s answers
Refer to caption: /html/2410.09732/assets/x13.png

Figure: Figure 17: Multi-choice example of category medical in image modality, with GPT-4o’s answers
Refer to caption: /html/2410.09732/assets/x14.png

Figure: Figure 18: Judgement example of category medical in image modality, with GPT-4o’s answers
Refer to caption: /html/2410.09732/assets/x15.png

Figure: Figure 19: Abnormal selection example of category animals in image modality, with GPT-4o’s answers
Refer to caption: /html/2410.09732/assets/x16.png

Figure: Figure 20: Abnormal selection example of category object in image modality, with GPT-4o’s answers
Refer to caption: /html/2410.09732/assets/x17.png

Figure: Figure 21: Abnormal detail explanation example of category object in image modality, with GPT-4o’s answers
Refer to caption: /html/2410.09732/assets/x18.png

Figure: Figure 22: Abnormal detail explanation example of category person in image modality, with GPT-4o’s answers
Refer to caption: /html/2410.09732/assets/x19.png

Figure: Figure 23: Abnormal selection example of category person in image modality, with GPT-4o’s answers
Refer to caption: /html/2410.09732/assets/x20.png

Figure: Figure 24: Multi-choice example of category satellite in image modality, with GPT-4o’s answers
Refer to caption: /html/2410.09732/assets/x21.png

Figure: Figure 25: Judgement example of category satellite in image modality, with GPT-4o’s answers
Refer to caption: /html/2410.09732/assets/x22.png

Figure: Figure 26: Abnormal detail explanation example of category scenery in image modality, with GPT-4o’s answers
Refer to caption: /html/2410.09732/assets/x23.png

Figure: Figure 27: Abnormal selection example of category scenery in image modality, with GPT-4o’s answers
Refer to caption: /html/2410.09732/assets/x24.png

### E.2 Video

Figure: Figure 28: Multi-choice example of category Abiotic in video modality, with GPT-4o’s answers
Refer to caption: /html/2410.09732/assets/x25.png

Figure: Figure 29: Multi-choice example of category Animals in video modality, with GPT-4o’s answers
Refer to caption: /html/2410.09732/assets/x26.png

Figure: Figure 30: Abnormal selection example of category Animals in video modality, with GPT-4o’s answers
Refer to caption: /html/2410.09732/assets/x27.png

Figure: Figure 31: Abnormal detail explanation example of category Human in video modality, with GPT-4o’s answers
Refer to caption: /html/2410.09732/assets/x28.png

Figure: Figure 32: Judgement example of category Human in video modality, with GPT-4o’s answers
Refer to caption: /html/2410.09732/assets/x29.png

Figure: Figure 33: Abnormal detail explanation example of category Human in video modality, with GPT-4o’s answers
Refer to caption: /html/2410.09732/assets/x30.png

Figure: Figure 34: Judgement example of category Human in video modality, with GPT-4o’s answers
Refer to caption: /html/2410.09732/assets/x31.png

Figure: Figure 35: Abnormal selection of category Scenery in video modality, with GPT-4o’s answers
Refer to caption: /html/2410.09732/assets/x32.png

### E.3 3D

Figure: Figure 36: Abnormal detail explanation example generated by Gaussian Splatting based methods in 3D modality, with GPT-4o’s answers
Refer to caption: /html/2410.09732/assets/x33.png

Figure: Figure 37: Judgement example generated by Gaussian Splatting based methods in 3D modality, with GPT-4o’s answers
Refer to caption: /html/2410.09732/assets/x34.png

Figure: Figure 38: Multi-choice example generated by Gaussian Splatting based methods in 3D modality, with GPT-4o’s answers
Refer to caption: /html/2410.09732/assets/x35.png

Figure: Figure 39: Multi-choice example generated by Gaussian Splatting based methods in 3D modality, with GPT-4o’s answers
Refer to caption: /html/2410.09732/assets/x36.png

Figure: Figure 40: Abnormal detail explanation example generated by Nerf based in 3D modality, with GPT-4o’s answers
Refer to caption: /html/2410.09732/assets/x37.png

Figure: Figure 41: Abnormal detail explanation example in 3D modality, with GPT-4o’s answers
Refer to caption: /html/2410.09732/assets/x38.png

Figure: Figure 42: Judgement example in 3D modality, with GPT-4o’s answers
Refer to caption: /html/2410.09732/assets/x39.png

Figure: Figure 43: Judgement example in 3D modality, with GPT-4o’s answers
Refer to caption: /html/2410.09732/assets/x40.png

### E.4 Audio

Figure: Figure 44: Judgement example of category Audio in audio modality, with LTU’s answers
Refer to caption: /html/2410.09732/assets/x41.png

Figure: Figure 45: Multi-choice example of category Music in audio modality, with LTU’s answers
Refer to caption: /html/2410.09732/assets/x42.png

Figure: Figure 46: Multi-choice example of category Singing in Audio modality, with Qwen-Audio’s answers
Refer to caption: /html/2410.09732/assets/x43.png

Figure: Figure 47: Multi-choice example of category Speech in audio modality, with AnyGPT’s answers
Refer to caption: /html/2410.09732/assets/x44.png

### E.5 Text

Figure: Figure 48: Multi-choice example of category Scientific Papers Abstract in text modality, with GPT-4o’s answers
Refer to caption: /html/2410.09732/assets/x45.png

Figure: Figure 49: Judgement example of category Scientific Papers Abstract in text modality, with GPT-4o’s answers
Refer to caption: /html/2410.09732/assets/x46.png

Figure: Figure 50: Multi-choice example of category Essay in text modality, with GPT-4o’s answers
Refer to caption: /html/2410.09732/assets/x47.png

Figure: Figure 51: Judgement example of category Essay in text modality, with GPT-4o’s answers
Refer to caption: /html/2410.09732/assets/x48.png

Figure: Figure 52: Multi-choice example of category News in text modality, with GPT-4o’s answers
Refer to caption: /html/2410.09732/assets/x49.png

Figure: Figure 53: Judgement example of category News in text modality, with GPT-4o’s answers
Refer to caption: /html/2410.09732/assets/x50.png

Figure: Figure 54: Multi-choice example of category Modern Literature in text modality, with GPT-4o’s answers
Refer to caption: /html/2410.09732/assets/x51.png

Figure: Figure 55: Judgement example of category Modern Literature in text modality, with GPT-4o’s answers
Refer to caption: /html/2410.09732/assets/x52.png

Figure: Figure 56: Judgement example of category Ancient Literature in text modality, with GPT-4o’s answers
Refer to caption: /html/2410.09732/assets/x53.png

Figure: Figure 57: Multi-choice example of category Ancient Literature in text modality, with GPT-4o’s answers
Refer to caption: /html/2410.09732/assets/x54.png

Figure: Figure 58: Multi-choice example of category Philosophy in text modality, with GPT-4o’s answers
Refer to caption: /html/2410.09732/assets/x55.png

Figure: Figure 59: Judgement example of category Philosophy in text modality, with GPT-4o’s answers
Refer to caption: /html/2410.09732/assets/x56.png

Figure: Figure 60: Multi-choice example of category Speech in text modality, with GPT-4o’s answers
Refer to caption: /html/2410.09732/assets/x57.png

Figure: Figure 61: Multi-choice example of category Speech in text modality, with GPT-4o’s answers
Refer to caption: /html/2410.09732/assets/x58.png

Figure: Figure 62: Multi-choice example of category Wikipedia in text modality, with GPT-4o’s answers
Refer to caption: /html/2410.09732/assets/x59.png

Figure: Figure 63: Judgement example of category Wikipedia in text modality, with GPT-4o’s answers
Refer to caption: /html/2410.09732/assets/x60.png

## Peer Reviews

confidence: 5
contribution: 3
initial_rating: 8
presentation: 3
questions: No follow-up questions.
soundness: 3
strengths: + LOKI is a novel, multimodal dataset.

+ The paper is easy to read and well-organised.

+ Comprehensive Evaluation and Validation.

+ Curates a diverse dataset with 18,000 questions across five modalities and 26 categories, providing a solid foundation for synthetic data detection evaluation.

+ Detailed Annotations.

+ Directly addresses the challenges of synthetic data proliferation, impacting security, misinformation, and content authenticity.

+ LOKI’s findings on LMM strengths and weaknesses have the potential to drive advancements in synthetic data detection and multimodal model development.
summary: The paper introduces a novel benchmark designed to evaluate the capability of LMMs in detecting synthetic data across multiple modalities, including video, image, 3D, text, and audio. LOKI is structured to provide diverse modalities, cover 26 detailed categories, and offer multi-level annotations, enabling tasks that range from basic authenticity judgments to fine-grained anomaly selection and explanation. The benchmark consists of 18,000 questions across various levels of difficulty, allowing for a comprehensive analysis of LMMs in detecting synthetic content. Additionally, the paper includes evaluations of 22 open-source and 6 closed-source LMMs, revealing both the potential and current limitations of these models in terms of detection accuracy and interpretability.
weaknesses: -  The benchmark lacks a robustness test against common real-world conditions like compression artifacts. To enhance real-world applicability, the authors could include performance evaluations on compressed data.

---

confidence: 4
contribution: 3
initial_rating: 8
presentation: 4
questions: Do you have a sense as to why the prompting strategies (FS, CoT) in general had a poor/neutral impact for most of the model/data pairs (Table 5)?
soundness: 3
strengths: The paper is well-written and easy to follow.

This work addresses a glaring and emergent need in a topic field: synthetic data detection for LLMs; I agree with the authors that there doesn't currently exist a comprehensive, multi-modal, nuanced dataset including explainability assessment for this domain area. 

Extensive examples and case studies provided in appendices. 

Many data domains are covered in this benchmark, including several categories and characteristics that are often underrepresented in synthetic data detection (e.g., satellite images, "abnormal details").
summary: The authors introduce  LOKI, a novel benchmark designed to evaluate the ability of LMMs to detect synthetic data across multiple modalities. With the concurrent preponderance of synthetic data and the rise of powerful LLMs, the authors aim to address the emerging research topic of LMM evaluation for synthetic data detection. LLMs can provide reasoning behind authenticity judgments, benefitting explainability. The focus of LOKI is to evaluate the performance of LLMs on synthetic data detection tasks. 

In particular, LOKI is aimed at addressing several shortcomings present in extant synthetic data evaluation datasets, including emphasizing human interpretability and multimodality. Overall, LOKI provides key improvements to synthetic data detection, including: diverse modalities, heterogenous categories, multi-level annotations and multimodal synthetic evaluation. framework.
weaknesses: While the differentiated modalities, categories and annotation levels are beneficial, the overall size of the dataset actually seems relatively small vis-a-vis related datasets (Table 1). 

It is unclear to me, how a user can methodically compare scores for different models across tasks/categories (e.g., in Table 2); perhaps the authors can address this, given the heterogenous and imbalanced nature of the data modalities and tasks, as well as the problem/domain "difficulty". 

As deepfake detection is one of the most prominent synthetic data detection categories today, I believe the benchmark would benefit from its inclusion.

---

confidence: 4
contribution: 3
initial_rating: 6
presentation: 3
questions: 1.	In the fine-grained anomaly selection task, all samples were manually reviewed to ensure question quality. However, how can the robustness of the task design be ensured in future large-scale applications?
2.	Could you provide additional insights into why Claude-3.5-Sonnet tends to misclassify synthetic images as real? For instance, does this result from limitations in the model's ability to recognize fine-grained abnormalities, or are there certain types of synthetic image features that are particularly challenging for the model?
soundness: 3
strengths: 1.	The paper is well written and easy to follow. The authors provide sufficient technical details for readers to understand their work.
2.	The benchmark designed by the authors encompasses a rich variety of modalities and diverse question types, enabling a comprehensive evaluation of LMM performance.
3.	The authors introduce a metric called the Normalized Bias Index (NBI) to quantify the performance differences of the model on natural and AI-generated data across different modalities, which is an innovative way to assess model bias.
summary: This paper introduces LOKI, a novel benchmark for evaluating large multimodal models (LMMs) in detecting synthetic data across multiple modalities, such as video, image, text, and audio.This benchmark features broad-level assessments, including judgment and multiple-choice tasks, alongside detailed anomaly selection and explanation exercises, providing an in-depth evaluation of large multimodal models (LMMs).
weaknesses: 1.	The current evaluation mainly relies on accuracy and NBI; however, at low recall rates, NBI may not adequately reflect model bias. Additionally, the design of NBI may be insufficient to comprehensively capture various types of bias exhibited by the model.
2.	The paper mentions that the model exhibits "bias" across different modalities. However, the specific causes of this bias are not thoroughly explored through experiments or comparative analysis. This conclusion may be based on surface-level observations without further investigation into whether the bias arises from data, model architecture, or task design.
3.	The paper mentions that the Chain-of-Thought (CoT) approach can impact model performance in image and 3D reasoning tasks.  However, it does not provide sufficient experimental details to clarify whether CoT significantly enhances performance across all types of tasks or if it is only effective for the specific tasks currently evaluated.
4.	It is suggested to discuss and compare more related works such as [1,2] in this paper.

[1] Detecting and Grounding Multi-Modal Media Manipulation and Beyond. TPAMI 2024.

[2] Detecting and Grounding Multi-Modal Media Manipulation. CVPR 2023.

---

confidence: 5
contribution: 4
initial_rating: 8
presentation: 3
questions: - Quality and Realism of Synthetic Data: How does the synthetic data in LOKI reflect the latest advancements in generative models? Are there measures taken to ensure that the synthetic data poses a realistic challenge for detection models?
- Prompting Strategies Implementation: Could you elaborate on how chain-of-thought and few-shot prompting were implemented across different models? Specifically, how did these strategies impact models that are less capable of handling long contexts or reasoning tasks?
soundness: 3
strengths: - Comprehensive Multimodal Benchmark: LOKI covers an extensive range of data modalities and subcategories.
- Inclusion of specialized domains like satellite imagery, medical images, and philosophical texts pushes the boundaries of traditional datasets and tests models in less-explored areas.
- Multi-Level Task Design: The benchmark doesn't just focus on binary classification but also includes tasks that assess models' abilities to explain their reasoning, promoting the development of interpretable AI systems.
- Highlighting the Importance of Explainability
- By testing perception, knowledge, and reasoning across modalities, LOKI contributes to the broader goal of advancing towards AGI.
summary: The paper introduces LOKI, a comprehensive benchmark designed to evaluate the capabilities of Large Multimodal Models in detecting synthetic data across multiple modalities. Recognizing the rapid advancement of AI-generated content and the associated risks of synthetic media proliferation, the authors aim to assess how well LMMs can discern real data from AI-generated counterparts.

LOKI encompasses a diverse set of data modalities, including video, image, 3D models, text, and audio, covering 26 detailed subcategories such as satellite images, medical images, philosophical texts, and various audio types like music and environmental sounds. The benchmark includes over 18,000 carefully curated questions with varying levels of difficulty.

The tasks within LOKI are multi-faceted:

Judgment Tasks: Binary classification to determine if a piece of data is real or AI-generated.
Multiple-Choice Questions: Selecting the AI-generated item from a set of options.
Abnormal Detail Selection: Identifying specific anomalies in synthetic data.
Abnormal Explanation Tasks: Providing explanations for why data is identified as synthetic.
The authors evaluated 22 open-source LMMs and 6 closed-source models (including GPT-4 and Gemini) using LOKI. Their findings highlight that while LMMs show promise in synthetic data detection and offer interpretability advantages over traditional expert models, they also exhibit significant limitations. Models tend to have biases, lack domain-specific knowledge, and display unbalanced performance across different modalities.
weaknesses: - Limited Performance in Certain Modalities: The benchmark reveals that LMMs perform poorly in modalities like 3D and audio, which may be due to the lack of available models or training data in these areas.
- Insufficient Details on Data Generation Methods: The paper could provide more in-depth information on how synthetic data was generated for each modality, which is crucial for reproducibility and understanding potential biases in the dataset.
- Evaluation of Few-Shot and Chain-of-Thought Prompting: The analysis of prompting strategies is somewhat limited.

Meta-review: 4x accept. This paper introduces a multimodal benchmark aimed at evaluating LMMs’ abilities to detect synthetic data across video, image, 3D, text, and audio modalities. The reviewers agree on the (1) comprehensive coverage of multiple data domains and tasks, (2) clear and well-structured presentation, (3) innovative metrics like NBI for bias assessment, and (4) detailed evaluations including explainability. However, they note (1) insufficient robustness tests under real-world conditions and compression artifacts, (2) limited analysis of deeper bias causes, (3) lack of detail on data generation methods, and (4) partial prompting strategy evaluations. The authors’ rebuttal includes new compression tests, clarifications on synthetic data generation, further bias analyses, and expanded prompting experiments, so the AC leans to accept this submission.

Decision: Accept (Spotlight)