On Learning Multi-Modal Forgery Representation for Diffusion Generated Video Detection

Authors: xiufeng  song
xiao  guo
jiache  zhang
qirui  li
lei  bai
xiaoming  liu
guangtao  zhai
xiaohong  liu
Venue: NeurIPS_cc_2024_Conference
Published: 2024-05-08 10:55:32

Abstract: Large numbers of synthesized videos from diffusion models pose threats to information security and authenticity, leading to an increasing demand for generated content detection. However, existing video-level detection algorithms primarily focus on detecting facial forgeries and often fail to identify diffusion-generated content with a diverse range of semantics. To advance the field of video forensics, we propose an innovative algorithm named Multi-Modal Detection(MM-Det) for detecting diffusion-generated videos. MM-Det utilizes the profound perceptual and comprehensive abilities of Large Multi-modal Models (LMMs) by generating a Multi-Modal Forgery Representation (MMFR) from LMM's multi-modal space, enhancing its ability to detect unseen forgery content. Besides, MM-Det leverages an In-and-Across Frame Attention (IAFA)  mechanism for feature augmentation in the spatio-temporal domain. A dynamic fusion strategy helps refine forgery representations for the fusion. Moreover, we construct a comprehensive diffusion video dataset, called Diffusion Video Forensics (DVF), across a wide range of forgery videos. MM-Det achieves state-of-the-art performance in DVF, demonstrating the effectiveness of our algorithm. Both source code and DVF are available at https://github.com/SparkleXFantasy/MM-Det.

Main paper: 
 introduction: 
 Figure 1 : Multi-Modal Detection (MM-Det) leverages features from spatiotemporal (ST) information ( ), a CLIP encoder [39] ( ), and an LMM ( ). The Fused feature ( ) achieves state-of-the-art performance in our Diffusion Video Forensics (DVF) dataset.Recent years have witnessed significant advancements in diffusion generative methods, which have led to the creation of extraordinarily visually compelling content in video generation [5, 4, 61] . Although the latest generated videos impress society with their versatility and stability, synthetic media also poses a risk of malicious attacks, such as counterfeit faces created by deepfakes [49] and falsifications in business, raising public concerns about information security and privacy. In response to such issues, researchers have made significant progress in forgery detection, addressing problems on image editing manipulation [60, 30, 17] and CNN-synthesized images [53, 40, 55, 36, 18] . To enhance the trustworthiness and reliability of current detectors in the face of evolving generative video methods, we aim to develop a generalizable detection method for diffusionbased generative videos.Briefly describe and inspect the image for forgery detection.It is of the sun over the sea with over-saturated color and lighting.Give concrete reasons why it is real or fake.It is fake, because the unrealistic color is a typical sign of forgery. Figure 2 : LMMs detect visual artifacts and anomalies, offering detailed textual reasoning that explains whether the image is generated using Artificial Intelligence (AI) techniques. The powerful representation in the visual domain enables LMMs to understand complex contexts within frames. Furthermore, their advanced language reasoning capability implicitly reveals image authenticity and provenance. For instance, the term like "consistent shape" refers to common features in authentic content, while "unrealistic color" signifies typical artifacts in forged content. This linguistic proficiency stems from the superior perception and comprehension abilities of LMMs, contributing to a generalizable multimodal feature space. By leveraging the visual understanding and textual reasoning abilities of LMMs, we construct a Multi-Modal Forgery Representation (MMFR).RealPreviously, the video forensics community emphasized more on developing facial forgery detection algorithms [57, 73, 10, 62, 70] , which may struggle to address recent fraudulent videos (e.g., sora, pika, etc.). Compared to facial forgery, diffusion-based generated content contains more diverse semantics, making it more challenging to distinguish diffusion forgery contents from real ones. Towards these challenges, a new thread of research on CNN-generated image detection has emerged [53, 40, 55, 36, 18] . These works aim to learn common generation traces in image-level content, but do not design specific mechanisms to capture temporal inconsistencies in videos.Therefore, previous defensive efforts might not be able to provide a video-level detection algorithm for newly emerged generated videos with diverse manipulation artifacts and visual contexts. Meanwhile, Large Multi-modal Models (LMMs) show unparalleled problem-solving ability [2, 24, 23, 75, 69, 28] , thanks to its powerful multi-modal representations. However, such representations are barely studied in the video forensics task.Motivated by the limitation of previous work and the unprecedented understanding ability of LMMs, we propose a video-level detection algorithm, named Multi-Modal Detection (MM-Det), to capture forgery traces based on an LMM-based multi-modal representation. MM-Det takes advantage of the perception and reasoning capabilities of LMMs to learn a generalizable forgery feature, as depicted in Fig. 2 . To the best of our knowledge, we are the first to use LMMs for video forensic work.Aside from multi-modal representations, two common sources of generative errors that can be leveraged for video discrimination are spatial artifacts and temporal inconsistencies. Our approach aims to effectively identify these two types of errors as an auxiliary feature in forgery detection.Inspired by the previous work [55, 33, 41 ] that shows the effectiveness of reconstruction for detecting diffusion images, we extend this idea into the video domain, amplifying diffusion artifacts both in spatial and temporal information. To capture such artifacts efficiently, we leverage a Vector Quantised-Variational AutoEncoder (VQ-VAE) [51] for a fast reconstruction process, as detailed in Fig. 3 . Moreover, we design a novel In-and-Across Frame Attention (IAFA) into a Transformerbased network, which balances frame-level forgery traces with information flow across frames, thus aggregating local and global features.Although diffusion methods demonstrate strong capabilities in video generation, the lack of public datasets on diffusion videos hinders research efforts in the video forensic community. In light of this, we have established a comprehensive dataset for diffusion-generated videos, named Diffusion Video Forensics (DVF). DVF includes generated content from a variety of diffusion models, featuring rich semantics and high quality, serving as a general benchmark for open-world video forensics tasks. ⋄ Our MM-Det achieves state-of-the-art detection performance on the DVF dataset. Also, a detailed analysis is provided to showcase the effectiveness of multi-modal representations in detecting forgeries, paving the way for compelling opportunities for using LMMs in future multi-media forensic research.Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?Answer: [Yes]Justification: The main claims fit with the paper's contributions and scope.Guidelines:• The answer NA means that the abstract and introduction do not include the claims made in the paper. • The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or NA answer to this question will not be perceived well by the reviewers. • The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings. • It is fine to include aspirational goals motivation as long as it is clear that these goals are not attained by the paper. 
related works: 
 Frame-level Detector Early work [53, 22, 13, 47, 65, 16] observed that forgery traces exist in images generated by AI techniques, and such traces are commonly used as evidence to distinguish diffusiongenerated content [40, 6, 7] and attribute if two images are generated by the same method [66, 37] . However, identifying unseen and diverse frequency-based clues in real-world scenarios is challenging. For that, existing frame-level forgery detectors concentrate on improving the generalization ability. For example, some works [36, 8, 17, 26, 29] introduced features from pre-trained CLIP [39] encoders for the forensic task to help reduce the overfitting issue on specific forgery types and increase the robustness towards detection [36, 8] and localization [17] . [64] and [12] proposed proactive methods to protect images from manipulation based on image watermarking and steganography. Also, reconstruction errors through the inversion process of DDIM [44] are studied by prior works [55, 33, 41, 32] for diffusion generative content detection. Moreover, the previous work [63, 46, 34, 18] develops specific techniques that increase the generalization to unseen forgeries. For example, HiFi-Net [18] proposes a tree structure to model the inherent hierarchical correlation among different forgery methods, NPR [46] devises a special representation as generative artifacts, and the training set diversity can also contribute to generalization ability [34] . Unlike prior works, our MM-Det leverages multi-modal reasoning to achieve a high level of generalization ability.Early video-level methods primarily focused on detecting facial forgery. For example, [25] learned the boundary artifacts between the original background and manipulated faces. [19] discriminated fake videos from the inconsistency of mouth motion. [73] designed a multi-attentional detector to capture deepfake features and artifacts. F3Net [38] captured global and local forgery traces in the frequency domain. [10, 15, 74, 56] explored temporal information and inconsistency from fake videos. Most recently, DD-VQA [70] formulates deepfake detection as a sentence-generation problem, largely improving the interpretation of deepfake detection. However, these studies are restricted to facial forgery methods, which are insufficient for the current defensive systems that address diverse content produced by diffusion models. Therefore, we develop MM-Det to detect diffusion video content, pushing forward the frontier of forgery video detection. Large Multi-modal Models (LMMs) LMMs possess generalizable problem-solving abilities in real-world tasks, including object detection [14] , semantic segmentation [67] and visual question answering [59] . [2, 23, 27, 28] studied feature alignment schemes to bridge visual and textual domains for LMMs. [68, 72, 71] extended the boundaries of LMMs to multi-modal downstream tasks. [45] aligns multi-modal features for capabilities on cross-domain behaviors. [58] developed a Large Language Model (LLM)-based feature extractor for cheap-fake detection. Inspired by these studies, we stimulate the powerful perceptual and reasoning ability of an LMM by introducing the multi-modal feature space in video forgery detection.Question: Does the paper discuss the limitations of the work performed by the authors?Answer: [Yes]Justification: The paper discusses the limitations of the work.Guidelines:• The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper. • The authors are encouraged to create a separate "Limitations" section in their paper.• The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be. • The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on a few datasets or with a few runs. In general, empirical results often depend on implicit assumptions, which should be articulated. • The authors should reflect on the factors that influence the performance of the approach.For example, a facial recognition algorithm may perform poorly when image resolution is low or images are taken in low lighting. Or a speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon. • The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size. • If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness. • While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an important role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations. 
methods: 
 In this section, we introduce the Multi-Modal Detection (MM-Det) framework for diffusion video detection, as depicted in Fig. 4 . More formally, Sec. 3.1 details a Large Multi-modal Model (LMM) branch that learns a Multi-Modal Forgery Representation (MMFR). Then Sec. 3.2 reports a Spatio-Temporal (ST) branch that utilizes In-and-Across Frame Attention (IAFA) to capture spatial artifacts and temporal inconsistencies in forged videos. Lastly, a dynamic fusion technique reported in Sec. 3.3 adaptively combines outputs from the LMM branch and ST branch.We propose a novel Multi-Modal Forgery Representation (MMFR) from the multi-modal space of LMMs in LMM branch. This representation utilizes the powerful perceptual and reasoning abilities of LMMs in the form of instruction-based conversations. Specifically, LMM branch is built on the top of LLaVA [28] , one representative LMM, which has two key components: a visual encoder (e.g., D v ), instantiated by visual encoders from the Contrastive Language-Image Pre-Training (CLIP) [39] , and the large language model D L (i.e., Llama 2 [50] ). Let us denote the input video as X ∈ R N ×H×W ×C that contains N frames, where each frame is represented as x ∈ R H×W ×C . First, x is fed to D v to obtain the corresponding visual representation F V ∈ R Z . Such F V not only contains rich semantics but also shows impressive generalization ability and robustness in the forgery detection task [36, 43, 9] . Then, a textual instruction T is sampled from pre-defined templates Q to guide the LMM on forgery detection reasoning. Both visual representation F V and instruction T are fed to D L , which generates enhanced visual representations (e.g., F L ). We convert F V into a sequence of visual tokens [28] (i.e.,•• • ••• •••H v = {h v,m } M m=1 ∈ R M ×D ), and T is transformed into textual tokens H t = {h t,o } O o=1 ∈ R O×D .Both H v and H t are taken as the input to D L , generating F L ∈ R S×D that can be tokenized into the language response providing reasoning (Fig. 2 ) about the authenticity of the input x. This procedure is formulated asEQUATIONwhere T guides the pre-trained D L in comprehending visual content (i.e., F V ), discerning the subset information from F V . This instruction T enables LMM branch to obtain the multi-modal representation that leverages the generalization ability from the pre-trained large language model Llama 2 (i.e., D L ), being different to prior work [36, 9] that only relies on F E .Lastly, we retrieve the final MMFR, denoted as F M ∈ R M ×Z , by concatenating F V and F L after a linear layer(i.e., PROJ), asEQUATIONTargeting capturing spatiotemporal artifacts in video tasks, we introduce a Spatial-Temporal (ST) branch that learns effective diffusion forgery representation at the video level. Through a reconstruction procedure, we amplify the diffusion traces in the frequency domain, which is then captured by In-and-Across Frame Attention (IAFA) to form an effective video-level feature.Similar to prior studies [55, 33] that discovered specific generative traces of diffusion models through reconstruction on diffusion-generated images, we utilize an Autoencoder to amplify diffusion traces in videos. The reconstruction procedure can be expressed as follows.More formally, denote the input video as X ∈ R N ×H×W ×C that contains N frames, in which each frame is represented as x ∈ R H×W ×C . We leverage a VQ-VAE [51] to obtain the reconstructed version of x, which is denoted as x ∈ R H×W ×C . The difference between x and x makes an effective indicator of showing if the input is generated by diffusion models, as depicted in Fig. 3 . Therefore, we jointly proceed x and x to the following proposed modules for learning an effective representation of discerning forgeries.It is worth mentioning that the prior approach [55] also adopts the idea of using the residual difference between original and reconstructed inputs to help forgery detection, but the reconstruction method requires multiple time-step denoising operations, which are computationally infeasible to reconstruct all frames from X. In contrast, our VQ-VAE-based reconstruction method only requires one single forward propagation to obtain reconstructed frame x, meanwhile preserving the effectiveness in indicating the discrepancy between real and fake inputs. Integration of Spatial and Temporal Information Same as the previous work [3, 35] that utilizes ViT to learn video-level information, we transform each input video frame (i.e., x) into L tokens, and propose IAFA for information aggregation, as depicted in Fig. 5a . Let us denote frame-level tokens as Patch-wise tokens (P-tokens), as they represent local information of one patch from x. More formally, the ith frame, x i is divided into L patches, and all patches are projected intoT i = {t j i } L j=1 ∈ R L×D, where D represents the dimension of each P-token. Also, to capture the global forgery information for each video frame, we introduce additional tokens called Frame-Centric tokens (FC-tokens). The FC-token is denoted as p i ∈ R D for frame x i and attends other tokens within the same frame x i .During the forward propagation, we conduct IAFA based on a Transformer, with each block containing two self-attentions and consecutively modeling the local and global forgeries at each video frame. Specifically, the first self-attention captures dependencies among P-tokens that restore local forgery clues. This is formulated by Eq. 3 that t j i ∈ R D attends the token t p q ∈ R D that represents pth token from qth frame x q . Consequently, given the ith frame i.e., x i , the second self-attention is conducted among the FC-token (i.e., {p i }) and P-tokens (i.e., {t 0 i , t 1 i , ... t L-1 i }) from the same frame, which encapsulates patch-wise forgery information into the global one for learning the more robust representation. We formulate this procedure in Eq. 4.EQUATIONEQUATIONwhere ATTN refers to the self-attention operation.We devise the dynamic fusion strategy (i.e., D f ) that combines spatiotemporal information from ST branch and MMFR (i.e., F f and F m ) for the final prediction, by adjusting their contributions based on forgeries from the input. More formally, D f (Fig. 5b ) learns channel-wise dependencies among forgery representations (e.g., F ST and F M ) via the attention mechanism, generating w ∈ R N +M as the output. This procedure can be expressed as w = D f (CONCAT{F ST , F M }). Also, w contains w ST ∈ R N and w M ∈ R M , representing learned channel-wise weights for F ST and F M , respectively. Such channel-wise weights are important in the fusion purpose, as they help emphasize useful information -we use w ST and w M to refine forgery representations asF ′ ST = F ST w ST and F ′ M = F M w M . Lastly, F ′ ST and F ′ M are concatenated into the fused representation F 0 ∈ R (M +N )×D ,which is used for the final scalar prediction s via the average pooling (i.e., AVG) and linear layers(i.e., PROJ):EQUATIONQuestion: For each theoretical result, does the paper provide the full set of assumptions and a complete (and correct) proof?Answer Justification: The paper provide sufficient information on the computer resources.Guidelines:• The answer NA means that the paper does not include experiments.• The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage. • The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute. • The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make it into the paper). 
diffusion video forensics (dvf) dataset: 
 We construct a large-scale dataset for the video forensic task named Diffusion Video Forensics (DVF), as shown in Fig. 6 . DVF contains 8 diffusion generative methods, including Stable Diffusion [42] , VideoCrafter1 [5] , Zeroscope, Sora, Pika, OpenSora, Stable Video, and Stable Video Diffusion [4] .To efficiently streamline the collection, we construct an effective automated pipeline that generates forgery videos based on real videos and prompts. Specifically, we start from two real video datasets, Internvid-10M [54] and Youtube-8M [1] . Real videos are sampled for rich semantic content, with their frames and captions used for generation. Fig. 7 a introduces the generation process of DVF. For open-sourced generation methods, a prompt is fed to a text-to-video method(i.e. VideoCrafter1, Zeroscope, OpenSora), or a frame is provided to an image-to-video method(Stable Video Diffusion) to generate the corresponding fake video. For commercial and close-sourced datasets(i.e. Stable Diffusion, Stable Video, Sora, Pika), forgery videos are collected from official websites and social media. In total, we collect 3, 938 fake videos and 2, 750 real videos in DVF. As shown in Fig. 7 b and 7 c, our dataset contains multiple resolutions and durations. The video number of each dataset varies from 0.1k to 2.8k, with the corresponding frame numbers from 4.2k to 784k. More details about DVF are provided in Appendix A.3.1. 
training strategy: 
 This section details our two-stage training strategy, in which we first finetune the LMM branch via instruction tuning [28] and then optimize the entire framework in an end-to-end manner.We first adapt LLaVA [28] to the forgery detection downstream task based on instruction tuning, an empirically effective way for various downstream tasks, which leverages LoRA [21] to improve the reasoning ability of Large Language Models(LLMs). For that, construct a large image-text paired dataset, named Rich Forgery Reasoning Dataset. Please refer to Appendix A.3.2 for more details. We use multi-turn conversations to fine-tune the LMM, enhancing its ability to identify and judge the authenticity of input images. Following the instruction tuning strategy of LLaVA [28] , we only fine-tune the projection layers and LLM in LLaVA. More formally, we formulate the objective function as the loss for an auto-regressive model, which is based on answer tokens from the LLM, as:EQUATIONwhere s i refers to the i th prediction token, T refers to the length of total prediction tokens, and θ 1 refers to the trainable parameters in the LMM.End-to-End Training After fine-tuning LLaVA, we use this model to form LMM branch of MM-Det, and then the entire model is trained in an end-to-end manner. Please note that all parameters in LMM branch are frozen to ensure the optimal multi-modal representation can be obtained. More formally, we denote MM-Det's final prediction scalar and the ground truth as s and y, respectively, and the model is optimized by the cross-entropy loss L as follows:EQUATIONwhere θ 2 refers to trainable parameters in both ST branch and dynamic fusion modules. For a fair comparison, we choose the following 10 recent detection methods as baselines. CN-NDet [53] applies a ResNet [20] as the backbone for forgery detection. F3Net [38] utilizes frequency traces left in forgery content. HiFi-Net [18] devise a specific hierarchical fine-grained learning scheme to learn a wide range of forgery traces. Clip-Raising [9] , Uni-FD [36] takes advantage of a pre-trained CLIP [39] as a training-free feature space. DIRE [55] detects diffusion images based on a reconstruction process of DDIM [44] . ViViT [3] , TALL [62] , and TS2-Net [31] take advantage of spatiotemporal information in various visual tasks. DE-FAKE [43] adopts visual and textual representations based on a CLIP encoder for image forgery detection. For the measurement, we choose AUC since it is a threshold-independent metric. 
video forgery detection performance: 
 In Tab. 1, our proposed MM-Det achieves SoTA performance in detecting diffusion video, surpassing the second-best method, i.e., HiFi-Net, by 6.7% in AUC scores. Specifically, for prior methods that are based on pre-trained CLIP features, such as Raising [8] and Universal FD [36] , they remain effective on certain types of diffusion content (i.e., Sora, Pika, Stable Diffusion), but fail on most others. Simple structures like CNN [53] exceed these CLIP-based methods after being fine-tuned on our proposed DVF, which proves the necessity of such datasets. As for our method, MM-Det outperforms other methods in most datasets. Compared with frequency-based forgery methods, e.g., F3Net [38] and CLIP-based methods [8, 36] , our method improves the performance from +0.7%(VideoCrafter1) to +6.3%(Sora). It is worth mentioning that HiFi-Net makes the second-best performer in our DVF dataset, achieving 89.0% AUC scores. We believe this indicates the multi-branch feature extractor used in HiFi-Net carries versatile forgery traces at multiple resolutions, enhancing the learning of the forgery invariant. The failure of frequency traces and CLIP features raises the need for more effective features. As for spatiotemporal baselines [3, 62, 31] , we outperform them by +7.6%(ViViT), +24.1%(TALL) and +23.0%(TS2-Net), demonstrating the effective features of MMFR and IAFA. At last, our detector improves by 22.6% to another multi-modal detector [57] , which utilizes visual information and corresponding captions for feature enhancement. It is shown that the introduction of MMFR is more generalizable than a simple combination of visual features and text descriptions in that the powerful perceptual and reasoning abilities of LMMs play a crucial role in discriminating between real and fake content. ViViT IAFA Figure 8 : Visualization of artifacts captured from our IAFA and ViViT [3] . We use activation maps to highlight spatial weights within each frame. All content is generated by VideoCrafter1. Features from the last layer of transformers are extracted for visualization.Tab. 2 shows the impact of individual modules proposed in MM-Det. Specifically, we use the Hybrid ViT [11] as the base model and incorporate it with the reconstruction procedure for diffusion trace amplification, which enhances the detection performance by +1.1% AUC score. Such a module raises the performance in OpenSora and Sora, revealing that frequency-based information benefits forgery detection on these methods. Detection performance is further increased by using IAFA, which strengthens the learning between in-frame and cross-frame information, increasing a +13.0% AUC score to the base model. The rise in performance indicates such temporal information benefits most types of forgery video detection. After that, Tab. 2's line 4 indicates the effectiveness of MMFR: a detector purely based on such representation can receive 90.5% performance in AUC, +14.0% higher than the base model. In addition, by merging MMFR (i.e., LMM branch) and ST branch, the performance rises by +3.2%. Finally, with the dynamic fusion strategy, our method receives an impressive 95.7% AUC score for all generative methods, higher than every single feature. These experiments highlight the necessities of each module in our framework. Moreover, an ablation study on LLMs is detailed in Appendix A.5 to prove the effectiveness of various LLMs in MM-Det.In the analysis of IAFA, we visualize feature activation maps from the last layer of ViT in the ST branch based on the L2-norm. We compare our feature maps with another spatiotemporal baseline, ViViT [3] , as depicted in Fig. 8 . While attention maps of ViViT are sparse and irregular, the ones of our IAFA have a tendency to concentrate on the segmentation of diffusion-generated objects, indicating that IAFA captures typical spatial forgery regions in frames. The attention mainly focuses on common forgery traces, such as blurred generative patterns and defective parts of objects, signaling that diffusion models might find it difficult to generate delicate content. The concentration of activation on certain informative objects discloses both spatial artifacts of existing generative methods, demonstrating the effectiveness of our proposed ST branch. Fig. 9 details the effectiveness of MMFR in LMM branch. First, as depicted in Fig. 9 a, we quantify the detection ability of features from each Transformer-based decoder layer in the LLM. Specifically, we evaluate both pre-trained LLaVA [28] and its fine-tuned version on the task of distinguishing real and fake frames. These frames are randomly sampled from 1, 000 real videos and equivalent fake ones from Stable Video Diffusion [42] . Layer-wise outputs from the large language model in LLaVA (i.e., Vicuna [50] ) are obtained -for i th layer in the LLM, we denote its output features as f o i , i ∈ [1, 33] . The K-Means clustering algorithm is adopted to evaluate the classification accuracy based on f o i . Empirically, we observe that features extracted from a fine-tuned LLaVA show promising classification accuracy in the last few layers, e.g., 22-nd or later layers. This phenomenon indicates that specific layers in LLMs indeed generate features that can be used for image forensic tasks. Such features are utilized in MMFR to exhibit high generalization ability towards diverse and unseen forgeries. Secondly, the comparison between pre-trained and fine-tuned LLaVA highlights the importance of downstream task-oriented instruction tuning for LMMs. This conclusion is consistent with the findings of prior works [27, 28, 69, 68] .In addition, shown in Fig. 9 b and 9 c, we analyze features from ST branch and LMM branch through t-SNE [52] . Both features achieve superior performance in separating real and forgery videos. In Fig. 9 b, spatiotemporal information forms a rough boundary between real and fake videos. This feature is effective for VideoCrafter1 [5] , Zeroscope, Stable Video, and Pika, whose durations and resolutions are similar to the training set. However, the detection performance might decrease on Sora and OpenSora with overlay in the clusters. We suppose that various resolutions and durations may compromise the generalization ability, magnifying the importance of a comprehensive dataset for these videos. Fig. 9 c demonstrates the more powerful feature from LMM branch. Samples from Zeroscope, Sora, and Pika are compacted into a denser area, indicating the ability of LLMs to conduct generalizable reasoning. Such features provide new insights for detection when spatial and temporal artifacts are not obvious among the latest forgery videos. 
conclusion: 
 In this work, we develop an effective video-level algorithm termed Multi-Modal Detection (MM-Det) for diffusion-generated video detection. MM-Det leverages a novel generalizable Multi-Modal Forgery Representation (MMFR) that is obtained from multi-modal spaces in LMMs. Specifically, the proposed MM-Det has two major branches: the LMM branch, which incorporates vision and text features from the fine-tuned foundation model, and the ST branch, which concentrates on modeling spatial-temporal information aggregated through In-and-Across Frame Attention. Extensive experiments demonstrate the effectiveness of our proposed detector. In addition, we establish a comprehensive dataset for various diffusion generative videos, which we hope will serve as a general benchmark for real-world video forensic tasks. 
code of ethics: 
 Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics https://neurips.cc/public/EthicsGuidelines? 
broader impacts: 
 Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?Answer: [Yes] Justification: The paper discusses both positive and negative social impacts in Boarder Impacts.Guidelines:• The answer NA means that there is no societal impact of the work performed.• If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact.• Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations. • The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster. • The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology. • If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML). 
safeguards: 
 Question: Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?Answer: [Yes] Justification: The paper has no risks.Guidelines:• The answer NA means that the paper poses no such risks.• Released models that have a high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters. • Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images. • We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make a best faith effort.12. Licenses for existing assets Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?Answer: [Yes]Justification: All creators and owners of assets are properly credited, and the license and terms of use explicitly mentioned and properly are respected.Guidelines:• The answer NA means that the paper does not use existing assets.• The authors should cite the original paper that produced the code package or dataset.• The authors should state which version of the asset is used and, if possible, include a URL. • The name of the license (e.g., CC-BY 4.0) should be included for each asset.• For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.• If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswithcode.com/datasets has curated licenses for some datasets. Their licensing guide can help determine the license of a dataset. • For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided. • If this information is not available online, the authors are encouraged to reach out to the asset's creators. 13. New Assets Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets? Answer: [Yes] Justification: New assets introduced in the paper are well documented, with the documentation provided alongside the assets. Guidelines:• The answer NA means that the paper does not release new assets.• Researchers should communicate the details of the dataset/code/model as part of their submissions via structured templates. This includes details about training, license, limitations, etc. • The paper should discuss whether and how consent was obtained from people whose asset is used. • We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution. • For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review. 
 
 Appendix: Although our proposed MM-Det advances in detecting fake videos, further issues are left to handle. First, as the landscape of video manipulation technology evolves, new techniques and tools will outpace existing detection methods. The gap between training data and real-world applications can lead to misleading results. Currently, our method is limited to fully synthesized diffusion videos, lacking the generalization to more delicate forgeries like partial manipulation. A possible reason is that small forgery traces disappear after multiple downsampling operations in the deep network of LMMs. Besides, the integration of a large language model into detection costs huge computational complexity, which is not an optimal choice for an environment with limited resources.In conclusion, while our algorithm makes a critical step forward in detecting fake videos, it faces significant challenges due to the rapid advancement of video manipulation technologies. Addressing these limitations requires further research to keep pace with the evolving techniques in digital content manipulation.In this work, our team has developed an effective algorithm to detect fake videos, a breakthrough that promises to fortify the authenticity of online media. In real-world social media where misinformation can spread rapidly, our method acts as a crucial safeguard by empowering platforms to flag and remove deceptive videos before they can mislead users. However, our methods may fail in extreme situations, such as blurred images or noisy images. The algorithm should be carefully treated to avoid misleading results. The long-term influence of our work protects public trust by ensuring the authenticity of digital content. Meanwhile, precaution is needed for a fair application.We propose a comprehensive dataset, named Diffusion Video Forensics (DVF), for diffusion video forensics, as shown in Tab. S1. DVF consists of fake videos generated from 8 different generation methods, covering text-to-video and image-to-video generative methods. In total, We make a collection of 2, 750 real videos and 3, 938 fake videos. Real videos are from YouTube and Internvid-10M [54] .We formally introduce the generation pipeline for video collection. Generation methods in DVF are divided into closed-sourced methods and open-sourced methods. For closed-sourced methods(Sora, Pika, Stable Diffusion [42] and Stable Video), we collect video samples from official websites and social media like TikTok to form the forgery video datasets. For open-sourced methods, the generation pipelines are divided into text-to-video (OpenSora, VideoCrafter1 [5] and Zeroscope) and image-to-video (Stable Video Diffusion [4] ). For a text-to-video generative method, real data derives from a text-image paired video dataset, Internvid-10M. Specifically, we fetch paired real videos R = {r 1 , r 2 , ..., r N } and corresponding captions C = {c 1 , c 2 , ..., c N }. We directly apply the Give concrete reasons why it is real/fake.Figure S1 : The overview of Rich Forgery Reasoning Dataset. To obtain text-image paired data on forgery reasoning, we take advantage of the powerful reasoning ability of Gemini [48] to generate ground truth for each image. First, an instruction T is sampled from a template, along with a frame f ∈ R H×W ×C fed into Gemini for forgery analysis and detection. The response r contains detailed judgment and reasoning on the content and authenticity of f (e.g., analyses on color and depth). Finally, f and r are collected as the text-image paired data to form Rich Forgery Reasoning Dataset.captions as prompts to generate fake video datasets F , such thatF = {f 1 , f 2 , ...f N }, f i = g t (c i ), i ∈ [1, N ],where g t denotes a text-to-video method. Both R and F are included in DVF as real and fake datasets. For image-to-video methods, real videos come from Youtube-8M [1] , which are denoted as R = {r 1 , r 2 , ..., r N }, r i ∈ R L×H×W ×C . For each video r i , a real frame x i ∈ R H×W ×C is randomly sampled from r i and serves as the conditional input for generation. The fake datasets are obtained asF = {f 1 , f 2 , ...f N }, f i = g im (r i ), i ∈ [1, N ],where g im denotes an image-to-video method.We construct a text-image paired dataset, called Rich Forgery Reasoning Dataset (RFRD), to support instruction-tuning LMMs on the forgery detection task, as shown in Fig. S1 . We start from the YouTube and Stable Video Diffusion dataset in DVF, where we select 1, 000 real videos and 1, 800 fake videos. Depending on the powerful reasoning ability of Gemini [48] v1.5 Pro, we follow the scheme in Fig. S1 to generate ground truth for frames. In total, 1, 921 real frames and 3, 579 fake frames are sampled to generate 5, 500 image-text paired textual descriptions. These descriptions are then cleaned and converted into 38k multi-turn conversations for fine-tuning LLaVA [28] in LMM branch of MM-Det.Hyperparameters of MM-Det We introduce the implementation of our MM-Det. In ST branch, we employ a Hybrid-ViT [11] to build the video-level feature encoder. We choose Hybrid-ViT-B, with a ViT-B/16 on top of a ResNet-50 backbone, the patch size 14 × 14, and the hidden size 768. We employ IAFA in each attention block of the ViT, with FC-tokens initialized as the class token of ViT. In the embedding stage, learnable spatial and temporal embeddings are introduced in the form of addition. All patches at the same position within frames share the same spatial embedding and patches with the same timestep share the same time embedding. A pre-trained VQ-VAE is applied to reconstruct videos, with hidden size d = 256 and the codebook size K = 512. We train the VQ-VAE on 50, 000 images from ImageNet. In LMM branch, we utilize LLaVA [28] v1.5, with a CLIP [39] encoder E of CLIP-ViT-L-patch14-336 and a large language model D of Vicuna-7b for reasoning. Our proposed MMFR is composed of the pooler output F V ∈ R 1024 from the last layer of E and the embedding of output F L from the last layer of D. To balance effectiveness and efficiency, we fix the length of output tokens into 64 and obtainF L ∈ R 64×4096 .Training and Inference As for the experimental resources in training and inference, we conduct all experiments using a single NVIDIA RTX 3090 GPU and a maximum of 200G memory.The training strategy of MM-Det is in two-stage. We first conduct instruction tuning for LLaVA in the LMM branch based on LoRA [21] . We start from a pre-trained LLaVA v1. for training until the model converges.For inference, we evaluate all models at the video level. For frame-level baselines, the score of an entire video is obtained as the average score of all frames. For video-level methods, successive clips are fed according to the corresponding window size, and the entire score is obtained as the average score of all clips. In addition, to leverage the efficiency for inference, MM-Det first caches MMFR for each video by conducting reasoning at the interval of 1 frame every 6 seconds. During inference, each video clip directly applies MMFR from the nearest cached frame as an approximation to reduce the huge computational cost.We adopt alternative LLMs in our MM-Det to evaluate different choices of language backbones. Performance is reported in Tab. S2. More formally, for a fair comparison, when using different LLMs, we maintain other components e.g., CLIP, the reconstruction procedure, IAFA, and the dynamic fusion of the original MM-Det remained. Specifically, the introduction of a combined vision and text space from Vicuna-7b in LLaVA improves the performance by +5.1%. As for the choice of LMMs, Vicuna-7b achieves a total 95.7% performance, +1.2% higher than Mistral-7b. We suppose this result may be attributed to different attention mechanisms in Vicuna and Mistral. Vicuna-13b gains a further improvement by +0.6% due to incremental parameters in capturing more effective multi-modal feature spaces. These results prove that our MMFR is effective and extensible to other language models.To analyze the robustness of our method, we conduct an additional evaluation of MM-Det based on common post-processing operations. We choose Gaussian blur with σ = 3, JPEG compression with quality Q = 90, resize with a ratio of 0.7, rotation with an angle of 90, and a mixture of all operations as unseen perturbations in real-world scenarios. Testing samples are selected from DVF to form a total of 500 real videos and 500 fake videos. As reported in Tab. S3, MM-Det meets a degradation of 2.5%(JPEG Compression) to 6.5%(Gaussian blur), with all performance above 89%. The results indicate the effectiveness of our method under these operations.NeurIPS Paper ChecklistJustification: The research conducted in the paper conforms, in every respect, with the NeurIPS Code of Ethics.Guidelines:• The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.• If the authors answer No, they should explain the special circumstances that require a deviation from the Code of Ethics. • The authors should make sure to preserve anonymity (e.g., if there is a special consideration due to laws or regulations in their jurisdiction). 


Decision: This manuscript studied AI-Generated video detection based on LLM and learning multi-model forgery representation. A new dataset is also presented to address the lack of benchmarks in AIGC detection.
The four reviewers generally appreciate the contributions of this paper and give positive comments.
As for the weaknesses posed in the review process, the reviewers also are satisfied with the responses from the authors.
Hence, the AC suggests to accept this manuscript.

Reviews: review: paper_summary: paper_summary: This paper proposed a video-level detection algorithm, named Multi-Modal Detection (MM-Det) for video forensics. MM-Det consists of Multi-Modal Forgery Representation (MMFR) that discriminates fake videos from real ones, In-and-Across Frame Attention (IAFA) that balances frame-level forgery traces with information flow across frames, and Dynamic Fusion Strategy for augmentation in features of high correlation and suppression for helpless ones integrates perception and reasoning abilities for video forensic work. In addition, this paper establishes a high-quality dataset including videos generated from various diffusion-based algorithms. Evaluation of several benchmarks confirms the effectiveness of MM-Det on general content from unseen diffusion models.
strength_weakness: strength_weakness: 1.	As for LLM, it isn't easy to see which module represents LLM from the paper and Figure 3. If LLM is decoupled and divided into several modules, the article should give explanations. 
2.	The author declares that IAFA conducts global and local attention alternately. However, the paper lacks adequate analysis and experiments to illustrate the functionality of IAFA. 
3.	Besides, LLM plays an important role in MM-Det. I think conducting an ablation study on LLM is necessary to see if different LLMs influence detection performance.
questions: questions: 1.	Which part in Figure 3 denotes LLM?
2.	Analysis and experiment to illustrate the functionality of IAFA. 
3.	Ablation study on LLM.
limitations: limitations: The author has fully discussed the limitations of this work and pointed out the direction of future work.
paper_summary: paper_summary: This paper proposes a method for detecting diffusion-generated fake videos using a multi-modal approach. Key contributions include:
A Multi-Modal Forgery Representation leveraging vision and language capabilities of large multimodal models 
An In-and-Across frame attention to capture spatial-temporal forgery traces
A fusion strategy to combine multi-modal features
A new dataset of diffusion-generated videos for benchmarking
The method outperforms existing approaches on cross-dataset evaluation.
strength_weakness: strength_weakness: The use of LMMs and complex Transformer architectures might impose high computational demands, which could limit practical deployment in resource-constrained environments.
Discussion on Failure Cases: The paper could benefit from a detailed discussion on scenarios where the proposed method fails or underperforms, which could guide future improvements and research.
Limited discussion of computational requirements and inference speed
Limited exploration of how the method generalizes to non-diffusion generated videos
it is unclear how the system performs under different operational conditions or with lower-quality videos.
questions: questions: How does the computational cost and inference speed compare to existing methods?
How well does the method generalize to detecting non-diffusion generated fake videos?
Could the authors discuss the transferability of the proposed MMFR to other forms of synthetic media detection, such as audio or text?
limitations: limitations: Potential biases in the dataset
Scalability to longer videos or different resolutions
Adaptability to rapidly evolving diffusion models
Potential for overfitting to current generation artifacts
paper_summary: paper_summary: This manuscript presents a new approach to detect fake videos generated with diffusion models. This approach is based on multimodality analysis and reports promising results on a database introduced by the author(s). These results are based on both frame and video levels. This work is well written and organized.
strength_weakness: strength_weakness: A clear justification for the complex approach is not provided, and readers expect more details.

I think that the method is limited in a global application to the recognition of videos generated with diffusion models and that we should use other methods for other types.
questions: questions: How can you add the method to a general method to detect all or most forgery video types?

What do you think will happen if you have created a video using other methods such as GAN methods?
limitations: limitations: These limitations are addressed by the author(s).
paper_summary: paper_summary: This work focuses on diffusion model detection, a core and popular research topic recently. It identifies limitations in previous studies that concentrate on fake face and image-level detection and explores the idea of using recent LMM to detect forgery. The proposed Multi-modal Forgery Representation (MMFR) leverages existing LMM and introduces a new in-and-across frame attention mechanism. Additionally, a new dataset is proposed, and empirical results demonstrate the effectiveness of the proposed methods.
strength_weakness: strength_weakness: 1. To me, the contribution of in-and-across frame attention seems limited. Providing a comparison to other video-level ViTs and more details about its differences from other works regarding ViT would make this contribution more convincing.

2. The DE-FAKE [R1] method also adopts multi-modal representations, but this work is neither discussed nor compared in the paper. 

 [R1: DE-FAKE: Detection and Attribution of Fake Images Generated by Text-to-Image Generation Models]

3. The proposed dataset is a significant contribution of the paper. Therefore, instead of individual frames, it would be more convincing to include sample video frames either in the supplementary material or on the project page.

4. The main performance table needs more details and a clarification on why it contains both frame and video comparisons.
questions: questions: Please refer to the weakness section.

1. The overall work is novel and demonstrates strong empirical performance, but I have minor concerns regarding its contributions, specifically the in-and-across attention mechanism and the dataset. 

2. Additionally, more related work should be discussed and compared to provide a comprehensive context.
limitations: limitations: Yes, the authors adequately discussed the limitations and societal impact of their work in Appendix.
paper_summary: paper_summary: To detect video forensics, the authors propose an innovative Multi-Modal Forgery Representation (MMFR) to discriminate fake videos from real ones. Besides, the authors establish a high-quality dataset including videos generated from various diffusion-based algorithems to evaluate the effectiness of the proposed detector.
strength_weakness: strength_weakness: (1) I didn't understand how the authors use VQVAE to amplify diffusion features, and it seems that this reconstruction branch is not shown in Figure 3.
(2) There have been some works [1,2] that incorporate temporal token in self-attention, is this different from IAFA? What are the advantages of the proposed method?  
[1] Wang J, Yang X, Li H, et al. Efficient video transformers with spatial-temporal token selection[C]//European Conference on Computer Vision. Cham. Springer Nature Switzerland, 2022: 69-86.  
[2] Liu Y, Xiong P, Xu L, et al. Ts2-net: Token shift and selection transformer for text-video retrieval[C]//European conference on computer vision. Cham: Springer Nature Switzerland, 2022: 319-335.  
(3) As shown in Section 4.2, the authors collected a training set and a test set to evaluate the performance of different methods. What is the relationship between these data and DVF? The article does not seem to describe the benefits of the DVF, e.g., how much performance improvement does the DVF bring to the authors' method, and how much performance improvement does the DVF bring to other methods?  
(4) Forgery video detection is challenging task, however, the authors' experimental content is too small and simple, only 2 tables of experimental content is difficult to fully verify the effectiveness of the proposed method. The authors need to do some other types of experiments to further validate the performance of the method.
questions: questions: From the visualization results, the colors of the fake image are more colorful, is the author's method still valid if the brightness is adjusted to make them similar to the real image?
limitations: limitations: Yes.

Comments: title: Thank you, Reviewer Niir
Thank you, Reviewer LBha
Thank you, Reviewer H1bu
Thank you, Reviewer X8oW
comment: Dear Reviewer Niir,

We are glad that our additional experiments solved your concerns. Thanks again for your valuable review. 

Best regards,

Authors of Submission 2768
Dear Reviewer LBha,
 
We are glad that our rebuttal has successfully addressed your concerns. We will include the clarifications into the revised manuscript. Thank you again for your valuable comments.

Best regards,

Authors of Submission 2768
Dear Reviewer H1bu,
 
Thank you for your recognition of our work. We greatly appreciate your valuable comments.

Best regards,

Authors of Submission 2768
Dear Reviewer X8oW,
 
We are glad that our rebuttal has addressed your concerns. Thanks again for your valuable comments. 

Best regards,

Authors of Submission 2768
The additional experiments solved my concerns, so I have increased my score to 6 (weak accept).
Thank you for your response. I keep the rating.
The author did additional experiments to solve my concerns, so I have increased my score to 6 (weak accept).
Thank you for the clear rebuttal. My concerns have been addressed. I have increased my score to 7 (accept). It would be good to include some of clarifications provided above into the revised version.