Visual Counter Turing Test (VCT^2): Discovering the Challenges for
  AI-Generated Image Detection and Introducing Visual AI Index (V_AI)
Authors: name: Nasrin Imanpour; name: Shashwat Bajpai; name: Subhankar Ghosh; name: Sainath Reddy Sankepally; name: Abhilekh Borah; name: Hasnat Md Abdullah; name: Nishoak Kosaraju; name: Shreyas Dixit; name: Ashhar Aziz; name: Shwetangshu Biswas; name: Vinija Jain; name: Aman Chadha; name: Amit Sheth; name: Amitava Das
Published: 2024-11-24 06:03:49

Abstract: The proliferation of AI techniques for image generation, coupled with their
increasing accessibility, has raised significant concerns about the potential
misuse of these images to spread misinformation. Recent AI-generated image
detection (AGID) methods include CNNDetection, NPR, DM Image Detection, Fake
Image Detection, DIRE, LASTED, GAN Image Detection, AIDE, SSP, DRCT, RINE,
OCC-CLIP, De-Fake, and Deep Fake Detection. However, we argue that the current
state-of-the-art AGID techniques are inadequate for effectively detecting
contemporary AI-generated images and advocate for a comprehensive reevaluation
of these methods. We introduce the Visual Counter Turing Test (VCT^2), a
benchmark comprising ~130K images generated by contemporary text-to-image
models (Stable Diffusion 2.1, Stable Diffusion XL, Stable Diffusion 3, DALL-E
3, and Midjourney 6). VCT^2 includes two sets of prompts sourced from tweets by
the New York Times Twitter account and captions from the MS COCO dataset. We
also evaluate the performance of the aforementioned AGID techniques on the
VCT^2 benchmark, highlighting their ineffectiveness in detecting AI-generated
images. As image-generative AI models continue to evolve, the need for a
quantifiable framework to evaluate these models becomes increasingly critical.
To meet this need, we propose the Visual AI Index (V_AI), which assesses
generated images from various visual perspectives, including texture complexity
and object coherence, setting a new standard for evaluating image-generative AI
models. To foster research in this domain, we make our
https://huggingface.co/datasets/anonymous1233/COCO_AI and
https://huggingface.co/datasets/anonymous1233/twitter_AI datasets publicly
available.