AIGCs Confuse AI Too: Investigating and Explaining Synthetic Image-induced Hallucinations in Large Vision-Language Models

Authors: yifei  gao
jiaqi  wang
zhiyu  lin
jitao  sang
Venue: acmmm_org_ACMMM_2024_Conference
Published: 2024-04-08 09:18:39

Abstract: The evolution of Artificial Intelligence Generated Contents (AIGCs) is advancing towards higher quality. The growing interactions with AIGCs present a new challenge to the data-driven AI community: While AI-generated contents have played a crucial role in a wide range of AI models, the potential hidden risks they introduce have not been thoroughly examined. Beyond human-oriented forgery detection, AI-generated content poses potential issues for AI models originally designed to process natural data. In this study, we underscore the exacerbated hallucination phenomena in Large Vision-Language Models (LVLMs) caused by AI-synthetic images. Remarkably, our findings shed light on a consistent AIGC hallucination bias: the object hallucinations induced by synthetic images are characterized by a greater quantity and a more uniform position distribution, even these synthetic images do not manifest unrealistic or additional relevant visual features compared to natural images. Moreover, our investigations on Q-former and Linear projector reveal that synthetic images may present token deviations after visual projection, thereby amplifying the hallucination bias.

Main paper: 
 introduction: 
 With the rapid evolution of generative model techniques, Artificial Intelligence Generated Contents (AIGCs) have ushered in a new era of prosperity [22, 26] . AIGCs are no longer mere outputs of generative models; rather, they encompass the information generated during human-model or model-model interactions [3] . This result in a large amount of synthetic content rapidly flooding into the Internet, and people may have interacted with synthetic content unconsciously.Nevertheless, the pervasive AI-generated content may give rise to several challenges. The first challenge to gain widespread attention is forgery detection [21] . This field aims to assist humans in distinguishing between natural and synthetic content and is considered a crucial aspect in AI safety. Particularly, a recent study [18] has indicated that the recognition of synthetic images has resulted in an approximate 40% human error rate, solidifying the fact that humans are easily confused by AIGCs. Another under-examined challenge is the impact of AI-generated content on AI models themselves. As synthetic data plays a more common role in the training and reasoning process [28] , the hidden risks of AIGCs to the AI models are urgent yet largely unexplored. Taking Figure 1 as an example, synthetic images with identical semantics are more likely to induce hallucinations in LVLMs than natural images. This is largely because these models' training data, architecture, and training processes are inherently designed for natural data. Applying models trained on natural data to synthetic datasets without complications could lead to unexpected outcomes. We are motivated to explore the topics of how synthetic data may impact AI models. In particular, this study focuses on the hallucination issues that synthetic images may cause in Large Vision-Language Models (LVLMs). Before addressing the core research question, we first establish a synthetic image-involved hallucination evaluation environment for LVLMs. Many current generative models adopt the Text-to-Image synthesis approach [22, 30] . However, the generation process leads to two primary issues: 1) semantic distortion [31] , where synthetic images lack authenticity (e.g., the finger problem [18] ) and; 2) semantic ambiguity [19] , where synthetic images lack consistency and struggle to respond to text prompts. Given the absence of an available synthetic image dataset for hallucination evaluation, mitigating the impact of the aforementioned issues is necessary. To this end, we introduce a Semantics Translation (ST) method, which begins with a natural image and employs 1) caption generation and revision, and 2) semantic filtering strategies to control the authenticity and consistency of the synthetic image, ensuring that the evaluation is not affected by the quality of the synthetic image.We translate two widely used hallucination evaluation datasets: POPE [13] and AMBER [23] , and delve into the hallucinations induced by synthetic images. We also compare them with the corresponding natural images. Surprisingly, our findings indicate that LVLMs have a bias towards synthetic images, as shown in Figure 1 . We refer to this phenomenon as synthetic image-induced hallucination bias (shorten as hallucination bias). Our further experiments reveal that the hallucination bias mainly exhibits 1) a greater quantity and 2) a more uniform position distribution of hallucinated content. Particularly, these phenomena are corroborated across different LVLMs and evaluation datasets. In other words, these LVLMs appear to adopt some inherent non-semantic shortcuts in synthetic images, which lead to a continuous impact on the extrapolation process. Then, we are committed to further studying how synthetic images confuse LVLMs. Drawing inspiration from the visual projection process of LVLMs, we examine two prevalent visual projection modules: Q-former and Linear. Specifically, our investigations shed light on the fact that 1) turning off the Q-former projection or 2) deepening the layers of the Linear projection can 1) effectively mitigate the token deviation of synthetic images and 2) narrow the synthetic image-induced hallucination bias. That is to say, LVLMs tuned in this way may generate less hallucinated content in response to synthetic images.Our core contributions are as follows: (1) In the context of the rapid development of AIGC, we explore the impact of synthetic images on the hallucination problem of LVLMs for the first time. To achieve this, we introduce a semantics translation method to establish a synthetic image-involved hallucination evaluation environment. (2) Extensive experiments uncover the synthetic image-induced hallucination bias of LVLMs, mainly manifesting as (i) a greater quantity, and (ii) a more uniform position distribution. (3) We provide an in-depth analysis on the synthetic image-induced hallucination bias from the perspective of visual-text alignment. Experimental results reveal that the current design of the visual projection module may cause the token deviation of synthetic images, thus resulting in the hallucination bias. 
related works: 
 AIGC and its Challenges: Recent advances in Artificial Intelligence Generated Contents (AIGCs) have profoundly transformed the approach to contents generation, and offered numerous benefits for humans in various aspects of life and work [25] . For instance, generative models such as Stable Diffusion [19] or ChatGPT-4 [1] can generate high-quality image or text information by adhering to the textual descriptions prompted by users. However, as AIGCs are progressively introduced into the online world and applied to society, it's essential to take note of certain potential hidden risks they may introduce. The first challenge to gain widespread attention is fake detection, which has been seen as the boundary of AI safety [21] . Another prevailing challenge stems from the insufficient examination on the impact of AI-generated content on AI models themselves. Specifically, certain researches have identified multifaceted risks associated with AIGCs, including issues related to security, bias, and privacy breaches [3, 4, 8, 24] . Furthermore, recent studies [6, 27] have demonstrated that synthetic data can introduce source bias in both text and cross-modal retrieval for web search, leading to an elevated ranking of AI-generated content (AIGCs). Therefore, within the context of the widespread application of AIGC, it is crucial to thoroughly investigate the potential risks it may pose.Hallucinations of LVLMs: Large Vision Language Models [2, 16, 29, 34] (LVLMs) are regarded as a natural extension of Large Language Models [32] (LLMs). Through rich visual-text instructionstuning, LVLMs have demonstrated remarkable progress in tackling complex multi-modal tasks, such as Visual Grounding and Visual Question Answering (VQA) [2, 16, 29, 34] . However, LVLMs are also plagued by the hallucination problem [10, 13, 17, 23] , in which the model either 1) depicts inaccurate objects or 2) entirely fabricates content from associated images. These phenomena represent a major bottleneck in their deployment and thus limit their practicality in many scenarios [5, 33] . Recent research has delved into The pipeline of semantics translation method. On the left side, we introduce caption generation and revision method to synthesize a correct description of the given natural image. Red represents the redundant or incorrect information within the initial caption. On the right side, we utilize image synthesis and filtering strategy to sample the final synthetic image, ensuring a strict correspondence to the revised caption and the input natural image. highlighted represents the redundant object in image synthesis process. The final synthetic image satisfies the criteria of authenticity and consistency.Figure 3 : The comparison of key object positions before and after the caption revision. Taking Stable Diffusion v1.5 as an example, where the accepted character limit is 77, the distribution of key objects in the revised caption generally satisfies the limits. hallucination problems from the perspective of evaluation and mitigation [17] . In terms of evaluation, POPE [13] defines the hallucination problem as a binary classification task, aiming to explore the model's perceptual ability with respect to specific objects present in the image. Meanwhile, AMBER [23] has contributed to the most comprehensive hallucination evaluation dataset by extending the binary evaluation approach of POPE and introducing generative task evaluation. The mitigation of hallucination generally involves three aspects: (1) Pre-processing, commonly achieved by providing highquality image-text pairs and well-designed instruction-tuning [14] ;(2) In-processing, where the model is enhanced by strengthening visual-textual feature learning [9] ; (3) Post-hoc processing, known for its superior scalability, usually alleviates model hallucinations in the decoding stage [11] . 
semantics translation: 
 We focus on ensuring accurate hallucination evaluation on synthetic images, with the precondition of excluding the quality influence.Specifically, a synthetic image should meet: 1) authenticity, where synthetic semantics should align with human cognition; and 2) consistency, where the synthetic image should accurately respond to the text prompt. In this section, we introduce a semantics translation method to synthesize high-quality images. As shown in Figure 2 , the synthesis process of semantics translation method is constrained under a natural image supervision through the following two steps: 1) Caption Generation and Revision, transforming visual semantics into detailed textual semantics(Section 3.1); and 2) Image Synthesis and Filtering, involving (i) image over-sampling and (ii) image filtering based on similarity (Section 3.2).In this subsection, we translate the visual semantics of the given natural image into the textual semantics. As shown in Figure 2 (left), we employ GPT-4V(ision) to capture the key semantics. To maintain accurate textual semantics, we revise the extracted information through GPT-3.5.Caption Generation: In order to ensure that the generative model receives text prompts closely aligned with the semantics of the given natural image, we employ GPT-4V to obtain coarse-grained captions. However, two notable issues persist in the generated captions: 1) redundant or missing information, where the generated semantics do not exist or fail to include the object that should be presented in the image; and 2) excessive caption length, where the generated captions often exceed the word limit accepted by most generative models. This results in the loss of semantic information beyond the word limit, thereby disrupting the consistency of semantics.Caption Revision: To mitigate the aforementioned issues, the generated caption are revised by GPT-3.5. Specifically, we provide manual annotations to assist GPT-3.5 in comprehending and revising the existence, quantity and the relation semantics in the scene. Detailed instruction is available in the Appendix. As shown in Figure 3 , the length of the revised caption is generally in line with the word limit set by the generative model, ensuring that all key semantic information can effectively prompt the generative model.Given the revised caption, we utilize Stable-Diffusion v1.5 to synthesize a set of candidate images. As shown in Figure 2 (right), we apply a filtering strategy to the candidate set, resulting in the final synthetic image with the most authentic and consistent semantics.Image Synthesis: The revised captions are employed as input prompts for image synthesis. We can more easily sample the synthetic image that are similar to natural image by increasing the sampling times. Therefore, we adopt an over-sampling strategy by conducting multiple generations with different random seeds to obtain a set of candidate images.Image Filtering: The filtering process includes two stages: (1) Ensuring authentic semantics in synthetic images, with a focus on avoiding (i) the depiction of objects not existing in natural images or (ii) introducing objects that contradict human cognition. To achieve this, we initially extract objects using automated segmentation tools [35] . Subsequently, we eliminate images displaying an excess or absence of objects in their annotation results when compared to the corresponding natural image. (2) Maintaining consistent semantics with natural images, with a focus on the similarity between synthetic and natural images. Specifically, we filter the candidate set from two dimensions: (i) Image perceptual similarity, referred to as the perceptual system's understanding of the similarity between two images (e.g., high-level semantics in terms of attributes and relations of objects). We use DreamSim [7] , which better corresponds to human perception, to measure the perceptual similarity between synthetic and natural images. (ii) Image semantic faithfulness, referred to as the alignment with textual annotations (e.g., existence-level semantics). Specifically, we compute the cosine similarity between the synthetic image and textual annotations through the CLIP [20] model. Detailed settings are available in the Appendix. Finally, we select the images with lower DreamSim scores and higher CLIP scores as the final synthetic images. 
hallucinations on synthetic images: 
 After excluding the influence of synthetic image quality issues on hallucination evaluation, this section delves into the synthetic imageinduced hallucination. To more intuitively reflect the impact of synthetic images, we also report the hallucination results on the corresponding natural images and conduct a fair comparison in the context of consistent semantics. Specifically, we first introduce the evaluation datasets and metrics to ensure a comprehensive examination (Section 4.1). Subsequently, we quantify the synthetic image-induced hallucinations across different LVLMs and datasets from perspective of hallucination quantity and position distribution (Section 4.2). Finally, we further investigate the effects on hallucination bias through experiments involving 1) prompt templates, and 2) generation temperatures (Section 4.3).Dataset: We selected two widely used datasets, POPE [13] and AM-BER [23] , as benchmarks for hallucination evaluation. Synthetic images corresponding to these datasets are obtained through semantics translation method. POPE focuses on existence-type hallucination, comprising 500 images with corresponding 9000 annotations. AM-BER offers a richer setting with diverse dataset scaling, reasoning tasks, and hallucination types. Specifically, AMBER 1) includes 1004 images with corresponding 15200 annotations; 2) assesses both generative and discriminative tasks reasoning and; 3) encompasses three types of model hallucination, including existence, attribute, and relation.We follow the settings in AMBER, where CHAIR and Cover are used to evaluate the hallucination. CHAIR measures the frequency of hallucinated objects in the responses, while the Cover refers to as the coverage of objects occurring in natural images. Generally, an ideal response should maintain a low hallucination level without sacrificing response quality too much, which means a lower CHAIR and a higher Cover.The hallucination evaluation for the discriminative task is usually defined as a binary classification. Considering the imbalanced distribution of yes and no answers in the question annotations, we referred to POPE and adopted various metrics, including Accuracy, Precision, Recall, and F1-score. Additionally, we report the 'Yes' ratio in POPE to reveal the confidence behavior of LVLMs.Model to be Evaluated: We conduct hallucination evaluation on the current mainstream LVLMs, including MiniGPT4 (13B) [34] , LLaVA-v1 (7B) [16] , LLaVA-v1.5 (7B) [15] , mPLUG-Owl (7B) [29] and Qwen-VL (13B) [2] .Observation on Hallucination Quantity: Table 1 and Table 2 present the evaluation results of the mainstream open-source LVLMs on both AMBER and POPE datasets. A consistent observation emerges that hallucinations induced by synthetic images are generally more pronounced than those observed in natural images. Additionally, we have the following noteworthy observations: (1) LVLMs are easily confused by synthetic images in generative tasks reasoning. This manifests particularly in i) an increased frequency of hallucinated objects and ii) the limited coverage of objects occurring in the image. Moreover, given the consistency constraint on global semantics between natural and synthetic images, it is counter-intuitive for the LVLMs to generate more hallucinations in response to synthetic images. This observation suggests that LVLMs may capture some non-semantic shortcuts beyond the capabilities of the human vision system. (2) The primary source of synthetic image-induced hallucinations in discriminative tasks reasoning stems from the attribute semantics. Table 2 provides a detailed comparison among three discriminative tasks, revealing that synthetic images induce higher hallucination results in attribute semantics. Our further analysis includes a comparison of hallucination numbers within each pair of synthetic and natural images across diverse attribute semantics, involving number, action, and state semantics of objects. As shown in Figure 4 , synthetic images induce higher hallucinations across all three attributes. (3) The synthetic image-induced model reasoning behavior appears to be under-confident. As shown in Table 1 , a surprising observation emerges that the 'Yes' ratio in LVLMs reasoning about synthetic images is much lower than that of natural images. This implies that the non-semantic shortcuts in synthetic images weaken the confidence of reasoning process. In other words, synthetic images are more likely to induce the model to say 'No'. However, on the AMBER result of existence task reasoning, Qwen-VL exhibits higher accuracy on synthetic images, yielding inconsistent behavior. We attribute this disparity to the nature of existence-type hallucination annotations in the AMBER dataset, which consist entirely of counterexample (i.e., questions that consider objects not present in the image). The results in POPE indicate that Qwen-VL has the lowest 'Yes' ratio among all evaluation models, suggesting a low confidence in reasoning discriminative tasks. Given this discrepancy takes advantage of AMBER's annotation to some extent, it does not impact the overall findings. Observation on Hallucination Position Distribution: We mainly focus on generative task reasoning and perform Kernel Density Estimation (KDE) examining the relative position distribution on both synthetic and natural image-induced hallucinated object. As shown in Figure 5 , we observe that in LVLMs' responses to natural images, hallucinated objects tend to appear more at the front of the response, corresponding to the peak of the density curve (i,e., blue distribution) located at the beginning of the relative position. In contrast, in responses to synthetic images, hallucinated objects are relatively uniformly distributed across various locations (i.e., purple distribution). This observation directly indicates that LVLMs usually generate more "security contents" at the end of the response. In contrast, synthetic images exhibit a continuous impact on the extrapolation process of LVLMs, thus resulting in higher hallucination results.Previous analyses have demonstrated that the hallucinations induced by synthetic images differ from those of natural images, characterized by a greater quantity and a more uniform distribution. We define the above phenomenon as the synthetic image-induced hallucination bias. In this subsection, we further investigate the effects on hallucination bias, specifically from the perspectives of prompt templates and generation temperature.Observation on Prompt Templates: For generative task, AMBER uses the most concise and commonly used generative prompt, "Describe this image" to obtain descriptions of images from LVLMs. Drawing inspiration from POPE's design, we opt "Generate a brief/ detailed caption of the image", separately, and explore the influence of prompt templates on the hallucination bias. Table 3 presents the evaluation results of different LVLMs under two prompt templates. A noticeable hallucination bias persists for synthetic images, regardless of whether the prompt template is designed for obtaining detailed or brief descriptions. Intriguingly, we observe that the long-text generation process appears to amplify the hallucination bias, indicating that the trend of increasing the quantity of hallucinated content in synthetic images surpasses that in natural images. Building upon this finding, we seek insights into the quantity evolution of hallucinated objects in the two types of images. As hypothesized in Figure 6 (a), the hallucination bias in position distribution reveals that synthetic images exert a continuous impact on the extrapolation process, leading to the ongoing generation of hallucinations. In contrast, natural images typically achieve the saturation of hallucinations before synthetic images, thus amplifying the hallucination bias. Nevertheless, the extrapolation length is typically predetermined before the reasoning (e.g., 'max_new_token' generally does not exceed 512), imposing an upper bound on the hallucinations quantity for both two types of images. Results on MiniGPT-4 confirm our hypothesis, as shown in Figure 6 (b) .Observation on Temperature: The temperature serves as a crucial hyper-parameter in controlling the randomness and creativity of the text generation process in LVLMs. We investigate the impact on hallucination bias by reasoning under different temperatures. As shown in Figure 7 , we observe that synthetic images consistently induce higher hallucinations, regardless of the temperature. Moreover, higher temperatures lead to an increased frequency of hallucinated objects, although it does not noticeably affect the coverage of objects present in the image. This is attributed to the fact that a higher temperature flattens the probability distribution, granting all words an equal chance of selection. Consequently, this increase the probability of generating hallucinated words without significantly altering the generation of objects present in the image. 
investigating the hallucination bias: viewpoint from visual projection module: 
 Previous discussions have confirmed that synthetic images indeed induce LVLMs to generate more hallucinated content. However, synthetic images also exhibit proficiency in successfully engaging in visual segmentation tasks, as evidenced by the consistency of segmentation annotations with natural images (refer to Section 3). The apparent contradiction between these two phenomena raises questions about how synthetic images confuse LVLMs. It is well-established that an image token is derived by the visual encoding and visual projection processes within LVLMs. Intuitively, the transformation process into an image token may amplify the impact of non-semantic shortcuts. At the same time, given the available difference in the visual representations between natural and synthetic images (e.g., Deepfake detection [21] seeks differences between real and fake faces in the feature space.), we are motivated to examine the effects of synthetic image on the visual projection process. Specifically, we analyze two types of LVLMs represented by Q-former (Section 5.1) and Linear (Section 5.2) projection, with a focus on 1) the relative distance between synthetic and natural images in different token spaces and; 2) the correlation with hallucination bias.Q-former, serving as a vision-language connector outfitted with learnable queries for efficient cross-attention mechanisms, stands as a central innovation in BLIP-2 [12] . To investigate the effect of synthetic images on the Q-former projection, in this subsection, we demonstrate the changes in 1) relative distance, and 2) hallucination bias before and after turning off Q-former visual projector.Experiment Setup: We deploy two variants, namely MiniGPT4-Vicuna0 and MiniGPT4-LLaMA-2-Chat, respectively. Both of them utilize the pre-trained BLIP-2 as the vision encoder. Nevertheless, the former retains the Q-former projection in extracting image tokens, while the latter abandons Q-former, aligning with our fundamental requirements. We employ kernel density estimation to visualize the distribution shift of the relative distance between synthetic and natural image tokens. Furthermore, both two LVLMs undergo hallucination evaluation on synthetic images. This exploration aims to uncover the relationship between the Q-former projection and hallucination bias.Observations on Relative Distance: The tokens of synthetic images deviate from those of natural images after Q-former projection. As illustrated in Figure 8 (a), it is observed that the distribution of relative distances significantly decreases when the Q-former is turned off. This suggests that two types of images, which are initially semantically close at the input level, also maintain proximity in the token space.Observations on Hallucination Bias: Synthetic image-induced hallucination bias is amplified by Q-former projection. As depicted in Figure 9 (a), LVLMs without Q-former exhibit a reduced hallucination bias in reasoning for both generative and discriminative tasks compared to preserving Q-former projections. Moreover, this phenomenon aligns with our observation on relative distance. When the Q-former projection is turned-off, the distribution of synthetic images and natural images in the token space gradually converges. Consequently, the issue of hallucinations in synthetic images is rectified to the same level as observed in natural images. Linear projection is a significant characteristic of the LVLMs represented by LLaVA architecture, and has emerged as the preferred projection module for many LVLMs owing to its simplicity and effective image-text alignment. Following the same settings on relative distance and hallucination bias mentioned in Section 5.1, this subsection presents two aforementioned observations as we deepen the layers of linear projection.Experiment Setup: We deploy two variants, namely LLaVA-Vicuna and LLaVA-LLaMA2-chat respectively. Both of them used pretrained CLIP-ViT-Large-34 as vision encoder. Nevertheless, the former utilizes the MLP (i.e., multi-layers linear projection) in extracting image tokens, while the latter retains single layer linear projection.Observations on Relative Distance: The tokens of synthetic images become closer to those of natural images after multi-layers linear projection. Figure 8 (b) reveals a narrowing of the distribution when progressing from a single linear projection to MLP, indicating that deepening the linear projection layer proves effective in reducing the token deviation between synthetic and natural images.Observations on Hallucination Bias: Synthetic image-induced hallucination bias can be reduced by deepening the layers of linear projection. As shown in Figure 9 (b), as the layers of Linear projection decrease, the hallucination bias of the synthetic image increases from the original 4.35 to 6.41. This indicates that the single layer linear projection, fine-tuned with natural text-image pairs, is more susceptible to the influence of synthetic images. Conversely, the MLP structure ensures a closer relative distance between the two types of images, resulting in less hallucination bias for the synthetic image. 
conclusion: 
 Despite the prosperity of generative models, the risks and challenges posed by AIGC cannot be overlooked. This paper pioneers an exploration into the impact of synthetic images on hallucination problems during the reasoning process of LVLMs. Extensive experimental results have confirmed a significant deviation between synthetic image-and natural image-induced hallucination, referring to as the hallucination bias. Further analyses have revealed that the divergence between synthetic and natural image tokens can be attributed to the visual projection module, leading to the amplification of relative distance and hallucination bias. Our future works will endeavor from the following aspects: 1) revealing the cause of synthetic imageinduced hallucination bias from the perspective of image synthesis mechanisms and; 2) mitigating the hallucination bias in the reasoning process of LVLMs on synthetic images.This paper also emphasizes the importance of understanding the differences between natural and synthetic data beyond the applications of forgery detection [21] . As AI-synthetic data becomes more prevalent, we may encounter the following scenarios: Scenarios one: Training with natural data and applying it to natural data. This is the primary focus of current research, where many tasks are well-solved under laboratory conditions. The proposing algorithms achieve performance close to or even beyond human benchmark. Scenarios two: Training with natural data and applying it to synthetic data. This is the scenario discussed in this paper. Scenarios three: Training with synthetic data and applying it to natural data. For instance, the widespread use of the ShareGPT dataset in training large language models, and the potential use of game engine-generated data by Sora. Synthetic data can compensate for the deficiencies of natural data and contribute to continuous improvement in model capabilities. This situation is expected to grow. Scenarios four: Training with synthetic data and applying it to synthetic data. Both scenarios two and three can be seen as generalized Out-Of-Distribution (O.O.D.) problems, which can be referred as "Natural-Synthetic O.O.D." They address the generalization from natural to synthetic data and vice versa. In fact, even in scenario four, some form of mixing natural and synthetic data for training should be considered. Therefore, understanding the differences between natural and synthetic data is not only essential for authenticity verification but also crucial for better utilization of synthetic data in training and effective interaction with synthetic data in applications. 
 
 Appendix:  


Decision: The paper received nearly positive ratings initially. The reviewers think the paper is novel and interesting. The main concerns are some of the claims are not convincing and more comprehensive analysis is needed. After the rebuttal, most of the reviewers' concerns have been addressed. AC made the acceptance decision.

Reviews: review: paper_summary: paper_summary: This paper examines the impact of AIGCs on Large Vision-Language Models (LVLMs).
strength_weakness: strength_weakness: 1. This paper tackles the crucial issue of visual hallucinations in LVLMs, a topic of great significance to the community.
2. The workload of this paper is sufficient.
limitations: limitations: 1. The writing of the paper has room for improvement in terms of organization and readability. 
2. The claim about the synthetic image quality issues (line 388) is not reasonable. Relying on the filtering process based on similarity scores cannot prove that the quality of the synthetic images is good enough. I think the influence of synthetic image quality issues still exists, resulting in bad evaluation in test period.
3. The overall pipeline is not that reasonable. GPT-4V, as an LVLM, may also exhibit hallucination in the caption period. Additionally, GPT-3.5 ignored the information "there are two trees in this image" and provided an incorrect revision. The content consistency between the natural images and their captions is questionable. 
4. The experiment results in Table 1 is not pronounced enough to support the conclusion. Especially considering that synthetic images are not as good as natural images.
5. Minor issues. GPT-4v was incorrectly labeled as GPT4-V in Figure 2.
paper_summary: paper_summary: This paper explores the impact of Artificial Intelligence Generated Contents (AIGCs) on Large Vision-Language Models (LVLMs), focusing on the phenomenon of hallucinations induced by synthetic images. The study reveals that synthetic images often cause LVLMs to hallucinate more frequently and uniformly compared to natural images. The paper introduces a semantics translation method to create a controlled environment for evaluating hallucinations and examines how different visual projection modules (Q-former and Linear projection) contribute to these hallucinations. The findings highlight the need for better understanding and mitigating the risks posed by synthetic data on AI models.
strength_weakness: strength_weakness: Novelty: The paper addresses a relatively unexplored area by investigating how synthetic images impact LVLMs, which is crucial given the increasing use of AIGCs in various applications. This focus on AI-to-AI interactions adds a new dimension to the existing research on AI-generated content.

Comprehensive Evaluation: The paper utilizes extensive experiments to compare hallucinations in LVLMs induced by synthetic and natural images. The use of multiple datasets (POPE and AMBER) and various evaluation metrics (e.g., CHAIR, Cover) provides a thorough assessment of the hallucination phenomenon.

Detailed Analysis: The investigation into the effects of different visual projection modules on hallucinations is detailed and insightful. The paper clearly explains how the Q-former and Linear projection modules influence the distribution and quantity of hallucinations, contributing to a deeper understanding of the underlying mechanisms.

Clarity: This paper is well-written and easy-to-follow.
limitations: limitations: Scope of Synthetic Data: While the paper provides valuable insights into the impact of synthetic images, it primarily focuses on specific types of synthetic data generated through semantics translation. The results might not be generalizable to all forms of synthetic data, such as those generated by other models or with different properties.

Potential Overfitting: The modifications proposed for the visual projection modules, such as turning off the Q-former or deepening the Linear projection, may address hallucinations in the specific experimental setups used but might not generalize well to other scenarios or datasets.
paper_summary: paper_summary: The paper examines how synthetic images induce hallucinations in AI models, particularly in Large Vision-Language Models (LVLMs). It highlights a significant issue where these models, trained primarily on natural images, exhibit biases and inaccuracies when interpreting synthetic images. The study introduces methods like Semantics Translation to generate and evaluate synthetic images that closely mimic natural images in terms of consistency and authenticity. Extensive experiments reveal that synthetic images tend to induce more frequent and uniformly distributed hallucinations across different LVLMs, a phenomenon termed as 'synthetic image-induced hallucination bias'. The paper also explores modifications in model architectures and image processing techniques to mitigate these biases, particularly focusing on visual projection components like the Q-former and Linear projection modules.
strength_weakness: strength_weakness: **S1: Robust Methodology** The introduction of the semantic translation method is a notable strength, as it ensures that the synthetic images used in experiments maintain high fidelity in terms of visual and semantic consistency with natural images. This methodological rigor allows for more accurate assessments of hallucinations.

**S2: Comprehensive Experiments** The paper conducts extensive experiments to quantify and analyze the hallucination biases, employing multiple datasets and metrics. It not only measures the frequency and distribution of hallucinations but also tests the models under various conditions to generalize findings.
limitations: limitations: **W1: Complexity in Implementation** While the methodologies used are robust, they might be complex for practical implementation outside of research environments. The process of generating and revising synthetic images to match natural images in terms of authenticity and consistency requires significant computational resources and expert knowledge.


**W2: Potential for Overfitting** By tightly controlling the generation and processing of synthetic images to mimic natural images, there's a risk of overfitting the models to the experimental setup, which might not entirely reflect real-world scenarios where synthetic images could be more varied.