Prompt-Consistency Image Generation (PCIG): A Unified Framework Integrating LLMs, Knowledge Graphs, and Controllable Diffusion Models

## Abstract

Abstract The rapid advancement of Text-to-Image(T2I) generative models has enabled the synthesis of high-quality images guided by textual descriptions. Despite this significant progress, these models are often susceptible in generating contents that contradict the input text, which poses a challenge to their reliability and practical deployment. To address this problem, we introduce a novel diffusion-based framework to significantly enhance the alignment of generated images with their corresponding descriptions, addressing the inconsistency between visual output and textual input. Our framework is built upon a comprehensive analysis of inconsistency phenomena, categorizing them based on their manifestation in the image. Leveraging a state-of-the-art large language module, we first extract objects and construct a knowledge graph to predict the locations of these objects in potentially generated images. We then integrate a state-of-the-art controllable image generation model with a visual text generation module to generate an image that is consistent with the original prompt, guided by the predicted object locations. Through extensive experiments on an advanced multimodal hallucination benchmark, we demonstrate the efficacy of our approach in accurately generating the images without the inconsistency with the original prompt. The code can be accessed via https://github.com/TruthAI-Lab/PCIG .

## 1 Introduction

The rapid advancement of Text-to-Image (T2I) generative models has revolutionized the field of computer vision, enabling the synthesis of high-quality images guided by textual descriptions. These models, such as DALL-E [, ], Stable Diffusion [], and GLIDE [], have shown remarkable progress in generating visually appealing and semantically relevant images. However, despite their impressive performance, these models often generate contents that contradict the input text, posing significant challenges to their reliability and practical deployment.

Figure: Figure 1: Selected samples generated by DALL-E 3. Each image represents one specific hallucination type. The inconsistency part for each image is highlighted in red.
Refer to caption: extracted/5622461/figure1.png

Inconsistencies between the visual output and textual input can manifest in various forms, such as mismatched object attributes (First image in Figure 1), inaccurate object placement or count (Fourth image in Figure 1), illegible or incorrect text within the image (Second image in Figure 1), and the inability to accurately depict real-world entities (Third image in Figure 1). These inconsistencies, also known as hallucinations, can severely impact the usefulness and trustworthiness of the generated images, especially in domains where accuracy is crucial, such as medical imaging [, ], autonomous vehicles [], and criminal investigation [].

Existing methods have attempted to address these challenges by improving the alignment between the input text and the generated image. Attention-based approaches [, ] have been proposed to better capture the relationships between words and visual features, while adversarial training techniques [] have been employed to enhance the realism and consistency of the generated images. However, these methods often struggle with complex scenes and fail to address the specific types of inconsistencies mentioned earlier.

To tackle these limitations, we introduce Prompt-Consistency Image Generation (PCIG), a novel diffusion-based framework that significantly enhances the alignment of generated images with their corresponding descriptions. PCIG addresses three key aspects of consistency: (1) general objects (GO), ensuring accurate depiction of object attributes and placement; (2) text within the image (TEXT), generating legible and correct text; and (3) objects that refer to proper nouns existing in the real world (PN), which cannot be directly generated by the model.

Our framework leverages state-of-the-art techniques in natural language processing and computer vision. We first employ large language models (LLMs) [] to extract objects from the input prompt and construct a knowledge graph to predict the locations of these objects in the generated image. LLMs, such as GPT-3 [] and BERT [], have shown remarkable capabilities in understanding and generating human language. By integrating LLMs into our framework, we enable a deeper understanding of the prompt and its relationships, guiding the subsequent image generation process.

Next, we utilize a controllable diffusion model [, , ] to generate an image consistent with the original prompt, guided by the predicted object locations. Controllable diffusion models allow for more fine-grained control over the image generation process by incorporating additional constraints or conditions. For general objects (GO), the model focuses on accurate attribute depiction and spatial arrangement. To handle text within the image (TEXT), we incorporate a visual text generation module [, , ] that specializes in rendering legible and semantically correct text. Recent advances in visual text generation have shown promising results in producing realistic and readable text in images. Finally, for objects referring to proper nouns (PN), we propose a novel approach that searches for representative images of the entities and seamlessly integrates them into the generated image.

Through extensive experiments on an advanced multimodal hallucination benchmark [], we demonstrate the efficacy of PCIG in generating images that align with the original prompt, significantly reducing inconsistencies across all three key aspects. Our unified framework achieves state-of-the-art performance, outperforming existing T2I models in terms of object hallucination accuracy, textual hallucination accuracy, and factual hallucination accuracy.

#### Contributions.

(1) We introduce PCIG, a novel framework that integrates LLMs, knowledge graphs, and controllable diffusion models to generate prompt-consistent images.
(2) We propose a comprehensive approach to address three key aspects of consistency: general objects, text within the image, and objects referring to proper nouns.
(3) We conduct extensive experiments on a multimodal hallucination benchmark, demonstrating the superiority of PCIG over existing T2I models in terms of consistency and accuracy.
(4) We provide insights into the effectiveness of integrating LLMs and knowledge graphs for prompt understanding and object localization in the image generation process.

## 2 Related Work

### 2.1 Text-to-image Diffusion Models

Denoising Diffusion Probabilistic Model [, ] and its subsequent studies [, , , , , ] have showcased impressive capabilities in generating high-quality images guided by textual prompts. These models employ iterative denoising steps starting from a random noise map to learn the process of text-to-image generation. Latent Diffusion Model (LDM) [] takes advantage of iterative denoising steps in a latent space, aiming to enhance text-to-image alignment and reduce training complexity while generating high-quality images from textual descriptions. Stable Diffusion and SDXL [] are applications of the Latent Diffusion method in text-to-image generation but trained with additional data and a powerful CLIP [] text encoder. DALL-E 2 [] and DALL-E 3 [], state-of-the-art text-to-image generation model developed by OpenAI, achieve photorealistic T2I generation using diffusion-based models.

### 2.2 Controllable Image Generation

As text description cannot precisely control the position of generated instances, Some controllable text-to-image generation methods [, , , , , , , ] introduce spatial conditioning controls to guide the image generation process. They extend the pre-trained T2I model [] to integrate layout information into the generation and achieve control of instances’ position. GLIGEN [], MIGC [], and InstanceDiffusion [] are state-of-the-art methods which can support controlled image generation using discrete conditions such as bounding boxes. By integrating spatial conditioning controls, these methods enable users to have control over the positioning of instances in generated images. This advancement allows for fine-grained manipulation and customization in the image generation process.

### 2.3 Visual Text Generation

Current mainstream text-to-image generation models, like Stable Diffusion, excel at producing high-quality images. However, they struggle to generate accurate and legible text on these images. To address this limitation, recent research studies [, , , , , ] have focused on integrating clear and readable text into images by introducing glyph conditions in the latent space. These advancements, particularly, GlyphControl [] and AnyText [], can be seamlessly plugged into existing diffusion models, allowing for more precise rendering of text on generated images.

### 2.4 Knowledge Graph and LLM

Knowledge Graph (KGs) are structured multirelational knowledge bases that typically contain a set of facts. Each fact in a KG is stored in the form of triplet $(s,r,o)$, where $s$ and $o$ represent the subject and object entities, respectively, and $r$ denotes the relation connecting the subject and object entity. KGs are crucial for various applications as they offer accurate explicit knowledge [, , , ]. LLM, pre-trained on the large-scale corpus, such as ChatGPT [] and GPT-4 [] have showcased their remarkable capabilities in engaging in human-like communication and understanding complex queries, bringing a trend of incorporating LLMs in various fields [, , , , , ].
By incorporating KGs, LLMs can benefit from the extensive knowledge stored in a structured and explicit manner. This integration enables LLMs to have a better understanding of the information contained in KGs, which also enhance the performance and interpretability of LLMs in various downstream tasks []. In our work, we leverage the knowledge retrieved from KGs to improve prompt analysis and object localization, enhancing the overall effectiveness of LLMs.

## 3 Method

Figure: Figure 2: The pipeline of our PCIG method, using the example "A blue basketball jersey with the Golden State Warriors logo and ’Stephen Curry’ written on it."
Refer to caption: extracted/5622461/PCIG.drawio.png

In this section, we first provide a detailed definition of consistency hallucination in Sec 3.1. Following that, we delve into the details of our framework with object extraction and classification in Sec 3.2, relation extraction in Sec 3.3, object localization in Sec 3.4, and non-hallucinatory image generation in Sec 3.5. More details of our PCIG framework can be seen in Figure 2.

### 3.1 Consistency Hallucination Definition

Before delving into the methodology, it is essential to first define the types of hallucinations more detailed. Based on the MHaluBench [] benchmark, our focus is centered upon four primary types of hallucinations that arise in text-to-image generation: (1) AH(attribute hallucinations), where the attributes of objects in the generated images are incongruous with the provided prompts; (2) OH(object hallucinations), where the number, placement, or other aspects of objects differ from the provided prompts; (3) SCH(scene-text hallucinations), where the textual content within the generated images does not align with the given prompts;(4) FH(factual hallucinations), where the depicted properties of objects contradict their real-world counterparts. While there are numerous other issues related to image hallucinations, this paper focuses chiefly on the aforementioned problems.

### 3.2 Object Extraction and Classification

Object Extraction. The initial step of the method involves a meticulous process of identifying objects and their attributes from the textual prompt. Given the initial prompt $P$, we extract the objects present, their quantities, and their specific properties and structured them as ${O=\{o_{i}\}_{i=1...N_{o}}}$ where $o_{i}$ represents a concise caption of object, comprising the combination of attribute and object name(i.e. A grazing sheep), and $N_{o}$ denotes the number of the objects identified from the initial prompt. This step is imperative since understanding the precise object information is necessary to accurately generate an image that embodies these exact details. Object Classification. Following the object extraction, the next is to classify each identified object in $O$ into three specific categories $C=\{GO,TEXT,PN\}$ that $O=\{o_{i},C\}_{i=1...N_{o}}$ where $GO,TEXT,PN$ represent general objects, text within the image, and objects that refer to proper nouns existing in the real world respectively. These categories are each linked to different types of hallucinations. $GO$ is associated with attribute and object hallucinations (AH and OH), $TEXT$ corresponds to scene-text hallucinations (SCH), and $PN$ is related to factual hallucinations (FH). This categorization is critical as it not only enables us to handle each hallucination problems separately for different types of objects but also lays the groundwork for subsequent relational and spatial analyses by clearly defining the nature and context of each object within the image.

### 3.3 Relation Extraction

Relationship Recognition. Once the objects are detected, GPT-4 determines the spatial relationships and interactions between the detected objects for the initial prompt. Let $R=\{r_{i}\}_{i=1...N_{r}}$ be the set of relationships identified from the initial prompt where $N_{r}$ is the number of relationships identified in the prompt. Triple Generation. Based on the detected objects $O$ and their relationships $R$, GPT-4 generates triples in the form of (object, predicate, object) to represent the identified relationships. The set of generated triples for the provided prompt is denoted as $T=\{t_{i}\}_{i=1...N_{t}}$, where $N_{t}$ is the number of triples in the initial prompt. Each triples is represented as $T=(O,R,O)$. For example, if the prompt depicts a young girl is wearing a pink dress, GPT-4 would generate a triple such as (Young Girl, is wearing, Pink Dress) where $\{YoungGirl,PinkDress\}\in O$ and $\{IsWearing\}\in R$. Knowledge Graph Construction. The knowledge graph the initial prompt is then constructed using the generated triples $T$ where the objects $O$ serve as the nodes and the relationships $R$ serve as the edges. The knowledge graph for the initial prompt is denoted as $G=(V,E)$ where $V=O$ is the set of nodes (objects) and $E$ is the set of edges with $R$ being the set of all possible relationships in the initial prompt. The construction of the knowledge graph using GPT-4 is a critical step in our method. It provides a structured and detailed representation of the initial prompt, capturing the relationships and interactions between objects in a way that goes beyond simple semantic features []. This enriched representation proves advantageous for subsequent steps of object localization and image generation.

### 3.4 Object Localization

Spatial Organization. Building upon the relationship extraction, this part focuses on the spatial organization of objects within the canvas. The process begins by identifying the node with the max degree $V_{m}$ in the knowledge graph $G$ constructed previously, highlighting it as the pivotal object in the image composition. Determining this anchor object is critical for orienting other objects in relation to it, ensuring a coherent and realistic spatial arrangement. Bounding Box Generation. The meticulous spatial arrangement extends to the precise calculation of each object’s placement in relation to the anchor point and throughout the canvas expanse. This phase demands a fine-tuned equilibrium to instill visual authenticity, taking into account factors like object scale and spatial positioning to construct bounding boxes that convincingly outline the locations of the entities. Let $BB=\{[x_{i},y_{i},w_{i},h_{i}]\}_{i=1...N_{o}}$ denote the set of bounding boxes for the objects, where the tuple $(x,y)$ signifies the object’s coordinates on the canvas, and $(w,h)$ indicates the object’s dimensions within the space. The bounding boxes are precisely structured, conforming to exact dimensional specifications and coordinate precisions, ensuring that every object is proportionately and accurately depicted within the generated image.

### 3.5 Prompt-Consistency Image Generation

Building upon the previously described steps, we have successfully secured a series of well-defined bounding boxes for every object on the canvas. Our objective is to leverage these bounding boxes to generate corresponding images wherein the positioning of objects closely mirrors the layout specified by $BB$. To this end, we employ a controllable text-to-image model as our primary framework of the model that is specifically designed to accept bounding boxes as input, enabling precise manipulation of image outcomes. A visual text generation module is incorporated with the model, designated to handle linguistic elements, forming the essence of our integrated system.

Our system categorizes inputs into three segments as mentioned in Sec. 3.2: GO, TEXT, and PN for image generation. For GO, we input both the bounding box and its caption directly into the main framework of the model. For TEXT, we input the textual content and its corresponding bounding box into a visual text generation module. This module incorporates narrative elements into the visual output. For PN, we use a search engine to find representative images of the objects. These images, along with their bounding boxes, are seamlessly integrated into the model’s primary input stream. By categorizing objects and applying specific generation paradigms, our model prevents the generation of images with hallucination features. This methodical approach results in photorealistic and prompt-consistency images, eliminating the challenges posed by hallucinations.

## 4 Experiments

### 4.1 Experiments Settings

#### Dataset.

Figure: Figure 3: Compared with multiple text-to-image generation methods. Our method shows comparable performance in all aspects.
Refer to caption: extracted/5622461/figure2.png

MHaluBench [] is a benchmark which encompasses the content from text-to-image generation, aiming to rigorously assess the advancements in multimodal hallucination detectors. The benchmark has been meticulously
curated to include 220 exemplars dedicated to Text-to-Image Generation with 158 are hallucinatory and 62 are non-hallucinatory. Specifically, it includes 137 prompts which will generate images with object and attribute hallucination potentially, 63 prompts with textual hallucination, 18 prompts with factual hallucination, 2 prompts with combination of factual hallucination and textual hallucination. In one exemplar, it contains the original prompt augmented through ChatGPT to include more specific information, the presence of hallucination, the analysis for hallucination, and the generated image based on the prompt.

#### Implement Details and Evaluation Metrics.

Our pipeline is training-free and comprises three pre-trained models. We employ the GPT-4 [] as the base LLMs to generate bounding box for identified objects and choose InstanceDiffusion [] as primary controllable text-to-image model while AnyText [] as text-generation module. We utilize UniHD [], which will return a label represents whether the input image with corresponding prompt is hallucinatory or not, as our hallucination detection method for generated images. We calculate the accuracy for each hallucination type mentioned above, including object hallucination accuracy (OH acc.), textual hallucination accuracy (TH acc.), factual hallucination accuracy (FH acc.), textual and factual hallucination accuracy (TFH acc.), and overall accuracy for evaluation.

**Table 1: Experimental results of our framework and various baseline on MHaluBench dataset.**
|  | Method | OH Acc.(%) | TH Acc.(%) | FH Acc.(%) | TFH Acc.(%) | Overall Acc.(%) |
| --- | --- | --- | --- | --- | --- | --- |
| Text-to-Image | SDv1.6 [] | 15.33 | 11.11 | 22.22 | 0.00 | 14.55 |
|  | SDXL [] | 18.98 | 9.52 | 8.33 | 0.00 | 15.91 |
|  | DALL-E 2 [] | 24.82 | 7.94 | 0.00 | 0.00 | 17.73 |
|  | DALL-E 3 [] | 60.58 | 26.98 | 9.99 | 0.00 | 45.45 |
| Layout-to-Image | GLIGEN [] | 88.32 | 7.94 | 22.22 | 0.00 | 59.09 |
|  | MIGC [] | 94.16 | 11.11 | 8.33 | 0.00 | 63.18 |
|  | InstanceDiffusion [] | 95.62 | 9.52 | 22.22 | 0.00 | 64.09 |
|  | PCIG (ours) | 94.89 | 82.54 | 77.78 | 50.00 | 89.55 |

#### Baseline.

Our baseline divides into two parts. The first part is the comparison with the most representative generative models, including Stable Diffusion v1.6 [], SDXL [], DALL-E 2 [], and DALL-E 3 [], which generate visually detailed images directly based on the prompt in the benchmark. The second part is the comparison with the state-of-the-art controllable text-to-image models, also named layout-to-image models, including GLIGEN [], MIGC [], and InstanceDiffusion [], which introduce spatial conditioning controls to guide the image generation process. We will use the bounding box generated in the first three steps to guide the process of image generation.

Figure: Figure 4: Ablation study on knowledge graph construction. Results become inaccurate in object locations when the proposed module is disable.
Refer to caption: extracted/5622461/figure3.png

Figure: Figure 5: Ablation study on object extraction. Results become inaccurate in object count and attribute when the proposed module is disable.
Refer to caption: extracted/5622461/figure4.png

Figure: Figure 6: Ablation study on text generation module. Results become inaccurate in visual text when the proposed module is disable.
Refer to caption: extracted/5622461/figure5.png

Figure: Figure 7: Bounding box generated by different LLM with original prompt "Six giraffes in a grassy plain with trees in the background.".
Refer to caption: extracted/5622461/figure6.png

**Table 2: Ablation results of our PCIG method on MHaluBench dataset.**
| Method | OH Acc.(%) | TH Acc.(%) | FH Acc.(%) | TFH Acc.(%) | Overall Acc.(%) |
| --- | --- | --- | --- | --- | --- |
| w/o KG extraction | 64.96 | 82.54 | 77.78 | 0.00 | 70.45 |
| w/o Object extraction | 75.91 | 7.94 | 11.11 | 0.00 | 50.45 |
| w/o Text module | 95.62 | 9.52 | 22.22 | 0.00 | 64.09 |
| model (ours) | 94.89 | 82.54 | 77.78 | 50.00 | 89.55 |

### 4.2 Experimental Results and Qualitative Analysis

#### Experimental Results.

Table 1 shows that PCIG outperforms the baseline models in all metrics. It is worth noting that the object hallucination accuracy of all text-to-image models, especially in Stable Diffusion [] and DALL-E 2 [], is extremely low. This suggests that these models struggle to generate images that align with the given prompts under such conditions. On the other hand, PCIG and other competitive layout-to-image models demonstrate exceptional abilities in accurately generating objects and their attributes in image generation. In terms of text hallucination accuracy, factual hallucination accuracy, and textual and factual hallucination accuracy, all baseline models perform poorly. In contrast, PCIG stands out from the rest. With the help of prompt analysis and text generation module, PCIG showcases exceptional performance in both text hallucination accuracy and factual hallucination accuracy. It surpasses the baseline models, highlighting its impressive capabilities in text generation and factual object generation. The corresponding prompt template is shown in Figure 8

#### Qualitative Analysis.

Through visualization of generated images, we compare our PCIG with competitive text-to-image generation models (SD [], SDXL [], DALL-E 2 [], and DALL-E 3 []). As depicted in Figure 3, The first row represents the prompts given to generate images and the left column represents the model used to generate images based on prompts. As a result, Stable Diffusion, SDXL, DALL-E 2, and DALL-E 3 shows different types of visualization errors during generation, including the inconsistency between the prompts and object attributes in generated images (column 1 and 2), the inconsistency between the prompts and object locations in generated images (column 3), the inconsistency between the prompts and the number of objects in generated images (column 4), text generation error (column 5), and factual object generation error (column 6). In contrast, Leveraging the capabilities of prompt analysis and text generation module, our PCIG presents accurate and vivid images consistent with the original prompts, as shown in the last row.

**Table 3: Comparison results of different base controllable text-to-image model with and without text module on MHaluBench dataset.**
| model | GLIGEN [] | MIGC [] | InstanceDiffusion [] |  |  |  |
| --- | --- | --- | --- | --- | --- | --- |
|  | OH Acc.(%) | TH Acc.(%) | OH Acc.(%) | TH Acc.(%) | OH Acc.(%) | TH Acc.(%) |
| Base. | 88.32 | 7.94 | 94.16 | 11.11 | 95.62 | 9.52 |
| Base. w/ text module | 89.05 | 76.19 | 94.16 | 79.37 | 94.89 | 82.54 |
| $\Delta$ | +0.73 | +68.25 | +0.00 | +68.25 | -4.27 | +73.02 |

### 4.3 Ablation Study

#### w/o KG extraction.

For w/o KG extraction, the ablation experiment locate the identified objects without relation extraction and knowledge graph construction, which means the lack of relation and spatial analysis for prompts. The corresponding prompt template is shown in Figure 9. Table 2 reveals that when the model lacks the guidance of a knowledge graph, it struggles to fully comprehend the relationship between identified objects. As a result, it is unable to provide accurate localization of objects. In Figure 4, the comparison between our method and the ablation results is depicted. The ablation experiments clearly illustrate the problems that arise when there is a lack of objects mentioned in the prompt (column 1 and 4). Additionally, they highlight the inaccuracies in the positional relationships between the objects (rest of column). These findings emphasize the importance of relation extraction and knowledge graph construction in prompt analysis.

#### w/o object extraction.

For w/o object extraction, the ablation experiment focuses on extracting relationships between objects without considering specific object information. The corresponding prompt template is shown in Figure 10. Table 2 clearly demonstrates that when the model lacks object information, it faces challenges in accurately identifying object attributes and the number of objects while extracting relationships between them. Furthermore, it also struggles in correctly identifying object categories when generating textual and factual object. Figure 5 presents a comparison between our method and the ablation results. The ablation experiment vividly highlights the problem of inconsistency between the number of objects in the generated image and the expected number of objects mentioned in the original prompt. Consequently, the model fails to provide precise object number and attribute information due to the absence of object guidance. which prove the importance of object extraction in prompt analysis.

#### w/o text module.

For w/o text module, the ablation experiment aimed to examine the impact of removing the text generation module in our model. The results, shown in Table 2, highlight that without the text generation module, the model faced challenges in generating accurate text. Figure 6 provides a visual comparison between our method and the ablation results. The ablation experiments demonstrate that errors, such as missing and incorrect text, were prevalent without the text generation module. These findings reinforce the significance of the text generation module in our approach.

#### Different base controllable text-to-image models.

In this section, we conducted ablation experiments using different baseline controllable text-to-image generation models, including GLIGEN [], MIGC [], and InstanceDiffusion [], with and without a text generation module. Table 3 displays the outcomes of ablation experiments conducted on various controllable T2I models, focusing on object hallucination accuracy and text hallucination accuracy. The findings reveal that utilizing different models, with or without a text generation module, both yields outstanding results for object hallucination accuracy. Furthermore, the presence of a text generation module significantly enhances text hallucination accuracy. This implies that the text generation module can be seamlessly integrated into different base models to improve text generation capabilities.

#### Different LLM for prompt analysis.

In this ablation experiment, we test the performance of different language models (LLMs) for prompt analysis. The LLMs we used are GPT4-turbo [], GPT3.5-turbo [], LLAMA2-7B [], LLAMA2-13B, and LLAMA2-70B. We measure the overall accuracies of these models, and the results are summarized in Table 4. According to the results, GPT4-turbo demonstrats the highest level of competitiveness among the LLMs tested. On the other hand, LLAMA2-7B performs the least effectively compared to the other models. Figure 7 displays the bounding boxes generated by different language models when analyzing the prompt "Six giraffes in a grassy plain with trees in the background". GPT4-turbo accurately identifies all objects and provides reasonable positions. GPT3.5-turbo can identify all objects, but the positions it generates are unreasonable. All LLAMA model fail to recognize objects and also generate unreasonable positions.

**Table 4: Comparison results of different LLM for our PCIG method on MHaluBench dataset.**
| Model | GPT4-turbo [] | LLAMA2-7B [] | LLAMA2-13B [] | LLAMA2-70B [] | GPT3.5-turbo [] |
| --- | --- | --- | --- | --- | --- |
| Overall Acc.(%) | 89.54 | 32.27 | 42.27 | 67.27 | 70.91 |
| $\Delta$ | +0.00 | -57.27 | -47.27 | -22.27 | -18.64 |

## 5 Conclusion

In this paper, we introduced the Prompt-Consistency Image Generation(PCIG), a effective approach that significantly enhances the alignment of generated images with their corresponding descriptions. Leveraging a state-of-the-art large language module, we make a comprehensive prompt analysis and generate bounding box for each identified objects. We further integrate a state-of-the-art controllable image generation model with a visual text generation module to generate an image guided by bounding box. We demonstrate our method could handle various type of object category based on the integration of text generation module and search engine. Both qualitative and quantitative results demonstrate our superior performance.

#### Limitations.

Our method uses GPT4-turbo as our LLM to finish object extraction, relation extraction, and object localization, which costs approximately 0.08$ in one generation process. Furthermore, our method have difficulties in generating images with complex relationship and interaction between objects as well as with small text. To address this concern, A more powerful basic diffusion model would be of great help.

## Appendix A Appendix / supplemental material

In this section, we first outline the prompt template in Figure 8
designed to guide the object extraction, relation extraction, and object localization. Then we present the prompt template designed for ablation study in Figure 9 and Figure 10. Furthermore, we present more results of our PCIG method in Figure 11.

Figure: Figure 8: Prompt template of prompt analysis in PCIG method.
Refer to caption: extracted/5622461/prompt.drawio.png

Figure: Figure 9: Prompt template of ablation study on knowledge graph construction in PCIG method.
Refer to caption: extracted/5622461/promptwokg.drawio.png

Figure: Figure 10: Prompt template of ablation study on object extraction in PCIG method.
Refer to caption: extracted/5622461/promptwooe.drawio.png

Figure: Figure 11: More results of our PCIG method.
Refer to caption: extracted/5622461/moreimage.png

Figure: Figure 12: More hallucination results of DALL-E 2.
Refer to caption: extracted/5622461/dallemoreimage.png

## Peer Reviews

confidence: 5
contribution: 2
initial_rating: 5
presentation: 2
questions: - Can the proposed method generalize to long and complex prompts?
soundness: 2
strengths: - The paper is easy to follow.
- The proposed can provide further insights into hallucinations problems in generative models.
summary: The paper proposes PCIG, a diffusion-based framework to significantly enhance the alignment of generated images with their
corresponding descriptions, addressing the inconsistency between visual output and textual input. The framework leverages an LLM to extract key information components from the prompt. This information is then integrates with a state-of-the-art controllable image generation model with a visual text generation module to generate an image that is consistent with the original prompt, guided by the predicted object locations. Extensive experiments have been performed on multimodal hallucination benchmarks.
weaknesses: - Firstly it seems the paper has been written hastily as I don't see much coherence in introduction section.
- The related work is not comprehensive. The authors have missed many existing related works. For instance, in the "LLM assisted generation" section, authors do not discuss [1] and [2] which are very relevant to the paper.
- The method used by the authors has a very limited novelty as the object extraction, attribute extraction, bbox generation from LLM has already been explored in [1] and [2]. Both these works are solving advanced problems and use this approach just as a small part of their method.
- Again the comparisons with many existing layout based methods is missing such as [1] and [2],
- The comparisons with baselines seem unfair. The proposed method uses GPT-4o Turbo while baselines either use existing open-source LLMs or other version of GPT (such as LayoutGPT) or do not use LLM at all. I do not think the scores in Table 1 are valid with these diverse settings.
- The authors should also compare their works with Prompt refinement papers such as  [3] and [4].
- Overall while the paper presents a good direction, there are major issues with respect to novely, missing comparisons and fairness of comparisons.



Reference

[1] Gani, Hanan, et al. "Llm blueprint: Enabling text-to-image generation with complex and detailed prompts." ICLR (2024).

[2] Lian, Long, et al. "Llm-grounded diffusion: Enhancing prompt understanding of text-to-image diffusion models with large language models." TMLR 2024.

[3] Hao, Yaru, et al. "Optimizing prompts for text-to-image generation." NeurIPS 2024.

[4] Zhan, J., Ai et al. Prompt Refinement with Image Pivot for Text-to-Image Generation. ACL 2024.

---

confidence: 5
contribution: 1
initial_rating: 3
presentation: 2
questions: My main concern is that this approach feels more like a temporary fix for the current T2I models rather than a solution to their fundamental limitations.

How much extra computation cost will be introduced?

How to address the errors introduced by the new components (like the built graph)?
soundness: 2
strengths: - The targeted problem is important 
- The solution is reasonable and works well in the benchmark evaluation
summary: This paper aims to address an important problem in current text-to-image generation models: the generated visual contents may not exactly follow the input text description. To solve this problem, this paper proposes a refinement method that first extracts object attribute description and object relation. Then a relation graph is built as input to a controlnet to improve consistency of the generated images with the text description.
weaknesses: - The proposed method does not solve the problem in a fundamental manner (like improve consistency of the image generation model itself). The proposed method involves multiple components. Although the overall pipeline seems reasonable, the involved components may introduce other accumulative errors which may hurt the quality of the generated images. 

- The proposed method will introduce extra computation cost for image generation model.

---

confidence: 4
contribution: 2
initial_rating: 5
presentation: 3
questions: See weaknesses.
soundness: 3
strengths: The strength of this work lies mainly in the fact that the tasks and questions raised are very important and meaningful. Specifically,
- This paper introduces PCIG, a novel framework that integrates LLMs, knowledge graphs, and controllable diffusion models to generate prompt-consistent images. 
- This work proposes a comprehensive approach to address three key aspects of consistency: general objects, text within the image, and objects referring to proper nouns. 
- Extensive experiments on a multimodal hallucination benchmark demonstrates the superiority of PCIG over existing T2I models in terms of consistency and accuracy. 
- Providing insights into the effectiveness of integrating LLMs and knowledge graphs for prompt understanding and object localization in the image generation process.
summary: This work introduces the Prompt-Consistency Image Generation(namely PCIG), a effective approach that significantly enhances the alignment of generated images with their corresponding descriptions. By leveraging a state-of-the-art large language module to make a comprehensive prompt analysis, this work first generate bounding box for each identified objects. Then it further integrates a state-of-the-art controllable diffusion model with a visual text generation module to generate an image guided by bounding box, which demonstrates the method could handle various type of object category based on the integration of text generation module and search engine. Both qualitative and quantitative results demonstrate the superior performance.
weaknesses: - **The innovativeness and practicality of the solution is somewhat limited.** This work raises a very important issue in the field of image generation - ``*generative hallucination*'', and classifies it into 4 categories and defines it well. However, the solution seems to be to introduce a lot of external components such as large models, diffusion models, knowledge graphs, object extraction models, relation extraction models, etc. to build a pipeline rather than an end-to-end solution. This leads to an important question: if one of the links, such as object extraction or relationship extraction, goes wrong, does it directly lead to model errors? Therefore, I think such a model is not innovative enough and is difficult to use in practice. Its reliability depends on the stability of each component and the adaptability of text prompts. For example, some complex scenarios (omitted subjects, professional terms, abbreviations, etc. are difficult to parse in a long textual prompt)

- **The details of the method are not clearly described.** In sections 3.2 to 3.5 of the method, the author focuses on "what to do" rather than "how to do it", so it is difficult to understand the details of the method, such as: Do the object extraction and relation extraction modules rely solely on a large language model?

- **Whether to support overlapping of hallucination categories.** I think generative hallucination is not a deterministic three-classification problem. It is very likely to have overlaps. For example, in the second example in Figure 1, the visual text presented is likely to need to ensure the consistency of object position, the correctness of scene text recognition, and the alignment with world knowledge (vocabulary or professional terms) at the same time. How would the model cope with it in this case?

- **Questions about the results in Table 1.** How is the TFH indicator calculated and what is its relationship with TH and FH? Why are the TH and FH metrics extremely low in other methods but very high in this work? Is this a fair comparison in the experimental setting and metric calculation? Are the TFHs of other methods all 0? Can the experiment be repeated and what about testing for statistical significance?

---

confidence: 4
contribution: 1
initial_rating: 3
presentation: 1
questions: 1. What exactly is "*Stable Diffusion v1.6*"? From what I know, *Stable Diffusion v1.5* [5] is the most widely used in the 1.x series. If v1.6 exists, could you provide its source or any documentation?

2. In Figure 3, the **second row's last column** shows results under the prompt "*A street sign with 'Main Street' written on it*." The output looks suspiciously like an image from the Hugging Face Diffusers documentation [6]. Can you clarify the validity of this comparison?

3. What about the time efficiency of the proposed method? Since this framework involves multiple stages (like calls to OpenAI API and online search engines), I assume the time cost could be higher. Could you provide a quantitative comparison with other methods?

4. In Table 1, you present quantitative results comparing layout-based methods and LLM-based methods, but the qualitative comparison doesn’t include these approaches. Why are these missing from the visual results?

5. Also in Table 1, there’s an odd repetition of numbers. For example, why do the results for *VPGen*, *Ranni*, and *LMD* seem identical? This repetition needs clarification.


**Reference**

[5] *https://huggingface.co/stable-diffusion-v1-5/stable-diffusion-v1-5*.

[6] *https://huggingface.co/docs/diffusers/en/using-diffusers/inpaint*.
soundness: 2
strengths: 1. The authors have summarized some of the key challenges in text-driven image generation, particularly around maintaining consistency in entities (Figure 1) and managing complex scene generation.
2. The proposed pipeline is relatively complete and utilizes LLM to refine entities and layouts, which helps improve text-consistent image generation.
summary: The paper presents a framework (PCIG) tackling the issue of text-to-image (T2I) generation inconsistencies. It highlights important challenges such as entity mismatches and handling complex scenes, and it proposes a pipeline that integrates large language models (LLMs), knowledge graphs and location prediction to improve overall generation quality.
weaknesses: I have some concerns about different aspects of the paper that I feel should be addressed:

1. **Framework Innovation**: Actually, what concerns me the most is that the proposed approach feels more like a combination of several established LLM strategies, such as “object extraction,” “relation extraction,” and “object localization,” rather than introducing novel research contributions. This makes it come across more as an industrial solution, which might not fit the innovation threshold for a conference like ICLR.

2. **Quantitative Results**: I find it difficult to be fully convinced by the quantitative results. The use of the existing *AnyText* [1] module to mitigate textual hallucination and factual hallucination (improved *TH Acc* and *FH acc*) seems to remove much of the novelty from your approach, as seen in Tables 1, 3.

3. **Misleading References**: 
In my view, the following statement is not rigorous (L50-L51). The mentioned works (*DDPM* [2] and *DDIM* [3]) weren't designed to impose extra conditions like text (an attention-based denoiser is indeed proposed in *DDPM*); rather, *LDM* [4] is what paved the way for capturing these text-driven image generation. This kind of statement may mislead the readers.
> *Attention-based approaches Ho et al. (2020); Song et al. (2020) have been proposed to better capture relationships between words and visual features,*

4. **Presentation Issues**: The citation style isn't consistent—using `\citep` instead of `\cite`. Additionally, there are quite a few grammatical errors and informal math expressions, making parts of the paper hard to follow.

5. **Inconsistent Terminology**: There’s a discrepancy with how LLM is referenced—in the abstract, it’s “*large language **module***,” but later, it’s called a “*large language **model***.” Consistency here would help avoid confusion and “*large language model*” is more formal.


**Reference**

[1] *Tuo, et al. "Anytext: Multilingual visual text generation and editing." arXiv (2023).*

[2] *Ho, et al. "Denoising diffusion probabilistic models." NeurIPS (2020)*.

[3] *Song, et al. "Denoising Diffusion Implicit Models." ICLR (2021)*.

[4] *Rombach, et al. "High-resolution image synthesis with latent diffusion models." CVPR (2022)*

---

confidence: 4
contribution: 2
initial_rating: 5
presentation: 3
questions: Please see weakness.
soundness: 3
strengths: The modular design of the framework shows potential for scalability and adaptability to more complex scenarios.

Consistency between the text prompt and the generated image is a well-recognized challenge in text-to-image generation. Solving this issue has significant implications for advancing practical applications of AI-generated content, making the PCIG framework highly valuable.

The paper presents extensive quantitative and qualitative experiments, as well as ablation studies, that clearly demonstrate PCIG’s prompt consistency.
summary: The paper introduces PCIG, a novel framework aimed at addressing inconsistencies in text-to-image generation. By integrating Large Language Models (LLMs), knowledge graphs, and controllable diffusion models, the proposed method enhances the alignment between generated images and their corresponding text prompts. The framework handles three key aspects of consistency: general objects, text within images, and objects that refer to proper nouns existing in the real world. Through extensive experimentation on the MHaluBench dataset, PCIG demonstrates significant improvements in both quantitive and qualtitive evaluations.
weaknesses: The primary limitation of this work lies in its novelty. PCIG appears to be a straightforward combination of pre-existing models and methods, offering little in terms of new insights or theoretical advancements. While the integration is effective, the framework lacks a significant innovative contribution that differentiates it from prior work.

The demo presented in Figure 2 still exhibits inconsistencies. For instance, the team name "Warriors" is misspelled as "VAARR," and Stephen Curry’s jersey number, which should be 30, is incorrect. These errors detract from the credibility of the framework’s performance in ensuring image-prompt consistency.

The experimental section lacks widely accepted image generation evaluation metrics, such as the Frechet Inception Distance (FID) and Inception Score (IS). Including these metrics would have provided a more comprehensive evaluation of the model's performance and allowed for better comparison with existing state-of-the-art methods.

Some experimental results, particularly in Table 3, raise concerns. For example, the OH ACC in the first column is unexpectedly higher when the text module is omitted. Moreover, the remove of KG extraction does not seem to improve TH or FH ACC, calling into question the contribution of these modules.