Image Generation from Contextually-Contradictory Prompts

## Abstract

Abstract. Text-to-image diffusion models excel at generating high-quality, diverse images from natural language prompts.
However, they often fail to produce semantically accurate results when the prompt contains concept combinations that contradict their learned priors.
We define this failure mode as contextual contradiction , where one concept implicitly negates another due to entangled associations learned during training.
To address this, we propose a stage-aware prompt decomposition framework that guides the denoising process using a sequence of proxy prompts.
Each proxy prompt is constructed to match the semantic content expected to emerge at a specific stage of denoising, while ensuring contextual coherence.
To construct these proxy prompts, we leverage a large language model (LLM) to analyze the target prompt, identify contradictions, and generate alternative expressions that preserve the original intent while resolving contextual conflicts.
By aligning prompt information with the denoising progression, our method enables fine-grained semantic control and accurate image generation in the presence of contextual contradictions. Experiments across a variety of challenging prompts show substantial improvements in alignment to the textual prompt.

## 1. Introduction

Text-to-image diffusion models have demonstrated remarkable capabilities in generating high-quality and diverse visual content from natural language prompts (Rombach et al., 2022; Ramesh et al., 2021; Ho et al., 2020). However, achieving precise semantic alignment between the generated image and the conditioning prompt remains an open challenge. This issue becomes particularly pronounced when the target prompt lies outside the model’s training distribution, such as prompts that combine semantically plausible yet statistically uncommon or unconventional concepts.
For example, as shown in Figure [1](https://arxiv.org/html/2506.01929v1#S0.F1), generating an image from the prompt “a rooster in a nest” often results in colorful birds sitting in a nest. This is due to the model’s strong prior entanglement between bird species and nests, which implicitly contradicts the notion of a rooster.

We refer to this phenomenon as Contextual Contradiction, a conflict between two concepts that arises not from direct semantic opposition, but from the model’s associations learned during training.
More precisely, we say that concept $A$ contextually contradicts concept $B$ if the model’s prior entangles $A$ with concept $C$, and $B$ contradicts $C$.
In Figure [1](https://arxiv.org/html/2506.01929v1#S0.F1), we illustrate this with a blowing dragon, which stands in contextual contradiction with the water due to its entanglement with fire.

The phenomenon of contextual contradiction in text-to-image models relates to the broader issue of spurious correlations in deep learning. Models often rely on shortcuts, which are predictive patterns in the training data that do not reflect true semantics (Geirhos et al., 2020). In NLP, models exploit syntactic heuristics rather than reasoning (McCoy et al., 2019), while in computer vision, they may depend on background cues rather than object features (Xiao et al., 2021; Singh et al., 2020). For example, classifiers may fail to recognize cows on beaches due to background biases (Beery et al., 2018).
In this paper, we identify similar biases in generative models, where contextual contradictions arise when prompts include plausible but uncommon concept combinations.

To address this issue, we build on the observation that the denoising process follows a coarse-to-fine progression, during which different semantic attributes (e.g., background, pose, shape, and texture) emerge at distinct stages (Chefer et al., 2023; Balaji et al., 2022; Patashnik et al., 2023).
Our key idea is to guide the model at each stage of denoising with the information most relevant to the type of content being formed at that point. To achieve this, we decompose the original prompt into a sequence of proxy prompts, each aligned with the attributes expected at a specific stage and designed to avoid contextual contradictions.

Ensuring that proxy prompts preserve the original intent while avoiding contextual contradictions requires a broad understanding of the real world. For example, it involves understanding that a bear is entangled with specific poses, such as walking on all fours or standing upright, which in turn, contradicts the handstand pose. Our method leverages a large language model (LLM) to analyze the target prompt, identify contextual contradictions, and construct suitable proxy prompts.

We demonstrate that, by using in-context examples and prompting the LLM to follow a reasoning process through a brief explanation, it can identify contextually contradictory concepts in a prompt and determine the appropriate stage of denoising at which each attribute should be introduced.
It can also suggest alternative, non-conflicting concepts that preserve the intended attributes and use them to construct stage-specific proxy prompts.
In doing so, the LLM effectively guides the model toward the intended meaning of the original prompt.

Through extensive experiments, we demonstrate the effectiveness of our method in generating images from contextually contradicting text prompts.
By introducing prompt information at targeted stages, our approach generates
precise combination of semantic attributes while avoiding undesired entanglement of unrelated attributes.
Compared to previous methods, our stage-dependent prompt decomposition leads to more faithful and semantically aligned generations.

## 2. Related Work

#### Learned Spurious Correlations

Machine learning models are known to be sensitive to spurious correlations in their training data (Geirhos et al., 2020; McCoy et al., 2019; Ye et al., 2024). This sensitivity often leads to a significant drop in performance when training-time associations between concepts do not hold at test time. Prior work has extensively studied this issue in discriminative computer vision tasks, showing, for example, that recognition models tend to rely heavily on background cues (Singh et al., 2020; Xiao et al., 2021; Beery et al., 2018).

In our work, we show that text-to-image models exhibit similar behavior.
When given contextually contradictory prompts – combinations of concepts that conflict with correlations seen during training – diffusion models often fail to generate images that accurately reflect the prompt. We evaluate this behavior using the Whoops! dataset (Bitton-Guetta et al., 2023), which contains prompts constructed by first describing two co-occurring elements, and then replacing one with a less compatible alternative. This results in scenarios that are unlikely to occur in the real world.

#### Semantic Alignment in Text-to-Image Synthesis

Text-to-image models often struggle to fully capture the semantic intent of input prompts,
particularly when prompts involve complex or internally conflicting concepts.
Previous works have analyzed common failure cases and proposed targeted improvements across various stages of the generation pipeline, including enhanced text embedding representations (Rassin et al., 2022; Feng et al., 2022; Tunanyan et al., 2023), refined attention mechanisms (Dahary et al., 2024) and guidance strategies that leverage attention maps for loss heuristics (Chefer et al., 2023; Rassin et al., 2023; Meral et al., 2024; Agarwal et al., 2023).
Despite these advances, existing methods often fail to handle prompts containing contradictory concepts arising from the model’s learned associations. Our work directly addresses this underexplored challenge, focusing on contextually contradictory prompts.

#### Multi-Prompt Conditioning Techniques

Conditioning diffusion models on multiple prompts has emerged as an effective strategy for improving control and compositionality.
One line of work, primarily focused on personalization, introduces distinct learned tokens at different layers of the model and at various denoising timesteps (Alaluf et al., 2023; Voynov et al., 2023). This design allows each token to capture different attributes of the personalized concept, leading to improved identity preservation.
Other approaches vary the prompt across timesteps to modulate specific visual properties, such as object shape (Liew et al., 2022; Patashnik et al., 2023), or alternate between rare and frequent object descriptions to improve attribute binding (Park et al., 2024).
Fine-grained spatial control has also been achieved by assigning sub-prompts to separate image regions (Yang et al., 2024).
Additionally, some methods leverage multiple diffusion models, each conditioned on different prompt attributes, and combine their outputs into a unified prediction (Liu et al., 2022; Bar-Tal et al., 2023).
Unlike most prior works,
we focus on utilizing multiple prompt to settle internal semantic tension, where concept combinations lead to contextual contradictions.

Figure: Figure 2. Our method navigates the denoising process using time-dependent proxy prompts. (Top) A large language model (LLM) is employed to decompose a target prompt with entangled or conflicting attributes into a sequence of proxy prompts and their corresponding time-step intervals. (Bottom) These intermediate prompts are injected into the diffusion process at the specified intervals. This time-dependent prompt conditioning aligns more closely with the model’s prior at each stage, enabling a semantically consistent and temporally coherent progression through the denoising process.
Refer to caption: x2.png

#### LLM-Guided Diffusion

LLMs have demonstrated strong capabilities in language understanding. Moreover, they possess a broad, implicit grasp of world knowledge, acquired through large-scale training on diverse textual data.
Recent approaches have leveraged these capabilities to guide diffusion model generation, often incorporating planning and reasoning to improve semantic alignment (Yang et al., 2024; Park et al., 2024; Hu et al., 2024).
In our work, we employ LLMs with in-context learning to identify contextual contradictions, generate proxy prompts, and determine the corresponding timestep ranges for conditioning, while promoting a reasoning process through brief, explanatory outputs.

## 3. Method

In this section, we begin by analyzing the coarse-to-fine behavior of the denoising process (Section [3.1](https://arxiv.org/html/2506.01929v1#S3.SS1)). Building on the insights from our analysis, we introduce our training-free framework for resolving contextual contradictions in text-to-image generation. As illustrated in Figure [2](https://arxiv.org/html/2506.01929v1#S2.F2), our approach consists of two main components:
(i) prompt decomposition (Section [3.2](https://arxiv.org/html/2506.01929v1#S3.SS2), top part of the figure), and
(ii) stage-aware prompt injection (Section [3.3](https://arxiv.org/html/2506.01929v1#S3.SS3), bottom part of the figure). In the following, we describe each of these components.

### 3.1. Coarse-to-Fine Denoising

As shown by previous works (Hertz et al., 2022; Balaji et al., 2022; Chefer et al., 2023; Patashnik et al., 2023), the denoising process of diffusion models operates in a coarse-to-fine manner: early steps determine low-frequency structures, while high-frequency details emerge in later steps.

This behavior is reflected in the training process, in which varying noise levels are applied across different timesteps, and the model is trained to reconstruct the original clean image from the noisy input.
High noise levels used at early timesteps yield low signal-to-noise ratios, restricting reconstruction to spatially smooth, low-frequency components, such as background or layout, which are recoverable even when the image is heavily corrupted.
At smaller timesteps, where the signal-to-noise ratio is higher, high-frequency features become more reliably reconstructible.

During inference, the noise is gradually removed as the model generates the image in a coarse-to-fine manner.
This coarse-to-fine nature of the image generation leads to two key observations:
(i) Irreversibility of details.
At each stage of denoising, the model commits to a certain level of structural detail. Once a given attribute (e.g., layout, coarse shape) has been established, it becomes effectively fixed and cannot be revised in later steps, even if it misaligns with the prompt.
(ii) Flexibility in high-frequency details.
In early stages, high-frequency details have not yet emerged and thus are unaffected by the prompt. This allows greater flexibility in guiding the model without impacting the final generated fine details.

We demonstrate these observations in Figure [3](https://arxiv.org/html/2506.01929v1#S3.F3), by examining the model’s $x_{0}$ predictions throughout the denoising process.
In the top row, the prompt is fixed to “A howling wolf and a flying bat at midday”. In the middle row, we use the proxy prompt “A howling dog and a flying bird at midday” during the early steps of denoising, and then switch to the same target prompt used in the top row. We replaced “wolf” with “dog” because the two animals are semantically similar and share similar low-frequency visual features. Moreover, “a howling dog” is not contextually associated with nighttime. For similar reasons, we replaced “bat” with “bird”.

The first row illustrates the irreversibility of details. Night colors and a large moon are established early in the denoising process, and although they do not match the prompt, the model is unable to correct them in later steps.

The second column shows the flexibility of high-frequency details.
Although different prompts were used for each row, the layout and object shapes are visually similar, reflecting the fact that distinguishing information between the target and proxy prompts – specifically regarding layout and object shape – has not yet emerged at this early stage of the denoising process.
Yet, since a howling dog, unlike a howling wolf, is not entangled with night, the background in the second row adheres more closely to the prompt.
By switching to the target prompt at this point in the process, we are still able to generate a wolf and a bat rather than a dog and a bird, while also aligning with the word “midday”.

The bottom row illustrates the importance of selecting an appropriate proxy prompt. Snakes and rats are associated with nighttime, which conflicts with the “midday” specification. In addition, their lack of visual similarity to the target objects leads to visual leakage between the bat and the wolf.

These observations motivate a stage-aware prompting strategy that enables precise control over early structural decisions, while allowing more flexible conditioning for attributes that are not yet generated.
Building on this, we now describe the two main components of our method: prompt decomposition and stage-aware prompt injection.

Figure: Figure 3. Coarse-to-fine denoising. $x_{0}$ predictions at the initial, intermediate, and final time steps. During the early denoising stages, where primarily low-frequency structures are generated, different prompts introducing distinct objects can produce visually similar layouts and objects. Once this structure is established, it becomes effectively frozen and cannot be modified in later steps. As shown in the bottom row, using incorrect placeholder objects in the proxy prompt can anchor erroneous visual information early on, leading to eventual misalignment to the target prompt.
Refer to caption: x3.png

### 3.2. Prompt Decomposition

Given a prompt $P$ that contains contextually contradictory concepts, we aim to construct a sequence of proxy prompts $\{p_{1},p_{2},...,p_{n}\}$ and corresponding timestep intervals $\{I_{1},I_{2},...,I_{n}\}$ that together reflect the intended semantics of $P$.
Each proxy prompt $p_{i}$ is designed to preserve the relevant semantics of $P$ for attributes or concepts that typically form during its assigned interval $I_{i}$. This is crucial due to the irreversibility of details in the generation process. At the same time, $p_{i}$ is chosen to avoid contradictory combinations that tend to emerge at that stage, leveraging the flexibility in high-frequency details.
This decomposition enables conditioning the diffusion model on contextually coherent content that evolves in tandem with the coarse-to-fine structure of the denoising process (see Figure [3](https://arxiv.org/html/2506.01929v1#S3.F3)).

To generate these proxy prompts and their associated intervals, we employ a large language model (LLM) with two complementary roles. First, we leverage the LLM’s broad world knowledge and reasoning capabilities to detect semantic contradictions and to propose contextually appropriate substitutes for conflicting concepts. Second, we use its instruction-following and in-context learning capabilities to infer the temporal staging of concepts, guided by examples and instructions that reflect the denoising behavior of diffusion models.

We implement this using a structured prompt template that includes both instructional text and representative input-output examples, along with textual explanations of the contextual contradictions in the original prompts. These examples illustrate successful decompositions that resolve contextual conflicts, as well as cases that require no decomposition, enabling the LLM to generalize from both scenarios. The full instruction prompt is provided in the appendix (see Table [4](https://arxiv.org/html/2506.01929v1#A1.T4)).

#### In-context Examples

Our in-context examples for guiding the LLM include the target prompt as input and the corresponding proxy prompts, each paired with its associated timestep interval, as output. Each example also includes a brief explanation describing the contextual contradiction in the prompt, which is part of the expected output. By requiring the LLM to produce this explanation, we encourage it to engage in a reasoning process, improving its ability to identify contradictions and generate coherent proxy prompts.
The examples cover a range of contradiction types, including semantic mismatches, spatial violations, behavioral implausibilities, and stylistic inconsistencies. We provide these examples in the appendix (see Table [5](https://arxiv.org/html/2506.01929v1#A1.T5)).

These examples were constructed by first identifying contextually-contradictory prompts that produce implausible generations in the base model (FLUX), particularly when concept conflicts appear early in the denoising process. Drawing on our coarse-to-fine analysis, we manually decomposed such prompts into proxy prompts sequences that stabilize low-frequency structure before gradually introducing high-frequency details. We used an LLM to generate natural language explanations and validate that each substitution preserve visual and contextual plausibility while resolving conflicts in a temporally coherent manner.

In total, we include 20 such examples, covering both effective decompositions and cases where minimal intervention was sufficient to preserve coherence. These examples guide the LLM in learning successful decomposition strategies, such as proxy substitution, temporal staging, and contradiction deferral.

#### Concept Substitution

A core mechanism of our prompt decomposition is concept substitution, which promotes contextual coherence by temporarily replacing a conflicting concept in the prompt with a structurally appropriate alternative (see Figure [3](https://arxiv.org/html/2506.01929v1#S3.F3)).
A simpler alternative for avoiding contextual contradictions is to remove the conflicting concept entirely. However, we observe that introducing an object only at a later stage, without a placeholder proxy, can affect the perceived size of that object (see Figure [4](https://arxiv.org/html/2506.01929v1#S3.F4)). In extreme cases, introducing the object too late may result in it being omitted from the generated image altogether. Therefore, we prefer the concept substitution strategy.

As demonstrated in Figure [3](https://arxiv.org/html/2506.01929v1#S3.F3), choosing the right substitution is key to the method’s success.
When the substituted concept is semantically appropriate and visually similar to the original object during the interval in which it is applied, the model correctly associates it with the original concept.
In contrast, poor substitutions – due to mismatches in scale, pose, location, or to inappropriate timing – can result in extra objects, layout errors, or failure to follow the prompt (see Figure [3](https://arxiv.org/html/2506.01929v1#S3.F3), bottom row, and Figure [5](https://arxiv.org/html/2506.01929v1#S3.F5)).

Figure: Figure 4. The effect of the object removal strategy. Each image is generated from a prompt that initially describes only the background (“a white sandy beach,” “a desert”). At the timestep indicated below each column, the foreground object (“a monkey,” “a snowman”) is added to the prompt. Removing the object from the initial prompt, without using a placeholder, significantly affects its apparent scale. Early insertion allows the model to allocate more spatial area, while late insertion confines the object to limited regions within the established layout.
Refer to caption: extracted/6503085/images/step_size_dim/monkey_on_the_beach/step1.jpg

### 3.3. Stage-aware Prompt Injection

Given a sequence of proxy prompts ${p_{1},p_{2},...,p_{n}}$, their corresponding timestep intervals ${I_{1},I_{2},...,I_{n}}$, and a text-to-image (T2I) diffusion model, we condition the model using different prompts throughout the denoising process. At each timestep $t$, we apply the prompt $p_{i}$ such that $t\in I_{i}$. By aligning each proxy prompt with a specific timestep interval, the denoising process is guided at each stage by concepts relevant to the level of detail emerging at that point. This enables the gradual construction of the image while avoiding conflicts with the model’s learned priors.

This injection mechanism integrates seamlessly into existing inference pipelines without any architectural modifications, and is compatible with a range of pretrained diffusion models. Differences in denoising schedules across models are handled entirely during prompt decomposition (Section [3.2](https://arxiv.org/html/2506.01929v1#S3.SS2)). We show examples with SD3 in the appendix (see Figure [11](https://arxiv.org/html/2506.01929v1#A1.F11)).

Figure: Figure 5. Impact of proxy object timing on generated image attributes. Changing the target object too early fails to disentangle the desired attribute, while modifying it too late prevents effective binding. Attribute control is most successful when applied within the correct time interval. In the top row, the initial proxy prompt is “A pillow fort in a bedroom”, while in the bottom row, the proxy prompt is “A mother duck guards three ducklings”.
Refer to caption: extracted/6503085/images/step_proxy/tent/static.jpg

## 4. Experiments and Results

In this section, we conduct qualitative (Section [4.1](https://arxiv.org/html/2506.01929v1#S4.SS1)) and quantitative (Section [4.2](https://arxiv.org/html/2506.01929v1#S4.SS2)) comparisons to assess the effectiveness of our method, along with ablation study (Section [4.3](https://arxiv.org/html/2506.01929v1#S4.SS3)) to evaluate the contribution of individual components. We also present some limitations of our method (Section [4.4](https://arxiv.org/html/2506.01929v1#S4.SS4)).

#### Implementation Details.

We use the state-of-the-art text-to-image model FLUX.1 [dev] (Labs, 2024) and GPT-4o as LLM to demonstrate the effectiveness of our approach. Inference is performed using $T=50$ sampling steps. In all experiments, we guide the LLM to limit the number of proxy prompts generated during decomposition to a maximum of three per input prompt.
We use the baselines hyperparameters as appeared in their original papers or implementation. To ensure a fair comparison, we also use $T=50$ sampling steps for all baseline methods.

#### Baselines

Since our method builds upon FLUX.1 [dev], we directly compare our results against this base model to clearly demonstrate the improvements achieved by our approach. Additionally, we include a comparison to the training-free method R2F (Park et al., 2024), which similarly employs a large language model (LLM) to alternate between different prompts during generation. As R2F is implemented based on SD3 (Esser et al., 2024), we also compare our results to this model. Lastly, we compare against the fine-tuned model Ella (Hu et al., 2024), which is currently publicly available with SD 1.5 as its base model.

#### Datasets

We evaluate our method using three datasets: Whoops! (Bitton-Guetta et al., 2023), Whoops-Hard, and ContraBench. Whoops! consists of 500 prompts paired with commonsense-defying images, designed to test visual reasoning and compositional understanding. While relevant to our task, many of its prompts are relatively easy for modern T2I models and fail to consistently reveal their limitations. To address this, we curate a more challenging dataset Whoops-Hard, a subset comprising 100 particularly difficult prompts from Whoops! (Bitton-Guetta et al., 2023). This challenging benchmark provides a stronger basis for evaluating text-to-image generation given the current performance of state-of-the-art methods. Furthermore, to test more challenging cases of semantic alignment, we introduce a new benchmark ContraBench, a curated set of 40 prompts crafted to induce strong contextual contradictions, including background-object mismatches, semantic tension, and scale conflicts. ContraBench serves as a focused, high-difficulty benchmark for evaluating text-image alignment on prompts that contain contextual contradictions. We will make both ContraBench and the Whoops-Hard subset publicly available.

Figure: Figure 6. Our method effectively resolves contextual contradictions across diverse forms, including inconsistent backgrounds, unnatural color assignments, and implausible actions. Compared to prior approaches, our results exhibit higher semantic coherence and more accurate prompt grounding under contradictory prompts.
Refer to caption: extracted/6503085/images/comparison/monkey_juggles/ella_Seed93198.jpg

### 4.1. Qualitative Results

#### Comparisons with baselines.

Figures [6](https://arxiv.org/html/2506.01929v1#S4.F6) and [9](https://arxiv.org/html/2506.01929v1#S5.F9) present qualitative comparisons on the Whoops! and ContraBench datasets. Across both benchmarks, the baseline methods FLUX, SD3, R2F, and Ella consistently exhibit characteristic failure modes when handling prompts with contextually contradictory concepts. For instance, as shown in Figure [6](https://arxiv.org/html/2506.01929v1#S4.F6), our method is the only one to successfully generate scenarios such as a blue Shrek or a monkey juggling tiny elephants.

For SD3 and FLUX, these prompts conflict with their learned priors, leading to failures in generating text-aligned images.
Ella was originally designed for long and complex prompts, not specifically for those containing contextual contradictions, and therefore struggles with challenging contradiction cases.
The training-free baseline R2F alternates between prompts at predefined timesteps, a strategy originally designed for attribute binding rather than addressing contextual contradictions. While it can reinforce rare concepts, this approach does not align prompt content with the stages at which semantic features emerge during denoising. As a result, it often produces hybrid concepts that merge incompatible elements from conflicting objects (see the owl and SpongeBob in Figure [9](https://arxiv.org/html/2506.01929v1#S5.F9)).

In contrast, our method produces semantically coherent outputs by introducing proxy prompts at denoising stages where the corresponding visual features are expected to emerge. This alignment enables the model to effectively handle conflicting concepts. Across both Whoops! and ContraBench, our approach consistently generates contextually contradictory elements with high accuracy.

To demonstrate the robustness of our approach, we provide a comparison across several random seeds in Figure [10](https://arxiv.org/html/2506.01929v1#S5.F10). The same seeds are used for all methods, showing that our method consistently outperforms the baselines.

#### Improve realism

Figure [7](https://arxiv.org/html/2506.01929v1#S4.F7) demonstrates how our method generates photorealistic and semantically coherent images for prompts with atypical attribute combinations. In contrast, FLUX often defaults to cartoon-like renderings, likely due to priors associating fantastical elements with imaginative visual styles. By using contradiction-free proxy prompts, our method enables the model to produce realistic outputs without explicitly requiring photorealism in the prompt.

Figure: Figure 7. FLUX tends to generate realistic images by default. However, when given unrealistic prompts, it often produces cartoon-like samples. In contrast, our method, which gradually resolves such prompts through coherent proxy stages, consistently generates realistic and semantically aligned images.
Refer to caption: extracted/6503085/images/cartoon/A_kangaroo_delivering_mail_on_a_bicycle/flux/Seed28144.jpg

### 4.2. Quantitative Results

Our method focuses on solving alignment with challenging and contradictory textual prompts. Therefore, we evaluate its performance by measuring prompt alignment. To do so, we use the GPT-4o vision-language model (VLM). The model is prompted to assign a score from 1 to 5 for each generated image, based on how well the image aligns with the given textual prompt. The exact evaluation prompt is provided in the appendix (See Table [6](https://arxiv.org/html/2506.01929v1#A1.T6)). For each prompt, we generate outputs using three fixed random seeds, which are applied consistently across all evaluated methods. The average score across the three seeds is then scaled to a range between 20 and 100 by multiplying by 20.

As can be seen in Table [1](https://arxiv.org/html/2506.01929v1#S4.T1), our method outperforms the baselines across all three benchmarks. Notably, using SD3 as the base model yields better prompt alignment than the state-of-the-art FLUX model under contradictory prompt settings, while FLUX demonstrates superior visual quality (See Figure [9](https://arxiv.org/html/2506.01929v1#S5.F9)). In contrast, our method, built on top of FLUX, achieves the highest prompt adherence while preserving its strong visual quality. This results in the best overall performance, combining high-fidelity generation with robust alignment to challenging prompts.

**Table 1. Quantitative evaluation on the Whoops dataset using the GPT-4o vision-language model. We report average alignment, where alignment reflects how well the image matches the prompt semantics, without looking at the quality of the image**
|  | Benchmarks |  |  |
| --- | --- | --- | --- |
| Models | Whoops | Whoops-Hard | ContraBench |
| Ella | 69.09 | 45.19 | 55.16 |
| SD3.0 | 82.63 | 55.73 | 57.5 |
| R2F (SD3.0) | 83.50 | 57.06 | 59.16 |
| FLUX | 78.85 | 44.3 | 57.16 |
| Ours (FLUX) | 85.10 | 62.13 | 66.16 |

**Table 2. User study results. Win rates of our method in text-image alignment and image quality, compared against each baseline method.**
|  | Ella | SD3.0 | R2F (SD3.0) | FLUX |
| --- | --- | --- | --- | --- |
| Alignment | $81\%$ | $70\%$ | $75\%$ | $81\%$ |
| Quality | $74\%$ | $72\%$ | $68\%$ | $63\%$ |

#### User study

VLM-based automatic metrics, such as those shown in Table [1](https://arxiv.org/html/2506.01929v1#S4.T1), often fail to detect subtle semantic inconsistencies, particularly when evaluating generations conditioned on complex or contradictory prompts. Moreover, these models do not adequately assess the visual quality of the generated images. To address these limitations, we conducted a user study aimed at evaluating both prompt adherence and visual quality from a human perspective. We randomly sampled 24 prompts from the Whoops! and ContraBench benchmarks. For each prompt, participants were shown two images: one generated by our method and the other by one of the baselines. They were asked to answer two questions: (1) which image most accurately reflected the prompt, and (2) which image had better visual quality.

In total, we collected responses from 61 users, resulting in 1,464 individual evaluations. The results of the user study are summarized in Table [2](https://arxiv.org/html/2506.01929v1#S4.T2), which reports the win rate of our method when compared pairwise against each baseline. These results highlight the superiority of our approach in handling contextually contradictory prompts, achieving both better prompt alignment and visual quality compared to baseline approaches.

Figure: Figure 8. Failure cases from Whoops!.
Refer to caption: extracted/6503085/images/failures/vase_Seed44885.jpg

### 4.3. Ablation Studies

To evaluate the contribution of each component in our method, we perform an ablation study on the Whoops-Hard benchmark by selectively removing or modifying individual elements of the method, and present the results in Table [3](https://arxiv.org/html/2506.01929v1#S4.T3).
Each configuration isolates a specific design choice to quantify its effect on alignment performance.
We observe that using in-context examples significantly improves the LLM’s ability to decompose and interpret contextually contradictory prompts, resulting in better text-image alignment. Removing the explanation requirement from the LLM impairs its reasoning capabilities and leads to a notable performance drop, suggesting that requiring the LLM to generate explicit explanations helps it make more coherent semantic decisions, ultimately improving prompt decomposition. Additionally, using only two proxy prompts, underperforms compared to the full method, highlighting the importance of introducing prompt information at targeted stages for robust alignment.

### 4.4. Limitations

In Figure [8](https://arxiv.org/html/2506.01929v1#S4.F8), we present three failure cases of our method on examples from the Whoops! benchmark. Since our approach relies on guiding the model through text alone, it cannot control properties that the underlying model inherently struggles with—such as generating specific quantities or enforcing precise orientations.

**Table 3. Ablation study on the Whoops-Hard Benchmark. We evaluate the impact of key components of our prompt decomposition method by: (1) removing in-context examples, (2) removing the explanation requirement, and (3) restricting the decomposition to at most two proxy prompts. Results show a significant drop in alignment when any of these components are removed.**
|  | static | w/o<br>in-context | w/o<br>reasoning | 2 proxy | Full |
| --- | --- | --- | --- | --- | --- |
| Alignment | 44.3 | 48.0 | 46.46 | 50.73 | 62.13 |

## 5. Conclusions

We have introduced a training-free framework for resolving contextual contradictions in text-to-image generation – that is, cases where seemingly plausible prompts are internally inconsistent because of strong, hidden biases in the model. Our method identifies and strategically separates such contradictions over time, enabling faithful rendering of prompts that would otherwise yield semantically inconsistent outputs.

At its core, our approach leverages the broad real-world understanding embedded in large language models to reason about the semantic plausibility of prompts. Rather than modifying the training data or fine-tuning the model, we harness this reasoning capacity at inference time to decompose prompts into a sequence of contextually coherent stages. These proxy prompts guide the generative process in alignment with the model’s prior expectations, helping it circumvent conflicts and maintain semantic fidelity.

Extensive experiments demonstrate that our stage-aware decomposition substantially improves alignment and image quality – especially for prompts that challenge the model’s default associations.
Future directions include refining our method using VLM-generated responses to improve proxy prompts generation dynamically during the generation process, and applying our method to text-to-video generation models.

## References

- (1)
- Agarwal et al. (2023)
Aishwarya Agarwal, Srikrishna Karanam, KJ Joseph, Apoorv Saxena, Koustava Goswami, and Balaji Vasan Srinivasan. 2023.
A-star: Test-time attention segregation and retention for text-to-image synthesis. In *Proceedings of the IEEE/CVF International Conference on Computer Vision*. 2283–2293.
- Alaluf et al. (2023)
Yuval Alaluf, Elad Richardson, Gal Metzer, and Daniel Cohen-Or. 2023.
A neural space-time representation for text-to-image personalization.
*ACM Transactions on Graphics (TOG)* 42, 6 (2023), 1–10.
- Balaji et al. (2022)
Yogesh Balaji, Seungjun Nah, Xun Huang, Arash Vahdat, Jiaming Song, Qinsheng Zhang, Karsten Kreis, Miika Aittala, Timo Aila, Samuli Laine, et al. 2022.
ediff-i: Text-to-image diffusion models with an ensemble of expert denoisers.
*arXiv preprint arXiv:2211.01324* (2022).
- Bar-Tal et al. (2023)
Omer Bar-Tal, Lior Yariv, Yaron Lipman, and Tali Dekel. 2023.
Multidiffusion: Fusing diffusion paths for controlled image generation.
(2023).
- Beery et al. (2018)
Sara Beery, Grant Van Horn, and Pietro Perona. 2018.
*Recognition in Terra Incognita*.
Springer International Publishing, 472–489.
[https://doi.org/10.1007/978-3-030-01270-0_28](https://doi.org/10.1007/978-3-030-01270-0_28)
- Bitton-Guetta et al. (2023)
Nitzan Bitton-Guetta, Yonatan Bitton, Jack Hessel, Ludwig Schmidt, Yuval Elovici, Gabriel Stanovsky, and Roy Schwartz. 2023.
Breaking Common Sense: WHOOPS! A Vision-and-Language Benchmark of Synthetic and Compositional Images. In *2023 IEEE/CVF International Conference on Computer Vision (ICCV)*. IEEE, 2616–2627.
[https://doi.org/10.1109/iccv51070.2023.00247](https://doi.org/10.1109/iccv51070.2023.00247)
- Chefer et al. (2023)
Hila Chefer, Yuval Alaluf, Yael Vinker, Lior Wolf, and Daniel Cohen-Or. 2023.
Attend-and-excite: Attention-based semantic guidance for text-to-image diffusion models.
*ACM transactions on Graphics (TOG)* 42, 4 (2023), 1–10.
- Dahary et al. (2024)
Omer Dahary, Or Patashnik, Kfir Aberman, and Daniel Cohen-Or. 2024.
Be yourself: Bounded attention for multi-subject text-to-image generation. In *European Conference on Computer Vision*. Springer, 432–448.
- Esser et al. (2024)
Patrick Esser, Sumith Kulal, Andreas Blattmann, Rahim Entezari, Jonas Müller, Harry Saini, Yam Levi, Dominik Lorenz, Axel Sauer, Frederic Boesel, et al. 2024.
Scaling rectified flow transformers for high-resolution image synthesis. In *Forty-first international conference on machine learning*.
- Feng et al. (2022)
Weixi Feng, Xuehai He, Tsu-Jui Fu, Varun Jampani, Arjun Akula, Pradyumna Narayana, Sugato Basu, Xin Eric Wang, and William Yang Wang. 2022.
Training-free structured diffusion guidance for compositional text-to-image synthesis.
*arXiv preprint arXiv:2212.05032* (2022).
- Geirhos et al. (2020)
Robert Geirhos, Jörn-Henrik Jacobsen, Claudio Michaelis, Richard Zemel, Wieland Brendel, Matthias Bethge, and Felix A. Wichmann. 2020.
Shortcut learning in deep neural networks.
*Nature Machine Intelligence* 2, 11 (Nov. 2020), 665–673.
[https://doi.org/10.1038/s42256-020-00257-z](https://doi.org/10.1038/s42256-020-00257-z)
- Hertz et al. (2022)
Amir Hertz, Ron Mokady, Jay Tenenbaum, Kfir Aberman, Yael Pritch, and Daniel Cohen-Or. 2022.
Prompt-to-prompt image editing with cross attention control.
*arXiv preprint arXiv:2208.01626* (2022).
- Ho et al. (2020)
Jonathan Ho, Ajay Jain, and Pieter Abbeel. 2020.
Denoising Diffusion Probabilistic Models.
*arXiv preprint arxiv:2006.11239* (2020).
- Hu et al. (2024)
Xiwei Hu, Rui Wang, Yixiao Fang, Bin Fu, Pei Cheng, and Gang Yu. 2024.
Ella: Equip diffusion models with llm for enhanced semantic alignment.
*arXiv preprint arXiv:2403.05135* (2024).
- Labs (2024)
Black Forest Labs. 2024.
FLUX.
[https://github.com/black-forest-labs/flux](https://github.com/black-forest-labs/flux).
- Liew et al. (2022)
Jun Hao Liew, Hanshu Yan, Daquan Zhou, and Jiashi Feng. 2022.
Magicmix: Semantic mixing with diffusion models.
*arXiv preprint arXiv:2210.16056* (2022).
- Liu et al. (2022)
Nan Liu, Shuang Li, Yilun Du, Antonio Torralba, and Joshua B Tenenbaum. 2022.
Compositional visual generation with composable diffusion models. In *European Conference on Computer Vision*. Springer, 423–439.
- McCoy et al. (2019)
Tom McCoy, Ellie Pavlick, and Tal Linzen. 2019.
Right for the Wrong Reasons: Diagnosing Syntactic Heuristics in Natural Language Inference. In *Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics*. Association for Computational Linguistics.
[https://doi.org/10.18653/v1/p19-1334](https://doi.org/10.18653/v1/p19-1334)
- Meral et al. (2024)
Tuna Han Salih Meral, Enis Simsar, Federico Tombari, and Pinar Yanardag. 2024.
Conform: Contrast is all you need for high-fidelity text-to-image diffusion models. In *Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition*. 9005–9014.
- Park et al. (2024)
Dongmin Park, Sebin Kim, Taehong Moon, Minkyu Kim, Kangwook Lee, and Jaewoong Cho. 2024.
Rare-to-Frequent: Unlocking Compositional Generation Power of Diffusion Models on Rare Concepts with LLM Guidance.
*arXiv preprint arXiv:2410.22376* (2024).
- Patashnik et al. (2023)
Or Patashnik, Daniel Garibi, Idan Azuri, Hadar Averbuch-Elor, and Daniel Cohen-Or. 2023.
Localizing object-level shape variations with text-to-image diffusion models. In *Proceedings of the IEEE/CVF international conference on computer vision*. 23051–23061.
- Ramesh et al. (2021)
Aditya Ramesh, Mikhail Pavlov, Gabriel Goh, Scott Gray, Chelsea Voss, Alec Radford, Mark Chen, and Ilya Sutskever. 2021.
Zero-Shot Text-to-Image Generation.
arXiv:2102.12092 [cs.CV]
[https://arxiv.org/abs/2102.12092](https://arxiv.org/abs/2102.12092)
- Rassin et al. (2023)
Royi Rassin, Eran Hirsch, Daniel Glickman, Shauli Ravfogel, Yoav Goldberg, and Gal Chechik. 2023.
Linguistic binding in diffusion models: Enhancing attribute correspondence through attention map alignment.
*Advances in Neural Information Processing Systems* 36 (2023), 3536–3559.
- Rassin et al. (2022)
Royi Rassin, Shauli Ravfogel, and Yoav Goldberg. 2022.
DALLE-2 is seeing double: Flaws in word-to-concept mapping in Text2Image models.
*arXiv preprint arXiv:2210.10606* (2022).
- Rombach et al. (2022)
Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Björn Ommer. 2022.
High-resolution image synthesis with latent diffusion models. In *Proceedings of the IEEE/CVF conference on computer vision and pattern recognition*. 10684–10695.
- Singh et al. (2020)
Krishna Kumar Singh, Dhruv Mahajan, Kristen Grauman, Yong Jae Lee, Matt Feiszli, and Deepti Ghadiyaram. 2020.
Don’t Judge an Object by Its Context: Learning to Overcome Contextual Bias. In *2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)*. IEEE.
[https://doi.org/10.1109/cvpr42600.2020.01108](https://doi.org/10.1109/cvpr42600.2020.01108)
- Tunanyan et al. (2023)
Hazarapet Tunanyan, Dejia Xu, Shant Navasardyan, Zhangyang Wang, and Humphrey Shi. 2023.
Multi-concept T2I-Zero: Tweaking only the text embeddings and nothing else.
*arXiv preprint arXiv:2310.07419* (2023).
- Voynov et al. (2023)
Andrey Voynov, Qinghao Chu, Daniel Cohen-Or, and Kfir Aberman. 2023.
p+: Extended textual conditioning in text-to-image generation.
*arXiv preprint arXiv:2303.09522* (2023).
- Xiao et al. (2021)
Kai Yuanqing Xiao, Logan Engstrom, Andrew Ilyas, and Aleksander Madry. 2021.
Noise or Signal: The Role of Image Backgrounds in Object Recognition. In *International Conference on Learning Representations*.
[https://openreview.net/forum?id=gl3D-xY7wLq](https://openreview.net/forum?id=gl3D-xY7wLq)
- Yang et al. (2024)
Ling Yang, Zhaochen Yu, Chenlin Meng, Minkai Xu, Stefano Ermon, and Bin Cui. 2024.
Mastering text-to-image diffusion: Recaptioning, planning, and generating with multimodal llms. In *Forty-first International Conference on Machine Learning*.
- Ye et al. (2024)
Wenqian Ye, Guangtao Zheng, Xu Cao, Yunsheng Ma, and Aidong Zhang. 2024.
Spurious correlations in machine learning: A survey.
*arXiv preprint arXiv:2402.12715* (2024).

## Appendix A Appendix

Table: Table 4. Full LLM prompt instruction for time step dependent prompt conditioning, used to decompose prompts by denoising stage.

<System Prompt>

You are an expert assistant in time step dependent prompt conditioning for diffusion models.

Your task is to decompose a complex or contextually contradictory prompt into up to three intermediate prompts that align with the model’s denoising stages — from background layout to object identity to fine detail.
Only introduce prompt transitions when needed.

Diffusion Semantics (Low $\rightarrow$ High Frequency Progression):

Steps 0–2: Scene layout and dominant color regions (e.g., sky, forest, sand tone)

Steps 3–6: Object shape, size, pose, and position

Steps 7–10: Object identity, material, and surface type (e.g., glass vs. rubber)

Steps 11–13+: Fine features and local details (e.g., tattoos, insects, facial detail)

Since denoising progresses from coarse to fine, it is crucial to stabilize large-scale visual structures (such as body shape, pose, and background) before introducing small or semantically charged elements (such as facial details, objects in hand, or surreal features).

Substitution Strategy:

1. Begin with high-level layout (background, geometry).
2. Use placeholder concepts if needed to stabilize layout before detailed insertions.
3. Substitutes must match in shape, size, and visual function.
4. Replace placeholders as soon as fidelity permits.
5. Do not maintain substitutions longer than needed.
6. If the prompt is visually coherent, return a single prompt with no decomposition.

Output Format:

a. Explanation: A short sentence explaining why decomposition is needed.

b. Final dictionary:

- The length of switch_ prompts_steps should be one less than prompts_list.
- Do not include any text outside this structure.

### A.1. LLM Instruction for Prompt Decomposition

Tables [4](https://arxiv.org/html/2506.01929v1#A1.T4) and [5](https://arxiv.org/html/2506.01929v1#A1.T5) detail the full LLM instruction used for our method’s decomposition, along with the corresponding in-context examples. In a single inference pass, our method detects contextual contradictions, generates proxy prompts, and assigns timestep intervals.

Table [8](https://arxiv.org/html/2506.01929v1#A1.T8) presents examples of our LLM input prompts, along with the corresponding output explanations and the decomposition into proxy prompts and timestep intervals.

Table: Table 5. In-context examples for time step dependent prompt conditioning, illustrating reasoning steps and prompt decomposition across denoising stages.

Input: A polar bear in a desert

Output:
a. Explanation: A polar bear is common in snowy scenes, not deserts. Since no suitable object proxy exists, the prompt starts with the desert alone before introducing the unlikely animal.
b. Final dictionary:

Input: A fox in a nursery

Output:
a. Explanation: A fox is uncommon in indoor scenes. Starting with a dog, then a visually similar breed (Shiba Inu), provides a natural proxy before introducing the fox in a childlike setting.
b. Final dictionary:

Input: A shoes tree in the meadow

Output:
a. Explanation: Treating a “shoes tree” like an “apple tree” allows the model to build structure and object layout using familiar patterns. Introducing apples first grounds the scene in a biologically plausible layout before shifting to the surreal concept of shoes as fruit-like objects.
b. Final dictionary:

Input: A red sports car parked on a mountain road

Output:
a. Explanation: This is a realistic and visually coherent scene, so no decomposition is needed.
b. Final dictionary:

### A.2. VLM evaluation

We evaluate text-to-image alignment for the different models on the benchmarks using the VLM GPT-4o. The instruction prompt used for the VLM is presented in Table [6](https://arxiv.org/html/2506.01929v1#A1.T6).

### A.3. ContraBench

To evaluate image generation under prompts containing contextual contradictions, we introduce ContraBench, presented in Table [7](https://arxiv.org/html/2506.01929v1#A1.T7). This dataset consists of 40 prompts generated using ChatGPT, each intentionally constructed to include visual or semantic inconsistencies challenge standard diffusion models. The prompts span a range of contradiction types, including physical implausibility, role violation, object misplacement, and surreal transformations. ContraBench serves as a targeted benchmark for assessing a model’s ability to produce visually coherent and semantically plausible images in the presence of contradictory concepts.

### A.4. Method Generalization to Different Models

Our method’s proxy prompt injection mechanism integrates seamlessly into existing inference pipelines without requiring any architectural modifications, and it is compatible with a range of pretrained diffusion models. Differences in denoising schedules across models are handled entirely during the prompt decomposition stage. See Figure [11](https://arxiv.org/html/2506.01929v1#A1.F11) for an example of how our method integrates with SD3 without any modifications.

Table: Table 6. VLM instruction for prompt alignment. Instructions provided to the VLM for scoring semantic alignment of generated images.

You are an assistant evaluating an image on how well it aligns with the meaning of a given text prompt.

The text prompt is: "{prompt}"

PROMPT ALIGNMENT (Semantic Fidelity)
Evaluate only the meaning conveyed by the image — ignore visual artifacts.
Focus on:

- •
Are the correct objects present and depicted in a way that clearly demonstrates their intended roles and actions from the prompt?
- •
Does the scene illustrate the intended situation or use-case in a concrete and functional way, rather than through symbolic, metaphorical, or hybrid representation?
- •
If the described usage or interaction is missing or unclear, alignment should be penalized.
- •
Focus strictly on the presence, roles, and relationships of the described elements — not on rendering quality.

Score from 1 to 5:

- •
5: Fully conveys the prompt’s meaning with correct elements
- •
4: Mostly accurate — main elements are correct, with minor conceptual or contextual issues
- •
3: Main subjects are present but important attributes or actions are missing or wrong
- •
2: Some relevant components are present, but key elements or intent are significantly misrepresented
- •
1: Does not reflect the prompt at all

Respond using this format:

> ### ALIGNMENT SCORE: <score> ### ALIGNMENT EXPLANATION: <explanation>

Figure: Figure 11. Generated images from our method applied on SD3.
Refer to caption: extracted/6503085/sup_dir/sd3_images/pacifier/sd3_Seed91661.jpg

Table: Table 7. ContraBench prompt set.

| ID | Prompt |
| --- | --- |
| 1 | A professional boxer does a split |
| 2 | A bear performing a handstand in the park |
| 3 | A photorealistic photo of SpongeBob SquarePants dancing ballet |
| 4 | A cowboy swimming competitively in an Olympic pool |
| 5 | A cruise ship parked in a bathtub |
| 6 | A man giving a piggyback ride to an elephant |
| 7 | A zebra climbing a tree |
| 8 | A coffee machine dispensing glitter |
| 9 | A vending machine in a human running posture |
| 10 | A ballerina aggressively flipping a table |
| 11 | A bathtub floating above a desert in a tornado |
| 12 | A monkey juggles tiny elephants |
| 13 | a woman has a marine haircut |
| 14 | A tower with two hands |
| 15 | An archer is shooting flowers with a bow |
| 16 | A baseball player backswing a yellow ball with a golf club |
| 17 | A barn built atop a skyscraper rooftop |
| 18 | A cat balancing a skyscraper on its nose |
| 19 | A cow grazing on a city rooftop |
| 20 | A fireplace burning inside an igloo |
| 21 | A mosquito pulling a royal carriage through Times Square |
| 22 | A grandma is ice skating on the roof |
| 23 | A muscular ferret in the woods |
| 24 | A house with a circular door |
| 25 | A photorealistic image of a bear ironing clothes in a laundry room |
| 26 | A pizza being used as an umbrella in the rain |
| 27 | A cubist lion hiding in a photorealistic jungle |
| 28 | A chicken is smiling |
| 29 | A realistic photo of an elephant wearing slippers |
| 30 | A computer mouse eating a piece of cheese |
| 31 | A horse taking a selfie with a smartphone |
| 32 | A sheep practicing yoga on a mat |
| 33 | a snake eating a small golden guitar |
| 34 | A soccer field painted on a grain of rice |
| 35 | A snake with feet |
| 36 | A woman brushing her teeth with a paintbrush |
| 37 | A horse with a hump |
| 38 | A hyperrealistic unicorn made of origami |
| 39 | A library printed on a butterfly’s wings |
| 40 | A bodybuilder balancing on pointe shoes |

**Table 8. Examples of prompt decomposition for contradictory prompts using our method.**
| Original Prompt | Explanation | Prompt Sequence | Switch Step(s) |
| --- | --- | --- | --- |
| A grown man has a baby’s pacifier in his mouth. | The grown man is established first for structure, while the pacifier, being a small but semantically significant detail, is added once the main identity is resolved. | (1)<br>A grown man with a small object in his mouth<br>(2)<br>A grown man has a baby’s pacifier in his mouth | 4 |
| A dragon is blowing water. | Dragons are more commonly depicted blowing fire. A proxy of white smoke is visually similar to water mist in texture, stabilizing the emission process before resolving the surreal water emission. | (1)<br>A dragon blowing white smoke<br>(2)<br>A dragon blowing water | 3 |
| A pizza with grape toppings. | Pizza with traditional toppings stabilizes the geometry and color before introducing the visually similar yet unusual grape topping. | (1)<br>A pizza with pepperoni toppings<br>(2)<br>A pizza with grape toppings | 3 |
| A coin floats on the surface of the water. | Coins typically sink in water, not float. Starting with a leaf—an object that naturally floats—ensures that this behavior within the scene is handled correctly before introducing the coin. | (1)<br>A leaf floats on the surface of the water<br>(2)<br>A coin floats on the surface of the water | 4 |
| A cockatoo parrot swimming in the ocean. | Cockatoos are birds and naturally do not swim; starting with a simple bird on water stabilizes position and motion. Progressing to a duck, before introducing the cockatoo parrot, eases the transition into the final surreal visual. | (1)<br>A duck swimming in the ocean<br>(2)<br>A parrot swimming in the ocean<br>(3)<br>A cockatoo parrot swimming in the ocean | 3, 6 |
| A woman writing with a dart. | Writing is typically associated with a pen or pencil. Starting with a pen stabilizes the action before introducing the unusual tool. | (1)<br>A woman writing with a pen<br>(2)<br>A woman writing with a dart | 3 |
| Shrek is blue. | Shrek is a distinct character with a recognizable green color. Using a simple b̈lue ogreïnitially sets the stage for a color change before fully introducing Shrek to ensure visual coherence. | (1)<br>A blue ogre<br>(2)<br>Shrek is blue | 3 |

## Peer Reviews

confidence: 3
contribution: 3
initial_rating: 6
presentation: 3
questions: How does the method perform with less capable, open-source LLMs (like LLaMA)? The paper briefly tests this, but is the performance drop acceptable for practical use?

Could this framework be extended beyond "contextual" contradictions to handle more direct logical or physical impossibilities (e.g., "a square apple")?

The LLM and diffusion model are separate. Have the authors considered a more tightly integrated design, where a VLM's understanding could directly influence the denoising process in real-time?
soundness: 3
strengths: The biggest strength is the elegant simplicity of the approach. It doesn't require retraining the base model, making it easy to apply to existing systems like FLUX or SD3. I was particularly impressed by the use of an LLM as a "planner" to resolve semantic conflicts—it feels like a natural and powerful fit. The results are convincing, showing clear improvements on challenging benchmarks where other methods produce nonsensical hybrids. It's also a major plus that the method doesn't degrade performance on standard, non-contradictory prompts.
summary: This paper tackles a common issue in text-to-image models: they often fail when prompts contain “contextually contradictory” concepts (e.g., a butterfly in a bee’s hive). The authors propose a method called Stage-Aware Prompting (SAP), which uses an LLM to break down the original prompt into simpler, non-conflicting proxy prompts. These are then used at different stages of the image generation process to guide the model toward more accurate results.
weaknesses: The approach's reliance on the LLM is also its main weakness. The quality of the final image is now dependent on the LLM's ability to correctly diagnose the contradiction and generate sensible proxies, which might not always be robust. Additionally, the method can only work around the base model's limitations; it won't fix fundamental issues like incorrect object counts or strange anatomy if the diffusion model itself struggles with them. The process for defining the timestep intervals for switching prompts also feels a bit heuristic and could benefit from a more principled design.

---

confidence: 5
contribution: 3
initial_rating: 2
presentation: 3
questions: 1. Details on the model used for generation and the attention visualization is not clear. From which layer are they taken or averaged? Which model is used? 
2. How does the approach generalize to scenarios with multiple foreground concepts or with text-centric visual information? 
3. How would the approach benefit from negative prompting? 
4. Can the authors comment on the evaluation strategy and its robustness.  Evaluating CLIP similarity can be considered?
soundness: 2
strengths: The paper proposes a training free approach for generating contextually contradictory prompts. 
The proposed approach outperforms FLUX, SD3.0, R2F, Ella, Annealing Guidance in both alignment and visual quality across multiple dataset.
The qualitative examples also show the effectiveness of the approach.
summary: This paper generates images for cases where contextual contradictions prevail. For example, a lion doing a  headstand would generally generate a human doing headstand with models like FLUX. To achieve this, based on the widely known and studied observation that diffusion models generate images in coarse to fine fashion, with low-level details in the early timesteps and finer details in the last timesteps, 
the paper leverages multi-stage prompting strategy. LLM is provided with instruction that given a prompt as input, it decomposes it into a series of prompts which are then input to the diffusion model for generation
The method is compared on Whoops dataset for contradictory generation against FLUX, SD3, annealing, R2F using LLM as a judge.
weaknesses: 1. The details on the attention mechanism in Figure 3 are not clear. 
2. Construction of in-context examples: The prompts considered have limited diversity with variations in the object and the background
3. The approach considers limited variations in the visual style with respect to which the modifications are performed. For example, a single instance of multi-object scenario will be challenging 
4. Comparison to approaches or negative prompting techniques: The paper focuses on progressive addition of concepts.  Negative prompting techniques can be considered. 
5. Evaluation is based solely on LLM as a judge. If same LLM used to create prompts and evaluate it can introduce bias from its own prompting strategy.

---

confidence: 4
contribution: 2
initial_rating: 2
presentation: 3
questions: 1. **Interval assignment.** Are stage intervals decided solely by the LLM? How do you ensure LLMs have a good understanding of the denoising process? Can you try a VLM that periodically inspects partially denoised images to decide when to advance to the next stage? But it should suffer from higher compute cost.  
2. **Benchmark construction.** You construct **Whoops-Hard** from **Whoops**. How are prompts selected? What criteria define “hard”?
3. **Cost/latency & robustness.** Your work uses llm as a component, can you show the cost of it?
soundness: 3
strengths: - **Clear figures.** Figures (e.g., Fig. 1, Fig. 5) effectively show the key motivation and the proposed pipeline.  
- **Strong results.** The method outperforms baselines by a meaningful margin in reported settings.  
- **Evaluation breadth.** The paper includes sufficient evaluation including human study.
summary: This paper investigates a failure mode in text-to-image diffusion models—contextual contradiction, where concepts in a prompt implicitly negate or conflict due to entangled priors. The authors introduce a stage-aware prompt decomposition pipeline: an LLM analyzes the input prompt, detects contradictions, and emits a sequence of proxy prompts aligned to different denoising stages, guiding sampling toward better semantic faithfulness. Experiments report sizable gains on contradiction-heavy prompts with both automatic metrics and human evaluation.
weaknesses: 1. **Problem prevalence** It’s unclear how prevalent “contextual contradiction” remains in SOTA systems. I tested gpt-image-1 with the prompt *“Bruce Lee is dressed in a yellow leotard and tutu practicing ballet”* and it performed reasonably well. Many evaluated backbones are ~1 year old. During the period, unified understanding and generation models, like Janus and Show-o surge and they often have great prompt alignment ability. Please evaluate on recent models using your contradiction prompts to quantify the gap and test your method on these models/
2. **Source of the issue: diffusion vs. text encoder.** Several tested backbones rely on relatively weaker text encoders (e.g., CLIP variants for SD). If LLM/LMM-based encoders already resolve many contradictions, the failure may stem more from **language encoding** than the **image generator**. Please:  
   - Compare with models that use LLMs/LMMs for prompt parsing (e.g., Qwen-image–style setups).  
   - Analyze whether your gains persist when a stronger language encoder is used.  
3. **Novelty positioning.** There is a bunch of work using LLMs to ground/composite prompts or alter conditioning across steps. Also, How is “contextual contradiction” distinct from known **bias/entanglement** issues (e.g., color or gender bias) beyond terminology.

---

confidence: 4
contribution: 2
initial_rating: 4
presentation: 3
questions: Overall, I like the task proposed in this paper; it is interesting. However, there are still some areas in this paper that need improvement, which are provided in the Weaknesses section. I am open to increasing my score if the authors can effectively address these concerns.
soundness: 2
strengths: 1. The paper clearly articulates contextual contradiction as a specific and challenging problem, distinct from general compositionality failures.
2. Using an LLM to temporally decompose a prompt into a series of proxy prompts that align with the diffusion model's coarse-to-fine generation process is a reasonable, training-free solution.
3. The authors provide a robust and convincing evaluation.
summary: The authors introduce the concept of contextual contradiction, a failure mode in text-to-image diffusion models where prompts fail not because of direct semantic opposition, but because one concept's learned associations conflict with another concept in the prompt. To solve this, the paper proposes Stage-Aware Prompting (SAP), a training-free framework. SAP is based on the insight that diffusion models generate images in a coarse-to-fine manner (e.g., layout, then shape, then details). The method uses a Large Language Model (LLM) to analyze the target prompt, identify contradictions, and decompose it into a sequence of proxy prompts. These proxies are injected at different timestep intervals of the denoising process to guide the model, first establishing a coherent layout/pose before introducing the contradictory concept. Results from both VLM-based scoring and human user studies show that SAP significantly improves prompt-image alignment compared to baselines.
weaknesses: 1. The method's success is critically dependent on the quality of the LLM (GPT-4o) and the 20 manually-crafted in-context examples provided to it. The ablation in Table 3 shows that removing these examples causes a severe performance drop. This indicates the method is less of a general framework and more of a highly-effective prompt engineering strategy, which may not generalize to contradiction types not covered by the 20 examples.
2. The same VLM (GPT-4o) is used as a core component of the method (to generate proxy prompts) and as the primary automated metric for evaluation. Does this introduce a risk of bias in the assessment?
3. The method requires the LLM to output specific timestep intervals (e.g., Steps 0-2, Steps 3-6). These intervals are based on a general heuristic provided in the system prompt, not on an empirical analysis of the specific diffusion models (FLUX, SD3). The paper shows that while the method is robust to minor shifts ($\pm2$ steps), it fails completely if the wrong stage is chosen (Figure 7). This suggests the heuristic must be correct, but its derivation is not well-justified.
4. While the new benchmarks are a strength, they are also small (40 prompts for ContraBench, 100 for Whoops-Hard). This makes it difficult to assess true generalizability.
5. The user study (Table 2) finds that users prefer SAP's outputs for Quality as well as Alignment. The paper does not explain why a prompt-editing method would improve fundamental image quality. This is likely due to resolving semantic conflicts, which prevents the model from generating messy, artifact-ridden hybrid concepts (as seen in baseline failures), but this link should be explicitly stated and analyzed.