CreatiDesign: A Unified Multi-Conditional Diffusion Transformer for Creative Graphic Design

## Abstract

Abstract Graphic design plays a vital role in visual communication across advertising, marketing, and multimedia entertainment.
Prior work has explored automated graphic design generation using diffusion models, aiming to streamline creative workflows and democratize design capabilities.
However, complex graphic design scenarios require accurately adhering to design intent specified by multiple heterogeneous user-provided elements ( e.g . images, layouts, and texts), which pose multi-condition control challenges for existing methods.
Specifically, previous single-condition control models demonstrate effectiveness only within their specialized domains but fail to generalize to other conditions, while existing multi-condition methods often lack fine-grained control over each sub-condition and compromise overall compositional harmony.
To address these limitations, we introduce CreatiDesign, a systematic solution for automated graphic design covering both model architecture and dataset construction.
First, we design a unified multi-condition driven architecture that enables flexible and precise integration of heterogeneous design elements with minimal architectural modifications to the base diffusion model.
Furthermore, to ensure that each condition precisely controls its designated image region and to avoid interference between conditions, we propose a multimodal attention mask mechanism.
Additionally, we develop a fully automated pipeline for constructing graphic design datasets, and introduce a new dataset with 400K samples featuring multi-condition annotations, along with a comprehensive benchmark.
Experimental results show that CreatiDesign outperforms existing models by a clear margin in faithfully adhering to user intent.

## 1 Introduction

Graphic design [47] is a fundamental vehicle for visual communication, affective perception, and brand identity across advertising, marketing, and multimedia entertainment.

Recently, diffusion models [35, 21, 90, 85] have achieved remarkable advances, especially in text-to-image generation [29, 102, 11, 77, 46], which can produce visually compelling and semantically rich images. Leveraging these models to automate graphic design—thereby streamlining creative workflows and democratizing design capabilities—has attracted increasing attention.

However, as illustrated in Figure [2](https://arxiv.org/html/2505.19114v2#S1.F2), graphic design generation poses unique challenges because it requires the precise control and harmonious arrangement of multiple heterogeneous elements, typically comprising three categories: \@slowromancapi@) Primary visual elements, which act as visual focal points and convey the central theme (*e.g*. product subjects, provided in image format); \@slowromancapii@) Secondary visual elements, which offer contextual support and enrich the composition (*e.g*. decorative objects, specified by semantic description and position in a layout); and \@slowromancapiii@) Textual elements, which directly convey essential information (*e.g*. slogans or product names, also provided as layout).
This multi-element nature introduces multi-condition control requirements for diffusion models, as it demands both semantic and spatial fidelity to users’ design intent.

While several works have explored unleashing the potential of diffusion models for automatic graphic design generation, three major challenges remain unresolved:
\@slowromancapi@) How to integrate multiple heterogeneous conditions in a unified manner. Previous expert models are typically tailored for only a single type of condition, and often fail to follow other conditions.
As illustrated in Figure [2](https://arxiv.org/html/2505.19114v2#S1.F2), image-driven models [101, 103, 53] focus exclusively on aligning with primary visual elements, whereas layout-driven models [77, 68, 113] are limited to following the semantic descriptions and spatial arrangements of secondary visual or textual elements. Such biased capability often leads to reduced fidelity to user intent, as highlighted by the red and purple masks.
\@slowromancapii@) How to preserve fine-grained controllability for each condition while achieving harmonious compositions. Existing multi-condition approaches [93, 105, 29, 4, 5] lack accurate control over each sub-condition and fail to effectively coordinate all elements, resulting in outputs that do not faithfully reflect user design intent.
\@slowromancapiii@) How to construct large-scale, multi-element graphic design datasets in an automated manner. Ready-to-use graphic design datasets with fine-grained, multi-condition annotations remain scarce, which naturally prevents models from learning design capabilities.

To this end, we propose CreatiDesign, a systematic solution for intelligent graphic design generation that addresses the aforementioned challenges through the following components:
\@slowromancapi@) Unified multi-condition driven architecture.
CreatiDesign preserves the strong generative capabilities of text-to-image diffusion models while unlocking their potential for graphic design with minimal architectural modifications. Specifically, the native image encoder is used to embed the multi-subject image condition into the latent space, while the semantic layout condition is processed by first extracting textual features via the native text encoder and then fusing them with positional information. After encoding the global prompt, noise image, image condition, and semantic layout condition into a unified feature space, native multimodal attention (MM-Attention) is applied to enable deep integration and interaction across modalities. This allows for unified and flexible multi-condition control over the generated content.
\@slowromancapii@) Efficient Multi-Condition Coordination.
To ensure that each heterogeneous condition precisely controls its designated image regions and to avoid mutual interference that could compromise the unique characteristics of each condition, we introduce carefully designed attention masks to regulate the interaction scope of each modality within the multimodal attention mechanism. This design enables each condition to independently and efficiently control its target region, while maintaining high overall compositional harmony.
\@slowromancapiii@) Automated Dataset Construction Pipeline.
We develop a fully automated pipeline for constructing graphic design datasets. This pipeline consists of design theme generation and rendering, conditional image generation, and multi-element annotation and filtering. It enables continuous scaling of the dataset, supporting robust and comprehensive model training. As a result, we construct a dataset containing 400K design samples with multi-condition annotations, along with a comprehensive benchmark for rigorous evaluation.

Figure: Figure 2: An overview of our motivation. Graphic design is a multi-condition driven generation task that requires the precise and harmonious arrangement of heterogeneous elements, including primary visual elements (provided as images with positions), as well as secondary visual and textual elements (both specified by semantic descriptions and positions). Previous methods either support only a single type of condition (*e.g*. image-driven or layout-driven models) or lack accurate control over each sub-condition(*e.g*. multi-condition driven models), resulting in failure to strictly adhere to user design intent, as highlighted by the red and purple masks.
Refer to caption: x2.png

## 2 Related Work

### 2.1 Text-to-Image Generation

Text-to-image (T2I) generation [85, 79, 8, 87, 14, 58, 76, 7, 26, 63] aims to generate visual content from textual descriptions, and has achieved remarkable progress in both visual quality and semantic alignment. Recent advances, such as SD3 series [23, 92], CogView4 [3], FLUX.1 [52], HiDream [2], and Seedream series [31, 28], have pushed the frontier further by leveraging Multimodal Diffusion Transformer architectures (MM-DiT).
Despite these advances, existing T2I models still struggle with fine-grained controllability, particularly in scenarios where users wish to specify precise subject identities or detailed compositional layouts.

### 2.2 Controllable Image Generation

To achieve precise control, a variety of conditional image generation paradigms have been proposed, including subject-driven [118, 72, 117, 86, 9, 19, 15, 42, 65, 91, 37, 111, 71, 112, 95, 51, 40, 88, 110, 116, 124, 69, 57, 96, 24, 17, 34, 33, 22, 41], layout-driven [56, 100, 122, 25, 20, 30, 89, 78, 55, 104, 45, 108, 120, 107, 36, 68, 16, 113, 77, 123], and so on. These expert models excel at controlling specific conditions—such as preserving the visual characteristics of the provided subjects or adhering to layout specifications—but often fail to follow other conditions. In response, multi-condition driven frameworks [93, 105, 29, 4, 5, 32, 59, 75, 98, 109, 48, 81, 38, 119, 84, 60, 94, 39, 54] have been introduced to jointly handle heterogeneous user-provided conditions. However, these unified approaches often lack accurate control over each sub-condition.

### 2.3 Automatic Graphic Design

Several works [29, 102, 11, 80, 46, 27, 62, 67, 99, 13, 121, 44, 43, 61, 66, 12, 97] have attempted to automate graphic design generation, aiming to streamline creative workflows and democratize design capabilities.
However, automatic graphic design introduces distinct challenges beyond general text-to-image or controllable image generation, requiring models to precisely preserve user-specified subjects, align secondary visual and textual elements with detailed semantic and spatial constraints, and maintain overall visual coherence. Despite recent progress, most existing methods struggle to meet all these demands simultaneously. This underscores the need for a unified, highly controllable, and harmonious solution, which is exactly the goal of this paper.

Figure: Figure 3: An overview of the architecture. CreatiDesign integrates image and semantic layout conditions through native multimodal attention. Multimodal attention mask ensures that each condition precisely controls its designated image regions while preventing leakage between conditions.
Refer to caption: x3.png

## 3 Method

### 3.1 Problem Formulation

This paper focuses on the task of graphic design generation, where each design typically comprises multiple heterogeneous elements provided by the user, such as primary visual elements, secondary visual elements, and textual elements, as illustrated in Figure [2](https://arxiv.org/html/2505.19114v2#S1.F2). The key challenge is to accurately and harmoniously integrate these user-specified elements—each representing distinct aspects of user intent—into the generated image. Formally, the task can be defined as: $I_{g}=f(P,I_{s},L)$,
where $I_{g}$ denotes the generated image, $P$ is the global prompt describing the overall image, $I_{s}$ represents the multi-subject image condition (*i.e*., a set of primary visual elements). $L$ denotes the semantic layout condition, which consists of $n$ elements, partitioned into two categories: secondary visual elements and textual elements. Each layout element is defined by a pair $(d_{i},b_{i})$, where $d_{i}$ is the semantic description and $b_{i}$ is the spatial position (bounding box), formally expressed as:

$$ $\displaystyle L=\{l_{i}=(d_{i},b_{i})\}_{i=0}^{n},\quad l_{i}\in\{\text{ secondary visual element, textual element}\}.$ (1) $$

In the following sections, we will introduce the key parts of CreatiDesign in detail.

### 3.2 Unified Multi-Condition Driven Architecture

In MM-DiT-based text-to-image models (*e.g*. FLUX.1 [52]), a text encoder (*e.g*. T5 [83]) is employed to tokenize and encode the input prompt into a sequence of text tokens, denoted as $h_{p}$. Concurrently, an image encoder (*e.g*. VAE [49]) is utilized to encode the ground-truth image into a latent representation $z$, which is subsequently partitioned into patches to obtain image tokens, denoted as $h_{z}$.
These text and image tokens are then fed into MM-Attention, which facilitates rich interactions between the textual and visual modalities, thereby enabling precise control over the image content. Our approach aims to retain the strong capabilities of T2I models while unlocking their potential for graphic design with minimal architectural modifications, as illustrated in Figure [3](https://arxiv.org/html/2505.19114v2#S2.F3).

#### Tokenize Multi-Subject Image Condition.

We first pad the multi-subject image condition with a background color (*e.g*. gray) and encode it using the native VAE. The encoded latent representation is then partitioned into patches to obtain the subject tokens $h_{s}$.

#### Tokenize Semantic Layout Condition.

For each element $l_{i}=(d_{i},b_{i})$ in the semantic layout condition, we utilize the native T5 text encoder to extract the semantic feature $h_{i}^{d}$ from $d_{i}$.
For the bounding box $b_{i}$, we apply Fourier positional encoding [70, 56] to obtain the spatial feature $h_{i}^{b}$. The final layout token $h_{i}^{l}$ is obtained by concatenating $h_{i}^{d}$ and $h_{i}^{b}$ along the feature dimension, followed by a layout encoder (*i.e*. MLP): $h_{i}^{l}=\mathrm{MLP}(\mathrm{Concat}(h_{i}^{d},h_{i}^{b}))$. In this way, layout tokens integrate semantic and spatial information.

#### Integrate Multi-Condition.

After encoding the prompt, noise image, multi-subject image condition, and semantic layout condition into tokens, denoted as $h^{p},h^{z},h^{s},h^{l}$, we concatenate them along the token dimension and feed the token sequence into a stack of MM-DiT Blocks. Each Block consists of linear projection layers (for Q, K, V), multimodal attention (MM-Attention), and feed-forward networks (FFN). Each type of tokens is linearly projected into its corresponding query, key, and value spaces: $Q^{*},K^{*},V^{*}=\mathrm{Linear}(h^{*})$, where $*$ denotes the modality (layout, prompt, image, or subject). For the layout tokens $h^{l}$ and subject tokens $h^{s}$, we further adapt their representations using LoRA modules deployed on the linear layer and adaptive layer normalization (AdaLN), enabling efficient fine-tuning and alignment.
The multimodal attention is then computed as:

$$ $\begin{gathered}{h^{l}}^{\prime},{h^{p}}^{\prime},{h^{z}}^{\prime},{h^{s}}^{ \prime}=\mathrm{Attention}([\mathbf{Q}^{l},\mathbf{Q}^{p},\mathbf{Q}^{z}, \mathbf{Q}^{s}],[\mathbf{K}^{l},\mathbf{K}^{p},\mathbf{K}^{z},\mathbf{K}^{s}], [\mathbf{V}^{l},\mathbf{V}^{p},\mathbf{V}^{z},\mathbf{V}^{s}]).\end{gathered}$ (2) $$

This design enables multiple conditions to control the image content.
To avoid positional embedding conflicts, such as between the noise image and image condition, or between the prompt and layout condition, we adopt positional encoding shifts to the image and layout condition tokens [95, 117] to ensure clear separation in the token space.
Overall, this architecture empowers the text-to-image model with multi-condition control capabilities through minimal architectural modifications.

### 3.3 Collaborative Multi-Condition Control

Multi-condition driven methods may suffer from degraded controllability over each sub-condition.
We attribute this to the fact that the sub-condition is not precisely bound to its corresponding image region and that there is semantic leakage among sub-conditions.
To address this, we introduce a multimodal attention mask within our unified architecture, consisting of a layout mask and a subject mask, as shown in Figure [3](https://arxiv.org/html/2505.19114v2#S2.F3).

#### Layout Attention Mask.

Given the user-specified bounding box $b_{i}$ for each semantic description $d_{i}$, we can precisely locate the target image region. Inspired by [10, 114], we construct a layout mask such that each layout token $h_{i}^{l}$ is only allowed to attend to and be attended by the image tokens $h_{i}^{z}$ within its corresponding bounding box. This explicit attention modulation enhances the spatial controllability.
Furthermore, we block interactions among layout tokens themselves, between layout tokens and subject tokens, and between layout tokens and prompt tokens, to prevent semantic leakage and to ensure that each layout token retains its unique characteristics.

#### Subject Attention Mask.

Based on the user-provided multi-subject image, we extract the spatial location of each subject to form a subject mask. Each subject token $h_{i}^{s}$ is only permitted to interact bidirectionally with the image tokens $h_{i}^{z}$ within its own mask region, thereby achieving precise subject injection.
In addition, to preserve the integrity and distinctive features of the subject token $h^{s}$, we block its interactions with all irrelevant tokens, including layout tokens $h^{l}$, prompt tokens $h^{p}$, and image tokens outside the target region of $h^{s}$.

With the proposed multimodal attention masks, CreatiDesign allows each condition to precisely and independently control its targeted image region without semantic leakage, thereby producing controllable and harmonious graphic designs that closely match user intent.

## 4 Graphic Design Datasets and Benchmark

### 4.1 Graphic Design Datasets

We propose a fully automatic dataset construction pipeline, as shown in Figure [4](https://arxiv.org/html/2505.19114v2#S4.F4), to address the scarcity of graphic design datasets with fine-grained, multi-condition annotations.

#### Design Theme Generation.

Based on a design keywords bank covering common graphic design elements (*e.g*. furniture, food, clothing *etc*.), we prompt a large language model (LLM, *e.g*. GPT-4) to act as a professional designer and generate design themes that include descriptions of primary visual elements, secondary visual elements, and textual elements.

#### Text Layer Rendering.

Based on the design theme, we follow the Hierarchical Layout Generation [18] (HLG) paradigm to generate a layout protocol of textual elements and a detailed background description. A rendering engine then converts the layout protocol into an RGBA image with accurately positioned foreground text.

Figure: Figure 4: Automated pipeline for graphic design dataset construction.
Refer to caption: x4.png

#### Foreground-based Image Generation.

To generate a visually coherent graphic design image, we draw inspiration from LayerDiffuse [115] and develop a foreground-conditioned image generation model.
Here, the RGBA text layer serves as the foreground, while the background is generated based on the aforementioned description. Specifically, we incorporate foreground-LoRA and background-LoRA modules into FLUX.1-dev and employ attention sharing to ensure seamless integration of foreground and background elements.

#### Entity Annotation.

We use GroundingSAM2 [1] to obtain bounding boxes and segmentation masks for all entities in the generated image. A vision-language model [73] (VLM) is then employed to generate fine-grained descriptions for each entity.
Entities are categorized as either primary or secondary visual elements. All primary visual elements are aggregated to form the multi-subject image condition, while secondary visual elements, together with the textual elements from the layout protocol, constitute the semantic layout condition.

Based on this automatic data construction pipeline, we synthesize graphic design samples at scale, alleviating the data bottleneck for model training. As a result, we construct a new dataset of 400K samples with annotations for various conditions.

### 4.2 Graphic Design Benchmark

To comprehensively evaluate graphic design generation under multiple conditions, we further construct a rigorous benchmark consisting of 1,000 carefully curated samples. This benchmark is designed to assess whether the generated results faithfully align with user intent—a critical requirement in practical graphic design scenarios.

The evaluation focuses on two key aspects, each with dedicated metrics:
\@slowromancapi@) Multi-Subject Preservation.
When given multi-subject image conditions (*i.e*. primary visual elements), it is crucial to strictly preserve the unique characteristics of each subject in the generated image.
To quantify this, we measure the similarity between each subject and its corresponding region (obtained via bounding box priors or detected by GroundingDINO [64]) in the generated image using both CLIP [82] similarity (CLIP-I) and DINO [74] similarity (DINO-I) scores. We further aggregate the DINO scores of all subjects by multiplication, denoted as M-DINO [101]. Unlike averaging, M-DINO is more sensitive to the failure of any single subject, providing a stricter assessment of subject preservation.
\@slowromancapii@) Semantic Layout Alignment.
For the semantic layout condition, specifying the positions and attributes of secondary visual elements and textual content, we assess alignment at spatial and semantic levels.
For secondary visual elements, we employ a vision-language model in a Visual Question Answering manner to assess the spatial, color, textual, and shape attributes of each entity in the generated image [113, 104]. For textual elements, we use PaddleOCR [6] to detect text and calculate sentence accuracy (Sen. Acc), normalized edit distance (NED; *i.e*. 1 minus the edit distance) [29], and IoU (spatial score) between detected and ground-truth texts.

## 5 Experiments

### 5.1 Experimental Setup

#### Dataset.

We train our models on the 400K synthetic graphic design samples described in Section [4.1](https://arxiv.org/html/2505.19114v2#S4.SS1).
The validation set contains 1,000 samples, covering diverse numbers of primary visual subjects and semantic layout annotations, enabling thorough evaluation of multi-condition controllability.

#### Evaluation Metrics.

As described in Section [4.2](https://arxiv.org/html/2505.19114v2#S4.SS2), we evaluate model performance from two perspectives—multi-subject preservation and semantic layout alignment—to assess whether the generated designs accurately fulfill user intent.
Additionally, to evaluate overall image quality, we report IR Score [106] and PickScore [50], which jointly capture prompt adherence, visual appeal, and compositional harmony across the entire image.

#### Implementation Details.

We fine-tune FLUX.1-dev using LoRA with 256 rank, introducing 491.5M extra parameters (4.1% of FLUX’s 12B).
We employ the AdamW optimizer with a fixed learning rate of 1e-4, training for 100,000 steps with a batch size of 8 on 8 H20-96G GPUs over 4 days.
We adopt a resolution bucketing strategy during training to support variable image sizes.
The image condition is set to half the target image size; each layout description is capped at 30 tokens, with up to 10 layouts per image.

**Table 1: Quantitative Results. We compare CreatiDesign with three types of previous SOTA models: multi-subject image-driven models, semantic layout-driven models, and multi-condition driven models. The best results are shown in bold, and the top-3 results are highlighted. Our proposed method significantly enhances the graphic design capabilities of the baseline, achieves top-tier performance across all metrics, and shows a clear lead in average score.**
|  | Multi-Subject Preservation | Semantic Layout Alignment | Image Quality | Avg. |  |  |  |  |  |  |  |  |  |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
|  | Primary Visual Elements | Secondary Visual Elements | Textual Elements |  |  |  |  |  |  |  |  |  |  |
|  | CLIP-I | DINO-I | M-DINO | Spatial | Color | Textual | Shape | Spatial | Sen. Acc | NED | IR | PickScore |  |
| UNO | 77.97 | 47.88 | 20.79 | 53.10 | 43.44 | 42.62 | 41.30 | 11.47 | 40.87 | 74.51 | 61.06 | 21.67 | 44.72 |
| MS-Diffusion | 84.75 | 74.13 | 44.34 | 49.54 | 33.37 | 34.89 | 34.79 | 1.01 | 0.00 | 10.21 | 46.64 | 21.03 | 36.23 |
| FLUX.1-Fill | 90.79 | 87.32 | 69.05 | 67.55 | 57.48 | 56.26 | 55.75 | 12.48 | 12.07 | 56.69 | 40.74 | 20.71 | 52.24 |
| CreatiLayout | 78.41 | 55.54 | 25.31 | 77.42 | 63.07 | 62.67 | 60.21 | 18.59 | 12.27 | 74.21 | 59.92 | 21.04 | 50.72 |
| HiCo | 72.45 | 34.47 | 11.59 | 79.69 | 61.48 | 61.17 | 60.17 | 1.01 | 0.00 | 14.98 | -36.16 | 19.58 | 31.70 |
| BizGen | 79.86 | 53.08 | 22.93 | 79.84 | 62.96 | 62.86 | 61.01 | 50.44 | 75.89 | 94.61 | 37.43 | 21.48 | 58.53 |
| AnyText2 | 74.68 | 34.86 | 12.22 | 36.63 | 27.58 | 27.16 | 26.53 | 53.95 | 9.56 | 48.26 | -25.95 | 20.24 | 28.81 |
| Emu2 | 73.96 | 45.17 | 19.92 | 60.81 | 45.37 | 46.20 | 44.06 | 0.20 | 0.00 | 13.81 | -1.84 | 20.18 | 30.65 |
| PosterMaker | 90.45 | 87.72 | 69.56 | 56.36 | 45.37 | 44.25 | 41.61 | 28.42 | 0.70 | 28.62 | 31.62 | 20.31 | 45.42 |
| OmniGen | 82.15 | 58.83 | 30.86 | 53.92 | 44.35 | 44.46 | 41.40 | 8.24 | 6.72 | 49.12 | 22.94 | 20.62 | 38.63 |
| Gemini2.0 | 81.46 | 57.23 | 29.68 | 59.41 | 52.29 | 52.49 | 50.36 | 16.60 | 71.38 | 88.71 | 28.52 | 21.23 | 50.78 |
| FLUX.1-dev | 75.93 | 44.59 | 17.76 | 60.02 | 47.1 | 46.19 | 44.76 | 13.25 | 57.95 | 81.52 | 59.45 | 21.48 | 47.50 |
| CreatiDesign | 89.39 | 86.48 | 65.75 | 78.94 | 66.02 | 66.94 | 65.82 | 56.90 | 78.30 | 94.68 | 60.02 | 21.49 | 69.28 |
| vs. Baseline | +13.46 | +41.89 | +47.99 | +18.92 | +18.92 | +20.75 | +21.06 | +43.65 | +20.35 | +13.16 | +1.17 | +0.01 | +21.78 |

### 5.2 Comparison with Prior Works

#### Baseline Methods.

We compare CreatiDesign with three types of previous SOTA models: multi-subject image-driven models [103, 101, 53], semantic layout-driven models [113, 68, 77, 97], and multi-condition driven models [93, 4, 105, 29].

#### Quantitative Comparison.

As shown in Table [1](https://arxiv.org/html/2505.19114v2#S5.T1), specialist models excel primarily on their targeted control conditions—image-driven models can preserve subjects, while layout-driven models can follow semantic layout control—but perform poorly when handling other conditions. Conversely, previous multi-condition models often lack fine-grained control over each sub-condition, resulting in lower subject preservation and semantic alignment.
In contrast, CreatiDesign achieves precise, balanced control across all conditions, as reflected in its top-tier performance across every sub-condition and clear lead in average scores.
Remarkably, this advanced graphic design capability is achieved with minor architectural modifications to the base model FLUX.1-dev and only 4.1% extra parameters were introduced, demonstrating both effectiveness and efficiency.

#### Qualitative Comparison.

To further illustrate the advantages of CreatiDesign, Figure [7](https://arxiv.org/html/2505.19114v2#S5.F7) presents qualitative comparisons on challenging cases with multiple subjects and complex layouts. Existing SOTA methods—including multi-condition driven models [5, 4, 105, 29] and single-condition experts [53, 77]—consistently fall short in faithfully fulfilling user intent. Previous multi-condition models exhibit limited precision in controlling sub-conditions, resulting in misplaced or inconsistent subjects (highlighted by purple masks), as well as content or spatial misalignment in the layout (highlighted by red masks).
Layout-driven models like BizGen [77] can follow the layout but struggle with subject consistency. Image-driven models such as FLUX.1-Fill [53] can preserve primary elements but often misplace or incorrectly render textual elements.
In contrast, CreatiDesign consistently preserves the identity and position of all primary subjects, precisely aligns secondary and textual elements within the layout, and ensures overall compositional harmony.

#### User Study.

To comprehensively assess the practical effectiveness of CreatiDesign, we conducted a user study involving feedback from both professional designers and general users. Specifically, we solicited 50 evaluation reports on 30 diverse graphic design samples, comparing our method with several state-of-the-art baselines. As illustrated in Figure [6](https://arxiv.org/html/2505.19114v2#S5.F6), participants rated the generated designs on a scale of 1 to 10 across multiple criteria, including adherence to multi-subject image conditions (position accuracy and subject preservation), alignment with semantic layout conditions (position accuracy, attribute accuracy and text accuracy), and overall perceptual quality (prompt following and visual quality). The statistical results demonstrate that CreatiDesign outperforms previous methods in fine-grained controllability and overall visual appeal, delivering superior user satisfaction in real-world graphic design scenarios.

Figure: Figure 5: User Study. CreatiDesign demonstrates top-tier performance in preserving multi-subject characteristics, strictly following semantic layout conditions, and achieving high overall image quality.
Refer to caption: x5.png

### 5.3 Free Lunch: Expanding to Editing Tasks

As illustrated in Figure [6](https://arxiv.org/html/2505.19114v2#S5.F6), CreatiDesign naturally extends beyond graphic design to a wide range of editing tasks without extra retraining. We demonstrate this capability via editing a series of movie posters. Initially, the user provides a global prompt, a multi-subject image condition (*e.g*. Hulk and Iron Man), and a semantic layout specifying elements and their spatial positions (*e.g*. Captain America and the “Avengers” title).
CreatiDesign generates a high-quality poster $P_{1}$ that precisely adheres to these controls. Subsequently, a sequence of editing operations is performed: first, by treating the previously generated poster $P_{1}$ as the new image condition and introducing a new text element “AGE OF ULTRON” with its desired position, CreatiDesign seamlessly inserts this subtitle to produce $P_{2}$; Next, by combining the Spider-Man image and its insertion mask with $P_{2}$ as the image condition, CreatiDesign generates $P_{3}$, achieving seamless integration of the new subject while preserving character fidelity and overall visual harmony; finally, by combining $P_{3}$ with the mask of the edited region as the image condition, the subtitle is modified to “INFINITY WAR” ($P_{4}$).
Throughout these editing processes, CreatiDesign consistently maintains subject identity, achieves accuracy layout control and overall visual harmony. In contrast, strong baselines such as Gemini2.0 frequently fail to preserve non-edited regions during sequential edits, often resulting in unwanted attribute changes to subjects or text, highlighting a lack of strict adherence to user intent.

Figure: Figure 7: Quantitative Results. Compared with previous multi-condition or single-condition models, CreatiDesign demonstrates stricter adherence to user intent, including high subject preservation and precise layout alignment. Purple masks: inconsistent or mispositioned subjects. Red mask: entities with incorrect semantics or locations. Gray mask: disharmonious background or foreground regions.
Refer to caption: x7.png

### 5.4 Ablation Study

Table [8](https://arxiv.org/html/2505.19114v2#S5.F8) and Figure [8](https://arxiv.org/html/2505.19114v2#S5.F8) evaluate the contributions of the three key components—Layout Encoder (LE), Layout Attention Mask (LAM), and Subject Attention Mask (SAM)—to the performance of CreativeDesign from quantitative and qualitative perspectives, respectively.
The LE fuses the semantic features of the textual description with Fourier-encoded positional features and further aligns them into layout tokens; removing LE leads to a clear drop in the accuracy of generated text.
The layout attention mask enables fine-grained spatial control by explicitly restricting each layout element to modulate only its designated image region and preventing semantic leakage across layout elements; removing LAM leads to imprecise placement of elements and increased confusion across different layout regions, as demonstrated by the decrease in spatial alignment and attribute accuracy.
Similarly, the subject attention mask ensures that each subject token only interacts with its corresponding image region and blocks interference from global prompts and layout conditions. Without SAM, we observe the degradation in subject consistency, such as the incorrect digits on clocks or altered popcorn color.
These results validate the effectiveness of each component in achieving faithful and controllable graphic design generation.

Figure: Table 2: Ablation study: quantitative analysis of key components in CreatiDesign.
Refer to caption: x8.png

## 6 Conclusion

In this paper, we presented CreatiDesign, a systematic solution that empowers diffusion transformers for intelligent and highly controllable graphic design generation. We designed a unified multi-condition driven architecture that seamlessly integrates heterogeneous design elements. Furthermore, we proposed a multimodal attention mask mechanism to ensure that each condition precisely controls its designated image region and to prevent interference between conditions. In addition, we introduced a fully automated pipeline for constructing large-scale, richly annotated graphic design datasets. Extensive experiments demonstrated that CreatiDesign outperforms previous methods in subject preservation, semantic layout alignment, and overall visual quality.

#### Limitation and Future Work.

CreatiDesign faces challenges in accurately preserving facial details and generating dense text, as our current dataset is not tailored for these scenarios. Improving performance in such cases, either through dataset enhancement or model-level advances, represents an important direction for future research.

## References

- ide [2024]
Grounded-sam-2.
[https://github.com/IDEA-Research/Grounded-SAM-2](https://github.com/IDEA-Research/Grounded-SAM-2), 2024.
- HiD [2025]
Hidream-i1.
[https://github.com/HiDream-ai/HiDream-I1](https://github.com/HiDream-ai/HiDream-I1), 2025.
- THU [2025]
Cogview4.
[https://github.com/THUDM/CogView4](https://github.com/THUDM/CogView4), 2025.
- goo [2025]
Gemini-2.0-flash.
[https://aistudio.google.com/prompts/new_chat?model=gemini-2.0-flash-exp](https://aistudio.google.com/prompts/new_chat?model=gemini-2.0-flash-exp), 2025.
- ope [2025]
4o image generation.
[https://openai.com/index/introducing-4o-image-generation/](https://openai.com/index/introducing-4o-image-generation/), 2025.
- pad [2025]
Paddleocr-v4.
[https://github.com/PaddlePaddle/PaddleOCR](https://github.com/PaddlePaddle/PaddleOCR), 2025.
- Bao et al. [2023]
Fan Bao, Shen Nie, Kaiwen Xue, Yue Cao, Chongxuan Li, Hang Su, and Jun Zhu.
All are worth words: A vit backbone for diffusion models.
In *CVPR*, 2023.
- Betker et al. [2023]
James Betker, Gabriel Goh, Li Jing, Tim Brooks, Jianfeng Wang, Linjie Li, Long Ouyang, Juntang Zhuang, Joyce Lee, Yufei Guo, et al.
Improving image generation with better captions.
*Computer Science. https://cdn. openai. com/papers/dall-e-3. pdf*, 2023.
- Cai et al. [2024]
Shengqu Cai, Eric Chan, Yunzhi Zhang, Leonidas Guibas, Jiajun Wu, and Gordon Wetzstein.
Diffusion self-distillation for zero-shot customized image generation.
*arXiv preprint arXiv:2411.18616*, 2024.
- Chen et al. [2024a]
Anthony Chen, Jianjin Xu, Wenzhao Zheng, Gaole Dai, Yida Wang, Renrui Zhang, Haofan Wang, and Shanghang Zhang.
Training-free regional prompting for diffusion transformers.
*arXiv preprint arXiv:2411.02395*, 2024a.
- Chen et al. [2025a]
Haoyu Chen, Xiaojie Xu, Wenbo Li, Jingjing Ren, Tian Ye, Songhua Liu, Ying-Cong Chen, Lei Zhu, and Xinchao Wang.
Posta: A go-to framework for customized artistic poster generation.
In *CVPR*, 2025a.
- Chen et al. [2024b]
Jingye Chen, Yupan Huang, Tengchao Lv, Lei Cui, Qifeng Chen, and Furu Wei.
Textdiffuser-2: Unleashing the power of language models for text rendering.
In *ECCV*, 2024b.
- Chen et al. [2024c]
Jingye Chen, Yupan Huang, Tengchao Lv, Lei Cui, Qifeng Chen, and Furu Wei.
Textdiffuser-2: Unleashing the power of language models for text rendering.
In *ECCV*, 2024c.
- Chen et al. [2024d]
Junsong Chen, YU Jincheng, GE Chongjian, Lewei Yao, Enze Xie, Zhongdao Wang, James Kwok, Ping Luo, Huchuan Lu, and Zhenguo Li.
Pixart-$\alpha$: Fast training of diffusion transformer for photorealistic text-to-image synthesis.
In *ICLR*, 2024d.
- Chen et al. [2025b]
Nan Chen, Mengqi Huang, Zhuowei Chen, Yang Zheng, Lei Zhang, and Zhendong Mao.
Customcontrast: A multilevel contrastive perspective for subject-driven text-to-image customization.
In *AAAI*, 2025b.
- Chen et al. [2023]
Xiaohui Chen, Yongfei Liu, Yingxiang Yang, Jianbo Yuan, Quanzeng You, Li-Ping Liu, and Hongxia Yang.
Reason out your layout: Evoking the layout master from large language models for text-to-image synthesis.
*arXiv preprint arXiv:2311.17126*, 2023.
- Chen et al. [2024e]
Xi Chen, Lianghua Huang, Yu Liu, Yujun Shen, Deli Zhao, and Hengshuang Zhao.
Anydoor: Zero-shot object-level image customization.
In *Proceedings of the IEEE/CVF conference on computer vision and pattern recognition*, pages 6593–6602, 2024e.
- Cheng et al. [2025]
Yutao Cheng, Zhao Zhang, Maoke Yang, Hui Nie, Chunyuan Li, Xinglong Wu, and Jie Shao.
Graphic design with large multimodal model.
In *AAAI*, 2025.
- Choi et al. [2025]
Yongjin Choi, Chanhun Park, and Seung Jun Baek.
Dynasyn: Multi-subject personalization enabling dynamic action synthesis.
In *AAAI*, 2025.
- Dahary et al. [2024]
Omer Dahary, Or Patashnik, Kfir Aberman, and Daniel Cohen-Or.
Be yourself: Bounded attention for multi-subject text-to-image generation.
In *ECCV*, 2024.
- Dhariwal and Nichol [2021]
Prafulla Dhariwal and Alexander Nichol.
Diffusion models beat gans on image synthesis.
In *NeurIPS*, 2021.
- Ding et al. [2024]
Ganggui Ding, Canyu Zhao, Wen Wang, Zhen Yang, Zide Liu, Hao Chen, and Chunhua Shen.
Freecustom: Tuning-free customized image generation for multi-concept composition.
In *CVPR*, 2024.
- Esser et al. [2024]
Patrick Esser, Sumith Kulal, Andreas Blattmann, Rahim Entezari, Jonas Müller, Harry Saini, Yam Levi, Dominik Lorenz, Axel Sauer, Frederic Boesel, et al.
Scaling rectified flow transformers for high-resolution image synthesis.
*arXiv preprint arXiv:2403.03206*, 2024.
- Feng et al. [2025]
Haoran Feng, Zehuan Huang, Lin Li, Hairong Lv, and Lu Sheng.
Personalize anything for free with diffusion transformer.
*arXiv preprint arXiv:2503.12590*, 2025.
- Feng et al. [2024]
Yutong Feng, Biao Gong, Di Chen, Yujun Shen, Yu Liu, and Jingren Zhou.
Ranni: Taming text-to-image diffusion for accurate instruction following.
In *CVPR*, 2024.
- Gao et al. [2024]
Peng Gao, Le Zhuo, Ziyi Lin, Chris Liu, Junsong Chen, Ruoyi Du, Enze Xie, Xu Luo, Longtian Qiu, Yuhang Zhang, et al.
Lumina-t2x: Transforming text into any modality, resolution, and duration via flow-based large diffusion transformers.
*arXiv preprint arXiv:2405.05945*, 2024.
- Gao et al. [2023]
Yifan Gao, Jinpeng Lin, Min Zhou, Chuanbin Liu, Hongtao Xie, Tiezheng Ge, and Yuning Jiang.
Textpainter: Multimodal text image generation with visual-harmony and text-comprehension for poster design.
In *ACMMM*, 2023.
- Gao et al. [2025a]
Yu Gao, Lixue Gong, Qiushan Guo, Xiaoxia Hou, Zhichao Lai, Fanshi Li, Liang Li, Xiaochen Lian, Chao Liao, Liyang Liu, et al.
Seedream 3.0 technical report.
*arXiv preprint arXiv:2504.11346*, 2025a.
- Gao et al. [2025b]
Yifan Gao, Zihang Lin, Chuanbin Liu, Min Zhou, Tiezheng Ge, Bo Zheng, and Hongtao Xie.
Postermaker: Towards high-quality product poster generation with accurate text rendering.
In *CVPR*, 2025b.
- Gong et al. [2024]
Biao Gong, Siteng Huang, Yutong Feng, Shiwei Zhang, Yuyuan Li, and Yu Liu.
Check locate rectify: A training-free layout calibration system for text-to-image generation.
In *CVPR*, 2024.
- Gong et al. [2025]
Lixue Gong, Xiaoxia Hou, Fanshi Li, Liang Li, Xiaochen Lian, Fei Liu, Liyang Liu, Wei Liu, Wei Lu, Yichun Shi, et al.
Seedream 2.0: A native chinese-english bilingual image generation foundation model.
*arXiv preprint arXiv:2503.07703*, 2025.
- Han et al. [2024]
Zhen Han, Zeyinzi Jiang, Yulin Pan, Jingfeng Zhang, Chaojie Mao, Chenwei Xie, Yu Liu, and Jingren Zhou.
Ace: All-round creator and editor following instructions via diffusion transformer.
*arXiv preprint arXiv:2410.00086*, 2024.
- He et al. [2024]
Junjie He, Yifeng Geng, and Liefeng Bo.
Uniportrait: A unified framework for identity-preserving single-and multi-human image personalization.
*arXiv preprint arXiv:2408.05939*, 2024.
- He et al. [2025]
Junjie He, Yuxiang Tuo, Binghui Chen, Chongyang Zhong, Yifeng Geng, and Liefeng Bo.
Anystory: Towards unified single and multiple subject personalization in text-to-image generation.
*arXiv preprint arXiv:2501.09503*, 2025.
- Ho et al. [2020]
Jonathan Ho, Ajay Jain, and Pieter Abbeel.
Denoising diffusion probabilistic models.
In *NeurIPS*, 2020.
- Hoe et al. [2024]
Jiun Tian Hoe, Xudong Jiang, Chee Seng Chan, Yap-Peng Tan, and Weipeng Hu.
Interactdiffusion: Interaction control in text-to-image diffusion models.
In *CVPR*, 2024.
- Hu et al. [2024]
Hexiang Hu, Kelvin CK Chan, Yu-Chuan Su, Wenhu Chen, Yandong Li, Kihyuk Sohn, Yang Zhao, Xue Ben, Boqing Gong, William Cohen, et al.
Instruct-imagen: Image generation with multi-modal instruction.
In *CVPR*, 2024.
- Hu et al. [2023]
Minghui Hu, Jianbin Zheng, Daqing Liu, Chuanxia Zheng, Chaoyue Wang, Dacheng Tao, and Tat-Jen Cham.
Cocktail: Mixing multi-modality control for text-conditional image generation.
In *NeurIPS*, 2023.
- Huang et al. [2024a]
Lianghua Huang, Wei Wang, Zhi-Fan Wu, Huanzhang Dou, Yupeng Shi, Yutong Feng, Chen Liang, Yu Liu, and Jingren Zhou.
Group diffusion transformers are unsupervised multitask learners.
*arXiv preprint arXiv:2410.15027*, 2024a.
- Huang et al. [2024b]
Lianghua Huang, Wei Wang, Zhi-Fan Wu, Yupeng Shi, Huanzhang Dou, Chen Liang, Yutong Feng, Yu Liu, and Jingren Zhou.
In-context lora for diffusion transformers.
*arXiv preprint arXiv:2410.23775*, 2024b.
- Huang et al. [2024c]
Mengqi Huang, Zhendong Mao, Mingcong Liu, Qian He, and Yongdong Zhang.
Realcustom: narrowing real text word for real-time open-domain text-to-image customization.
In *CVPR*, 2024c.
- Huang et al. [2025]
Qihan Huang, Siming Fu, Jinlong Liu, Hao Jiang, Yipeng Yu, and Jie Song.
Resolving multi-condition confusion for finetuning-free personalized image generation.
In *AAAI*, 2025.
- Inoue et al. [2023a]
Naoto Inoue, Kotaro Kikuchi, Edgar Simo-Serra, Mayu Otani, and Kota Yamaguchi.
Towards flexible multi-modal document models.
In *CVPR*, 2023a.
- Inoue et al. [2023b]
Naoto Inoue, Kotaro Kikuchi, Edgar Simo-Serra, Mayu Otani, and Kota Yamaguchi.
Layoutdm: Discrete diffusion model for controllable layout generation.
In *CVPR*, 2023b.
- Jia et al. [2024]
Chengyou Jia, Minnan Luo, Zhuohang Dang, Guang Dai, Xiaojun Chang, Mengmeng Wang, and Jingdong Wang.
Ssmg: Spatial-semantic map guided diffusion model for free-form layout-to-image generation.
In *AAAI*, 2024.
- Jia et al. [2023]
Peidong Jia, Chenxuan Li, Yuhui Yuan, Zeyu Liu, Yichao Shen, Bohan Chen, Xingru Chen, Yinglin Zheng, Dong Chen, Ji Li, et al.
Cole: A hierarchical generation framework for multi-layered and editable graphic design.
*arXiv preprint arXiv:2311.16974*, 2023.
- Jobling and Crowley [1996]
Paul Jobling and David Crowley.
Graphic design: reproduction and representation since 1800.
1996.
- Kim et al. [2023]
Sungnyun Kim, Junsoo Lee, Kibeom Hong, Daesik Kim, and Namhyuk Ahn.
Diffblender: Scalable and composable multimodal text-to-image diffusion models.
*arXiv preprint arXiv:2305.15194*, 2023.
- Kingma [2013]
Diederik P Kingma.
Auto-encoding variational bayes.
*arXiv preprint arXiv:1312.6114*, 2013.
- Kirstain et al. [2023]
Yuval Kirstain, Adam Polyak, Uriel Singer, Shahbuland Matiana, Joe Penna, and Omer Levy.
Pick-a-pic: An open dataset of user preferences for text-to-image generation.
*arXiv preprint arXiv:2305.01569*, 2023.
- Kumari et al. [2025]
Nupur Kumari, Xi Yin, Jun-Yan Zhu, Ishan Misra, and Samaneh Azadi.
Generating multi-image synthetic data for text-to-image customization.
*arXiv preprint arXiv:2502.01720*, 2025.
- Labs [2024]
Black Forest Labs.
Flux.1-dev.
[https://blackforestlabs.ai/announcing-black-forest-labs](https://blackforestlabs.ai/announcing-black-forest-labs), 2024.
- Labs [2025]
Black Forest Labs.
Flux.1-fill-dev.
[https://huggingface.co/black-forest-labs/FLUX.1-Fill-dev](https://huggingface.co/black-forest-labs/FLUX.1-Fill-dev), 2025.
- Le et al. [2024]
Duong H Le, Tuan Pham, Sangho Lee, Christopher Clark, Aniruddha Kembhavi, Stephan Mandt, Ranjay Krishna, and Jiasen Lu.
One diffusion to generate them all.
*arXiv preprint arXiv:2411.16318*, 2024.
- Lee et al. [2024]
Phillip Y Lee, Taehoon Yoon, and Minhyuk Sung.
Groundit: Grounding diffusion transformers via noisy patch transplantation.
*arXiv preprint arXiv:2410.20474*, 2024.
- Li et al. [2023]
Yuheng Li, Haotian Liu, Qingyang Wu, Fangzhou Mu, Jianwei Yang, Jianfeng Gao, Chunyuan Li, and Yong Jae Lee.
Gligen: Open-set grounded text-to-image generation.
In *CVPR*, 2023.
- Li et al. [2025a]
Yaowei Li, Lingen Li, Zhaoyang Zhang, Xiaoyu Li, Guangzhi Wang, Hongxiang Li, Xiaodong Cun, Ying Shan, and Yuexian Zou.
Blobctrl: A unified and flexible framework for element-level image generation and editing.
*arXiv preprint arXiv:2503.13434*, 2025a.
- Li et al. [2024]
Zhimin Li, Jianwei Zhang, Qin Lin, Jiangfeng Xiong, Yanxin Long, Xinchi Deng, Yingfang Zhang, Xingchao Liu, Minbin Huang, Zedong Xiao, et al.
Hunyuan-dit: A powerful multi-resolution diffusion transformer with fine-grained chinese understanding.
*arXiv e-prints*, 2024.
- Li et al. [2025b]
Zhong-Yu Li, Ruoyi Du, Juncheng Yan, Le Zhuo, Zhen Li, Peng Gao, Zhanyu Ma, and Ming-Ming Cheng.
Visualcloze: A universal image generation framework via visual in-context learning.
*arXiv preprint arXiv:2504.07960*, 2025b.
- Lin et al. [2024]
Han Lin, Jaemin Cho, Abhay Zala, and Mohit Bansal.
Ctrl-adapter: An efficient and versatile framework for adapting diverse controls to any diffusion model.
*arXiv preprint arXiv:2404.09967*, 2024.
- Lin et al. [2023a]
Jiawei Lin, Jiaqi Guo, Shizhao Sun, Zijiang Yang, Jian-Guang Lou, and Dongmei Zhang.
Layoutprompter: awaken the design ability of large language models.
*NeurIPS*, 2023a.
- Lin et al. [2023b]
Jinpeng Lin, Min Zhou, Ye Ma, Yifan Gao, Chenxi Fei, Yangjian Chen, Zhang Yu, and Tiezheng Ge.
Autoposter: A highly automatic and content-aware design system for advertising poster generation.
In *ACMMM*, 2023b.
- Liu et al. [2024a]
Bingchen Liu, Ehsan Akhgari, Alexander Visheratin, Aleks Kamko, Linmiao Xu, Shivam Shrirao, Joao Souza, Suhail Doshi, and Daiqing Li.
Playground v3: Improving text-to-image alignment with deep-fusion large language models.
*arXiv preprint arXiv:2409.10695*, 2024a.
- Liu et al. [2023]
Shilong Liu, Zhaoyang Zeng, Tianhe Ren, Feng Li, Hao Zhang, Jie Yang, Qing Jiang, Chunyuan Li, Jianwei Yang, Hang Su, et al.
Grounding dino: Marrying dino with grounded pre-training for open-set object detection.
*arXiv preprint arXiv:2303.05499*, 2023.
- Liu et al. [2025]
Tao Liu, Kai Wang, Senmao Li, Joost van de Weijer, Fahad Shahbaz Khan, Shiqi Yang, Yaxing Wang, Jian Yang, and Ming-Ming Cheng.
One-prompt-one-story: Free-lunch consistent text-to-image generation using a single prompt.
*arXiv preprint arXiv:2501.13554*, 2025.
- Liu et al. [2024b]
Zeyu Liu, Weicong Liang, Zhanhao Liang, Chong Luo, Ji Li, Gao Huang, and Yuhui Yuan.
Glyph-byt5: A customized text encoder for accurate visual text rendering.
In *ECCV*, 2024b.
- Ma et al. [2025a]
Jian Ma, Yonglin Deng, Chen Chen, Nanyang Du, Haonan Lu, and Zhenyu Yang.
Glyphdraw2: Automatic generation of complex glyph posters with diffusion models and large language models.
In *AAAI*, 2025a.
- Ma et al. [2025b]
Yuhang Ma, Shanyuan Liu, Ao Ma, Xiaoyu Wu, Dawei Leng, and Yuhui Yin.
Hico: Hierarchical controllable diffusion model for layout-to-image generation.
In *NeurIPS*, 2025b.
- Miao et al. [2025]
Yingmao Miao, Zhanpeng Huang, Rui Han, Zibin Wang, Chenhao Lin, and Chao Shen.
Shining yourself: High-fidelity ornaments virtual try-on with diffusion model.
*arXiv preprint arXiv:2503.16065*, 2025.
- Mildenhall et al. [2021]
Ben Mildenhall, Pratul P Srinivasan, Matthew Tancik, Jonathan T Barron, Ravi Ramamoorthi, and Ren Ng.
Nerf: Representing scenes as neural radiance fields for view synthesis.
*Communications of the ACM*, 2021.
- Mou et al. [2024]
Chong Mou, Xintao Wang, Liangbin Xie, Yanze Wu, Jian Zhang, Zhongang Qi, and Ying Shan.
T2i-adapter: Learning adapters to dig out more controllable ability for text-to-image diffusion models.
In *AAAI*, 2024.
- Mou et al. [2025]
Chong Mou, Yanze Wu, Wenxu Wu, Zinan Guo, Pengze Zhang, Yufeng Cheng, Yiming Luo, Fei Ding, Shiwen Zhang, Xinghui Li, et al.
Dreamo: A unified framework for image customization.
*arXiv preprint arXiv:2504.16915*, 2025.
- OpenBMB [2024]
OpenBMB.
Minicpm-v-2.6.
[https://huggingface.co/openbmb/MiniCPM-V-2_6](https://huggingface.co/openbmb/MiniCPM-V-2_6), 2024.
- Oquab et al. [2023]
Maxime Oquab, Timothée Darcet, Théo Moutakanni, Huy Vo, Marc Szafraniec, Vasil Khalidov, Pierre Fernandez, Daniel Haziza, Francisco Massa, Alaaeldin El-Nouby, et al.
Dinov2: Learning robust visual features without supervision.
*arXiv preprint arXiv:2304.07193*, 2023.
- Pan et al. [2025]
Yanjie Pan, Qingdong He, Zhengkai Jiang, Pengcheng Xu, Chaoyi Wang, Jinlong Peng, Haoxuan Wang, Yun Cao, Zhenye Gan, Mingmin Chi, et al.
Pixelponder: Dynamic patch adaptation for enhanced multi-conditional text-to-image generation.
*arXiv preprint arXiv:2503.06684*, 2025.
- Peebles and Xie [2023]
William Peebles and Saining Xie.
Scalable diffusion models with transformers.
In *CVPR*, 2023.
- Peng et al. [2025]
Yuyang Peng, Shishi Xiao, Keming Wu, Qisheng Liao, Bohan Chen, Kevin Lin, Danqing Huang, Ji Li, and Yuhui Yuan.
Bizgen: Advancing article-level visual text rendering for infographics generation.
In *CVPR*, 2025.
- Phung et al. [2024]
Quynh Phung, Songwei Ge, and Jia-Bin Huang.
Grounded text-to-image synthesis with attention refocusing.
In *CVPR*, 2024.
- Podell et al. [2024]
Dustin Podell, Zion English, Kyle Lacey, Andreas Blattmann, Tim Dockhorn, Jonas Müller, Joe Penna, and Robin Rombach.
Sdxl: Improving latent diffusion models for high-resolution image synthesis.
In *ICLR*, 2024.
- Pu et al. [2025]
Yifan Pu, Yiming Zhao, Zhicong Tang, Ruihong Yin, Haoxing Ye, Yuhui Yuan, Dong Chen, Jianmin Bao, Sirui Zhang, Yanbin Wang, et al.
Art: Anonymous region transformer for variable multi-layer transparent image generation.
In *CVPR*, 2025.
- Qin et al. [2023]
Can Qin, Shu Zhang, Ning Yu, Yihao Feng, Xinyi Yang, Yingbo Zhou, Huan Wang, Juan Carlos Niebles, Caiming Xiong, Silvio Savarese, et al.
Unicontrol: A unified diffusion model for controllable visual generation in the wild.
*arXiv preprint arXiv:2305.11147*, 2023.
- Radford et al. [2021]
Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al.
Learning transferable visual models from natural language supervision.
In *ICLM*, 2021.
- Raffel et al. [2020]
Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J Liu.
Exploring the limits of transfer learning with a unified text-to-text transformer.
*JMLR*, 2020.
- Ran et al. [2024]
Lingmin Ran, Xiaodong Cun, Jia-Wei Liu, Rui Zhao, Song Zijie, Xintao Wang, Jussi Keppo, and Mike Zheng Shou.
X-adapter: Adding universal compatibility of plugins for upgraded diffusion model.
In *CVPR*, pages 8775–8784, 2024.
- Rombach et al. [2022]
Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Björn Ommer.
High-resolution image synthesis with latent diffusion models.
In *CVPR*, 2022.
- Ruiz et al. [2023]
Nataniel Ruiz, Yuanzhen Li, Varun Jampani, Yael Pritch, Michael Rubinstein, and Kfir Aberman.
Dreambooth: Fine tuning text-to-image diffusion models for subject-driven generation.
In *CVPR*, 2023.
- Saharia et al. [2022]
Chitwan Saharia, William Chan, Saurabh Saxena, Lala Li, Jay Whang, Emily L Denton, Kamyar Ghasemipour, Raphael Gontijo Lopes, Burcu Karagol Ayan, Tim Salimans, et al.
Photorealistic text-to-image diffusion models with deep language understanding.
In *NeurIPS*, 2022.
- Shin et al. [2024]
Chaehun Shin, Jooyoung Choi, Heeseung Kim, and Sungroh Yoon.
Large-scale text-to-image model with inpainting is a zero-shot subject-driven image generator.
*arXiv preprint arXiv:2411.15466*, 2024.
- Shirakawa and Uchida [2024]
Takahiro Shirakawa and Seiichi Uchida.
Noisecollage: A layout-aware text-to-image diffusion model based on noise cropping and merging.
In *CVPR*, 2024.
- Song et al. [2021]
Jiaming Song, Chenlin Meng, and Stefano Ermon.
Denoising diffusion implicit models.
In *ICLR*, 2021.
- Song et al. [2024]
Kunpeng Song, Yizhe Zhu, Bingchen Liu, Qing Yan, Ahmed Elgammal, and Xiao Yang.
Moma: Multimodal llm adapter for fast personalized image generation.
In *ECCV*, 2024.
- stability.ai [2024]
stability.ai.
Stable diffusion 3.5.
[https://stability.ai/news/introducing-stable-diffusion-3-5](https://stability.ai/news/introducing-stable-diffusion-3-5), 2024.
- Sun et al. [2024a]
Quan Sun, Yufeng Cui, Xiaosong Zhang, Fan Zhang, Qiying Yu, Yueze Wang, Yongming Rao, Jingjing Liu, Tiejun Huang, and Xinlong Wang.
Generative multimodal models are in-context learners.
In *CVPR*, 2024a.
- Sun et al. [2024b]
Yanan Sun, Yanchen Liu, Yinhao Tang, Wenjie Pei, and Kai Chen.
Anycontrol: create your artwork with versatile control on text-to-image generation.
In *ECCV*, 2024b.
- Tan et al. [2024]
Zhenxiong Tan, Songhua Liu, Xingyi Yang, Qiaochu Xue, and Xinchao Wang.
Ominicontrol: Minimal and universal control for diffusion transformer.
*arXiv preprint arXiv:2411.15098*, 2024.
- Tarrés et al. [2025]
Gemma Canet Tarrés, Zhe Lin, Zhifei Zhang, He Zhang, Andrew Gilbert, John Collomosse, and Soo Ye Kim.
Multitwine: Multi-object compositing with text and layout control.
*arXiv preprint arXiv:2502.05165*, 2025.
- Tuo et al. [2024]
Yuxiang Tuo, Yifeng Geng, and Liefeng Bo.
Anytext2: Visual text generation and editing with customizable attributes.
*arXiv preprint arXiv:2411.15245*, 2024.
- Wang et al. [2025a]
Haoxuan Wang, Jinlong Peng, Qingdong He, Hao Yang, Ying Jin, Jiafu Wu, Xiaobin Hu, Yanjie Pan, Zhenye Gan, Mingmin Chi, et al.
Unicombine: Unified multi-conditional combination with diffusion transformer.
*arXiv preprint arXiv:2503.09277*, 2025a.
- Wang et al. [2024a]
Shaodong Wang, Yunyang Ge, Liuhan Chen, Haiyang Zhou, Qian Wang, Xinhua Cheng, and Li Yuan.
Prompt2poster: Automatically artistic chinese poster creation from prompt only.
In *ACMMM*, 2024a.
- Wang et al. [2024b]
Xudong Wang, Trevor Darrell, Sai Saketh Rambhatla, Rohit Girdhar, and Ishan Misra.
Instancediffusion: Instance-level control for image generation.
In *CVPR*, 2024b.
- Wang et al. [2024c]
Xierui Wang, Siming Fu, Qihan Huang, Wanggui He, and Hao Jiang.
Ms-diffusion: Multi-subject zero-shot image personalization with layout guidance.
*arXiv preprint arXiv:2406.07209*, 2024c.
- Wang et al. [2025b]
Zhendong Wang, Jianmin Bao, Shuyang Gu, Dong Chen, Wengang Zhou, and Houqiang Li.
Designdiffusion: High-quality text-to-design image generation with diffusion models.
In *CVPR*, 2025b.
- Wu et al. [2025]
Shaojin Wu, Mengqi Huang, Wenxu Wu, Yufeng Cheng, Fei Ding, and Qian He.
Less-to-more generalization: Unlocking more controllability by in-context generation.
*arXiv preprint arXiv:2504.02160*, 2025.
- Wu et al. [2024]
Yinwei Wu, Xianpan Zhou, Bing Ma, Xuefeng Su, Kai Ma, and Xinchao Wang.
Ifadapter: Instance feature control for grounded text-to-image generation.
*arXiv preprint arXiv:2409.08240*, 2024.
- Xiao et al. [2024]
Shitao Xiao, Yueze Wang, Junjie Zhou, Huaying Yuan, Xingrun Xing, Ruiran Yan, Chaofan Li, Shuting Wang, Tiejun Huang, and Zheng Liu.
Omnigen: Unified image generation.
*arXiv preprint arXiv:2409.11340*, 2024.
- Xu et al. [2023]
Jiazheng Xu, Xiao Liu, Yuchen Wu, Yuxuan Tong, Qinkai Li, Ming Ding, Jie Tang, and Yuxiao Dong.
Imagereward: Learning and evaluating human preferences for text-to-image generation.
In *NeurIPS*, 2023.
- Xue et al. [2023]
Han Xue, Zhiwu Huang, Qianru Sun, Li Song, and Wenjun Zhang.
Freestyle layout-to-image synthesis.
In *CVPR*, 2023.
- Yang et al. [2023]
Binbin Yang, Yi Luo, Ziliang Chen, Guangrun Wang, Xiaodan Liang, and Liang Lin.
Law-diffusion: Complex scene generation by diffusion with layouts.
In *ICCV*, 2023.
- Yang et al. [2025a]
Hongji Yang, Wencheng Han, Yucheng Zhou, and Jianbing Shen.
Dc-controlnet: Decoupling inter-and intra-element conditions in image generation with diffusion models.
*arXiv preprint arXiv:2502.14779*, 2025a.
- Yang et al. [2025b]
Han Yang, Chuanguang Yang, Qiuli Wang, Zhulin An, Weilun Feng, Libo Huang, and Yongjun Xu.
Multi-party collaborative attention control for image customization.
*arXiv preprint arXiv:2505.01428*, 2025b.
- Ye et al. [2023]
Hu Ye, Jun Zhang, Sibo Liu, Xiao Han, and Wei Yang.
Ip-adapter: Text compatible image prompt adapter for text-to-image diffusion models.
*arXiv preprint arXiv:2308.06721*, 2023.
- Zeng et al. [2024]
Yu Zeng, Vishal M Patel, Haochen Wang, Xun Huang, Ting-Chun Wang, Ming-Yu Liu, and Yogesh Balaji.
Jedi: Joint-image diffusion models for finetuning-free personalized text-to-image generation.
In *CVPR*, 2024.
- Zhang et al. [2024]
Hui Zhang, Dexiang Hong, Yitong Wang, Jie Shao, Xinglong Wu, Zuxuan Wu, and Yu-Gang Jiang.
Creatilayout: Siamese multimodal diffusion transformer for creative layout-to-image generation.
*arXiv preprint arXiv:2412.03859*, 2024.
- Zhang et al. [2025a]
Hong Zhang, Zhongjie Duan, Xingjun Wang, Yingda Chen, and Yu Zhang.
Eligen: Entity-level controlled image generation with regional attention.
*arXiv preprint arXiv:2501.01097*, 2025a.
- Zhang and Agrawala [2024]
Lvmin Zhang and Maneesh Agrawala.
Transparent image layer diffusion using latent transparency.
*arXiv preprint arXiv:2402.17113*, 2024.
- Zhang et al. [2025b]
Yinhan Zhang, Yue Ma, Bingyuan Wang, Qifeng Chen, and Zeyu Wang.
Magiccolor: Multi-instance sketch colorization.
*arXiv preprint arXiv:2503.16948*, 2025b.
- Zhang et al. [2025c]
Yuxuan Zhang, Yirui Yuan, Yiren Song, Haofan Wang, and Jiaming Liu.
Easycontrol: Adding efficient and flexible control for diffusion transformer.
*arXiv preprint arXiv:2503.07027*, 2025c.
- Zhang et al. [2025d]
Zechuan Zhang, Ji Xie, Yu Lu, Zongxin Yang, and Yi Yang.
In-context edit: Enabling instructional image editing with in-context generation in large scale diffusion transformer.
*arXiv preprint arXiv:2504.20690*, 2025d.
- Zhao et al. [2023]
Shihao Zhao, Dongdong Chen, Yen-Chun Chen, Jianmin Bao, Shaozhe Hao, Lu Yuan, and Kwan-Yee K Wong.
Uni-controlnet: All-in-one control to text-to-image diffusion models.
*NeurIPS*, 2023.
- Zheng et al. [2023]
Guangcong Zheng, Xianpan Zhou, Xuewei Li, Zhongang Qi, Ying Shan, and Xi Li.
Layoutdiffusion: Controllable diffusion model for layout-to-image generation.
In *CVPR*, 2023.
- Zhong et al. [2019]
Xu Zhong, Jianbin Tang, and Antonio Jimeno Yepes.
Publaynet: largest dataset ever for document layout analysis.
In *ICDAR*, 2019.
- Zhou et al. [2024]
Dewei Zhou, You Li, Fan Ma, Xiaoting Zhang, and Yi Yang.
Migc: Multi-instance generation controller for text-to-image synthesis.
In *CVPR*, 2024.
- Zhou et al. [2025]
Dewei Zhou, Mingwei Li, Zongxin Yang, and Yi Yang.
Dreamrenderer: Taming multi-instance attribute control in large-scale text-to-image models.
*arXiv preprint arXiv:2503.12885*, 2025.
- Zhu et al. [2025]
Chenyang Zhu, Kai Li, Yue Ma, Chunming He, and Xiu Li.
Multibooth: Towards generating all your concepts in an image from text.
In *AAAI*, 2025.

## Peer Reviews

confidence: 4
contribution: 2
initial_rating: 4
presentation: 3
questions: Please see the weaknesses.
soundness: 2
strengths: - The paper tackles a real-world problem with clear applications in graphic design creation. 
- It contributes a large-scale dataset with multi-condition annotations. 
- The curated benchmark can well evaluate the design quality by including multiple metrics covering different aspects.
- The experimental results show clear improvements of CreatiDesign over baselines.
summary: This paper presents CreatiDesign, a unified multi-conditional diffusion transformer for graphic design generation. CreatiDesign can receive inputs from different modalities: keyword, layout, prompt, image, etc. To support the task setting, the authors develop an automated pipeline for constructing a 400K graphic design dataset with multi-condition annotations. Using the dataset, they fine-tune the FLUX.1-dev model, where they modify the original attention strategy to enable multi-condition interaction. Experimental results demonstrate CreatiDesign's superior performance over existing methods in design generation.
weaknesses: 1. The technical contribution of CreatiDesign is limited. The used attention masking, LoRA fine-tuning are straightforward applications of existing techniques.
2. The model setting requires users to specify exact positions for the elements, making final quality heavily dependent on user design expertise. However, most users may lack such knowledge, thereby limiting its practical applicability. 
3. In the experiments, it seems that the paper only evaluates scenarios where all conditions are simultaneously provided. It would be great if the authors could conduct more experiments using the same model: (1) text-to-design generation (intention-to-design without visual inputs), (2) content-aware text layout generation (placing text on given background images), etc. Evaluating diverse conditioning inputs would demonstrate practical utility across different design workflows and strengthen the effectiveness of CreatiDesign.
4. The dataset details are not included in the paper, such as the element number, prompt length, diversity, quality, etc.
5. How does CreatiDesign's inference time compare to other methods?

---

confidence: 3
contribution: 2
initial_rating: 6
presentation: 2
questions: Can this method be extended to support placing given visual elements in appropriate locations in the design? This is similar to traditional layout generation/variation problem. The visual knowledge can give an edge over geometry based methods.
soundness: 3
strengths: + a multimodal controllable generation model developed based on Flex T2I model
+ block-wise multi-modality attention achieves both controllability and prevents information leakage
+ a new benchmark dataset with curated high quality 1k set
summary: This paper introduces CreatiDesign, a multi-condition diffusion model designed for automated graphic design generation. ​ It integrates flexible multimodal inputs, including global prompts, rendered text, foreground visuals, background elements, and their layout positions. ​ A masked attention mechanism ensures precise control, aligning layout tokens with corresponding visual elements while preventing interference. 
The authors developed a new dataset with 400K synthetically generated samples and a benchmark of 1,000 annotated samples with ground truth labels. CreatiDesign is shown to outperform existing models in various design metrics.
weaknesses: - DiT framework is similar to previous work: Art: Anonymous region transformer for variable multi-layer transparent image generation. In CVPR, 2025.
- controllability is limited, only with given images and precise layout condition, text layer is not vector font. It would be useful to allow flexible layout position in output.
- the design images in the benchmark is synthetically constructed, the quality is not guaranteed.

---

confidence: 4
contribution: 2
initial_rating: 4
presentation: 2
questions: It is tedious for a designer to determine the position of bbox of each text in a poster,  why not directly predict the position of these boxes as did in recent CVPR papers, for instance, Unsupervised domain adaption with pixel-level discriminator for image-aware layout generation in CVPR. Overall, this paper leans towards a graphic-design rendering paper more than a design paper, since the layout should be input by a user.
soundness: 2
strengths: 1.  The proposed multimodal attention mask is effective in control the behavior of the diffusion model as verified in the ablation study. Both layout attention mask and subject attention mask are important to the quality of the graphical design. 
2. The diffusion model trained with multi-condition dataset can automatically support editing by using the image generated in the previous steps as the condition. While the STOA T-2-I models can support editing to some extent, the controllability of the proposed method should be better by associated bounding box in the graphic design.
summary: This paper presents a multi-conditioned creative graphic design method, where the conditions include semantic descriptions, positions and b-boxes of objects and texts to be placed on the image. The method encodes each condition into tokens and then employ a  multimodal attention mask to prevent the attention procedure from information confusion or leaking. Experimental results demonstrates the designed posters are in high quality.
weaknesses: The overall pipeline is built upon masked attention mechanism. However, masked attention mechanism is widely used in model design. For example, in NLP, casual attention is often used to avoid a token taking information from future tokens. From the perspective of novelty,  I hesitate to give this paper a high score.  In addition, the advantage of the proposed method are limited to the graphical design due to the specific definition of input conditions.

---

confidence: 4
contribution: 3
initial_rating: 6
presentation: 3
questions: 1. For entity annotation, take Fig.2 as an example, how to separate different entities as primary or secondary visual elements? Will the generated graphic design be affected by considering different entities as primary or secondary visual elements?
2. The proposed dataset is constructed via a fully automatic pipeline. Several existing models are used for design theme generation, text layer rendering, and foreground-based image generation. How to ensure the quality of the constructed dataset without involving human annotations or double-checking?
soundness: 2
strengths: The proposed method that integrates multiple heterogeneous conditions in a unified manner for graphic design generation is reasonable and effective. The proposed large-scale, multi-element graphic design dataset could also be useful for future research in the community if released.
summary: This paper focuses on the graphic design generation task. A unified multi-condition driven architecture is proposed by integrating different user-provided elements with a multimodal attention mask mechanism. A new graphic design dataset with 400k samples featuring multi-condition annotations, along with a comprehensive benchmark. The experimental results demonstrate the effectiveness of the proposed method in adhering to user intent.
weaknesses: 1. The current evaluations mainly compare the proposed method with existing MLLMs for controllable image generation. However, there is a domain gap between natural images and graphic designs. To make fair comparisons, existing multi-condition or single-condition graphic design generation works, e.g., [1-4], should also be discussed and compared.
2. The user study may not sufficiently demonstrate the practical effectiveness of the proposed method. The motivation of this paper is to ensure the strict adherence between the generated design and the user intent. Thus, the user study should investigate from the above criteria and ask participants to create inputs by themselves, rather than predefining inputs and following similar evaluation metrics used in quantitative comparisons.
3. Since the proposed dataset is also the contribution of this paper, the ablated versions of the automated pipeline for graphic design dataset construction should also be studied. For example, using different methods or changing the module order for the data construction process.

[1] Hsu, HsiaoYuan, and Yuxin Peng. "PosterO: Structuring Layout Trees to Enable Language Models in Generalized Content-Aware Layout Generation." CVPR 2025.

[2] Horita, Daichi, et al. "Retrieval-augmented layout transformer for content-aware layout generation." CVPR 2024.

[3] Seol, Jaejung, Seojun Kim, and Jaejun Yoo. "Posterllama: Bridging design ability of language model to content-aware layout generation." ECCV 2024.

[4] Yang, Tao, et al. "Posterllava: Constructing a unified multi-modal layout generator with llm." arXiv preprint arXiv:2406.02884.


Decision: Accept (Poster)