AttriCtrl: A Generalizable Framework for Controlling Semantic Attribute Intensity in Diffusion Models
Authors: Die Chen
Zhongjie Duan
Zhiwen Li
Cen Chen
Daoyuan Chen
Yaliang Li
Yinda Chen
Venue: ICLR 2026 Poster (2026)
Keywords: Diffusion; Control Generation
Primary area: generative models

Abstract: Diffusion models have recently become the dominant paradigm for image generation, yet existing systems struggle to interpret and follow numeric instructions for adjusting semantic attributes. 
In real-world creative scenarios, especially when precise control over aesthetic attributes is required, current methods fail to provide such controllability. 
This limitation partly arises from the subjective and context-dependent nature of aesthetic judgments, but more fundamentally stems from the fact that current text encoders are designed for discrete tokens rather than continuous values. 
Meanwhile, efforts on aesthetic alignment, often leveraging reinforcement learning, direct preference optimization, or architectural modifications, primarily align models with a global notion of human preference. While these approaches improve user experience, they overlook the multifaceted and compositional nature of aesthetics, underscoring the need for explicit disentanglement and independent control of aesthetic attributes.
To address this gap, we introduce AttriCtrl, a lightweight framework for continuous aesthetic intensity control in diffusion models. 
It first decomposes relevant aesthetic attributes, then quantifies them through a hybrid strategy that maps both concrete and abstract dimensions onto a unified $[0,1]$ scale. A plug-and-play value encoder is then used to transform user-specified values into model-interpretable embeddings for controllable generation.
Experiments show that AttriCtrl achieves accurate and continuous control over both single and multiple aesthetic attributes, significantly enhancing personalization and diversity.
Crucially, it is implemented as a lightweight adapter while keeping the diffusion model frozen, ensuring seamless integration with existing frameworks such as ControlNet at negligible computational cost.

Published as a conference paper at ICLR 2026
ATTRICTRL: A GENERALIZABLE FRAMEWORK FOR CONTROLLING SEMANTIC ATTRIBUTE INTENSITY IN DIFFUSION MODELS
Die Chen' Zhongjic Duan
Zhiwen Li
Cen Chen
Daoyuan Chen
Yaliang Li² Yingda Chen?
Schooo of Data Science and Engineering. East China Normal University Alibaba Group
dchenêstu. sernnnsttueccu.... ecnu. edu. en duanzhongjie.dz dwweeaiiaaaa-bbaa com
ahivenli@stu.ecnu edu. cenchenddase www.eddaseeeee.....oo.ooooe edu. daoyuanchen, cwwphhhhhhnncyyaallaelalaaa inc. com yaliang li@alibaba -ataaabaainnc.... com yingda. www.eachleaaaa-nnnnnnn com
ABSTRACT
Diffusion models have recently become the dominant paradigm for image gen- eration, yet existing systems struggle to interpret and follow numeric instructions for adjusting semantic attributes. In real-world creative scenarios, especially when precise control over aesthetic attributes required, current methods fail to provide such controllability. This limitation partly arises from the subjective and context- dependent nature of aesthetic judgments, but more fundamentally stems from the fact that current text encoders designed for discrete tokens rather than tinuous values. Meanwhile, efforts on aesthetic alignment, often leveraging rein- forcement learning, direct preference optimization, architectural modifications, primarily align models with a global notion of human preference. While these approaches improve user experience, they overlook the multifaceted and compo- sitional nature of aesthetics, underscoring the need for explicit disentanglement and independent control of aesthetic attributes. To address this gap, we introduce AtriCCtl,, a lightweight framework for continuous aesthetic intensity control in diffusion models. first defines relevant aesthetic attributes, then quantifies them through hybrid strategy that maps both concrete and abstract dimensions onto a unified [o, 1] scale. user-specified values into model cooeel-iteepreetable embeddings for controllable gener- ation. Experiments show that AttriCtrl achieves accurate and continuous control over both single and multiple aesthetic attributes, significantly enhaneing person- alization and diversity. Crucially, it is implemented as lightweight adapter while keeping the diffusion model frozen, ensuring seamless integration with existing frameworks such as ControlNet at negligible computational cost.
blug-and-play value encoder then used to transform
INTRODUCTION
Diffusson models have emerged as a dominant paradigm in image generation due to their stable training dynamics and strong generative performance (Nichol et al. 202T]] Ho et al. 2020 Ramesh
et al. 2022). Building on these advances, large-scale pretrained variants and their control frame- works have recently pushed the frontier of personalized and controllable image synthesis (Zhang et al. 2023 Hertz et al. 2022 Ye et al. 2023 Mou et al. 2024). Despite this remarkable progress, current systems remain limited their ability to understand and follow numeric instructions for adjusting semantic attributes, especially in scenarios that demand precise control over aesthetic at- tributes. which severely constrains their applicability in real-world creative workflows. Consider a professional photographer who wishes to adjust image's atmosphere by making it exactly 20% dimmer rather than issuing a vague request like "make darker." Or a children's book illustrator who needs generate visuals for different age groups, requiring precise control over the degree of cartoon-like abstraction. Similarly, content creators often need subtle refinements such as "slightly
"Corresoonding author.

---

Published as a conference paper at ICLR 2026
Effects of Add to Prompt and Control with Kontext
Add
Control with Kontext
Centrol with Kontest
Brightness dog"
Detail
Origeal
brightson Erightnee
106
Effects of AttriCtrl
Brighttees
Detail
Realism
Safety
Figure 1: Overview. Methods such as 'Add to Prompt' and 'Control with Kontext' fail to establish stable or reliable attribute control. In contrast, our proposed AttriCtrl enables fine-grained control over aesthetic attributes by modulating their intensity in the generated image.
sharper textures" "a touch more realism." " As shown in Figure current models fail to interpret such comparative or gradable instructions, leading to outputs that are misaligned with user intent.
From an application perspective, this limitation stems from the subjective and context-dependent nature of aesthetic preferences, as judgments can vary widely across individuals or even for the same user depending on emotion task. More fundamentally, this limitation stems from mismatch: current text encoders are designed for discrete tokens rather than continuous values, which makes it inherently difficult capture and control aesthetic intent (Raffel al. 2020 Radford et al. 2021).
To align generative models with human preferences, recent work has sought to use feedback- optimization. Reinforcement learning (Kirstain al. 2023 Liang et al. 2024) and direct prefer- ence optimization (DPO) (Fan et al. [0233 Wallace et al. 2024) leverage human-labeled data to train reward models that bias generation toward preferred outcomes, but these approaches rely on high-quality annotations and incur significant computational costs. Alternative efforts aim to im- prove model architecture by integrating modular components (Si al.) 2024). However, the core limitation of all these methods is that they operate under global preference alignment paradigm, implicitly assuming single optimal target. This overlooks the multifaceted and context-dependent nature of aesthetic judgment and lacks mechanisms for disentangling and precisely controlling indi- vidual attributes. Other strategies, such as latent-space interpolation (He et al. 2024).. blend features between two discrete endpoints, but lack explicit guidance the attribute's semantic manifold, producing artifacts or collapsing structures. This motivates a central research question: How can generative models disentangle aesthetic attributes, understand them as continuous values, and smoothly navigate their intensity in a user-controllable manner? We address this challenge in the context of aesthetic intensity control. Instead of relying on undifferentiated preference signals, we explicitly decompose aesthetic attributes, quantify them along continuous dimensions, and en- able users modulate their intensity through numeric instructions.
To this end, we introduce AttriCtrl, a framework comprising two key components. First, it quanti- fies aesthetic attributes through a hybrid strategy that combines direct metrics with vision-language semantic similarity, capturing both concrete (e.g., brightness, detail) and abstract (e.g.. realism, safety) dimensions. Second, it provides a lightweight, plug-and-play adapter for continuous and fine-grained modulation of aesthetic intensity. Specifically, AttriCtrl incorporates value encoder that maps scalar intensity values into semantically meaningful embeddings. This encoder trained on curated subsets of image-tes, text pairs while keeping the base diffusion model frozen, incurring min- imal computational overhead and allowing seamless integration into existing controllable generation pipelines. By learning continuous and navigable trajectory for each attribute within the model's
2

---

Published as a conference paper at ICLR 2026
conditioning space, approach achieves disentangled, attribute-spedt specific control vectors, enabling smooth transitions and precise modulation while remaining independent of other factors.
Extensive experiments show that AttriCtrl delivers accurate and continuous control across single or multiple attributes, improving both personalization and diversity. Furthermore, its compatibility with widely adopted frameworks such ControlNet (Zhang et al., 2023) highlights its versatility and potential for real-world deployment. While our study focuses on aesthetic attributes, the value encoder paradigm can be generalized to any semantic attribute (e.g., object count, size ratio, color temperature), positioning it as a foundation for a broader class of scalar-conditioned controls. Our code are published httpp::/// httpthtthhuhccooo com/ wttp.//////2////ttt..eeo.
2 RELATED WORK
Controllable Generation. Controllable generation has become a central research direction in the recent progress of diffusion models (Sohl-Dickstein et al. [015 Nichol al. [022]. J. aiming to pro- vide users with finer control and greater customization over the image synthesis process. A widely
explored form of control relies on natural language prompts (Brooks et al. 2023 Avrahami et al.,
2022 Hertz et al. [022]] Voynov et al. 2023b] Xiao et al.]
2025b) [Sheynin et al.
2024; Batifol
cct al. 2025). In particular, methods such as Prompt-to-Prompt (Hertz et al. (022) and P+ (Voynov
2023b) manipulate cross-attention layers to steer the semantic content of generated images. Moreover, this paradigm has been extended by instruction-
al.
image editing frameworks, includ- ing InstructPix2Pix (Brooks al. 2023). EMU-Edit (Sheynin al., 2024), and OmniGen (Xiao) ct al. 2025b). These approaches enable users to perform precise and intuitive image modifications through natural language commands, thereby improving both usability and contextual flexibility. Another complementary line of work explores explicit structural signals, such as depth maps, edge maps. sketches. and segmentation masks (Voynov et al. 2023a Meng et al. 2021 Kumari et al.)
2023 Ruiz et al.] [2023] Xiao et al.] [025aa] Zhhng et al. 2023 Mou et al.J 2024). Representative methods include ControlNet (Zhang et al. 2023) and T21 T21 121-Adppter (Mou et al.) 2024), which attach
lightweight auxiliary modules to pretrained diffusion models without retraining the core network.
Aesthetic Modeling, While these controllable generation techniques enable fine-grained control over semantic content, they remain limited in manipulating aesthetic or numerical attributes. Several approaches have attempted to introduce aesthetic control. For example, methods like DPOK (Fan) et al. 2023) and Diffusion-DPO (Wallace et al.] 2024) adapt direct preference optimization to fine- tune diffusion models based on human feedback, which requires substantial human annotations and computational resources. From a model-architecture perspective, FreeU (Si et al. 2024) enhances the U-Net backbone of diffusion models to preserve gigh-frequency details and visual quality with- out incurring additional computational cost. However, all these approaches focus improving the global preference alignment for single optimal target, rather than decomposing it into fine-grained attributes. They largely overlook the fact that aesthetic preferences tifaceted. When it comes to controlling attribute intensity, the most straightforward strategy is to directly specify the desired values the prompt or instruction. Yet text encoders are often insensi- tive to such numerical information (Raffel et al. 2020; Radford et al. 202I), which makes it difficult to produce consistent and comparable results, especially in the absence of a unified definition of at- tribute intensity. A related work, AID (He et al. 2024). attempts to interpolate between two images by applying weighted operations to attention layers. However, this method operates without any explicit guidance along the attribute manifold, which often leads to visual artifacts. Therefore, our AutiCttl proposed to address these limitations and enable controllable generation along specific aesthetic attributes with adjustable intensity.
inherently dynamic and mul-
3 METHOD
To achieve precise control over aesthetic attributes, we decompose the problem into two components. In Section 3.1 quantify each attribute and normalize its raw measurement into a scalar within 1]. In Section 3.2, we introduce a lightweight value encoder that converts these scalars into semantically meaningful embeddings, injected into the diffusion process to guide generation.
3

---

Published as a conference paper at ICLR 2026
Brightocc
Ron Th ncc 3J
Ran Maann
Rin To B.
Ham Trylect
Rin
Ran Valen
Rus Safery
Rin
New Valai 6.2
Rom Value dat Norra Valuer
Baw Wila 00
Rew Vila 6.13
Rew Vila 0.55 Natue
How Yake Valles
How howkek 4.8 Narra Valler
R.!.
Raw Silacou Narta
Row 16l
Narta Valie
Figure 2: Examples of aesthetic attribute intensities in the training dataset. We show the raw values computed via quantitative metries and the normalized values after value mapping, scaled the [0, 1].
3.1 AESTHETIC ATTRIBUTE QUANTIFICATION
We define four semantic attributes that are closely related to human perceptual preferences: bright- ness, detail, realism and safety. The inclusion of the safety semantic attribute motivated by the need for tiered safety control to accommodate users across different age groups. Given the sensitive nature of safety-related content, recommend that its intensity level be configured exclusively by system administrators rather than exposed directly to end users.
To quantify these attributes, we adopt a hybrid strategy. For conerete attributes such as bright- ness and detail, we apply direct metric-based estimation. For more abstract and semantic attributes like realism and safety, we leverage pretrained vision-language models to compute cross mooal similarity between images and descriptive text prompts. Figure presents examples of attribute quantification results on a subset of the training dataset.
Direct Estimation. Brightness is estimated in the HSV (Hue, Saturation, Value) color space. We extract the Value channel, which directly corresponds to perceived brightness, and compute its mean pixel intensity normalized by 255, the maximum possible value in 8-bit encoding. This produces a raw brightness value within the range [0, 1], where 0 indicates complete darkness and indicates maximum brightness. Formally, for image I, the brightness intensity value is defined as:
Brightness
U
H.W W
255
where Uiy denotes the value channel of the pixel (i,j) in the HSV representation, and H, W are the height and width of the image, respectively.
For detail, adopt Shannon entropy the quantification metric, which find be an effective and computationally efficient proxy for perceptual detail in our training regime, validated through human studies and shown outperform alternatives like frequency-domain analysis in our exper- iments (see Appendix A for detailed comparison and justification). While entropy may be in- fluenced by noise and does not explicitly capture structural complexity, it nevertheless provides reliable correlate of textural richness in natural images. This suitability stems from the fact that visual detail arises not from isolated structures but from cumulative variations in luminance across the image. High entropy values indicate rich diversity of luminance levels, typically corresponding to structural elements such as edges, textures, shadows, and fine details, which aligns closely with human sensitivity to local contrast and intensity changes. To compute this measure, the image is first converted to grayscale to remove chromatic variations and emphasize structural content. his- togram over 256 grayscale levels is then constructed and normalized into probability distribution. The raw intensity value of detail is defined as the entropy of this distribution:
256
Detail
Entropy(Hist(())
Σ pa log(ps),
(2)
k=
where P& denotes the probability of the k-th grayscale intensity level in image
4

---

Published as a conference paper at ICLR 2026
Similarity-Bane Based Estimation. For abstract aesthetic attributes such realism and safety, direct quantification is inherently difficult because they rely on high-level semantic understanding and contextual interpretation. To address this challenge, leverage the multimodal representation capabilities of pretrained vision-language models, specifically CLIP (OpenAl (022), which encode both images and text into shared emhedding space. We compute the cosine similarity between an image embedding e and a set of carefully crafted textual prompts describing the target attribute, and use the resulting value as proxy for the attribute's ' intensity. The similarity is defined as:
sim(er,
e1 ET
| ||TT||
To quantify realism, we define set of positive and negative prompts. The positive prompt is "a real photograph, realistic details and natural lighting" (Cpes), and the negative prompt is "a cartoon image, human-created artistic representation, such an illustration or painting" (Cneg). Through empirical evaluation of several prompt pairs, we find this contrasting set to yield the most stable and perceptually aligned realism scores across our dataset. We encode these prompts into text embed- dings eps and Eneg using the text encoder of CLIP and obtain the image embedding via its image encoder. The realism intensity value is then defined as:
Realism sim(er, (pos) - sim((er Eneg); I1
where higher values indicate stronger semantic alignment with realistic photographic content.
For safery, defining a comprehensive positive description of safe content is inherently challenging, as safety is typically characterized not by the presence of acceptable elements, but by the absence of harmful or inappropriate ones. Rather than attempting to define an absolute notion of safety, we align with a pre-defined standard by leveraging the internal safety checker of Stable Diffusion (Comp Vis 2022). Specifically, we extract its textual embedding e, for unsafe concepts (e.g., explicit nudity) and compute the cosine similarity with the image embedding et as:
(5) where the threshold t defines the maximum allowable similarity (set to 0.19, consistent with the checker's 's default classification boundary). To ensure consistency with the directionality of other aesthetic attributes, we invert the value by taking its negative. This transformation centers the score around the safety threshold & and inverts it, such that scores greater than zero correspond to safe images, with higher values indicating a larger margin of safety.
Safoty -siim(er, ex) - t),
Value Mapping. Our goal is to make attribute values both uniformly distributed for training and comparable across different attributes. To this end, we first address distribution imbalance within each attribute. The empirical value range is divided into 10 equal-width bins based on dataset statistics, and balanced sampling strategy is applied: underrepresented bins oversampled with replacement, while overrepresented bins are randomly downsampled. This procedure yields a more uniform set of training samples while preserving the ordinal structure of the original distribution. After balancing, the raw attribute values v; are normalized onto a shared [0, 1] scale via rank- normalization, enabling consistent multi-atrribue control. Specifically, given raw value æ, from collection of n training samples (II,..., En), we compute its normalized counterpart as:
rank - 0.5 E [0,
norm
(6)
where rank(z;) denotes the average rank of x; among the n samples.
3.2 TAILORED AESTHETIC CONTROL
Preliminaries. Diffusion models learn to generate images by modeling the denoising process of a latent variable corrupted by Gaussian noise. Given an input image 1, it is first encoded by a variational encoder E into a latent representation = E(!). During training, Gaussian noise is progressively added to 2 over timesteps, producing a noisy latent 2, at each step. A denoising network Eœ is then trained to predict the added noise, enabling the model to gradually reconstruct the original image from pure noise. For eoxt-to-image generation, a text prompt is encoded into contextual embedding e using text encoder. This embedding integrated into the denoising
5

---

Published as a conference paper at ICLR 2026
Single Attribute
Attributes
Prompt
Value Encuder
85
Valao
Brigeness
Detal 64
Redicc
0.8
Neewekid intodding
Value Ennsder
Value Encoder
Value Encoder
Value Encoder 3
Value Enooder
MLP
User
SEU
Hiuer
Ennteking
Figure 3: Framework. We trains value encoder that maps normalized attribute intensity value to multi-scale representations, which are concatenated with text prompts and injected into the DIT.
process via attention modules, allowing the model to align image synthesis with the semantics of the prompt. After denoising, the latent is passed through decoder to reconstruct the final image.
Inspired by previous works (Mou al. 2024), we design an independent value encoder to transform the normalized intensity value ar into a learnable token sequence for each aesthetic attribute, as illustrated in Figure Specifically, the encoding begins with sinusoidal embedding mechanism. originally used for timestep encoding in diffusion models. Its smooth in- terpolation properties make it well-suited for converting continuous aesthetic values into structured high-dimensional vectors. The resulting embedding is passed through a two-layer multilayer per- ceptron (MLP) with SiLU activations and becomes hidden representation, which is then duplicated and expanded into a fixed-length sequence. This expansion into a sequence of tokens is crucial, as allows the model's self-attention layers process the scalar intensity value in distributed and relational manner, analogous to how it interprets sequence of text tokens. We then add a learnable positional embedding to the repeated representations obtain enables the model assign dis- tinct functional roles to each token in the sequence, creating more expressive representation than single conditioning vector would allow. As result, the encoder can capture richer attribute-specific information, which facilitates more expressive and fine-grained control during generation. The final embedding v is concatenated along the sequence dimension of the text embedding c, forming a joint representation that enters the backbone of the diffusion model. The training objective becomes:
Value Encoder.
((0) = E.,.c.et [Ilc- flc-éélfe Eo(z,, t)](
The design of value encoder allows aesthetic control information to be seamlessly integrated, pro- viding strong compatibility and extensibility for downstream tasks.
Multi- Aalli--Attriutte Composition. A straightforward approach to multi-attribute aesthetic control is to merge single-attribute datasets and jointly train value encoders for all attributes. However, this often results in data imbalance across attributes, hindering training stability and convergence. To address this, we adopt modular strategy: each aesthetic attribute is first encoded independently using its corresponding value encoder trained on single-attribute data. At inference time, the resulting em- beddings are concatenated in sequence and appended the text embedding. This composite embed- ding enables joint conditioning on multiple aesthetic dimensions within unified framework. Such a modular design leverages the composability of independently trained value encoders, allowing for flexible, plug-and-ple integration of aesthetic attributes while minimizing mutual interference.
4 EXPERIMENTS
In this section, we present systematic experiments to evaluate the effectiveness of AttriCtrl across multiple aesthetic attributes, as well as compatibility with existing controllable frameworks.
4.1 EXPERIMENTAL SETUP
Implementation Details. We adopt FLUX Labs 2025) the base model and integrate it with our proposed value encoder module. The encoder is optimized using AdamW with fixed learning rate of 1 x 10-5 and outputs fixed-length sequence of 32 tokens. This architecture, particularly the

---

Published as a conference paper at ICLR 2026
Table 1: Left: We measure control accuracy using the average absolute difference (AvgDiff J) between the target and result attribute intensity values. Right: User preference study. Participants were shown sequences of images with increasing attribute intensity from each method and asked to select the one demonstrating the most accurate, smooth, and high-quality progression (N=10
participants, 100 comparisons).
Control Accuracy (AvgDiff L)
User Study (The proportion of selected 1)
Method
Bright.
Detail
Realism
Avg
Method
Bright.
Detail
Realism
Avg
Kontext
0.294
0.420
0.270
0.328
Kontext
0.021
0.006
0.006
0.011
W-Emb
0.327
0.436
0.271
0.345
W-Emb
0.024
0.015
0.015
0.018
AID-in
0.214
0.361
0.227
0.267
AID-in
0.074
0.067
0.076
0.072
AID-out 0.214
0.361
0.227
0.267
AlD-out
0.047
0.058
0.064
0.056
Ours
0.141
0.191
0.192
0.175
Ours
0.835
0.852
0.839
0.842
sequence length and the use of positional embeddings, was determined through ablation studies to provide the optimal balance of representational capacity and control accuracy (see Appendix for details). All experiments are conducted on four NVIDIA A100 GPUs. During inference, images are generated at resolution of 1024x1024 using 30 denoising steps.
Datasets and Metrics. We use EliGen (Zhang al., 2025) as the training corpus and sample 155K image-text pairs with high semantic diversity. For validation, we adopt GenEval (Ghosh et al., 2023), which consists of 553 prompts. Each prompt from this benchmark combined with eight different random seeds to produce eight images, with a randomly sampled target attribute intensity value Ptarget E [0, 1] assigned to each image during generation. The raw attribute value is extracted from the generated image and normalized to Uresalt [0, 1] via quantile-based mapping derived from the training set. This mapping aligns each predicted raw value with the closest match in the training distribution. Control accuracy is measured using the average absolute difference between the target and generated attribute values:
N
AvgDiff
1
(i) Marrgtt
(i) resullt
(8)
N
While we mainly focus three aesthetic attributes (brightness, realism, and detail), also extend our framework to safety control. To train the safety value encoder, we construct a dedicated dataset. Specifically, we NSFW adversarial prompts from RAB (Tsai images, and generate another 50K safe images using the neutral prompt "A person wearing clothes." Each image assigned raw safety value, and these values are discretized into 10 equal-width bins. We then apply resampling to obtain exactly 10K samples per bin. For evaluation, we adopt the I2P prompt set (Schramowski et al. 2023) and generate one image for each of its 4703 prompts, with the target safety intensity value fixed to to enforce maximum suppression of unsafe content. The performance measured by the removal rate (RR), defined as:
2023) generate 50K unsafe
RR
(9)
Th
where no is the number of unsafe images generated by the base model and is the number of unsafe outputs after applying safety control. higher RR indicates stronger suppression.
Baselines. As there is no existing method for fine fine-grainee attribute intensity control, we compare
our approach with several representative control strategies: (1) Prompt-based control via instruction- driven generation (Kontext). (2) Interpolation- -based control (AID), including AID-in (interpolating within the key and value result of attention) and AlD-out (interpolating on attention outputs). (3) Weighted encoding (W-Emb), where we train two fixed embeddings using the top-2000 and bottom- 2000 images in attribute intensity, and linearly combine them at inference as w Chigh +11 111-0 w Ctow- For safety, compare with existing concept erasure methods, including NP (Ho & Salimans, 2022) SLD (Schramowski et al. 2023). and ESD (Gandikota et al., (203)). All baselines are implemented on top of Flux and detailed configurations of all baselines provided in Appendix

---

Published as a conference paper at ICLR 2026
Brightmess
Detail
Realism
9.383
Target 0464
BBB 3.30
Bain w9
Bam 1110
MMB 0407
Iwin F-XX
IMR 2-35
W-Emb
TAAL
ATD-in
ATD-out
IKh 0.115
Iith 0.39
DM 0:300
B9th 3.372
DMP 3.24
DVR DAST
INR E50
DRR 2.33
BAlL UA022
Figure 4: Qualitative comparison of different control methods. Given target attribute intensity value, we visualize the absolute difference (Diff J) between the generated images and the target.
NP
SLDD
ESD
Ours
Expased Body Parts (in FLUX)
Total(291)
0.12
|0,33
0.54
0.58
Buttocks(41)
3reast2Br
0.12
0,29
0.78
0.53
Female_ (2naala)enitallaa22)
Male, Breasttr3)
0.33
0.88
0.97
0.44
(alla
0.55
207
2
2
0
N0
Figure 5: Performance comparison on the 12P dataset. AtrrCCrl achieves total removal rate (RR
1) of 57.7%, outperforming all baselines, including ESD (53.9%), SLD (32.6%) and NP (11.6%).
4.2 SINGLE-ATTRIBUTE CONTROL.
Control Accuracy of Aesthetic Attributes. As shown the left side of Table [ report the AvgDiff performance all baseline methods. Representative qualitative examples are provided in Figure where all methods evaluated under identical random seeds and target intensity values; within each method, images are generated under different seeds and intensity values. The figure also presents the absolute difference (Diff) between the generated and target intensity values for each image, where smaller Diff' values indicate better control.
Among the baselines, the next-best approach (AID-in/out) achieves moderate accuracy, with remaining above 0.21 across all attributes. However, as shown in several cases (Appendix D). we observe that its interpolation process occasionally degrades generation quality, producing artifacts such as halos or structural collapse. We attribute this to the lack of explicit attribute guidance during intermediate steps, which can lead entanglement of multiple attributes during generation. Kon- text and W-Emb rely prompt-based based or static embedding strategies, showing only weak control ability. Their AvgD values exceed 0.32, indicating poor precision in attribute targeting. These ob- servations highlight the necessity of explicitly training the model to recognize intermediate attribute intensities, enabling it to build a continuous notion of graded variation rather than relying solely on endpoint content. In contrast, aesthetic attributes, demonstrating substantially higher control accuracy than all baselines while maintaining smooth content transitions and high image quality. This confirms its effectiveness in
AttriCtrl consistently achieves the lowest AvgDiff across all three
8

---

Published as a conference paper at ICLR 2026
Brightans
Brightaos
bnghtwe Julie E
Figure 6: Qualitative results of multi-attribute control. We generate images by jointly varying com- binations attribute intensity values in a coordinated manner.
Brightnsss
Detall
Realion
EIGGa Controlac
Figure 7: Compatibility of AttriCtrl with mainstream control frameworks.
capturing fine- fine-grained variations in attribute intensity and translating them into accurate and stable controllable generation. More qualitative examples can be found Appendix H
User Study. To further assess the perceptual quality of attribute control, we conduct user study under double-blind setup (details Appendix ED. The results are summarized on the right side of Table I In each trial, participants are presented with five image sequences generated from the same base prompt with progressively increasing target intensity value, corresponding to four baseline methods and our proposed AttriCtrl. A total of 10 participants evaluate 100 such comparisons, covering all three aesthetic attributes (brightness, detail, and realism) across diverse intensity ranges. Our method is overwhelmingly preferred by participants, demonstrating substantial advantage in generating visually coherent and controllable attribute variations.
Inappropriate Content Suppression. As shown in Figure 5 AttriCtrl significantly outperforms established concept erasure methods like SLD and ESD in terms of removal rate on the I2P bench- mark, achieving RR of 57.7%. This demonstrates its high efficacy for content suppression tasks, offering powerful alternative to existing safety mechanisms. In Appendix F we conduct experi- ments the COCO-10K dataset (Lin al. 2014) to compute the CLIP score and FID, examining its potential influence on unrelated concepts.
4.3 MULTI-ATTRIBUTE CONTROL AND APPLICATIONS OF ATTRICTRL
Multi-attribute Control. To further validate the flexibility of our framework, we extend it from aingle-attribute multi-attribute control, enabling simultaneous adjustment of multiple aesthetic properties within a single generation process. As shown in Figure 6, we jointly vary the target intensity values of brightness, detail, and realism in coordinated manner. The results demonstrate that method produces images that change smoothly along each axis, while maintaining content consistency across different combinations of attribute intensity values. We also observe a slight coupling between realism and detail: images with higher realism values tend to exhibit moderately increased detail. This correlation is likely inherent to the training data, where photorealistic images naturally contain more fine mn--inae textures than stylized ones.
Applications. As illustrated in Figure 7 we demonstrate the compatibility of AttriCtrl with existing conditional generation frameworks by integrating it into representative pipelines: ControlNet

---

Published as conference paper at ICLR 2026
Zhang et al. 2023) and Eligen (Zhang al. 2025). Across both scenarios, AttriCtrll produces fine-
grained aesthetic transitions without disrupting the underlying content or structure, demonstrating its effectiveness as flexible plug-and-pll and-play module for diverse conditional control settings.
CONCLUSION AND FUTURE WORK
In this paper, we introduced AttriCtrl, lightweight framework for fine-grained aesthetic control in diffusion models. We observed reduced precision when prompts contain strong attribute modi- fiers (e.g., "hyper-realistie hyperlapse lighting") "), suggesting future work on the interplay between natural-language semanties and scalar control (Appendix G Beyond aestheties, AttriCtrl gener- alizes to diverse attributes-froo object count and geometry to abstract factors like temperature or motion blur-by mapping normalized values into learnable token sequences. More broadly, points toward disentangled, compositional control, where modular controllers can be combined at inference, paving the way for "mixing-cones -like generative systems. key frontier is defining robust proxy metrics for subjective notions such composition, tone, narrative coherence.
ACKNOWLEDGMENTS
This work was supported by the Guizhou Provincial Program on Commercialization of Scientific and Technological Achievements (Qiankchezhongyindi [2025] No. 006) and Alibaba Group through the Alibaba Innovation Research Program.
ETHICS STATEMENT
This work investigates a generalizable framework for controlling semantic attribute intensity dif- fusion models. Our goal is to enhance transparency and agency in generative systems, while also contributing to safer and more reliable outputs. By providing fine- grainrd interpretable control approach supports responsible deployment of diffusion
aesthetic and semantic attributes, models across creative and practical applications. We view this work as step toward aligning generative AI with human preferences and societal values.
REPRODUCIBILITY STATEMENT
To ensure the reproducibility of our experiments, we provide an anonymous link to the source code and data for review. Once this paper is accepted, we will make the code and data publicly available to researchers in the community.
REFERENCES
Omri Avrahami, Dani Lischinski, and Ohad Fried. Blended diffusion for text-driven editing of natural images. In Proceedingy the IEEE/CVF conference on computer vision and pattern recognition, pp. 18208-18218, 2022.
Stephen Batifol, Andreas Blattmann, Frederic Boesel, Saksham Consul, Cyril Diagne, Tim Dock- horn, Jack English, Zion English, Patrick Esser, Sumith Kulal, et al. Flux. 1 kontext: Flow match- ing for in-context image generation and editing latent space. arXiv e-prints, pp. arXiv-2506, 2025.
Tim Brooks, Aleksander Holynski, and Alexei A Efros. Instructpix2pix: Learning to follow image editing instructions. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 18392-18402, 2023.
CimpVis.
http://hhgginnffecccoooommixaabb htttae///buuginnfffaac..o///oooo
diffusson-safet--checkk,, 2022.
URL
10

---

Published as a conference paper at ICLR 2026
Ying Fan, Olivia Watkins, Yuqing Du, Hao Liu, Moonkyung Ryu, Craig Boutilier, Pieter Abbeel, Mohammad Ghavamzadeh, Kangwook Lee, and Kimin Lee. Dpok: Reinforcement learning for fine-tuning text-to-image diffusion models. Advances in Neural Information Processing Systems, 36.79858-79885, 2023.
Rohit Gandikota, Joanna Materzynska, Jaden from diffusion models. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp.
and David Bau. Erasing concepts
2023.
Dhruba Ghosh, Hanmanch Hajishirzi, and Ludwig Schmidt. Geneval: An object-focuser framework for evaluating text-to-image alignment. Advances in Neural Information Processing Systems, 36: 52132-52152, 2023.
Qiyuan He, Jinghao Wang, Ziwei Liu, and Angela Yao. Aid: Attention interpolation of text-to-image diffusion. arXiv preprint arXiv:2403. /7924, 2024.
Amir Hertz, Ron Mokady, Jay Tenenbaum, Kfir Aberman, Yacl Pritch, and Daniel Cohen-Or. Prompt-to-prompt image editing with cross attention control. arXiv preprint arXiv:2208.01626, 2022.
Jonathan Ho and Tim Salimans. arXiv:2207. 12598, 2022.
Classificr-free
diffusion guidance.
arXiv preprint
Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffusion probabilistic models. Advances in neural information processing systems, 33:6840-6857 1851, 2020.
Yuval Kirstain, Adam Polyak, Uriel Singer, Shahbuland Matiana, Joe Penna, and Omer Levy. Pick- a-pic: An open dataset of user preferences for text-to-image generation. Advances in neural information processing systems, 36:36652-36663, 2023.
Nupur Kumari, Bingliang Zhang, Richard Zhang, Eli Shechtman, and Jun-Yan Zhu. Multi-concept customization of text-to-image diffusion. In Proceedings of the IEEE/CVF conference on com- puter vision and pattern recognition, pp. 1931-1941, 2023.
Black Forest Labs. Flux, github http://llcck-fssst-aaaalal
2025.
URL https://
Youwei Liang, Junfeng He, Gang Li, Peizhao Li, Arseniy Klimovskiy, Nicholas Carolan, Jiao Sun, Jordi Pont-Tuset, Sarah Young, Feng Yang, et al. Rich human feedback for text-to-image genera- tion. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 19401-19411, 2024.
Tsung-Y Yi Lin, Michael Maire, Serge Belongie, James Hays, Pietro Perona, Deva Ramanan, Piotr Dollár, and C. Lawrence Zitnick. Microsoft COCO: Common objects in context. In Computer Vision ECCV 2014 13th European Conference, Zurich, Switzerland, September 6-12, 2014, Proceedings, Part V. pp. 740-755 2014.
Chenlin Meng, Yutong He, Yang Song, Jiaming Song, Jiajun Wu, Jun- Yan Zhu, and Stefano Ermon. Sdedit: Guided image synthesis and editing with stochastic differential equations. arXiv preprint arXiv:2108.0107: 2021.
Chong Mou, Xintao Wang. Liangbin Xie, Yanze Wu, Jian Zhang, Zhongang Qi, and Ying Shan. T2i-adapter: Learning adapters to dig out more controllable ability for text-to-image diffusion models. In Proceedings of the AAAI conference artificial intelligence, volume 38, pp. 4296- 4304, 2024.
Alex Nichol, Prafulla Dhariwal, Aditya Ramesh, Pranav Shyam, Pamela Mishkin, Bob McGrew, Ilya Sutskever, and Mark Chen. Glide: Towards photorealistic image generation and editing with text-guided diffusion models. arXiv preprint 22 2021.
OpenAl. hia-vii-larree-aachh14 http::/hhngginnnaee..oooooiiiiop patch14, 2022. URL

---

Published as a conference paper at ICLR 2026
Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervision. In International conference on machine learning, pp- 8748-8763. PMLR, 2021.
Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J Liu. Exploring the limits of transfer learning with a unified text-to-text transformer. Journal of machine learning research, 21(140):: 17,, 2020.
Aditya Ramesh, Prafulla Dhariwal, Alex Nichol, Casey Chu, and Mark Chen. Hierarchical text- conditional image generation with clip latents. arXiv preprint arXiv:2204.06125, 1(2):3 2022.
Nataniel Ruiz, Yuanzhen Li, Varun Jampani, Yael Pritch, Michael Rubinstein, and Kfir Aberman. Dreambooth: Fine tuning text-to-image diffusion models for subject-driven generation. In Pro- ceedings of the IEEE/CVF conference computer vision and pattern recognition, pp. 22500- 22510, 2023.
Patrick Schramowski, Manuel Brack, Bjöm Deiseroth, and Kristian Kersting. Safe latent diffusion: Mitigating inappropriate degeneration in diffusion models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. pp. 22522-22531, 2023.
Shelly Sheymin, Adam Polyak, Uriel Singer, Yuval Kirstain, Amit Zohar, Oron Ashual, Devi Parikh, and Yaniv Taigman. Emu edit: Precise image editing via recognition and generation tasks. Pro- ceedings of the IEEE/CVE Conference Computer Vision and Puttern Recognition, pp. 8871- 8879, 2024.
Chenyang Si, Ziqi Huang, Yuming Jiang, and Ziwei Liu. Freeu: Free lunch in diffusion u-net. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp- 4733-4743, 2024.
Jascha Sohl-Dickstein, Eric Weiss, Niru Maheswaranathan, and Surya Ganguli. Deep unsupervised learning using nonequilibrium thermodynamies. In International conference on machine learn- ing. pp. 2256-2265, PMLR, 2015.
Yu-Lin Tsai, Chia-Y Yi Hsu, Chulin Xie, Chih-Hsun Lin, Jia-You Chen, Bo Li, Pin- Yu Chen, Chia-Mu Yu, and Chun- Ying Huang. Ring-a-bell! how reliable are concept removal methods for diffusion models? arXiv preprint arXiv:2310. 10012, 2023.
Andrey Voynov, Kfir Aberman, and Daniel Cohen-Or. Sketch-guides text-to-image diffusion mod- els. In ACM SIGGRAPH 2023 conference proceedings, pp. 1-11, 2023a.
Andrey Voynov, Qinghao Chu, Daniel Cohen-Or, and Kfir Aberman. p+: Extended textual condi- tioning in text-to-image generation. arXiv preprint arXivv2303.09522 2023b.
Bram Wallace, Meihua Dang, Rafael Rafailov, Linqi Zhou, Aaron Lou, Senthil Purushwalkam, Stefano Ermon, Caiming Xiong, Shafiq Joty, and Nikhil Naik. Diffusion model alignment using direct preference optimization. In Proceedings of the IEEF/CVF Conference on Computer Vision and Pattern Recognition, pp. 8228 8238, 2024.
Guangxuan Xiao, Tianwei Yin, William T Freeman, Frédo Durand, and Song Han. Fastcomposer: Tuning-free free multi-subject image generation with localized attention. International Journal of Computer Vision, 133(3):1155-1144 2025a.
Shitao Xiao, Yueze Wang, Junjie Zhou, Huaying Yuan, Xingrun Xing, Ruiran Yan, Chaofan Li, Shuting Wang, Tiejun Huang, and Zheng Liu. Omnigen: Unified image generation. In Proceed- ings of the Computer Vision and Pattern Recognition Conference, pp. 13294-13304, 2025b.
Hu Ye, Jun Zhang, Sibo Liu. Xiao Han, and Wei Yang. Ip-adapter: Text compatible image prompt adapter for text-to-image diffusion models. arXiv preprint arXiv:2308..m 06721, 2023.
Hong Zhang, Zhongjie Duan, Xingjun Wang, Yingda Chen, and Yu Zhang. Eligen: Eetel controlled image generation with regional attention. arXiv preprint arXiv:2501 000097, 2025.
Lvmin Zhang, Anyi Rao, and Maneesh Agrawala. Adding conditional control text-to-image diffusion models. In Proceedings of the IEEE/CVF international conference on computer vision, pp. 3836-3847, 2023.
12

---

Published as a conference paper at ICLR 2026
A DISCUSSION ON QUANTIFICATION OF DETAIL ATTRIBUTES
For the quantification of detail, we adopt Shannon entropy as the primary metric. The underlying principle is intuitive: when pixel intensities concentrated within narrow range of gray levels, such as in uniform backgrounds, the histogram becomes highly predictable, yielding entropy val- close zero and indicating visually simple, detail-poor regions. In contrast, when intensities are broadly and evenly distributed across all 256 grayscale levels, the distribution reaches max- imal uncertainty, with entropy approaching its theoretical upper bound of log2(256) 8. This reflects visually rich content characterized by diverse luminance variations. We acknowledge that Shannon entropy an imperfect proxy for perceptual detail, as it may confounded by noise and does not explicitly capture structural complexity. Nevertheless, it provides an effective and com- putationally efficient correlate of textural richness natural images. To validate this choice, we considered alternatives such as frequency-domain analysis (c.g., spectral power) and local contrast metries (e.g., standard deviation of the Laplacian). For evaluation, we selected 100 representative images and computed detail scores with all three metries. Images were ranked from high to low for each method, and a panel of ten human experts performed voting to judge perceptual alignment. Entropy consistently outperformed the alternatives, being unanimously identified the most reli- able indicator. As shown in Figure R) which illustrates the top and bottom five images under each metric, entropy demonstrated greater robustness across diverse image contents and exhibited lower sensitivity to global illumination changes.
Spretzal Power
Tep3
Tips
Busan:
Butien
Bunen.
Betees
Bamall
Standard Deviation of the Laplacian
Trp.
Tay
Top
Tey
Tip-5
Batton
Baten
Berson
Batsel
Estropy
Figure 8: Examples of quantification of different detail metries.
B ABLATION STUDIES
conduct ablation experiments to investigate the design choices of the proposed value encoder, focusing on two key factors: the sequence length of the encoded tokens and the use of positional encoding. The results are reported Table and Table 3 measured by the AvgDiff metric across three aesthetic attributes (brightness, detail, and realism). As shown in Table 2 increasing the number tokens progressively improves control accuracy. While shorter sequences (1 or tokens) lead to relatively high errors, extending the length 32 tokens yields the best performance. Further increasing the length to 64 tokens does not bring noticeable gains, suggesting that the benefit of longer sequences saturates beyond a certain length. This indicates that 32 tokens strike a good balance between representation capacity and efficiency. Moreover, Table 3 shows that introducing positional encoding further enhances control accuracy, as helps the model better distinguish the roles of individual tokens, thereby improving the expressiveness of the encoded representations.
C BASELINE CONFIGURATION
To comprehensively evaluate the effectiveness our method, we introduce four representative base- lines: Kontext, W-Emb, AID-in, and AlD-out. Their configurations summarized follows:
13

---

Published as a conference paper at ICLR 2026
Table Ablation on the number tokens evaluated by AvgDiff
Number of Tokens
Brightness
Detail
Realism
token
0.257
0.295
0.235
8 token
0.185
0.206
0.193
16 token
0.183
0.253
0.197
32 token
0.141
0.191
0.192
64 token
0.171
0.178
0.196
Table 3: Ablation on the use of positional encoding evaluated by AvgDiff J.
Positional encoding Brightness
Detail
Realism
None With
0.181
0.213
0.228
0.141
0.191
0.192
This baseline adopts the instruction- based control mechanism used in FLUX. It manipulates the aesthetic attributes by directly appending natural-languee instructions to the prompt, such as "Make it value% level of [attribute]" where [attribute] be detail, brightness, or realism.
Kontext.
W-Emb. We collect the top 2,000 and bottom 2,000 image text pairs for each attribute from the AttriCtrl training set, and train attribute-specific embeddings under the same architec- ture as AttriCtrl. During generation, these embeddings are injected and linearly weighted according to the target attribute intensity.
AID-in / AlD-out. These two baselines generate intermediate images by interpolating be- tween two endpoint prompts through an attention-hase interpolation mechanism, with the warm-ratio parameter fixed at 0.6. For each target attribute, design two prompts representing opposite extremes: for brightness, "darker" versus "brighter": for detail, "minimal" versus "detailed"; and for realism, "cartoony" versus "photorealistic. Dur- ing inference, the model first produces endpoint images conditioned on these prompts, and then synthesizes intermediate results by proportionally blending their attention maps ac- cording to the desired attribute intensity. This allows the system gradually transition between two extremes of given aesthetic attribute.
D FAILURE CASES IN THE AID BASELINE
During experiments, we observe that both AID-in and AID-out occasionally produce artifacts such as halos and ghosting, as shown in Figure 9 We attribute this to the absence of explicit attribute conditioning in intermediate steps. Without clear semantic guidance, the model may blend multiple conflicting attributes simultaneously, resulting in visual degradation structural collapse in the generated images.
E
USER STUDY
We further conduct user study to evaluate the perceptual quality and controllability of different methods. Ten expert participants are invited to complete 100 single-choice questions, including 34 for brightness, 33 for detail, and 33 for realism. Each question presents five image sequences generated from the same prompt and random seed, covering different attribute intensities from the four baselines and our method. Participants were asked to select the sequence that best met the following criteria: (1) smooth and continuous variation across attribute levels, (2) high visual quality and coherence within the sequence, and (3) accurate reflection of the intended attribute changes. Examples of the questions used in the study are shown in Figure 10
F EVALUATION ON UNRELATED CONCEPTS
To examine whether our method unintentionally affects the generation of unrelated concepts while suppressing inappropriate ones, conduct additional evaluation the COCO dataset al.
14

---

Published as a conference paper at ICLR 2026
Aid-in
AlD-out
Figure 9: Typical failure cases of the AID-in and AlD-out baselines. Both methods occasionally produce severe visual artifacts such as halos and ghosting.
Brightness
Detail
Realism
Kontest
Nontest
Komtest
W-Fabb
W-Emb
W-Emb
Afl-in
AID-In
AID-out
AlD-out
ATD-out
Ours
Ours
Ours
Figure 10: Example questions from the user study. Each question presents five image sequences generated from the same prompt and random seed, showing different attribute intensities produced by four baselines and our method.
2014). Specifically, we sample 10K prompts from COCO captions and use them to generate images. We then assess the results with two widely adopted metries: Fréchet Inception Distance (FID), which measures the distributional similarity between generated and real images, and CLIP Score, which evaluates image-ted text alignment. As shown in Table 4 our method achieves competitive FID and CLIP Score compared to the baselines, demonstrating that it does not impair the model's ability capture and represent unrelated concepts. These results highlight the robustness of AttriCtrl in preserving general generation quality beyond the targeted attribute suppression.
Table 4: Evaluation on unrelated concepts using FID
and CLIP Score t.
Metric
NP
SLD
ESD
Ours
CLIP Score
0.337
0.334
0.318
0.317
FID 39.322
40.016
35.665
29.963
G DISCUSSION AND FUTURE WORK
Our experiments based on the recent FLUX model, a DiT-based architecture. Future work could explore the adaptability of AttriCtrl to U-Net based diffusion models like Stable Diffusion, which would further validate its architectural agnosticism.
As shown in Figure 11 we observe that control becomes less precise when the prompt itself al- ready contains attribute-related modifiers, such as requesting a "hyper-realistic hyperlapse lighting." Quantitatively, the model constrains the attribute intensity within semantically coherent range, reflecting a prioritization of semantic fidelity over rigid adherence to explicit instructions. This be- havior aligns naturally with real-world usage, where intent is embedded in natural language prompts. Additionally, since our safety dimension defined relative to the Stable Diffusion safety checker, its effectiveness is inherently bounded by the coverage and biases of this reference model.
15

---

Published as a conference paper at ICLR 2026
Brightness
Target: 0.10
Target: 0.30
Target: 0,50
Target: 0.70
Target: 0.90
Value: 0.41
Value: 00.47
Value: 0.52
Value: 0.54
Value: 0.58
A steampunk- flouring markkeplace a tropical sky, balloons powered by steam cores selling clockwork parroes and tea infased with moonlightt intricate brass machinery details.
Realism
Target: 0.10
Target: 0.30
Target: 0,50
Target: 0.70
Target: 0.90
Value: 0.38
Value: 0.41
Value: 0.42
Value: 0.44
Value: 0..6
derelict spaceship crash- landed in an Arctic glacier, its metallic careass overgnown with crystalline flora, regenented
pollinating flawers, hyper-realistic hyperiapee lighting.
Figure 11: Examples of interaction between prompt semantics and attribute control.
In other words, AttriCtrl does not establish an absolute notion of safety, but instead aligns control- lability with specific, predefined standard.
Broader Impact and Future Directions. thetic control. The core token sequence via a value encoder a general and powerful paradigm for fine-grained conditioning in diffusion models. This paves the way for controlling vast range of previously inaccessible, quantifiable attributes. One can envision future work applying this framework to pre- cisely specify the number of objects in a scene, adjust the geometrie properties (e.g., aspect ratio, roundness) of a generated element, or even manipulate abstract physical parameters like simulated temperature or motion blur. Furthermore, our work highlights a promising path toward learning highly disentangled and compositional representations. The ability to independently train and then combine attribute controllers at inference time suggests future of truly modular, "mixing-console like generative systems. This opens up compelling new research avenue: systematically exploring robust proxy metries for complex, subjective, or abstract concepts. Devising effective ways to quan- tify notions like "creative composition", "emotional tone", or "narrative coherence" remains challenging but exciting frontier, for which AttriCtrl provides a foundational control mechanism.
The contributions AttriCtrl extend far beyond aes- mapping normalized scalar value into dedicated, Iearnable
H MORE EXAMPLES
Figure 12
presents additional qualitative examples of our method controlling the strength of three aesthetic attributes and safety. These results further demonstrate the flexibility and effectiveness of our approach in achieving fin-grained aesthetic control.
I UNIVERSALITY ACROSS DIFFERENT ARCHITECTURAL MODELS
To demonstrate the universality of AttrCCtrl, we present qualitative results generated using three widely adopted diffusion backbones: Stable Diffusion v1.4, Stable Diffusion XL, and Stable Dif- fusion v3.0. As shown in Figure 13 AttriCtrl consistently enables fine-gn fine-grained control over target attributes across all three architectures. This indicates that our method can integrated into various diffusion models with minimal architectural modifications.
16

---

Published as a conference paper at ICLR 2026
Brightness
Detail
Realism
Figure 12: More examples of single attribute intensity control.
Brightness
Detail
SD v1
SD
Figure 13: Qualitative results of AttriCtrl across different diffusion architectures.
J USE OF LLM
This document was supported by the use of large language models (LLMs), including tools such ChatGPT and Qwen, during the preparation of this document. These models were used for purposes such as language polishing, improving sentence fluency, and proofreading, only after the core content had been written by the authors. No part of the technical reasoning. data analysis, ideas was generated or influenced by L.MMs. The initial
interpretation of results, or development drafts, structure, and key content of all sections were entirely authored by humans. The models were not involved in any decision-making process related to methodology, results, or conclusions.
17


Reviews:
[{"review_id": "UINHp7hG68", "reviewer": "Reviewer_jx4y", "summary": "The paper introduces *AttriCtrl*, a novel method which allows fine-grained control over the strength of a set of textual attributes (specifically *brightness, detail, realism, safety*) in the context of text-to-image diffusion models. Given one attribute a user can decide in a 0-1 scale how strongly that attribute should be present in the final generation. For instance, by selecting the attribute *realism* the user can smoothly interpolate from less realistic to more realistic generations. The method relies on the training of specialized encoders (one for each attribute) which, given a 0-1 score, output a set of tokens which are appended to the textual ones to condition the generation. Experiments are conducted using the FLUX model and prompts from GenEval. Given the novelty of the task of fine-grained attribute control, AttriCtrl is compared to some baselines proposed by this same work and new evaluation metrics are proposed for the task. A user study is conducted to further evaluate the performances. Quantitative and qualitative results as well as the user study generally prove the effectiveness of the proposed method.", "questions": "1. In line 26, the phrasing *decompose relevant aesthetic attributes* may suggest that AttriCtrl can disentangle and control an arbitrary number of attributes, while in practice it focuses on four (*brightness*, *detail*, *realism*, *safety*). Could this part be clarified or rephrased?\n2. How long does the training of each attribute encoder take?\n3. Have you experimented also with different models other than FLUX? It would be interesting to understand whether an encoder learnt with FLUX can be used with another model for instance. This would make the method easy to apply by only training the encoders once.\n4. Minor notes:\n    - The *safety* attribute seems to be missing in the list of line 158.\n    - *Ours* should not be capitalized in line 416.", "limitations": "", "rating": 8, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 4, "strengths": "1. **Original and relevant task.** The paper clearly defines and tackles the underexplored task of *continuous aesthetic intensity control*, introducing a principled framework that goes beyond binary or discrete conditioning.\n2. **Sound and well-motivated design.** The use of independent value encoders for each attribute is simple yet effective, allowing plug-and-play control without retraining the diffusion backbone.\n3. **Comprehensive evaluation.** The paper presents quantitative, qualitative, and user study evidence to compare over baselines, along with ablations clarifying key design choices. It also provides a clear experimental framework to facilitate future research in this direction.\n4. **Clarity and reproducibility.** The paper is clearly written, well structured, and supported by released code and detailed appendices.", "weaknesses": "1. **Scalability.** Each attribute requires a separate encoder; while this modularity is elegant, scaling to a larger number of attributes could be computationally expensive as a new encoder must be trained for each attribute.\n2. **Content preservation.** In some qualitative examples (e.g., Fig. 4, brightness adjustment of the cow), changes in attribute intensity alter semantic content (the cow becomes white for higher brightness values), which may not be desirable.\n3. **Limited backbone diversity.** Experiments are conducted only on FLUX; demonstrating transferability to other architectures (e.g., Stable Diffusion) would strengthen the work.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "E1LyW0KLX6", "reviewer": "Reviewer_VVmk", "summary": "AttriCtrl is a small add-on that lets you directly control how a text-to-image model renders certain qualities—like brightness, detail, realism, or safety—by giving it a numeric value (e.g., brightness = 0.7). It turns that number into a few tokens and appends them to the normal text prompt, while keeping the big diffusion model frozen, so it’s easy to plug in. For concrete attributes, it uses simple measurable definitions (e.g., image brightness); for abstract ones, it relies on CLIP similarity. The method works for one or multiple attributes at once, shows clear, smooth control in examples, and beats prior “prompting” baselines on a main accuracy metric and user preference tests. Overall, it’s a practical, lightweight way to get consistent, continuous control without retraining the main model.", "questions": "- Could you add an “oracle” numeric-prompt tuning baseline (e.g., prompt templates tuned per attribute) and a CLIP-guided latent interpolation variant to situate AttriCtrl’s gains?\n- Since Safety(I) depends on the SD checker prior (eq. 5), can you add an alt-prior (e.g., LAION NSFW classifier) to test sensitivity? Also report RR per category (as in Fig. 5) with confidence intervals. \n- Please add control baselines that also aim to steer generation while preserving content regarding Table 4.", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "- Clear, modular mechanism (frozen base, tiny encoder) that composes with ControlNet/T2I-Adapter; qualitative demos on multi-attribute control are convincing (Figs. 6–7). \n- Simple, explicit quantification of attributes (eqs. 1–6) with visualized normalization; practical and reproducible.\n- Positive headline numbers: Table 1 shows lower AvgDiff vs AID/Kontext/W-Emb and strong user preference; safety shows higher RR than NP/SLD/ESD (Fig. 5).", "weaknesses": "- Robustness under conflicting prompts is acknowledged but not quantified. Appendix G notes degradation when prompts contain strong attribute modifiers (e.g., “hyper-realistic…”) and shows Fig. 11, but lacks a quantified analyses.\n- Safety depends on SD checker prior. Safety is defined relative to the Stable Diffusion safety checker; authors note bounded effectiveness (Appendix G). This is a limitation if the checker’s coverage/bias shifts.\n- Unrelated-concept quality is only lightly assessed. Table 4 (COCO-10K: CLIP/FID) is encouraging, but the comparison set is safety baselines (NP/SLD/ESD), not control baselines.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "HHlxc1Y8lH", "reviewer": "Reviewer_LHsn", "summary": "This paper addresses the task of controlling specific attributes (e.g., brightness, detail) in image generation. It proposes a method that defines and manipulates these attributes, presenting comparative experiments against other methods like AID. The main contribution lies in formulating this control task and providing a comparative analysis based on the proposed attribute metrics.", "questions": "1.  **Conceptual Foundation:** Could the authors clarify the fundamental concept? Are you controlling \"semantic\" or \"aesthetic\" attributes? How do your definitions of attributes like `brightness` and `detail` align with or differ from established definitions in image analysis and aesthetic assessment? Specifically, why was average pixel intensity chosen over illumination-aware models for brightness?\n2.  **Technical Contribution:** Given that the attribute calculations are based on existing techniques, what is the specific novel technical contribution of the method proposed in this paper? Is it primarily the composition and application pipeline?\n3.  **Experimental Comparisons:**\n    *   Could you provide a more balanced comparison with AID, including a statistical analysis of performance and failure cases for both methods, rather than just selected examples? The results in Figure 4 seem to favor AID; can you comment on this?\n4.  **Metric Design:** The results in Figure 4 show that your metrics can lead to unrealistic outputs (e.g., small image size for low detail, color change for brightness). Do you agree that these metrics have inherent limitations? Have you considered designing or incorporating more sophisticated, perceptually-aligned metrics to address these issues?\n5.  **Clarification and Reproducibility:**\n    *   How exactly is the final aesthetic intensity embedding `v` obtained? Please provide a clear mathematical formulation or description.", "limitations": "", "rating": 4, "confidence": 4, "soundness": 2, "presentation": 2, "contribution": 2, "strengths": "*   **Interesting Problem:** The core problem identified in Figure 1 is valid and represents an interesting challenge in controllable generation.\n*   **Initial Validation:** The experimental results indicate that the proposed method shows some advantages in controlling the defined attributes compared to certain baselines.", "weaknesses": "*   **Lack of Conceptual Rigor and Novelty:**\n    *   **Inconsistent/Terminological Flaws:** A significant weakness is the lack of conceptual clarity. The title mentions \"semantic attributes\" while the abstract discusses \"aesthetic attributes,\" which are incorrectly applied to concepts like `detail` and `safety`. The definition of `brightness` (changing object color vs. illumination) is problematic and not aligned with standard understandings in image processing or aesthetics.\n    *   **Unnovel Technical Core:** The technical approach for calculating and utilizing the attribute metrics is acknowledged as being based on existing, relatively straightforward techniques (e.g., average pixel intensity for brightness, entropy for detail). The paper fails to clearly articulate what the specific novel technical contribution is beyond the application of these existing metrics.\n*   **Inadequate Experimental Validation and Comparisons:**\n Based on Figure 4, I believe the AID method performs the best. AID's results demonstrate precise control over the `brightness`, `detail`, and `realism` attributes, while excellently maintaining content consistency during attribute manipulation.\n    *   Regarding `brightness`, AID modifies the ambient lighting rather than the cow's color. This discrepancy is inherently linked to the brightness calculation method used in this paper (average pixel intensity), which does not correspond to illumination in the imaging process.\n    *   Using entropy as a complexity metric leads to excessively small objects in low-complexity controls, which is unreasonable from either an aesthetic or user need perspective.\n    *   Concerning `realism`, the first result from the proposed method has low realism, whereas all AID results appear highly realistic. The `Diff` metric also does not accurately reflect the corresponding quality.\n    *  The experiments in the appendix primarily demonstrate AID's capability in attribute control and semantic consistency. Although cases where AID fails are listed, based on my knowledge of generative models, the proposed method likely also has failure cases. Isolated examples cannot fully substantiate the proposed method's superiority.\n\n    *   **Metrics Lead to Artifacts:** The chosen attribute metrics themselves are a source of limitations, as evidenced by Figure 4, where they cause undesirable effects (e.g., unnaturally small images for low complexity, unrealistic brightness changes). This suggests the core formulation may be flawed.\n    *   **Insufficient Evidence for Superiority:** The claim of superiority over AID is not fully convincing. The appendix shows AID failures but does not provide a fair comparative analysis of failure cases for the proposed method. The visual results in Figure 4 suggest AID may perform better in terms of control precision and content preservation.\n*   **Issues with Presentation and Reproducibility:**\n    *   **Clarity:** The explanation is sometimes unclear, such as the definition and derivation of the final embedding `v`. Variable naming is non-standard (`$normalized_i$`).\n    *   **Reproducibility:** The provided code link is broken.\n    *   **Scholarly Rigor:** Section 3.2 lacks necessary citations for the techniques used.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "a7plR144JS", "reviewer": "Reviewer_3gSF", "summary": "While diffusion models have been showing really good results in image generation, there might still be cases where they are unable to follow instructions for adjusting visual inputs. This issue arises because text encoders handle words, not continuous values, and current aesthetic alignment methods only capture broad preferences. The proposed framework, AttriCtrl, enables fine-grained, continuous control of aesthetic features by mapping both concrete and abstract traits onto a [0, 1] scale. It uses a plug-and-play value encoder to translate user inputs into interpretable embeddings, allowing smooth and accurate adjustments. AttriCtrl integrates with existing systems like ControlNet while keeping the base model unchanged and computationally efficient.", "questions": "I do not have particular questions for the authors, but I would be very interested to see if they can properly address the mentioned weaknesses. If so, I might be willing to increase my score.", "limitations": "", "rating": 4, "confidence": 3, "soundness": 2, "presentation": 2, "contribution": 2, "strengths": "I think the paper has the following strengths:\n\n1) Most of the results seem to be visually nice, which shows that the method can have some good usability.\n\n2) It is very nice that the method can be combined with the likes of ControlNet and Eligen.\n\n3) The paper is quite readable, and a motivated user should be able to understand it.", "weaknesses": "I think the paper can be further improved in these aspects.\n\n1) By far my main issue with the paper is the lack of extensive comparisons with the other methods. I understand that for some tasks, this is very hard to do, considering that there aren't clear comparisons. However, for this reason, the authors should have done more evaluations in the task of nudity erasure where there are many other methods and benchmarks. In particular, the authors could have compared with:\n\n[A] Gong et al., Reliable and efficient concept erasure of textto-image diffusion models, 2024\n\n[B] Schramowski et al.,  Safe latent diffusion: Mitigating inappropriate degeneration in diffusion models, 2023\n\n[C] Yoon et al., SAFREE: training-free and adaptive guard for safe text-to-image and video generation, 2024\n\n[D] Lyu et al., One-dimensional adapter to rule them all: Concepts, diffusion models and erasing applications, 2024\n\n[E] Gaintseva et al., CASteer: Steering Diffusion Models for Controllable Generation, 2025\n\nI understand that erasure is not the main focus of the paper, but considering how well-established (and important) it is, it can serve as a proxy on quantitative evaluation of the method.\n\n2) Similarly, for erasure, the authors could have done a user study.\n\n3) Some parts of the paper feel like padding. In particular, almost all equations seem trivial such as doing a normalization, or computing entropy. This does not make the paper be more sophisticated, in fact, it almost have the opposite effect.\n\n4) The authors should have tested their method with more diffusion backbones, currently it is only in Flux. In particular, I would have been very interested to see how it works in SDXL, SD-3/3.5 and SANA.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)