GroundingBooth: Grounding Text-to-Image Customization
Authors: Zhexiao Xiong
Wei Xiong
Jing Shi
He Zhang
Yizhi Song
Nathan Jacobs
Venue: Accepted by TMLR ()

Abstract: Recent approaches in text-to-image customization have primarily focused on preserving the identity of the input subject, but often fail to control the spatial location and size of objects. We introduce GroundingBooth, which achieves zero-shot, instance-level spatial grounding on both foreground subjects and background objects in the text-to-image customization task. Our proposed grounding module and subject-grounded cross-attention layer enable the creation of personalized images with accurate layout alignment, identity preservation, and strong text-image coherence. In addition, our model seamlessly supports personalization with multiple subjects. Our model shows strong results in both layout-guided image synthesis and text-to-image customization tasks. The project page is available at https://groundingbooth.github.io.

Published in Transactions on Machine Learning Research (02/2026)
GroundingBoothi Grounding Text-to-lmage Customization
Zhexiao Xiong+
Wei Xiong Jing Shi²
He Zhang
Yizhi Song Nathan Jacobs'
Washington University in St. Louis Adobe Research
Purdue University
Reviewed OpenReview: attee://pppaaeorrr...r/////rrmrt
Abstract
Recent approaches in text-to-image customization have primarily focused on preserving the identity of the input subject, but often fail to control the spatial location and size of objects. We introduce GroundingBooth, which achieves zero-shot, instance-level spatial grounding on both foreground subjects and background objects in the text-to-image customization task. Our proposed grounding module and subject-grounded cross-attention layer enable the creation of personalized images with accurate layout alignment, identity preservation, and strong text-image coherence. In addition, our model seamlessly supports personal- ization with multiple subjects. Our model shows strong results in both layout-guided image synthesis and toxt-to-image customization tasks. The project page available at nttps://groundibbothh htw.::/nnooooooiiiu.......
1 Introduction
Text-to-image customization, also known subject-driven image synthesis or personalized text-to-image generation, is a task of generating diverse variants of a subject from a set of images with the same identity. Text-to-image customization has achieved significant progress during the past few years, allowing for more advanced image manipulation.
Earlier approaches like Dreambooth RRuiz al. 2023). Textual Inversion (Gal al. 2022 and Custom Diffusion (Kumari et al. 2023) address this task by finetuning specifie model for given subject in the test phase, which is time-consuming and not scalable. Recent approaches like ELITE (Wei et al. 2023) and InstantBooth (Shi et al., 2023) eliminate test-time-finet finet-fin by learning general image encoder for the subject. Although these methods improve the efficiency of inference, they mainly focus on preserving the identity of the subject, yet fail to accurately control the spatial locations of subjects and background objects. In real-world scenarios of image customization, it is a crucial user need to achieve fine-grained and accurate layout control on each of the generated objects for more flexible image manipulation.
To address this issue, in this paper, investigate more fundamental task, grounded tert-to-image cus- lomizalion, which extends the existing text-lo-image customization task by enabling spatial grounding con- trollability over both the foreground subjects and background objects. The imput of this task includes a prompt, images of subjects, and optional bounding boxes of the subjects and background text entities. The generated image is expected to be prompt-aligned, identity preserved for the subjects, and ayygueelligned for all the grounded subjects and background objects. It is challenging satisfy all these requirements this task simultaneously.
Several related studies have enabled layout control in text-to-imane generation (Zheng et al. 2023 Li et al. 2023) However, they cannot preserve the identity of the subjects. A distinct line of research (Chen et al.
2023b Song et al. 2022 2024) has demonstrated control over the input subject's placement in image com- position tasks. However, they are neither capable of text-to-image senthesis nor able to control the spatial location the background objects.
To fully address our task, we propose GroundingBooth a general framework for grounded text-to-image customization. Specifically, to enable layout control, we propose grounding module that ensures both

---

Published in Transactions on Machine Learning Research (02/2026)
(a) Single-subject Grounded Text-to-lmage Customization
stuffed "a stuffled onimal with a city the background
toy jungle"
toy
sunflowers around
backpock on top of wooden floor"
backpack with the back background"
to the beach!
"o the snow
(b) Multi-subject Grounded Text-to-lmage Customization
toy and sneaker with chair + and table in the snow"
"A dog candle with + and fireplace
chair
table
rug fireplace
christmas eve"
Grounded Entities
Grounded Entities
Figure 1: We propose BootndingBooth, a framework for grounded text-to-image customization. Ground- ingBooth supports: (a) grounded single-subject customization, and (b) joint grounded customization for multi-subjects and text entities. GroundingBooth achieves prompt following, layout grounding for both sub- jects and background objects, and identily preservation of subjects simullaneoussyy.
the foreground subjects and background objects adhere to the input bounding boxes. Moreover, we observe that without specific design, the appearance the generated subject tends to blended with its surrounding background objects generated from prompts Xiao el al. (2023). To resolve this issue and further improve the identity preservation of the subject, we propose a subject-grounded cross-attention layer that disentan- gles the subject-driven foreground generation and text-driven background generation, effectively preventing the erroneous blending of visual concepts. As shown in Fig. [ our framework not only achieves grounded text-to-image customization with a single subject (Fig. [ (a)). but also supports multi-subject customization (Fig. [ (b)): users can input multiple subjects along with their bounding boxes, and our model can generate each subject in the exact target region with identity preservation and scene harmonization. Meanwhile, our model also allows for the grounding of multiple background objects (Fig. I (b)). We summarize our contributions below:
We propose Grouniiggoooh, general framework for the grounded text-to-image customization lask. Our model achieves layout control for both foreground subjects and background objects while preserving subject identity. Furthermore, supports multi-subject customization.
We propose subject-grounded cross-attention layer, which disentangles the foreground subject generation and text-driven background generation through cross-attention manipulation, thus pre- venting erroneous context blending.
Our model outperforms existing works in text-image alignment, identity preservation, and layout alignment.
2 Related Work
Text-to-Image Customization Text-to-image customization, also known as personalized text-to-image generation subject-driven text-to-image generation, aims to generate images from set of subject images
2

---

Published in Transactions on Machine Learning Research (02/2026)
and a text prompt that describes the image content (Chen et al. 2023a Pan et al.
2024 Xiao et al 2023
Wang et al. 2024a; Avrahami et al., 2023). In this task, the specific identity of the input reference images is defined as a subject or a concept. Existing image customization works can be categorized into three major types. The first type is cest-time-finctuning methods (Ruiz et al.) 2023 Gal et al. 2022 Kumari et al. 2023). These methods lune a specific diffusion model on a few subject images so that the model is adapted to a new identifier token representing the subject. This type of methods is computationally intensive. The second type is encoder-based customization methods (Arar et al., 2023 Wei al., 2023 Shi et al.) 2023; Zhang et al. 2024), which eliminates test-time finetuning by pretraining the diffusion model equipped with an image encoder so that it can generalize to new subjects during inference. These methods can achieve much faster image customization. The third type (Roich et al., 2021 Gal et al. 2023) is a combination of the first two methods, which learns a general image encoder to encode the identity of the subject and then finetunes the model for a few steps to further improve the results. Recent works (Tan et al. 2024 Zhang et al., 2025 Wang et al. (025)) also propose using LoRA finctuning on Diffusion Transformer (DiT) architectures achieve subject-driver customization.
subject variants and are limited in controlling the layout of the generated scenes. A related work Break-A-Scene (Avrahami et al. 2023) enables personalized local region editing of an image. Their task differs from ours in that as an image editing method, they can only specify few local regions to modify, failing to fully control the layout of the full image. In contrast, our model achieves a comprehensive spatial grounding on both the foreground subjects and background contents. MS-Diffusion Wang et al.] (2024b) also achieves layout control of given subjects. However, they fail to spatially control the background contents. Our model not only grounds the subjects, but also fully controls the layout of the background contents with text entities as guidance.
Most existing image customization methods focus on synthesizing
Grounded Imagee Generation Given a layout containing bounding boxes labeled with object categorics or text entities, grounded text-to-image generation aims to generate the corresponding image that aligns with the layout. Traditional grounded text-to-image generation such LostGAN (Sun & Wui
2019), LAMA (Li et al. 2021 and PLGAN (Wang et al. 2022 are based generative adversarial networks
(G.Ns). Recently, diffusion- based methods (Rombach et al. 2022 Zheng et al. 2023 Li et al.
2023 Zhang
et al. 2023: Wang et al., 2024c) have made attempts to add layout control for image generation. For example,
2023 injects LayoutDiffusion (Zheng et al. 2023 uses a patch-based fusion method. GLIGEN (Li et al.
grounded embeddings into gated Transformer layers. ControlNet (Zhang et al. 2023) uses copied encoders and zero convolutions. InstanceDiffusion (Wang et al. 2024c) allows for multiple formats of location control. LayoutGPT (Feng et al.) (2024) and LayoutLLM- 'Qu et al. 2023) use LLM as guidance. However, existing methods can only perform text-to-image generation without subject-driven generation and identity preservation. In contrast, our model achieves identity preservation of subjects while aligning the layout.
3 Our Approach
Given one or multiple background-free images X' = {z1, ... Fm where each image Em E R hxxxx3 represents a subject, and their larget bounding box locations Lx = (1x; P2 ...,,,.",77 text entities T = 12,,2,.... , with their target locations Lr = 12, ..... 17), and the overall text prompt P, we aim to generate a customized image é such that the subjects can be seamlessly placed inside the desired bounding box with natural poses and accurate identity, and the background objects generated from text-box pairs are positioned at the correct locations.
To rigorously distinguish the roles of these inputs, we define this task under Layout- Paradigm. In this formulation, the bounding boxes (Lx, Lr) function as hard spatial constraints that define the mandatory generation regions, whereas the text prompt P and subject images X' serve as semantic references that define the content. Consequently, our architocture is designed to structurally prioritize layout constraints over conflicting spatial cues embedded in the text prompt.. Here 1 or 19. refers to the bounding box coordinates of subject or a text entity, which can be represented as [Emin: Vmin: Tmax ymax] The generated image
1 BBakkrroudd-f free images refer images with background removed. We obtain them with SAM (Kirillov et
2023
2 Heee each text entity is referred to a text tag, such as "chair" and "hat".
3

---

Published in Transactions on Machine Learning Research (02/2026)
Step Feature Extraction
Step 2: Grounded Feature Integration
The Isys, boot und cobolestone street
ond
x(P-1)
Grounding Module
Se Aute hoo
Gated Sol Motrror
Cross Wintoon
Sub Grouddad Aoo.-ttttnn Sotoooo
Traneformer Blocks
Grounding
FFN
bibbv Grooderr
Layort
ID Tokens of Surject
Igz
anNv Erocde
U-Net
Input
Figure Inference pipeline of Groundiggoooh. contains two steps: (1) Feature extraction. We use the CLIP encoder and DINOv2 encoder lo extract prompt and image tokens, respectively. We use our proposed Grounding Module extract grounding tokens from layout and text entities.
(2) Grounded Prros--Attetionn Layer in cach transformer block to integrate the subject image tokens, text tokens, and grounding tokens. Note that the model is trained with a single subject per image, but generalizes well to multiple subjects during inference.
feature integration. We propose Subject-Grounder
f can be calculated as:
GroundingBoot P
The pipeline our proposed GroundingBooth model is shown in Fig. 2 We first extract grounded text tokens from lext and layout impuls, and image tokens from subject images, as described Sec. 3.1 Then we integrate these tokens with our proposed grouude-grouded cross-attention layer, as described in Sec. 3.2
Sec. 3.3 and Sec. 3.4 reveal the details of model training and inference, rospectively.
3.1 Feature Extraction
Feature Extraction of Prompt and Subject Images We first extract text tokens from the input prompt. using the CLIP lext encoder and identily tokens from the subject images using DINOv2 (Oquab et al. 2023). For each subject image, we extract 257 identity tokens which are composed of global image class token and 256 local patch tokens. We reshape the feature dimension of each image token to 768 through linear projection layer.
Grounding Module To control the layout of the foreground and background objects, propose grounding module to jointly ground text. and image tokens through positional encoding. Fig. 3 shows its the overall structure. Specifically, it contains two branches: 1) In the text entity branch (bottom), the bounding boxes of the background objects LT are passed through a Fourier encoder to obtain the text Fourier embeddings the text entities, which are then concatenated with the text tokens in the feature space to obtain the grounded text embeddings. 2) In the subject image branch (upper), the bounding boxes of the subject Lx are also passed through Fourier encoder to extract the subject Fourier embeddings, which are then concatenated with the subject image tokens in the embedding space to obtain the grounded subject embeddings. At the end of the following two branches, the grounded text embeddings and subject image embeddings are projected via lincar layers and then concatenated in the embedding space to form the final grounding Lokens. Given the text entities T and subject images X,, the grounding process is formulated as:
TT,20 = MLP((Wrea(T)) Fourier([T)),
MLP(Won Fourier([x))) (oorrea))))
(2)
where Fourier represents the Fourier embedding Tancik et al. 2020 MLP(...) is a multi-layer perceptron, [. is concatenation operation, and ((T,2) is the grounding tokens. Vtent and doty denote to the text encoder and image encoder, respectively. The generated grounding token (T.X) an intogration of the layout information of text entities, layout information of subject images, and the rich vision feature of the subject. We then inject the grounding tokens through gated self-attention layer (Li al. 2023 that is
4

---

Published in Transactions on Machine Learning Research (02/2026)
DROa
Ouject
Fourier Encoding
Subject Fourier Embeddings
Concat
Grounded Subject Embeddings
MLP
Layout
Reference
Fourier Encoding
Text Fourier Embeddings
Concat
Grounded Text Embeddings
MLP
Grounding Tokens
of Entilos
Grounding Module
CUI You Sood
opple shoe
Text Toxaris
Figure Grounding Module: Our grounding module takes both the prompt-layout pairs and reference object-layout pairs as input. For the foreground reference object, both CLIP text token and the DINOv2 image class token utilized.
newly introduced into each transformer block of the diffusion U-Net, in-between the original self-attention layer and cross-attention layer of the original block. We formulate the gated attention layer as:
== u tanh(>) (SelfAtta ([T,ACT<<))))
(3)
where is a learnable scalar initialized as 0, A (T.x) is the grounding token and is the output of the self-attention layer. During training, the model adaptively learns adjust the weight y of the grounding module, which ensures stable training and balances the weight between the grounding token and the visual features.
3.2 Grounded Feature Integration
On fusing the text and image features, existing text-to-image customization methods usually directly con- catenate the text and image tokens in the cross-attention layers, leading to several issues: First, the generated subject and the background objects generated from the prompt and text entities can be unnaturally blended, as we observe in our experiments. Second, in the circumstances where two bounding boxes belong to the same class, the model cannot distinguish whether each bounding box belongs to a subject image or a text entity, resulting in misplacement the subject. Moreover, this type of fusion strategy usually cannot handle the customization of multiple subjects. To address all these issues, we propose a subject-grout goolect-roonded cross- attention layer to specifically disentangle the generation process of subjects and background objects. The details of our module are illustrated in Fig.
Layer In this layer, both the DINOv2 image tokens and bounding box of the subject Lub are taken as inputs. The queries K and values Q are calculated from the image tokens. We first compute the affinity matrix A through A = Q- K and obtain A E hwwk where h w indicates the resolution of the Teature map in the atlention layer. As we have the object layout Lund it is straight restrict the injection of image tokens only inside the region of the target bounding box. Therefore, we reshape the layout luub to h and generate the cross-attention mask, which is formulated as:
Subject-Grounder Grounded Cross-Attentio
MLayoutli.j] = O, oc,
[ Loud. [i $ Loub
where MLayoutli.j] represents the mask value of position [i.j] in rectified attention score maps.
The mask contains the explicit location information of the subject. It encourages the accurate placement of the subject and avoids information leakage from other objects. After obtaining the mask, we it to
5

---

Published in Transactions on Machine Learning Research (02/2026)
Resize
Layout Mask
Layout Mask1
MR
x (N
1)
Input Image Feature
Output Image Feature
Attention Score Maps
Rectified Attention Map A'
Rectification
ID tokens of Subject
ID tokens of Subject
Matrix Multiplication
Figure 4: Groudet- Attention Q, K, and V are visual query, key, and value respectively, and A is the affinity matrix.
constrain the spatial distribution of the attention maps by rectifying the attention, and obtain the mask- rectified affinity matrix A' through A' A + M Loyout- Then multiply the masked affine matrix A' with V to obtain the subject-grounded cross-attention output feub. The whole subject-grounded cross-attention layer is formulated as:
Saub softmax (QK"+
+ MLayoou Vd V V.
(5)
For the training samples where there is a lack of subject image, Magout is set to all O, then the masked cross-attention degrades into normal cross attention. Through subject-grounde grounded cross-attention layer, the information of each subject is restricted to be integrated within the corresponding bounding box. This ensures not only the independence between the generation of foreground subjects and background objects, but also the independence among multiple subjects. Owing to this, our model scamlessly enables the customization of multiple subjects. In summary, our proposed layer prevents information leakage and ensures accurate layout alignment of subjects.
3.3 Model Training
During training, for each image, we input only one subject image and its bounding box to the model, along with several text entities with their corresponding bounding boxes. The number of entities per training image is limited to 10 and we drop the rest ones. For portion of training samples that do not contain any valid subject images or text entities, we set the token of the subject image to be zero embeddings, and the layout of the subject to be all zeros. We keep the text encoder and DINOv2 image encoder frozen and merely fine-tune the multi-layer perceptron after the image encoder, the gated self-attention layers, the subject-grounded cross-attention layers, and the multi-layer perceptron after the DINOv2 image encoder.
3.4
Model Inference
Although our model is trained on singlo-subject data, it can be scamlessly extended to achieve multi-subject customization without retraining. In the inference stage, assume we have N subjects. As shown in Fig. J the vision token each subject will be injected into the corresponding bounding box region via the subject- grounded cross-attention layer. As we analyzed in section 3.2 the subject-grounded cross-attention layer encourages the independence of generating each subject, preventing potential false blending of visual con- cepts, e.g., the unnatural blending of two objects in the overlapping regions. Tt. also guarantees accurate layout control on all the subjects.
4 Experiment
Datasets We mix several datasets for training. For image pairs the same object, we use (1) multi-view data, MVIngNet YYu et al. 2023) and (2) video instance segmentation dataset Refer-Yon Tube-VOS (Seo)

---

Published in Transactions on Machine Learning Research (02/2026)
Table Comparison with existing methods on Dreambench. The top two results for each metric are highlighted with first, second background, respectively.
CLIP-T 1 CLIP-I 1 DINO t
BLIP-Diffusion
[L et al.
2024
0.2824
0.8894
0.7625
ELITE
[eij et al.
2023
0.2461
0.8926
0.7391
Kosmnss-
G
Pan et al
2023
0.2864
0.8452
0.6933
A-eclipse
Patel et al.
2024
0.2767
0.8901
0.7734
AnyDoor
Chen et a..
2023b
0.2416
0.9029
0.7781
GLIGEN
Li
2023
0.2898
0.8520
0.6890
Custom
Yuan et al.
2023
0.2815
0.9090
0.7526
MSDiffusion
Wang et
2024b)
0.3029
0.8982
0.7267
Ours
0.2931
0.9169
0.7950
et 2020 MVIngNet contains 6.5 million frames from 219,188 videos across 238 object categories, with fine-grained annotations of object masks. Refer-YouTTube- VOS dataset contains 3,978 high-resolution YouTube videos with 131k high-quality manual annotations and 15k language expressions. Following Any- Door (Chen et al. (202b)), for each object, we randomly selected two different frames from the same video clip to form a training pair. We apply the object mask on one frame to obtain the background-free object as the input subject image. We use the other frame as the ground-truth, and use its bounding boxes as the layout input. For single-image data, we use (3) LVIS (Gupta et al.) 2019), a well-known dataset for fino-grained large vocabulary instance segmentation, including 118,287 images from 1,203 categorics, and (4) Openlmages v7 dataset (Kuznetsova et al.) 2020),, which we only select the images with instance segmenta- tion annotations for training. We use the tround-truth segmentation mask and crop the image to obtain the background-free subject images. For each sample of single image data, select only the instance bounding boxes with top-10 largest areas to compose the layout, and choose the subject that has the largest area as the subject. for training.
Evaluation Metrics We calculate the CLIP-I RRafford et al. 2021 score and DINO (Caron et al. 2021 score assess the identity preservation performance of the subjects and use CLIP-T (Rdlord et al. 2021 score to evaluate the text alignment of the generated image. For evaluation of the model's grounding ability, we use AP and AP based on a pretrained YOLOv8 (Jocher et al., 2023) object detection model.
4.1 Single Subject Customization
We compare our work with existing state-of-the-art works on DreamBench (Ruiz et al. 2023 for the cus- tomization of a single subject. We mainly compare our model with existing encoder-insed customization methods, as our work falls in this line of research. In this experiment, we use the bounding box of the subject in the ground-truth image the imput layout. The qualitative and quantitative results are shown in Fig. and Table. D respectively. Overall, our method shows significantly better performance in layout align- ment, subject identity preservation, and text alignment. DLif-Diffusion (Li et al.
2024 ELITE (Wei et al.
2023), X-eclipse (Patel et al.) 2024) and MLLM-based method Kosmos-G Pan et al. 2023 fail to maintain accurate identity of the subjects. They also lack the ability of precise layout control. Any Door (Chen et al. 2023b) is designed for image composition. It can only generate subjects on a given background, but un- able to generate the background contents from texts. Although previous grounded text-to-image generation methods like GLIGEN (Li et al.
2023) can achieve layout control, it cannot preserve the identity of the
subjects. NustomNet (Yuan et al. 2023 achieves pose control to some extent. However, it highly relies
the pretrained model Zero123 (Liu et al. 2023). limiting the resolution of its generated image to be 256 x 256. Moreover, there can be obvious artifacts around the boundary of the generated subject. MS-Diffusion (Wang) et al. 2024b) can achieve grounded customization. However, it fails to maintain an accurate identity of the subject. We note that Brotndingooot and MS-Diffusion rely on different backbones, so the obscrved gap may reflect both the conditioning design and the underlying base model. Accordingly the results indicate different senggthh:: MS-Diffusion achieves higher CLIP-T Dream Bench, while GroundingBooth provides
7

---

Published in Transactions on Machine Learning Research (02/2026)
Prompt stuffed osima tree and cuturrn loovss in the background
Eclipse
AnyDoor
Diffuston
"A boot on hop of dirt
aninal white
cebolestonne street"
dog wearing
adowalk creaded street
Figure 5: Visual comparison with existing methods for the single-subject customization task. Zoom in to see the details.
Grounded Entities
Grounded Entities
backpack and bast with and mauntain in the background"
"A dog with sea and sky background"
lake
sky
"A toy dog
'A and vose with plant and vintage lantenn on quaint balcony"
with ocean and soiling ship
+
+
under sunset sky"
ocean
sailing
ship
plant
lantern
vintage
*A toy and cantoon
bockpack
and stuffed
with wheat field and house Gogh's countryside style"
animal +
and garden bench
with wildflowers
wild- flowers
garden
bench
wheat field
house
sunny day"
Figure 6: Multi-subject customization on DreamBench objects. Zoom in to see the details.
stronger identity preservation and more reliable joint grounding of foreground subjects and background text entities.
We observe that previous wewggrouddin--aaee customization methods tend generate objects that very large and in the center of the image, which increases the CLIP-I score and DINO score during evaluation. However, in real-world scenarios, users may want more control over the subject size in the generated images. They may also choose to generate a larger background with detailed textual information. In such cases, non-grounding customization methods fail generate the desired result. Results Fig. demonsttaate that our method achieves stronger identity preservation and more accurate layout alignment. We encourage the readers to view more visualizations in the Appendix.
8

---

Published in Transactions on Machine Learning Research (02/2026)
Table 2: Quantitative results of multi-subject customization.
CLIP-T M-CLIP-I 1 M-DINO
A-eclipse Patel et al.
2024
0.2735
0.8837
0.7428
MSDiffusion (Wang et al.
2024b)
0.2887
0.8865
0.7153
Ours
0.2905
0.9048
0.7556
Table 3: Quantitative results image customization with complex layout as inpuls on MS-COCO validation set. In this setting, we compare our method with methods only trained on COCO.
CLIP-T CLIP-1 f DINO 1 FID AP APso
LAMA (Li al. 2021
0.2507
0.8441
0.7330
69.50
13.1
18.2
UniControl (Qin et al.
2023
0.3143
0.8425
0.7598
42.22
4.53
12.8
LayoutDifftusin [Zheng et a..
2023
0,2738
0.8655
0.8033
37.90
23.4
37.3
GLIGEN (Li et al.
2023)
0.2899
0.8688
0.7792
33.14
23.9
38.2
InstanceDiifusion Wang et al.
2024c
0.2914
0.8391
0.7939
37.57
36.1
50.3
Ours
0.2968
0.9095
0.8592
25.63
37.4
52.6
4.2 Sulti-Subject Customization
cross-attention layer, our model seamlessly supports the customization of multiple subjects. Fig. O shows the qualitative results of multi-subject customization. In this experiment, there are also multiple text entities along with their bounding boxes to describe the background contents. We observe that when inputting multiple subjects such as a bag and a boot, along with the layout of the background text entities such as the mountain and the lake, our model successfully generates the subjects and background with an accurate layout alignment for each visual concept. The identities of the subjects are preserved and the overall image is well-aligned with the prompt. Moreover, in several cases, even when the bounding boxes of the foroground objects have a large overlap with the background text entities, the model can disentangle subject-driven foreground generation from text-driven background generation, effectively avoiding context blending.
With our proposed
To evaluate the model's identity preservation performance on multi-subject customization quantitatively, we first compute the DINO score between each input subject and the generated image, then calculate the average score. For clarity, name this score Multi-DINO (M-DINO). Similarly, we follow this process but use CLIP-I score instcad to obtain the Multi-CLIP-I (M-CLIP-I) score. In practice, we randomly select 2 subjects from DreamBench, and composite layout for them as the inputs, then evaluate the models. We compare our model with baselines that support multi-subject customization. Results in Table. 2 show that our model achieves better text alignment and identity preservation in multi-subject customization.
4.3 Customization with Complex Layout
We evaluate our model's performance on the COCO (Lin et al. 2014) validation set, where the input layout and text entities are very complex. For each testing image, we use the largest object as the reference object (i.e., the subject), and the remaining text entities as background entities. To quantify the model's grounding ability, adopt YOLOv8 (Jocher et al., 2023) the object detection method, and test the evaluation results using COCO's official evaluation metrics(ALe AP and APs0). Quantitative and qualitative results are shown in Table. 3 and Fig. 7 respectively. Results show that even if we input complex layouts and text entities, our model can still generate high-quality scenes with precise layout alignment for all the objects and regions, and accurate identity prescrvation for the subjects, while preserving the text alignment. Compared with existing layout-to-image generation methods, our model shows a competitive accuracy in grounding the visual concepts and remarkable improvement on identity preservation.
9

---

Published in Transactions on Machine Learning Research (02/2020)
Caption-Tag
Reference Object
of Reference object
LayoutDiffusion
InstanceDiffusion
"The red, double
decker bus
Sprng
driving
other
bus
"These platters display healthy
food choices of
two entrees with
Available
vegetable and fruit'
bowl
"A kötchenn filled
Not Available
top oven and refrigenator*
refrigerator
"Three
in field near bushes'
zebra
front wooden bench garden'
Available
bench
"Two brown teddy
bears sitting side side'
teddy bear
Figure 7: Visual results of image customization with complex layout and text entities as conditions the COCO validation set. Note that LayoutDiftusion (Zheng et al., 2023) is only conducted on the COCO dataset with filtered annotations, so some of its results are not available.
"a teddybear wearing hat in the snow"
Figure 8: More results about object interactions.
4.4 Further Comparison
Fig. provides a qualitative comparison among our method MS-Diffusion, tical prompts and layout specifications. Notice that Mano-Banana-Pro is the stato-of-tho-art closc-source model. As shown in the examples, both MS-Diffusion and background text entities at their designated regions and exhibits noticeable spatial deviations for foreground objects, whereas our method achieves accurate grounding for both foreground reference subjects and back- ground text entities with more reliable spatial control.
under iden-
struggle consistently ground
We also compare our result on VLM-based metrics in Table. 6 Specifically, we adopt VIEScore, which leverages a large vision-language model assess the semantic alignment between the input prompt and the generated image. In our experiment, we use GPT-4o as VLM judge. Compared with previous methods, our method achieves competitive performance in prompt-following ability.
10

---

Published in Transactions on Machine Learning Research (02/2026)
MIS Dit-Dittuionn
Nano Banana Pro
Ours
"A backpack and stuffed animal +
with wildflowers and gorden bench in sunny day"
wild- flowers
garden
bench
"A and cartoon toy
with wheat field and house a + Van Gogh's
wheat field
countryside style"
house
Figure 9: Qualitative comparison with MS-Diffusion and Mano-Banana-Pro. MS-Diffusion and Nano- Banana-Pro struggles to consistently ground background text entities (e.g., wildflowers, garden bench, wheat field, house) at the specified locations, and its foreground layout often exhibits noticeable spatial deviations. In contrast, our method achieves accurate grounding for both foreground reference subjects and background text entities, while providing more reliable spatial control aligned with the input bounding boxes.
Table 4: Ablation study for model components Dreambench. GM: Grounding Module. SG-
Table 5: Ablation Study for model components on MS-
COCO Validation Set. GM: Grounding Module. SG-CA:
CA: Subject-Grounded Cross- Attention.
Subject-Grounded Cross- Attention.
CLIP-T
CLIP-I f DINO f
CLIP-T 1 CLIP-I f DINO 1
J AP
APso
w/o GM 0.2762
0.8578
0.7049
w// GM
0.2796
0.8605
0.7740
40.63
22.1
28.5
w/o SG-CA
0.2878
0.8616
0.7065
w/o SG-CA
0.2884
0.8707
0.7970
31.29
28.5
38.6
Full
0.2931
0.9169
0.7950
Full
0.2968
0.9095
0.8592
25.63
37.4
52.6
4.5 Results on Object Interaction
Owing to the accurate layout control and identity preservation of multiple subjects, our model naturally enables realistic object interactions. Fig. g highlights its ability to seamlessly composite reference objects and faithfully capture their interactions.
4.6 Ablation Study
We conduct the ablation study to validate the effectiveness of proposed components: the subject- grounded cross-attention layer and the grounding module. Table. 4 and Table. 5 present the quantitative results on DreamBench and COCO, respectively. We observe that both components play a vital role in improving the model's capacities of identity preservation, layout alignment, and text alignment.
In addition, our model also seamlessly support several simpler tasks by dropping some conditions, including pure text-to-image synthesis, pure layout-guided text-to-image synthesis, and the traditional personalizod text-to-image synthesis. We put the related analysis and results in appendix.
We also compared our full model against variant that replaces Fourier positional encoding with standard Lincar Coordinate Encoding (i.c., directly projecting normalized bounding box coordinates via an MLP) in Table. Empirical results demonst that substituting Fourier encoding with linear encoding leads to a noticeable degradation in layout alignment, as reflected by a decline in the YOLO-AP score. This confirms that Fourier Positional Encoding essential for mapping positional information into a high-frequency feature space, thereby overcoming the spectral bias of neural networks and enabling the model strictly adhere bounding box constraints.
11

---

Published in Transactions on Machine Learning Research (02/2026)
Table 6: Quantitative results on VLM-Based Metrics.
Table 7: Ablation Study for different position
SC 1 PQ 1 Overall
encoding method on MS-COCO Validation Set.
AP
Alto
CustomNet
Yuan et 2023
7.805
4.472
5.769
MSDiffusion
Waang et al.
2024b
7,850
7.256
7.453
Linear
7.1
11.5
Ours
7.891
7.989
7.690
Fourier
37.4
52.6
Table 8: User Study based Drnch:ench: In the questions, the user is presented side-by-side comparisons of our generated image and another image randomly chosen from one of the basclines. The results in the table show user preference percentage.
Ours
CustomNet
Ours
AnyDoor
Ours
GLIGEN
Identity
60.78
39.22
59.31
40.69
72.81
27.19
Grounding
56.86
43.14
64.21
35.79
58.25
41.75
Text Alignment
51.96
48.03
73.52
26.47
55.34
44.66
Overall Quality
54.41
45.58
62.25
37.74
58.74
41.26
4.7
User Study
Table. shows the preference results comparing model with cxisting models (Chen et al.) 2023b;
Yuan et al. 2023 Li et al. 2023) on DreamBench. Specifically, given the same input, we first generate results with each model. Then we ask the users to make side-by-side comparison between our result and a randomly chosen result from the baselines regarding identity preservation, text alignment, grounding ability, and overall image quality. We collect user responses using Amazon Mechanical Turk. Results show that participants have significantly higher preference over our method. We put more details appendix.
5 Conclusion
We present GroundingBooth, general framework for the grounded text-to-image customization task. Our model achieves accurate layoul grounding for both image subjects and lext entities while preserving the details of the subject and maintaining text-image aligmment, outperforming existing methods. Our results suggest that the proposed grounding module and the subject-grounded cross-attention layer are effective in generating distinct objects within each bounding box and improving the identity of the subjects. We hope our research can motivate the exploration of more identity-preservinn and controllable foundation generative models, enabling more advanced visual editing.
References
Moab Arar, Rinon Gal, Yuval Atzmon, Gal Chechik, Daniel Cohen-Or, Ariel Shamir, and Amit H. Bermano. Domain-agnostic tuning-encoder for fast personalization of text-to-image models. In SIGGRAPH Asia 2023 Conference Papers, pp. 1 10, 2023.
Omri Avrahami, Kfir Aberman, Ohad Fried, Daniel Cohen-On and Dani Lischinski. Break-a-scene: Ex- tracting multiple concepts from a single image. In SIGGRAPH Asia 2023 Conference Papers, pp. 1-12, 2023.
Mathilde Caron, Hugo Touvron, Ishan Misra, Hervé Jégou, Julien Mairal, Piotr Bojanowski, and Armand Joulin. Emerging properties in self-supervised vision transformers. In Proceedings of the IEEE/CVI international conference on computer vision, pp. 9650-9660, 2021.
Li Chen, Mengyi Zhao, Yiheng Liu, Mingxu Ding, Yangyang Song, Shizun Wang, Xu Wang, Hao Yang, Jing Liu, Kang Du, al. Photoverse: Tuning-fre free image customization with text-to-image diffusion models, 2023a.
12

---

Published in Transactions on Machine Learning Research (02/2026)
Xi Chen, Lianghua Huang, Yu Liu, Yujun Shen, Deli Zhao, and Hengshuang Zhao. Anydoor: Zero-shot object-level image customization. aixiv preprint or arXiv:337.004481, 2023b.
Weixi Feng, Wanrong Zhu, Tsu-jui Fu, Varun Jampani, Arjun Akula, Xuehai He, Sugato Basu, Xin Fric Wang, and William Yang Wang. Layoutgpt: Compositional visual planning and generation with large language models. Advances in Newral Information Processing Systems, 36, 2024.
Rinon Gal, Yuval Alaluf, Yuval Atzmon, Or Patashnik, Amit H. Bermano, Gal Chechik, and Daniel Cohen- Or. An image is worth one word: Personalizing text-to-image generation using textual inversion, 2022.
URL hntps: org/ abs//208.....2 2208. .06618
Rinon Gal, Moab Arar, Yuval Atzmon, Amit H Bermano, Gal Chechik, and Daniel Cohen-Or. Encoder-based domain tuning for fast personalization of text-to-image models. ACM Transactions on Graphics (TOG), 42(4))1-13, 2023.
Agrim Gupta, Piotr Dollar, and Ross Girshick. LVIS: A dataset for large vocabulary instance segmentation. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2019.
Glenn Jocher, Ayush Chaurasia, and Jing Qiu. Ultralytics yolo, January 2023. URL ultrallyticcsultrralyyicc
com/
Alexander Kirillov, Eric Mintun, Nikhila Ravi, Hanzi Mao, Chloe Rolland, Laura Gustafson, Tete Xiao, Spencer Whitchead, Alexander C Berg, Wan-Yen Lo, et al. Segment anything. In Proceedlings of the IEEE/CVP International Conference on Computer Vision, pp. 4015 4026, 2023.
Nupur Kumari, Bingliang Zhang, Richard Zhang, Eli Shechtman, and Jun-Yan Zhu. Multi-concept cus- tomization of text-to-image diffusion. 2023.
Alina Kuznelsova, Hassan Rom, Neil Alldrin, Jasper Uijlings, Ivan Krasin, Jordi Pont-Tuset, Shahab Kamali, Stefan Popov, Matteo Malloci, Alexander Kolesnikov, Tom Duerig, and Vittorio Ferrari. The open images dataset v4: Unified image classification, object detection, and visual relationship detection at scale. IJCV, 2020.
Dongxu Li, Jumnan Li, and Steven Hoi. Blip-diffusion: Pre-trained subject representation for controllable text-to-image generation and editing, Advances in Neural Information Processing Systems, 36, 2024.
Yuheng Li, Haotian Liu, Qingyang Wu, Fangzhou Mu, Jianwei Yang, Jianfeng Gao, Chunyuan Li, and Yong Jac Loc. Gligen: Open-set grounded text-to-image generation. pp. 22511-22521, 2023.
Zejian Li, Jingyu Wu, Immanuel Koh, Yongchuan Tang, and Lingyun Sun. Image synthesis from layout with locality-aware mask adaption. In IEEE International Conference on Computer Vision (ICCV), pp. 13819-13828. IEEE, 2021.
Tsung-Yi Yi Lin, Michael Maire, Serge Belongie, James Hays, Pictro Perona, Deva Ramanan, Piotr Dollár, and C Lawrence Zitnick. Microsoft coco: Common objects in context. In Computer vision ECCV 2014: 13th European conference, zurich, Switzerland, September 6-12, 2014, proceedings, part v 13, pp. 740 755. Springer, 2014.
Ruoshi Liu, Rundi Wu, Basile Van Hoorick, Pavel Tokmakov, Sergey 'Zakharov, and Carl Vondrick. Zero- 1-to-3: Zero-shot one image to 3d object. In Proceedings of the TEEE/CVE international conference computer vision, pp. 9298-9309, 2023.
Maxime Oquab, Timothée Darcet, Théo Moutakanni, Huy Vo, Mare Szafraniec, Vasil Khalidov, Pierre Fernandez, Daniel Haziza, Francisco Massa, Alaacldin El-Nouby, et al. Dinov2: Learning robust visual features without supervision. arXiv preprint ar XrXiv22304.07193, 2023.
Xichen Pan, Li Dong, Shaohan Huang, Zhiliang Peng, Wenhu Chen, and Furu Wei. Kosmos-g: Generating images in context with multimodal large language models. Xix preprint arXi:22100008 020999, 2023.
13

---

Published in Transactions on Machine Learning Research (02/2026)
Yulin Pan, Chaojie Mao, Zeyinzi Jiang, Zhen Han, and Jingfeng Zhang. Locate, assign, refine: Taming customized image inpainting with text-subject guidance. arXiv preprint or Krx:24403. 19534, 2024.
Maitreya Patel, Sangmin Jung, Chitta Baral, and Yezhou Yang. X-eclipse: Multi-concepl personalized text-to-image diffusion models by leveraging clip latent space. arXio preprint 1tXiu:2402.05195, 2024.
Can Qin, Shu Zhang, Ning Yu, Yihao Feng, Xinyi Yang, Yingbo Zhou, Huan Wang, Juan Carlos Niebles, Caiming Xiong, Silvio Savarese, et al. Unicontrol: A unified diffusion model for controllable visual gener- ation in the wild. arXiv preprint arXiv:2305. 11147, 2023.
Leigang Qu, Shengqiong Wu, Hao Fei, Liqiang Nic, and Tat-Seng Chua. Layoutllm-t2i: Eliciting layout guidance from Imm for text-to-image generation. In Proceedings of the 31st ACM International Conference on Multimediaa, pp. 643 654, 2023.
Alec Radford, Jong Wook Kim, Chris Hallacy Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervision. In International conference machine learning, pp. 8748-8763. PMLR, 2021.
Daniel Roich, Ron Mokady, Amit H Bermano, and Daniel Cohen-Or. Pivotal tuning for latent-based editing of real images. ACM Trans. Graph., 2021.
Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Björn Ommer. High-resolution image synthesis with latent diffusion models. 2022.
Nataniel Ruiz, Yuanzhen Li, Varun Jampani, Yael Pritch, Michael Rubinstein, and Kfir Aberman. Dream- booth: Fine tuning text-to-image diffusion models for subject-driven generation. In Proceedings of the IRFE/CVF conference on compuler vision and pattern recognition, pp. 22500-22510, 2023.
Seonguk Seo, Joon- Young Lee, and Bohyung Han. Urvos: Unified referring video object segmentation net- work with a large-scale benchmark. In Computer Vision ECCV 2020: 16th European Conference, Glas- gow, UK, August 23 28, 2020, Proceedings, Part XV, pp. 208 223, 2020. doi: 10.1007/997--3-00--55555
13. URL https://doi. org/10.
Jing Shi, Wei Xiong, Zhe Lin, and Hyun Joon Jung. Instantbooth: Personalized text-to-image generation without test-time finetuning. arXiv preprint arXiv:2304. 034411 2023.
Yizhi Song, Zhifei Zhang, Zhe Lin, Scott Cohen, Brian Price, Jianming Zhang, Soo Ye Kim, and Daniel Aliaga. Objectstitch: Generative object compositing. arXiv preprint XXi::2222.00992 2022.
Yizhi Song, Zhifei Zhang, Zhe Lin, Scott Cohen, Brian Price, Jianming Zhang, Soo Ye Kim, He Zhang, Wei Xiong, and Daniel Aliaga. Imprint: Generative object compositing by learning identity-preserving representation. In Proceedings of the IEEE/CVT Conference on Compuler Vision and Pallern Recognition, pp. 8048 8058, 2024.
Wei Sun and Tianfu Wu. Image synthesis from reconfigurable layout and style. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 10531-10540,, 2019.
Zhenxiong Tan, Songhua Liu, Xingyi Yang, Qiaochu Xue, and Xinchao Wang. Ominicontrol: Minimal and universal control for diffusion transformer. arXiv preprint 1rXiv:2411.15098 2024.
Matthew Tancik, Pratul Srinivasan, Ben Mildenhall, Sara Fridovich-Kan Nithin Raghavan, Utkarsh Singhal, Ravi Ranamoorthi, Jonathan Barron, and Ren Ng. Fourier features let networks learn high frequency functions in low dimensional domains. Advances in neural information processing systems, 33:7537-75. 2020.
Bo Wang, Tao Wu, Minfeng Zhu, and Peng Du. Interactive image synthesis with panoptic layout generation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 7783-7792, 2022.
14

---

Published in Transactions on Machine Learning Research (02/2026)
Haoxuan Wang, Jinlong Peng, Qingdong He, Hao Yang. Ying Jin, Jiafu Wu, Xiaobin Hu, Yanjie Pan, Zhenye Gan, Mingmin Chi, et al. Unicombine: Unified multi-conditional combination with diffusion transformer. arXiv preprint arXic:2503.09277 2025.
Qixun Wang, Xu Bai, Haofan Wang, Zekui Qin, and Anthony Chen. Instantid: Zero-shot identity-preserving generation in seconds. arXiv preprint 2711:2201..77519, 2024a.
Xiaowei Wang, Siming Fu, Qihan Huang, Wanggui He, and Hao Jiang. Ms-diffusion: Multi-subject zero-shot image personalization with layout guidance. ar preprint arXiv:2406. 0000200600209, 2024b.
Xudong Wang, Trevor Darrell, Sai Saketh Rambhatla, Rohit Girdhar, and Ishan Misra. Instancediffusion: Instance-level control for image generation. In Proceedings of the IEEE/CVF Conference on Compuler Vision and Pattern Recognition, pp. 6232 6242, 2024c.
Yuxiang Wei, Yabo Zhang, Zhilong Ji, Jinfeng Bai, Lei Zhang, and Wangmeng Zuo. Elite: Encoding visual concepts into textual embeddings for customized text-to-image generation. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 15943 15953, 2023.
Guangxuan Xiao, Tianwei Yin, William T Freeman, Frédo Durand, and Song Han. Fastcomposer: Tuning- free multi-subject image generation with localized altention. xrxiv preprint Xrv: N00:23355.0431, 2023.
Xianggang Yu, Mutian Xu, Yidan Zhang, Haolin Liu, Chongjie Ye, Yushuang Wu, Zizheng Yan, Tianyou Liang, Guanying Chen, Shuguang Cui, and Xiaoguang Han. Mvimgnet: large-scale dataset of multi-view images. In CVPR, 2023.
Ziyang Yuan, Mingdeng Cao, Xintao Wang, Zhongang Qi, Chun Yuan, and Ying Shan. Customnet: Zero- shot object customization with variable-vieepoints in text-to-image diffusion models. arXiv preprint arXiv:2310.19784, 2023.
Lvmin Zhang, Anyi Rao, and Maneesh Agrawala. Adding conditional control to text-to-image diffusion models. 2023.
Yuxuan Zhang, Yiren Song, Jiaming Liu, Rui Wang, Jinpeng Yu, Hao Tang, Huaxia Li, Xu Tang, Yao Hu, Han Pan, et al. Ssr-encoder: Encoding selective subject representation for subject-driven generation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 8069 8078, 2024.
Yuxuan Zhang, Yirui Yuan, Yiren Song, Haofan Wang, and Jiaming Lin. Easycontrol: Adding efficient and flexible control for diffusion transformer. arXiv preprinl arXiv:2503.070 07027, 2025.
Guangcong Zheng, Xianpan Zhou, Xuewei Li, Zhongang Qi, Ying Shan, and Xi Li. Layoutdiffusion: Con- trollable diffusion model for layout-lo-image generation. 2023.
15


Reviews:
[{"review_id": "g6vsoZdPM0", "reviewer": "Reviewer_C87i", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "No", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This paper proposes GroundingBooth, a framework for grounded text-to-image customization. The task aims to generate images where: (1) foreground subjects preserve the identity of input reference images, (2) both subjects and background text entities adhere to specified bounding box layouts, and (3) the overall image aligns with the text prompt.\n\nThe technical contributions include:\n* A subject-grounded cross-attention layer that uses layout masks to restrict the injection of subject features within corresponding bounding boxes, preventing context blending between foreground and background.\n* The framework supports multi-subject customization at inference time despite being trained on single-subject data.\n\nStrengths:\n* The task formulation of jointly grounding both subject-driven foreground and text-driven background is practically relevant.\n* Comprehensive experiments on DreamBench and COCO with multiple evaluation metrics (CLIP-T, CLIP-I, DINO, AP, FID).\n* The paper is clearly written and the method is easy to follow.\n\nWeaknesses:\n* Limited technical novelty: the gated self-attention is directly from GLIGEN, and the subject-grounded cross-attention is similar to existing masked attention mechanisms in prior works.\n* Generated subjects exhibit nearly identical poses to input reference images, contradicting claims of generating \"natural poses.\"\n* The distinction from MS-Diffusion (already accepted at ICLR 2025) is not convincingly demonstrated.", "claims_and_evidence_yn": "No", "claims_explanation": "1. The claimed novelty of the subject-grounded cross-attention layer is limited.The paper presents this as a key contribution, but the core technique of using layout masks to constrain cross-attention regions (Eq.4-5) has been explored in prior works:\n* \"Be Yourself: Bounded Attention for Multi-Subject Text-to-Image Generation\" (ECCV 2024)\n* InstanceDiffusion (CVPR 2024) uses masked attention for instance-level control\n* \"Training-Free Layout Control with Cross-Attention Guidance\" (WACV 2024)\n\n2. The distinction from MS-Diffusion is not convincingly supported. The paper claims MS-Diffusion \"fails to spatially control the background contents\" (Sec 2). However:\n* Table 1 shows MS-Diffusion achieves higher CLIP-T (0.3029) than GroundingBooth (0.2931), suggesting better text alignment including background descriptions.\n* No direct experiment demonstrates GroundingBooth's superior background grounding capability.\n\n3. The claim of generating subjects with \"natural poses\" (Sec 3) is not supported by visual evidence. In Fig.1, Fig.5, and Fig.6, the generated subjects consistently maintain nearly identical poses to their input reference images.", "audience_yn": "Yes", "audience_explanation": "The problem of jointly controlling layout and preserving subject identity in text-to-image generation is practically important for applications such as  E-commerce product visualization and controllable image editing.", "requested_changes": "1. Clarify novelty over prior masked attention methods. Provide detailed technical comparison with Be Yourself (ECCV 2024), InstanceDiffusion (CVPR 2024), and Training-Free Layout Control (WACV 2024). Explain what is fundamentally new in the subject-grounded cross-attention beyond applying layout masks to attention.\n\n2. Provide convincing evidence for background grounding superiority over MS-Diffusion. Include:\n    * Direct comparison on background object placement accuracy\n    * Ablation showing that background text entities are correctly grounded\n    * Update the \"concurrent work\" framing since MS-Diffusion predates this submission\n3. Address the pose consistency issue. Provide examples where generated subjects exhibit pose variations adapted to the scene context, or explicitly acknowledge this as a limitation. Currently, the visual results contradict the claim of generating \"natural poses.\"", "broader_impact_concerns": "No major ethical concerns are identified with this work. The method focuses on personalized image generation with layout control, which has applications in creative tools and content creation. As with all subject-driven generation methods, there is potential for misuse in generating misleading imagery of objects/products. However, this is a general concern for the field rather than specific to this work.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "5uU21V1F4m", "reviewer": "Reviewer_xcvP", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "Summary:\n\nThe paper introduces grounded diffusion-based text-to-image customization, where both: foreground subjects (from reference images), and background objects (from text entities) are placed at specified bounding boxes, while preserving subject identity and following the text prompt. GroundingBooth proposes a grounding module that injects bounding-box information to guide the placement of subjects and background elements, along with a subject-grounded cross-attention layer that separates subject appearance modeling from background synthesis, reducing visual entanglement and supporting accurate identity-preserving generation for multiple subjects.\n\n\nStrength:\n* The paper is well written, easy to follow, and clearly motivates the problem and proposed solution.\n* GroundingBooth demonstrates effective layout control for both foreground subjects and background objects while maintaining strong subject identity preservation.\n* Although trained only on single-subject customization, the model generalizes well to multi-subject customization at inference time.\n\nWeakness:\n* The method assumes strong consistency between free-form text prompts and box-level layout conditions. However, in realistic scenarios, these two sources may conflict (e.g., scale, relative position, or relational semantics). Since the proposed subject-grounded cross-attention enforces hard spatial masking, the model appears unable to reconcile such conflicts. Could the authors clarify how mismatches are handled, and whether any hard-negative or conflict-aware training was considered? An analysis of failure cases under prompt–layout inconsistency would significantly highlight the effective of GroundingBooth\n* The method enforces layout constraints via hard masking, but does not appear to offer any adjustable mechanism to balance or arbitrate between free-form text prompts and bounding-box conditions. Could the authors clarify whether any parameters exist to control the relative influence of text versus layout? Instead of a single learnable scalar $\\gamma$ in the gated self-attention for global grounding tokens.\n* In Fig. 5 (“the boot”, “stuffed animal”, etc.), I observe that the size and aspect ratio of the bounding box significantly influence the generated object’s pose and shape. This suggests that the box acts not only as a localization constraint but also as an implicit geometric prior. Could the authors clarify whether the generated objects are expected to be invariant to bounding box scale/aspect ratio, and if not, how users should choose boxes to avoid unintended pose distortion?\n* The method supports multi-subject customization by injecting multiple subject-grounded attentions. However, the paper does not clarify whether the order of subject bounding boxes affects generation, particularly when boxes overlap or are spatially adjacent.\n* Could the authors clarify how inference latency and memory usage scale with the number of subject and text-entity bounding boxes? \n* The grounding module uses Fourier positional encoding for bounding boxes. Could the authors elaborate on the motivation for this choice, and whether simpler positional encodings (e.g., normalized box coordinates or learned embeddings) were considered?\n* The method relies on SAM to extract background-free subject crops, yet the crop tightness can significantly affect the identity features extracted by DINOv2. Here, IoU refers to the overlap between the SAM crop and the object extent, where higher IoU corresponds to tighter, object-centric crops and lower IoU indicates global viewing of the object. Tighter crops may improve identity purity but lose pose and contextual cues, while looser crops introduce feature extraction biases. Have the authors evaluated performance sensitivity to crop scale or IoU, and considered multiscale or margin-randomized cropping to improve robustness across objects with varying IoU?", "claims_and_evidence_yn": "Yes", "claims_explanation": "The paper presents clear mathematical formulations alongside comprehensive quantitative and qualitative results. Overall, the experimental evidence appears well grounded and consistent with the proposed architecture and the object-centric dataset preparation, supporting the validity of the authors’ claims.", "audience_yn": "Yes", "audience_explanation": "The paper would be of interest to individuals in the TMLR audience working on text-to-image generation, controllable diffusion models, and personalized image synthesis, particularly those interested in combining identity preservation with spatial grounding. The proposed architecture and empirical findings provide useful insights for researchers studying controllability, compositionality, and object-centric conditioning in generative models.", "requested_changes": "I recommend the authors to address one-by-one each of the recommendation listed below:\n\n* The current method assumes strong consistency between free-form text prompts and box-level layout constraints, yet these may conflict in realistic scenarios (e.g., scale, relative position, relational semantics). I'd suggest to clarify how such mismatches are handled and whether conflict-aware or hard-negative training strategies were considered. An explicit failure-case analysis under prompt–layout inconsistency would help better characterize the strengths and limitations of GroundingBooth.\n\n\n* While a learnable scalar $\\gamma$ modulates the contribution of global grounding tokens, it does not offer fine-grained or instance-level arbitration between text prompts and bounding-box conditions. Is there any mechanism worth discussing to balance / weight the text and layout constraints. Could the authors clarify whether any parameters / module exist to control the relative influence of text versus layout\n\n* Qualitative results (e.g., Fig. 5) suggest that bounding box size and aspect ratio influence object pose and shape, indicating that boxes act as implicit geometric priors rather than pure localization cues. The authors should clarify whether invariance to box scale/aspect ratio is expected, and provide guidance on how users should choose bounding boxes to avoid unintended pose distortion.\n\n* Although the method supports multiple subjects, it is unclear whether the order of subject bounding boxes affects generation, particularly in cases of overlapping or adjacent boxes. Clarifying whether the model is order-invariant, or specifying a recommended ordering strategy, would improve reproducibility and usability.\n\n*  How the inference latency and memory usage scale with the number of subject and text-entity bounding boxes, given the per-subject grounded cross-attention applied across U-Net blocks and diffusion steps.\n\n* The grounding module adopts Fourier positional encoding for bounding boxes, but the motivation for this choice is not fully articulated. I'd suggest discussing why simpler encodings (e.g., normalized box coordinates or learned embeddings) are insufficient, or by providing an ablation study.\n\n* Since subject identity is extracted from SAM-based crops, performance may be sensitive to crop tightness (IoU), where tighter crops favor identity purity while looser crops preserve contextual cues. The authors are recommended to analyze performance as a function of crop IoU and consider multiscale or margin-randomized cropping to improve robustness across diverse object sizes and contexts. An ablation study would ground the statement!", "broader_impact_concerns": "No", "additional_comments": "No", "extra_scores": {}, "extra_text": {}}, {"review_id": "0RxKUPXKYP", "reviewer": "Reviewer_MPBw", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This paper proposed GroundingBooth, a grounded text-to-image customization framework that jointly achieves layout control for both foreground subjects and background objects, while preserving subject identity and supporting multi-subject customization. It introduces a grounding module using Fourier-encoded bounding boxes for both text entities and subjects, and a subject-grounded cross-attention layer that spatially masks subject tokens to disentangle subject generation from text-driven background generation and avoid concept blending. The model can be trained once without per-subject finetuning, demonstrates improved text alignment, identity preservation, and grounding metrics over prior open methods on DreamBench and COCO. Ablations show both new modules are crucial to the gains.​", "claims_and_evidence_yn": "Yes", "claims_explanation": "The paper is generally well written and easy to follow, and the motivation and problem setup are clearly stated. The method, experiments, and ablations are technically sound.", "audience_yn": "Yes", "audience_explanation": "Text-to-Image Customization is a long studied problem since the textual inversion / dreambooth paper (2022), and it has been a hot topic. However, most of the methods required per-subject finetuning. a few representative prior works like Gligen and Anydoor has provided solutions to achieve zero-shot Text-to-Image Customization, while GroundingBooth provided another route to achieve similar performance and thus potentially interesting to TMLR's audiences.", "requested_changes": "C1) According to the official TMLR template, its not allowed to put arbitrary content (including images/figures) before the abstract unlike other conference templates. The teaser figure needs to be moved under introduction.\n\nC2) There is no section that benchmarks against or even qualitatively compares to close-source SOTA models (like Nano banana), so it is hard to gauge how close GroundingBooth is to close-source SOTA models. I think it would strengthen the work to include either a qualitative comparison or a discussion of (i) the current performance gap to such models, or (ii) scenarios where GroundingBooth's explicit grounding and identity preservation provide advantages that typical proprietary models do not currently offer.\n\nC3) It took me a non-trivial amount of effort to reconstruct how GroundingBooth differs architecturally from prior methods such as DreamBooth-style customization, encoder-based methods, and GLIGEN-like grounded diffusion. While the text and current figures describe the proposed modules, the differences to baselines are spread across sections. I recommend adding a simple crisp side-by-side architectural diagram that contrasts GroundingBooth with representative prior approaches. This would let readers quickly grasp the architectural novelty without having to piece it together from multiple paragraphs.\n\nC4) It would strengthen the evaluation to add a VLM-based metric such as VIEScore (or at least report it on a subset on DreamBench++), as VLM-based metric has become a popular evaluation method for evaluating generated images.", "broader_impact_concerns": "/", "additional_comments": "/", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Decision: Accept (poster)