ParallelEdits: Efficient Multi-Aspect Text-Driven Image Editing with Attention Grouping
Authors: Mingzhen Huang
Jialing Cai
Shan Jia
Vishnu Suresh Lokhande
Siwei Lyu
Venue: NeurIPS 2024 poster (2024)
Keywords: Diffusion Models
Image Editing; Multi-aspect editing;consistency models
Primary area: diffusion_based_models

Abstract: Text-driven image synthesis has made significant advancements with the development of diffusion models, transforming how visual content is generated from text prompts. Despite these advances, text-driven image editing, a key area in computer graphics, faces unique challenges. A major challenge is making simultaneous edits across multiple objects or attributes. Applying these methods sequentially for multi-attribute edits increases computational demands and efficiency losses. 
    In this paper, we address these challenges with significant contributions. Our main contribution is the development of ParallelEdits, a method that seamlessly manages simultaneous edits across multiple attributes. In contrast to previous approaches, ParallelEdits not only preserves the quality of single attribute edits but also significantly improves the performance of multitasking edits. This is achieved through innovative attention distribution mechanism and multi-branch design that operates across several processing heads. 
    Additionally, we introduce the PIE-Bench++ dataset, an expansion of the original PIE-Bench dataset, to better support evaluating image-editing tasks involving multiple objects and attributes simultaneously. This dataset is a benchmark for evaluating text-driven image editing methods in multifaceted scenarios.

ParallelEdits Efficient Multi-Aspect Text-Driven Image Editing with Attention Grouping
Mingzhen Huang, Jialing Cai, Shan Jia, Vishnu Suresh Lokhandex, Siwei Lyu+ University at Buffalo, State University of New York, USA
Source prompt: A striped brown cat sits on weoden planks ogainst wooden wall
Source prompt: man sitting in boot is silhouetted against surset with mountain in the background
Source Image
InfEdit
Direct Inversion
PorallelEdits (Ours)
Number of edited
aspects
@
@ beach
c curtain
grey cat
ducks
bright sun city
>
mon
@ necktie
sunglasses
ball
X bridge
sallboat
wool nug
lawn
+ fireworks
evening glow
curled
castle
Figure 1: Multi-aspect text-driven image editing, Multiple edits in images pose significant challenge in existing models (such as Directlnverison II] and InfEdit (2)). their performance downgrades with an increasing number of aspects. In contrast, our ParallelFdits seconds. The symhol @ denotes a swap action, the symbol @ denotes an object addition action, and symbol @ denotes an object deletion, Arrows (-+) on the image highlight the aspects edited by our method.
achieve precise multi-aspect image editing in
Abstract
Text-driven image synthesis has made significant advancements with the devel- opment of diffusion models, transforming how visual content is generated from text prompts. Despite these advances, text-driven image editing, a key area computer graphies, faces unique challenges. A major challenge is making si- multaneous edits across multiple objects or attributes. Applying these methods sequentially for multi-aspect edits increases computational demands and efficiency losses. In this paper, we address these challenges with significant contributions. Our main contribution the development of ParallelEdits, method that seam- lessly manages simultaneous edits across multiple attributes. In contrast to previous approaches, ParallelEdits not only preserves the quality of single attribute edits but also significantly improves the performance of multitasking edits. This is achieved through innovative attention distribution mechanism and multi-branch design that operates across several processing heads. Additionally, we introduce the PIE-Bench+ dataset, an expansion of the original PIE-Bench dataset, better support evaluating image-editing tasks involving multiple objects and attributes si- multaneously. This dataset benchmark for evaluating text-driven image editing methods in multifaceted scenarios. Codes are available at: https://ming/hhnn httpgeghbbboopooctss///lll/llu/
"Corresponding authors
38th Conference on Neural Information Processing Systems (NcurIPS 2024).

---

Introduction
Recently, text-driven image editing has experienced remarkable growth, driven by advances in diffusion-based image generative models. This technique involves modifying existing images based on textual prompts to alter objects, their attributes, and the relationships among various objects. The can produce edited images that closely match the semantic content described in the prompts while keeping the rest of the image unchanged. Unlike early image editing approaches that required image matting to precisely extract foreground objects using alpha mattes [5]. text-driven editing offers less labor-intensive alternative. User-provided textual prompts guide the edits, with auxiliary inputs like masks facilitating localized modifications [6].
latest methods [
While these methods have showcased promising results, existing methods typically focus on editing a single aspect in the source image. An "aspect" refers to a specific attribute or entity within the textual prompt that describes the image and can be modified, such object type, color, material, pose, or relationship. However, the ability to edit multiple aspects through text prompts is rarely explored. We introduce the concept of multi-aspect text-driven image editing to address this gap. Multi-aspect image editing is essential due to the rich and diverse content and structure of digital images, well as the varied requirements of users. For example, it always occurs that users wish to modify multiple attributes or regions in an image, such as adding a necktie to a cat and changing the background wall to beach (Fig. [ Left), or removing man and replacing mountain with castle in the right example, Unlike traditional editing methods (e.g.. (12) that focus on single aspect, multi-aspect editing allows users to manipulate various aspects simultaneously. Different from full text-to-image synthesis 7 [ which involves creating content from scratch, multi-aspect editing works with the source image ensure essential content preservation. bridges the gap between single-aspect editing and full synthesis, catering to a wide range of editing scenarios.
However, observe that directly applying the single-aspect text-driven image editing methods in cases where multiple image aspects must be modified often does not yield satisfactory results. A straightforward solution to this problem is apply the single aspect editing method sequentially- we can order the aspects to be modified and use a single-aspect editing method to change the aspects one by one. Although sequential applications of single-aspect text-driven image editing methods can modify multiple aspects of an image, they may introduce significantly higher computational overhead. More importantly, the order of the aspects modified may affect the quality changes later aspects may undo the early ones or accumulate the errors and artifacts, thus reducing the effectiveness of the final editing results, as the last two rows of Fig. [ and Table show.
In this work, we introduce ParallelEdits as an efficient and effective solution to the problem of multi-aspect text-driven image editing. This method is based on a crucial insight that the editing step can occur in parallel with the image's diffusion steps. Therefore, in ParallelEdits, we build image aspect editing into the diffusion steps accelerate the editing process. Parralle/Edits is based on an architecture with a fixed number of additional branches dedicated to handling rigid, non-rigid, and style changes. This design ensures scalability independent of the number of prompt aspects altered. In addition, we employ an attention aggregator to accurately assess editing difficulty and route aspects to appropriate branches within the ParallelEdits framework, ensuring precise and efficient editing. To enable subsequent research and evaluation of multi-aspes text-driven image editing methods, we also build the PIE-Bench++ dataset, an extension of the PIE-Bench M that has 700 images with detailed text prompts and tailored to facilitate simultancous edits across multiple image aspects. We propose evaluation metrics and benchmark different text-driven image editing methods on PlE-Bench++. The ParallelEdits outperforms the state-of offte--f-he-arr image editing methods on PIE-Bench++.
2
Related Works
Diffusion Models for Text-Driven Image Editing. Text-driven image editing aims to manipulate local regions of an image based on textual prompts. The editing has two main goals: ensuring the edits align with provided instructions and preserving essential content. Diffusion models 99 have gained popularity as a preferred image editing model for their capacity for generating high- quality samples by incorporating diverse conditions, especially using text [10 2 14 I1.
This involves transforming the images into the latent space and generating regions using diffusion models conditioned by the text prompt while ensuring accurate reconstruction of unmodified regions

---

during editing. To avoid the edited image deviating from original image, early text-driven image editing typically requires user-specified masks as additional condition [15 or training [8]200)
to guided the editing process, which constrain their potential zero-shot application. To address this limitation, recent editing models, such as InfEdit 22. PnP [21]. Direct Inversion [[] follow the work Prompt-to-Promp (P2P) [3, which proposed to obtain an attention map from the cross attention process and either swap or refine the attention map from text prompt for image editing. This design automatically obtains the editing mask and only allows image editing using text prompt. Another method, MasaCtrl 4 converts existing self-attention in diffusion models into mutual self-attention for non-rigid consistent image synthesis and editing, enabling query correlated local contents and textures from source images for consistency.
Multi-Aspect Image Editing. While current image editing models have shown promising results in their text-driven image editing benchmarks, we have observed that they work well on single-attribute editing while struggling to edit multiple aspects, especially when editing multiple objects (as shown in Fig. We attribute this limitation to the following reasons. First, existing methods use the attention mask to direct where edits should be made. With multiple attributes, the editing area may expand significantly, incorporating extensive semantic information or scattered regions that are challenging to edit using a single mask. Second, employing a fixed mask from cross-attention maps struggles with edits involving changes in region size (such pose adjustments), while using an adaptive mask faces challenges in maintaining edit fidelity. Therefore, integrating various attention masks for accurate multi-attribute editing presents challenging technical problem. Early studies [22] 23 have employed GAN models such as StyleGAN2 [24] to edit multiple attributes in faces. The
editing is realized by training the GAN model with supervised multi-class training and training dataset of image and attribute vector pairs. This solution heavily relies on the training sets and has limitations in generalizing to new editing types. Few recent works achieve multi-aspect editing with additional inputs: (25) leverages rich text to edit multiple objects and (26) pre-processes the image with grounding to localize multiple edited regions for multi-aspect editing. However, the editing performance highly relies on additional input beyond plain text, either from user input or other off-the-shelf models. A recent work 27 proposes iterative multi-granula image editor, where diffusion model can faithfully follow a series of image editing instructions from a user. However, this interactive editing pipeline will result in significant computational overhead.
Image Editing with Multiple Branches. In the literature 4 3 image editing processes have been conducted by implementing a branch approach. This methodology involves segregating source and target branches throughout the editing process. Specifically, the source branch is reverted to 20, while the trajectory of the target branch is iteratively adjusted. By computing the distance from the source branch, the calibration of the target branch occurs each time-step. Our observation underscores the disparity between the effectiveness of a dual branch in enhancing the editing process and its failure in multi-aspect editing. A singular target branch proves inadequate in calibrating fully from the source branch, leading to imperfect incorporation of all aspects into the image. Hence, our primary proposition advocates for multi-aspect editing by utilizing multiple target branches. Each target branch's 's trajectory is meticulously calibrated, with simpler concepts addressed in the initial branches and more complex aspects deferred to subsequent ones. In the following section, we will delve deeper into this concept.
3 Diffuusin--aseed Image Generation and Editing
We are provided with an image sample xo which transforms the latent space via an encoder/decoder pair E/D, such that Fg = E(xo). Here, Zy represents the latent representation of the image Zi-- With slight abuse of notation, denotes the reconstructed version of 20. These operations are integral to the latent diffuusinn model [9]. The diffusion process constitutes two steps: the forward step incrementally adds zero-meat white Gaussian noise with vire-narying variance to the latent vector 2 according to discrete-time
approximate the reconstructed image To as D(žo), where 2o
2 = Vaizo + vI- ayee with ~~ AN 1).
(I)
01:7 represents variance schedule for / drawn from the interval [1, .). The variance schedule can be different, such as linear or cosine quadratic (28). The backward step is an iterative process to remove
'Diffusion process is rigorously detined as a continuous timeenouu----eeeeeeee stochastic differential equation, but in practice often implemented with discrete-time updates.
3

---

the noise from the data progressively. Using the same variance schedule a1:7 as in the forward step, noise schedule 1:: and a parameterized noise prediction network Eo with coefficients Gpeed v
Ott-1 of, and Cnoise U2. the backward step corresponds to the following process: /1
Cdir
2t- Gpres.f CdirEe
Croise Et
with Et 10(,
predicting Zg
adjust aloog 21 random noise
The noise schedule σ1. comprises hyperparameters requiring careful selection based on factors like image dimensions or desired performance (29) [30]. In the framework of Denoising Diffusion Implicit Models (DDIM) B1], the function fe is employed for the prediction and reconstruction of
2o. based on the input 2,. Specifically, we have 2g = fa(z,. t)
Var 2t
A--ai
Co (Zt. t).
Vor
have been introduced to expedite the generation process through a consistent distillation approach. These models exhibit a self-consistency property, ensuring that samples along the same trajectory map to the same initial point. Specifically, the function fa is rendered self-consistent by satisfying fo(ze,t) t) 20 for given sample 2 at timestep t. As a result, the self-consistency property yields a closed-for form solution for the noise predictor <g. We denote this particular 60 co00s, which is derived as coons 2z /0rv0 vT-o.
Consistency Models for Inversion-free Image Editing. Consistency models
Since ccoms not parameterized and contains the ground-truth zo. Xu el al. propose starting directly with random noise, i.e., 27 ~ (0, I), at the last time-step T, which is particularly advantageous for image-ediinng tasks as eliminates the need for inversion from 20 27. Therefore, starting with
2, = ~ (0, I). the sampling process proceeds as follows:
-o-vil-a,"" Where, e, is given by vI-ai
VayzD
Va,
Noise is added 2,, i.e, 2, va,2 + VI a,E where E IO.
After many iterations, the final output z. Furthermore, 2 demonstrates that the dual-branch paradigm (involving source and target branch) used in image editing tasks can be executed in inversion-free manner. We will delve into this, along with our method description, in Section 4.2.2
Multi-Aspect Image Editing
4.1 Problem Definition
The input to the multi-aspect image editing task includes source image (Iare), the source prompt, and set of edits to be applied to the source image, indicating the changes from the source prompt to target prompt. text prompt (whether source or target) comprises several independent tokens, of which only a subset is editable. We refer to these editable tokens as Aspects.
Definition 4.1 (Aspect). We define an 7th aspect A in the source prompt (or the 5th aspect A in the target prompt) any entity that be substituted, deleted, or inserted into the text prompt, resulting in a meaningful sentence structure.
Several examples of tokens corresponding to aspects or not are given in Fig. 3 In other words, aspects correspond single or multiple tokens representing object color, pose, material, content, background, image style, etc. An editing operation bi+U between the editing pair (Airre: Alade) as Ei-j E { O O, }.. Here, @ denotes swap action, denotes object addition action, e denotes object deletion, and Ø indicates no change in the aspect. Such an editing operation can be inferred directly by appropriately mapping the source and target prompts, or it can be provided as metadata 3 34]. The editing task is considered successful if the edited source image, Iedts reflects the required edits while preserving the unaffected aspects of the original image.
4.2 Method
Figure Joutliness the overall pipeline of our method, which has three steps. In the first step (Sec. 4.2. we perform aspect grouping using attention maps generated by running few iterations of the diffusion process. The aspects the source image are put into up to N groups, each processed by a distinct branch. The second step (Sec. 4.2.2) demonstrates how each branch, which receives a specific group of aspects, performs inversion-free editing. In the last step (Sec. [42.3), we perform the necessary adjustments for enabling cross- brasch interaction and elucidate the significance of such interaction.

---

Source, prampt:
sabbut
non urtiing in beet sihcustted eguisst the surser with
background
non ditting in sallboot is ethoustted agannet the evening glow with duder the water tridge barkground
prarpt:
Ediinn
K Self Arreshin
k
G K M
Source image
Aspact Grouping
duold
Ectting
Edited Image
Beanch
nn
bridee
glow
Editing
Branch
X
Q
Sourcce Branch
Figure 2: Pipeline. Our method, ParallelEdits, takes a source image, source prompt, and target prompt as input and produces an edited image. The target prompt specifies the edits needed in the source image. Attention maps for all edited aspects are first collected. Aspect Grouping (see Section 4.2.I) categorizes each aspect into one of N groups (in the above figure, N 5). Each group is then assigned branch and the branch-level updates detailed in Section 4.2.2 Each branch can viewed either a rigid editing branch, non-rigid editing branch, or global editing branch. Finally, adjustments to query/key/valuc at the self-attention and iross-attention layers are made, as illustrated in the figure and described in Section 4.2.3
4.2.1 Aspect Grouping
We would like to group aspects in prompt into N distinct groups using the cross-attention maps of the diffusion UNet [35] to character- ize the spatial layouts as in previous studies (36). Given editing operation Æ between the source aspect Agre and the target aspect Addts we obtain the corresponding attention maps from both the source and target prompts Mi are and M respectively. The atten- tion map M is defined by the query feature Q and key feature from the cross-attention as (QRT
silhouetted agoinst surset mountoin
A Arre
Abre
At...
Airo
Alps
backeround
Abre
Rigid
Rigid
Rin-RRiid
Non-Rigid
Branch
Branch
Branch 5
AP
AT
A
A
bmidee
evening glax
M softmax
The binarized atten-
Vi
tion map M is obtained by normalizing M and thresholding its values. Our aspect grouping proceeds in two steps,
Aise
Ria
Ale
Alan
Torget prompti
apainat
Step 1. Assign type for every editing op- eration (Ei We consider three possible types of edits, in line with previous works [4] namely a global edit, a local rigid edit or lo- cal non-rigid edit. Rigid local edits, such changing an object's color or texture, do not al- ter the layout of objects. Conversely, non-rigid local edits modify the layout of objects, such as adding deleting objects changing object poses. Global edits affect background and style changes. The type assignment for the editing operation (Ei+)) is determined by the following rules:
Aas
Ale
Ala
with ducha on the autee and bridne the backgraar
A
Al
Ardi
Figure 3: Aspects and Aspect Grouping. In text prompt, there are multiple independent tokens, with only some being editable, known aspects and underlined
above example. These aspects can be added, deleted, or swapped between the source and target prompts. Pairs of source and target aspects are grouped into branches, and the methodology for aspect grouping explained in Section 4.2.1
global edit non-rigid edit rigid edit
T(M 2 By (E(Mett))
(y
((Mines Masd <
(3)
local edit 7(Mea) < By (EMe))))
((M M
Here, & represent mloU
(37. while 7 indicates the alpha mattes attention maps. and are tunable hyperparameters. For further details, please refer to the supplementary Sec.
5

---

Step 2. Categorize every editing operation (E into N groups. For each editing operation (E*+)) of specific type, assess whether e(M edt: M 2 À to determine if there exists substantial overlap between any pair of attention maps of that type. If significant overlap detected, the attention maps grouped together. On the other hand, if attention maps are isolated like the "boat" and "mountain" in Fig. Blare categorized into separate groups due to small overall. Therefore, have total of N groups. Each group has dedicated branch, resulting in total of N > branches.
4.2.2 Inversion-Free Multi-Branch Editing
We use a set of N branches indexed by n. These branches are in addition to a source branch (also shown in Figure 2 that undergoes DDCM sampling process [2). The n branch is calibrated its (n -1 1)th branch, and the first branch is calibrated to the source branch. The N -wyy target branch calibration occur simultaneously, saving significant compute time. For the DDCM sampling process of the thth branch, it has the form of Section
Step
Updating n brunch
z(n)alt
( noisy latent
v1 ar (e(n))") e(n 1) jeck
<(n), Joons
1)/vaF
edited latent
parameterized noise
consistency noise
Let us break down Eq. step by step. n= representing the source branch, we have =(1) = se and e(1),²t = e.t Also, = =(1)=") = =,,,, which at time step T TI, is random noise drawn from (0, I). Similarly, when all the required aspect edits after repeating for E (Ti, T2, } timesteps. The noise addition
N, =(N) represents the final calibrated/edited image containing
on any target branch remains the same Step
2 i.e., 2(11,, (dl = va,=(n) edit v1 d'eé where
~ (0, 1). For 1 < n < NN, we have e(n) rr) ( (n) )=ll T), where eq represents a parameterized noise predictor network (details in the Appendix Sec. D A key observation is that the difference
in the parameterized noise at the n branch and (n th branch is utilized calculate z(n) in Finally, c(n). ) defined by c(n))
(z(n) pdtt va,2(n 1)/ //VT-/VTaa a,, Unlike the dual-branch setup in (2). the reference initial input is the estimated latent from the previous branch at a previous diffusion denoising iteration as indicated by ă(n 1)ed.
4.2.3
Interactions
For rigid local branches, the cross-attention map M from the previous branch is either switched or injected into the current branch, akin to the method used in P2P B] This approach facilitates local edits while preserving structural consistency. For non-rigid local branches, we observe that the query features in the shallow layers of UNet (35] can effectively query correlated local contents and textures from the prior branch's 's latent features, ensuring consistency. Consequently, the key and value features from the prior branch are retained in the current branch to maintain consistent editing. We use a non-rigid editing branch to manage non-rigid local edits. the current branch n, textures from the previous branch (n 1) are preserved by replacing the Kn-1 and Vn features from the last branch with the K,, and V, features in the current branch. Only the query features are preserved to maintain layout semantic correspondence. Additionally, the attention mask M from the previous branch's cross-attention layer is used to guide the editing process by adding it to Mn. thereby converting the object layout from M, to M, This step crucial for object removal or shape modification edits, where the object mask is derived from the previous branch. For all global branches, there is no replacement of attention features or masks, and the attention mask is not used to guide the editing process, as the entire image is intended to be altered.
5 Experiments
PIE-Bench++ Dataset. We introduce a new dataset, PIE-Bench+++, derived from PIE-Bench 1 and dedicated evaluate the performance multi-aspect image editing. The PIE-BBnch dataset contains 700 images and prompts with single-aspect editing including object-level manipulations (addition,

---

mhon
Dalon
molo
Hello,
Figure 4: Qualitative results ParallelEdits. We denote the edits in arrows with edit actions and aspects for each pair of images. The last image pair is a failure case Paralle/Edits.
deletion, alteration), attribute-level manipulations (changes in content, pose, color, and material), and image-level manipulations that modify background and overall style. Our PIEz-Becc+++ extends PIE-Bench by enabling multi-aspect edits: 57% of our dataset have aspect edits per prompt, 19% have more than two edits, and the remaining 24% have a signle aspect edit. For additional details and examples of the PIE-Bench++ dataset, please refer to the supplementary material.
Evaluation Metrics. We introduce two new metrics designed for evaluating multi-aspect text-driven image editing, alongside standard evaluation metrics.
(a) Aspect Accuracy-LLavA Drawing inspiration from the remarkable capability of large vision language models comprehending intricate semantics within images, we propose to innovatively leverage them as "omniscient" agent equipped with extensive knowledge understand various attributes of images. We use the LLaVA [38] model, trained on visual grounding tasks, to evaluate the accuracy of multi-aspect image editing. Given text prompt with multiple aspects, such as "A [pink] [taxi] with [colorful] [flowers] on top". , we provide the following prompt with the edited image to the LLaVA model: "Does the image match the elements in J: A [pink] [taxi] with [colorful] [flowers] on top? Return a list of numbers where is matched and is unmatched." " We then parse the returned list and compute its average determine the aspect accuracy. We name this evaluation metric as AspAcc-LLaMa Examples and detailed explanations this evaluation metric are available the supplementary material.
(b) Aspect Accuracy-CLIP. We also use the similarity of the CLIP 33 to evaluate if an attribute has been successfully edited. Given an edited image Tedt and the target prompt Pedt with edited aspects Acat, every time we remove an aspect A'ar from Pedr and revert it back to Aire as Peat- We then extract the CLIP (39] similarity between the edited image Leat and two prompts, i... $1 CLIP(Iedd, Peat) and 82 CLIP(Ieat Pedr). We expect 81 $2 if the aspect A has been successfully edited. Thus, the aspect accuracy is & when a total of k, aspects have been successfully edited among k target edits. Note that in the case of an edited or added object that also involves changes in attributes (such color or material), we consider a successful edit only if both the object and its attributes have been successfully modified. We name this metric as AspAce- CLIP.
(c) Standard Metrics. Several standard metrics widely used for evaluating text-image similarity and image quality are considered, including PSNR, LPIPS [40,, MSE, and SSIM [41). We also the CLIP [39] score to measure the image-text alignment performance. Additionally, the bi-directional CLIP (D-CLIP) score 42 is reported, which is formulated follows:
cos (CLIP img (Ieat) - CLIP imng (Iare), CLIP text (Peat) CLIP text (Pare))
5.1
Quantitative Results
We first conduct experiment on the PIE-Bench++ dataset to compare our method with the state-of-the- art text-driven image editing methods combining their corresponding inversion method leads to best
7

---

Source Irage
MessCrii
P2
ZnfEd1
Rich-aacte
SPT 4V
PeraldEalts (Ours)
6ermon Shephard shands
coller
Sernan Shepherd mouth
leather coller autumn
Source
MasaChel
mossy branch sponst bockground
"A yellon bulch stands on flowers against sterry right background
Source Image
Mul--eet Inversion
Dinect on
IrdEdr
Pap
6PT-4V
Purs (Curs)
"o walking in with mountaine the backgrourd
"a walking
backigrond!
Source Irage
Invension
DinectTrwersion
InfEar
6PT-4V
Eard (Curs)
"white dumplings on brown wooden bow!
"white cupookes block metol bow?
Figure 5: Qualitative results comparison. Current methods fail to edit multiple aspects effectively, using sequential edits (noted *). Methods marked with taking additional inputs other than source image and plain text.
StyleD MasaCtri P2P
DI
NTI
InfEdit
PnP
DI*
P2p*
InfEdit
PnPs
Ours
1 D-CT.IP t
24.02
23.37
24.00
24.40
24.03
24.44
24.90
22.80
25.13
25.17
25.39
25.70
8.43
7.68
11.43
13.23
12.08
11.02
11.83
2.74
8.30
11.77
11.85
20.70
(secs/sample) +
382.98
12.70
33.72
29.70
145.29
32.51
100.98
121.32
11.82
122.81
4.98
AspAcc-CLIP (56)
32.37
34.05
26.14
31.95
42.19
42.38
44.91
28.23
38.96
42.38
48.20
$1.05
AspAce-LLaVA (%)
53.79
55.79
54.42
59.80
61.36
46.24
55.21
61.90
63,80
65.19
Table 1: Comparison results in multi-aspeet image editing on the PIE--encc+++ dataset. Computational efficiency is abhreviated as Eff., and denotes the method using sequential editing. The best performance is highlighted bold and the second best performance is underlined.
performance, including D 4 DDDM--Propt--o-Promyy (P2P) 3 DDIM+
Play (PnP) (21). StyleDiffusion (StyleD) 43 +P2P, Null-text Inversion (NTI) PPPP DirectInveri- son (DD[I+PuP, and InfEdit [2]. An intuitive way to improve off-the-shelf image editing methods is to apply the single-aspect editing method sequentially. We follow 2 to adapt existing image editing methods into sequential editing processes, where these methods are applied multiple times to achieve multi-aspect editing. Each time, only one aspect is edited. [ the metrics in terms of text-image similarity (i.e., CILP and D-CLIP scores), computational efficiency, and aspect accuracy. Our ParallelEdits model outperforms all baselines in editing effectiveness, with slightly longer runtime than the InfEdii model. Even though sequential editing better aligns target prompt than their vanilla methods, it significantly increases computational overhead and may propagate editing errors over time. Moreover, although the sequential editing conducted the latent space, would introduce more noise and artifacts to the edited image. Hence, their performance in all editing quality metrics was inferior to our method.
5.2 Qualitative Results
Fig. presents several examples of our method's multi-aspect editing on the PIE--ench+++ dataset. The results demonstrate the effectiveness of our method in handling multiple and varied types of

---

Ours
DivectInwersion
P2P
InfEdit
Ours
DirectInversion
P2P InfEdit
08.0
CLIP (%)
60
40-
40,9
50.0
56.0
007 a
35.0
943.0
LLaVA (%)
53
63
61.9
63.4
16 Ssg 40.
62.0
58.4
60.9
50
1 esseett
aspect
2 espects
3+ espects
1 espect
espect
2 esspets
3+ aspects
PIE-Bench
PIE-Bench+*
PIE-Bench
PIE-Bench-*
Figure 6: Comparison across different numbers of editing aspects. We also include the comparison in PIF:-Bench dataset. Our proposed method robust to different numbers of editing aspects.
Background Preservation
Aspect Preservation % CLIPt LLaVA f
Methods
PSNRT
LPIPS x103
MSE 104 +
SSIM 102 t
P2P 3
18.48
16.64
188.26 231.83
345.07 190.07
73.55 / 69.17
20.72/ 23.48
66.59/ 72.60 75.65 78.77
PnP 21
22.73
22554
103.16/ 120.87
75.97 102.47
80.73 / 78.85
20.79 / 25.59
InfEdit 2
24.61 / 24.09
103,99
107.43
160.54 163.72
78.85 / 79.64
24.69 25.04
75.90 78.05 80.70
Ours
26.13
95.87
113.86
82.35
25.49
Table 2: Comparison results in terms of background and aspects preservation. The results from sequential editing noted as green. ParallelEdits achieves state-of-the-art performance on multi-aspect editing while preserving the background and content consistency.
edits across diverse image content. Fig. 5 further compares method with several state-of- fftthh--rr art models and one popular multi-modal large language model, GPT-4V 44 by providing the source image, source prompt, and target prompt to guide the image editing. The Rich-text 25 model differs from other models, which uses rich-text prompt to edit the image generated from the plain (source) text prompt. The results show that current image editing models even with sequential editing fail to edit multiple aspects, while modail large language models fail to preserve the content of source image. Our method achieves visually convincing results by successfully editing different attributes with good content preservation.
5.3 Ablation Study and Analysis
(a) Impact of Editing Aspect Number. We first examine the performance of our ParallelEdits and baseline methods on various editing aspect numbers by comparing CLIP and LLa VA VA-base aspect accuracies on the original PIE-Bench [ and our PlE-Bench++ datasets. The bar charts in Fig. [ show the outstanding performance of our method across all settings, including single-aspect editing on two datasets and multi-aspect editing. Takeaway: the proposed ParallelEdits demonstrates robusiness across varying numbers of editing aspects.
(b) Evaluation on Perservation. We follow [ to evaluate the background preservation. We first use the PSNR, LPIPS (4)). MSE and SSIM [4] evaluate the background preservation. We measure that metric on a subset of images of our proposed PIE-Bench+++ dataset where the background can be well defined in that image, e.g., no image style or background editing, and the background is visible after aspect editing. The results are shown in Table where we compare our method with the top performance methods Table T Moreover, we adopt the similar way calculating the AspAcc-LLaVA to prompt LLaVA [38 for evaluating how the unchanged aspect preserves in the edited image. We also calculate the CLIP 39 score between the target image and the text prompt after removing all edited aspects. The results are reported in Table 2 noted as CLIP and LLaVA, respectively. Takeaway: preservation even maintained ParallelEdits.
(c) Branches numbers and aspect grouping. To demonstrate the effectiveness of our bulti-branch design and early aspect grouping, we design additional ablation studies for our method in threefold. (1) We only use one single non-rigid branch conduct all edits; (2) we remove the aspect categorization
process from the pipeline and use the same non-rigid branch for each edit; (3) we adopt one single branch for different type of edits without using any auxillary branches which results total of three
branches (also see Section B for more details). Takeaway: As shown in Table 3 the multi-branch design and aspect grouping play a significant role enhancing the performance of our proposed ParatlelEdits.

---

with aspect categorization
with aspect with auxillary
Similarity %
Aspect Accuracy %
grouping
branch
CLIPP
D-CLIPT
CLIPT
IJaVA
24.32
10.45
40.97
57.67
V
25.14
11.97
46,66
58.37
ParallelFdiis
24.50
12.33
48.08
61.22
25.70
20.70
51.05
65.19
Table 3: Ablation studies on branch numbers and aspect grouping.
Change Color
Add Object
Delete
Asepet Ace-CLIP
Object
Content
Pose
Material
Background
Style
Object
P2P 3
33.13
20.00
25.83
34.17
31.67
30.63
19.38
22.29
11.88
MasaCri Ctrl
4
40.83
23.75
40.83
20.00
30.83
26.88
29.38
37.08
28.96
NTI 45
48.13
41.25
23.75
51.25
24.17
51.25
22.50
40.42
32.08
DirectInversion
40.63
26.25
23.33
40.00
25.42
32.50
25.00
30.00
20.83
In[Edit 2
36.24
33.33
25.41
41.67
27.50
48.75
41.88
50.63
45,41
PnP [21
44.38
27.29
27.91
49.17
32,91
52.50
55.63
44.38
42.08
ParallelEdits
51.46
44.16
39.58
60.00
47.50
60.00
50.00
56.04
52.08
Table 4: Comparison on each category in TTE-Bench++. Our ParallelEdits achieves the best performance on most of the categories from the dataset.
(d) Performance comparison on each category. Recall that our dataset includes nine different categories for editing. We compare the performance of baseline models and approach across the nine categories, as presented in Table Takeaway; Our proposed ParallelEdits achieves state-of-the- art performance across most categories.
Limitations and Failure Cases. The proposed ParallelEdits has several limitations. First, it cannot handle the text editing in the image, as shown in the last image pair of Fig. 4 Second, ParallelEdits fails to edit dramatic background changes, as examples shown in the supplementary material.
6 Conclusion
In this work, we propose new research task, multi-aspect text-driven image editing, to modify multiple object types, attributes, and relationships. We introduce dedicated method, ParallelEdits, to multi-aspect text-driven image editing as an effective and efficient solution to this problem. Due to the lack of evaluation benchmark, we introduce PIE-Bench++. improved version of PIE-Bench [ tailored for simultaneous multiple-aspe edits within images. ParallelEdits achieves better quality and performance than existing methods on proposed PIE- PIE-Bench+++ Our work introduces ParallelEdits, novel approach that adeptly handles multiple attribute edits simultaneously, preserving the quality of edits across single and multiple attributes through unique attention grouping mechanism without adding computational complexity. There are several future works we would like to explore. First, different aspects of an image have a specific semantic order. Editing these aspects according to their intrinsic order will simplify the editing process. Secondly, the current ParallelEdits still has limitations, as shown Fig. It will be of interest study approaches to improve these aspects.
Ethics Statement. In anticipation of contributing to the academic community, plan to make the dataset and associated code publicly available for research. Nonetheless, we acknowledge the potential for misuse, particularly by those aiming to generate misinformation using our methodology. We will release our code under an open-source license with explicit stipulations to mitigate this risk. These conditions will prohibit the distribution of harmful, offensive, or dehumanizing content or negatively representing individuals, their environments, cultures, religions, and so forth through the use of our model weights.
Acknowledgement. This work supported in part by the National Science Foundation (NSF) Projects under grants SaTC-2153112 12, No. N8.1822190, and TIP-2137871. Prof. Lokhande thanks support provided by University at Buffalo Startup funds. We thank Sudhir Kumar Yarram for the insightful discussions on the project.
10

---

References
[1] Xuan Ju, Ailing Zeng, Yuxuan Bian, Shaoteng Liu, and Qiang Xu. Direct inversion: Boosting diffusion-based editing with 3 lines of code. 2023.
[2] Sihan Xu, Yidong Huang, Jiayi Pan, Ziqiao Ma, and Joyce Chai. Inversion-free image editing with natural language. arXiv preprint arXiv:2312.049953 2023.
[3] Amir Hertz, Ron Mokady, Jay Tenenbaum, Kfir Aberman, Yacl Pritch, and Daniel Cohen-On Prompt-to-prompl image editing with cross attention control. 2022.
[4] Mingdeng Cao, Xintao Wang, Zhongang Qi, Ying Shan, Xiaohu Qie, and Yinqiang Zheng. Masactrl: Tuning-free free mutual self-attention control for consistent image synthesis and editing. In Proceedings of the IEEF/CVF International Conference on Computer Vision (ICCV), pages October 2023.
[5] Jizhizi Li, Jing Zhang, and Dacheng Tao. Referring image matting. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 22448-22451 2023.
[6] Yuhao Liu, Jiake Xie, Xiao Shi, Yu Qiao, Yujie Huang, Yong Tang, and Xin Yang. Tripartite information mining and integration for image matting. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 7555-7564, 2021.
[7] Shuyang Gu, Dong Chen, Jianmin Bao, Fang Wen, Bo Zhang, Dongdong Chen, Lu Yuan, and Baining Guo. Vector quantized diffusion model for text-to-image synthesis. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 10696-10706. 2022.
[8] Ming Ding, Wendi Zheng, Wenyi Hong, and Jie Tang. Cogview2: Faster and better text-to- image generation via hicrarchical transformers. Advances in Neural Information Processing Systems, 35:16890-16900, 2022.
[9] Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Björn Ommer High- resolution image synthesis with latent diffusion models. In Proceedings of the IEEF/CVF Conference on Computer Vision and Pattern Recognition, pages 10684-10695, 2022.
[10] Guillaume Couairon, Jakob Verbeek, Holger Schwenk, and Matthicu Cord. Diffedit: Diffusion- based semantic image editing with mask guidance. In The Eleventh International Conference on Learning Representations, 2023.
[11] Gaurav Parmar, Krishna Kumar Singh, Richard Zhang, Yijun Li, Jingwan Lu, and Jun-Yan Zhu. Zero-shot image-to-image translation. In ACM SIGGRAPH 2023 Conference Proceedings, pages 1-11, 2023.
[12] Bahjat Kawar, Shiran Zada, Oran Lang, Omer Tov, Huiwen Chang, Tali Dekel, Inbar Mosseri, and Michal Irani. Imagic: Text-based real image editing with diffusion models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 6007-6017, 2023.
[13) Guillaume Couairon, Jakob Verbeek, Holger Schwenk, and Matthieu Cord. Diffedit: Diffusion- based semantic image editing with mask guidance. arXiv preprint arXiv:2210. /427, 2022.
[14] Thao Nguyen, Yuheng Li, Utkarsh Ojha, and Yong Jae Lee. Visual instruction inversion: Image editing via image prompting. Advances in Neural Information Processing Systems, 36, 2023.
[15] Andreas Lugmayr, Martin Danelljan, Andres Romero, Fisher Yu, Radu Timofte, and Lue Van Gool. Repaint: Inpainting using denoising diffusion probabilistic models. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition, pages 11461-11471, 2022.
[16] Omri Avrahami, Dani Lischinski, and Ohad Fried. Blended diffusion for text-driven editing of natural images. Proceedings of the IEFF/CV Conference on Computer Vision and Pattern Recognition, pages 18208-18216 2022.
11

---

[17] Alex Nichol, Prafulla Dhariwal, Aditya Ramesh, Pranav Shyam, Pamela Mishkin, Bob McGrew, Hya Sutskever, and Mark Chen. Glide: Towards photorealistic image generation and editing with text-guided diffusion models. arXiv preprini arXiv:21 10741, 2021.
[18] Jooyoung Choi, Sungwon Kim, Yonghyun Jeong. Youngjune Gwon, and Sungroh Yoon. IIvr: Conditioning method for denoising diffusion probabilistie models. arXiv preprini 2xXi:2108.002938 2021.
[19] Sihan Xu, Ziqiao Ma, Yidong Huang, Honglak Lee, and Joyce Chai. Cyclenet: Rethinking cycle consistency text-guided diffusion for image manipulation. Advances in Neural Information Processing Systems, 36, 2024.
[20] Min Zhao, Fan Bao, Chongxuan Li, and Jun Zhu. Egsde: Unpaired image-to-image translation via energy-guided stochastic differential equations. Advances in Neural Information Processing Systems, 35:3609-3623, 2022.
[21] Narek Tumanyan, Michal Geyer, Shai Bagon, and Tali Dekel. Plug-and-ple diffusion features for text-driven image-to-image translation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 1921-1930, 2023.
[22] Hao Wang, Guosheng Lin, Ana García del Molino, Anran Wang, Zehuan Yuan, Chunyan Miao, and Jiashi Feng. Maniclip: Multi-atribbute face manipulation from text. arXiv preprint arXiv:2210.00m 10.00455 2022.
[23] Siavash Khodadadeh, Shabnam Ghadar, Saeid Motiaan, Wei-An Lin, Ladislau Bölöni, and Ratheesh Kalarot. Latent to latent: A learned mapper for identity preserving editing of multiple face attributes in on Applications of Computer Vision, pages
images. In Proceedings of the IEEE/CVF Winter Conference 2022.
[24] Tero Karras, Samuli Laine, Miika Aittala, Janne Hellsten, Jaakko Lehtinen, and Timo Aila. Analyzing and improving the image quality of stylegan. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 8110-8119, 2020.
[25] songwei Ge. Taesung Park, Jun-Yan Zhu, and Jia-Bin Huang. Expressive text-to-image generation with rich text. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 7545-7556, 2023.
[26] Hangeol Chang, Jinho Chang, and Jong Chul Ye. Ground-a-score: Scaling up the score distillation for multi-attribute editing, arXiv preprint 2rXiv:2403.135511 2024.
[27] KJ Joseph, Prateksha Udhayanan, Tripti Shukla, Aishwarya Agarwal, Srikrishna Karanam, Koustava Goswami, and Balaji Vasan Srinivasan. Iterative multi-granular image editing using diffusion models. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pages 8107-8116, 2024.
[28] Alexander Quinn Nichol and Prafulla Dhariwal. Improved denoising diffusion probabilistic models. In International Conference on Machine Learning, pages 8162-8171. PMLR, 2021.
[29] Ting Chen. On the importance of noise scheduling for diffusion models. arXiv preprint 10X7 2023.
[30] Tero Karras, Miika Aittala, Timo Aila, and Samuli Laine Elucidating the design space of diffusion-bbsed generative models. Advances in Neural Information Processing Systems, 35:26565-26577, 2022.
[31] Jiaming Song, Chenlin Meng, and Stefano Ermon. Denoising diffusion implicit models. arXiv preprint arXiv:2010.02500. 2020.
[32] Yang Song, Prafulla Dhariwal, Mark Chen, and Ilya Sutskever. Consistency models. arXiv preprint arXiv:2303..11469 2023.
[33] Simian Luo, Yigin Tan, Longbo Huang, Jian Li, and Hang Zhao. Latent consistency models: Synthesizing high-resolution images with few-step inference. arXiv preprint arX.v:2310.04378. 2023.
12

---

[34] Ron Mokady, Amir Hertz, Kfir Aberman, Yael Pritch, and Daniel Cohen- Or. Null-text inversion for editing real images using guided diffusion models. In Proceedings of the IEEE/CVP Conference Computer Vision and Pattern Recognition, pages 6038-6047, 2023.
[35] Olaf Ronneberger, Philipp Fischer, and Thomas Brox. U-net: Convolutional networks for biomedical image segmentation. In Medical Image Computing and Computer-Assisted Inceveention-MICCA. 2015: 18th International Conference, Munich, Germany, October 5-9, 2015, Proceedings, Part III 18, pages 234-241. Springer, 2015.
[36] Raphael Tang, Linging Liu, Akshat Pandey, Zhiying Jiang, Gefei Yang, Karun Kumar, Pontus Stenetorp, Jimmy Lin, and Ferhan Ture. What the daam: Interpreting stable diffusion using cross attention. arXiv preprint (ixxiv22210048000 2022.
[37] Tsung- Yi Lin, Michael Maire, Serge Belongie, James Hays, Pietro Perona, Deva Ramanan, Piotr Dollár, and C Lawrence Zitnick. Microsoft coco: Common objects in context. In European conference on computer vision, pages 740-755. Springer, 2014.
[38] Haotian Liu, Chunyuan Li, Qingyang Wu, and Yong Jae Lee. Visual instruction tuning. In NeurIPS, 2023.
[39] Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervision. In International conference on machine learning.
pages 8748 8763. PMLR, 2021.
[40] Richard Zhang, Phillip Isola, Alexei Efros, Eli Shechtman, and Oliver Wang. The unrea- sonable effectiveness of deep features as perceptual metric. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 586-595, 2018.
[41] Zhou Wang, Alan C Bovik, Hamid Sheikh, and Eero Simoncelli. Image quality assessment: from error visibility to structural similarity. IEEE transactions on image processing, 13(4:600 612, 2004.
[42] Chen Henry Wu and Fernando De la Torre. Unifying diffusion models' latent space, with applications to cyclediffusion and guidance. arXiv preprint urXiv:2210005525 2022.
[43] Senmao Li, Joost van de Weijer, Taihang Hu, Fahad Shahbaz Khan, Qibin Hou, Yaxing Wang, and Jian Yang. Stylediffusion: Prompt-embedding inversion for text-based editing. 2023.
[44] OpenAl. Gpt-4v(ision) system card. 2023.
[45] Ron Mokady, Amir Hertz, Kfir Aberman, Yael Pritch, and Daniel Cohen-Or. Null-text inversion for editing real images using guided diffusion models. arXiv preprint arXiv:2211.0979. 2022.
[46] Narek Tumanyan, Omer Bar-Tal, Shai Bagon, and Tali Dekel. Splicing vit features for semantic appearance transfer. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition, pages 10748-10757,
13

---

Appendix
A ParallelEdits: The Algorithm
In this section we provide Algorithm!! [ Early Aspect Grouping and Algorithm ParallelEdits on particular branch. These algorithms describe the overall idea behind ParallelEdits. They are also pictorially illustrated in Figures and of the main paper. Let us denote an arbitrary branch and the timestep in the diffusion process by n and respectively. Firstly, in Algorithm | we demonstrate how Early Aspect Grouping is conducted over the attention maps. Recall that we refer to this as "early" aspect grouping because only a few steps (maximum of 5) are sufficient to perform the grouping. This phase of ParallelEdits takes as an input, the edit action set {E*+)) and the corresponding cross-attention maps for every token A are and outputs the grouped edit actions set A Recall from Section 4 of the paper that F E { ① O Ø), with X denoting swap action, Q denoting an add action, O denoting aspect deletion, and Ø indicating no change the aspect. Once grouped edit actions set is computed, it is fed into Algorithm to conduct multi-aspect editing and obtain the edited latent features. Algorithm 2, we implement several operations on the attention masks, similar to the P2P method 3 and describe them follows.
Replace: Swapping token attention mask Mn-1 1 in the prompt from previous branch, overriding Mni
Refine: Injecting only the attention mask that corresponds to the unchanged part of the prompt from M Mni
Retain:
Keeping the attention mask Mn unchanged.
Algorithm Early Aspect Grouping
Input: Edit action set {E Cross attention maps {M} rigid-edit + {}, non-rigid-edit + (}. global-edit
2: for A'aii E {EEit} do
3:
if ((Meed 2 (T(EMMaat) }) then global-edit + global-edit+ + {E*
This is global edit
4:
5: else if 6(M "MM .M.r) < 0 then
This is rigid edit
6: for Aldt rrgiid-dit do
7:
if' mloU(A edes Ei 2 0) then
D A is a set of grouped edit actions
8:
Ala + Acds + E Ei
9:
else
10:
rigid-edit + rigid-edit + Ei-j end if
11:
12:
end for else if of MMsss Mi ) then for Artr non-rigid-edit do
This non-rigid edit
15:
if m ede Ei-s 2 8) then
16:
A + Ac E
17:
else
18:
end non-rigid-edit non-rigid-edit E* end for
19:
20:
21:
end
22:
end for
Output: Grouped edit actions set (Ac
B Some More Details on ParallelEdits
In the literature 4 3, image editing processes have been conducted through the implementation of
dual-branc' approach. This method involves utilizing source and target branches for editing.
14

---

Algorithm 2 ParallelEdits on a Particular Branch
Input:
Denoising UNet Eg.
Grouped edit action A
Output from early aspect grouping
Latent feature in previous branch and previous timestep 2 2n
Cross attention maps {M}.
Self attention features Qn 1 -n Edit type list: rigid-edit, non-rigid-edit, global-edit
1: Mn EE -h-1___0_0___
2: if A E global-edit then
This is global edit
3: (e)ain(MM)
Do not switch attention maps for global edits
4: else if Aget non-rigid-edit then
This is non-rigid edit
5:
rep.ace(Mnn1..M,
6: else if A rigid-edit then
This is rigid edit
7: (Qn. Kn, Vn t (Qn. Kn-1, Vn
8:
refine(MMn-1,MM.
9: end if
Mn < binariize Mm)
11: 2 Mn = + (1 MMa) ) 2
Output: Latent feature 2/
Specifically, the source branch is reverted to 2o. while the trajectory of the target branch is iteratively adjusted. By computing the distance from the source branch and cons with Latent Consistency Model [32] the target branch is calibrated at each time step.
Our experiments, as seen in Section of the main paper, show the ineffectiveness procedure for multi-aspect editing tasks. Specifically, single target branch is inadequate, leading to imperfection in the target image. Thereby we advocate multi-aspect editing through the use of multiple target branches. Each target branch handles group of aspects, with simpler aspects such as non-rigid local edits directed to initial branches, and more complex aspects such as rigid local edits deferred to subsequent ones. Note that however, all the branches operate simultaneously.
dual-branch
Auxiliary Rigid Non-Rigid Branches. In the main paper, was noted that there was one dedicated branch for each type of edit: non-rigid, rigid, and global edit. The Early Aspect Grouping algorithm]] T classifies aspects into these three categories. Our experiments revealed that sometimes, due low overlap between attention maps, aspects may not always be grouped into dedicated rigid or non-rigid branches. In such cases, becomes necessary include an auxiliary branch to handle the ungrouped aspects. Therefore, ParallelEdits may involve single rigid branch and additional auxiliary branches to manage ungrouped aspects, and similarly, a single non-rigid branch and supplementary auxiliary branches to address ungrouped aspects. An ablation study auxiliary branches is provided in Table 3
C More Details on Evaluation Metrics
In this section, we describe more details of our evaluation metrics.
LLaVA VA aspect accuracy. We show how we leverage LLaVA 38
to evaluate the multi-aspect editing accuracy in Fig. 7 and Fig. 8 also prompt LLaVA 38 for explanation to human readers. LLaVA [8] could provide detailed summary for the image and also explanations for the mismatched between edited aspects and image.
Other evaluation metrics. Moreover, even though the Structure Distance [66 has been used in PIE-Bench [1] to evaluate structure between source and target image while ignoring appearance information, it could not serve as good evaluation metric for multi-aspect editing. This is due to the fact that the structure of multi-aspect edited target image may necessitate substantial modifications, particularly when it involves adding removing multiple objects.
15

---

User
[saliboott) sikouutted the (snow) mountoin backgrounds, Execty fo formet whene is matched unmatched, return The list longth
LLavA
LLaVA
The metches the elements the description fellowe
imge, which is slhousthed against evening The glow viablle the creating beautfful backdrap for the salleot.
valbeat
The mountoine the bockground are covered with snow, wnich consistert with the description
The everal scone is wnteer scene, indicated by the snew cow-cooerdd nountons and the coln, cold appxoronce of the water.
Therefore, all elements in the description are motched by the mage, resuttng is let of enes.
Dees this image natch the e elements in man [standing] in best siloouetted agoinst the susset and [ducki)] on the water with [Ales mountain]] in the bockground" Exectly formcl
matiched unmetched,, return only.
length equal
LLMA
Please explenation.
LLavA
The image motches al the elements the given discoription:
mon standing i
alhovetted spoinst sunset.
bockground fectures Tharefore, the elaments the desiciption all present the and the lisn would be 11
desoribed
image elements EP
lake mountain] background"
Evertly follow the formot where matched unnatched, only.
The liat langth
LLVA
LLaVAA
The image aPooes e boot os lake with house and mountoin in the background. The mountain oppears to be covered with soo, which could be desorbed "ice mousthie" The elements the brockets motch description provided
image.
house The mountain bookground covened with which could desumbed "lo mountoin
beckigound.
Therefore, the elements the brazkets motch the description, and the lut would be 11.1313
Figure 7: Examples of prompting LLL.V for aspect accuracy measurement in cases of successful editing. LL effectively illustrate if and how the target image and edits misaligned.
16

---

User
piont] and (word "Hello,, Worlddl) I writtenn netebookk i (slock tobll)"
Exectly follow the neturn fornat a list where matched and O unnetched, neturn list only.
The let length is asual to 3.
LLaVA
[.3.0]
LLaVA
Cortonlyy explonation
image and
provided
The
commoniy
type
glessy green
lenves, So, this elxment matched.
"[word 'Hele, World' is written en the natebask - The no notebeek in the image hos the word "Halls, World" written it However, the format specities that word should be writton on the natebook, which not the case hore. The word actually written on whiheboard or similrr surface is placed front of notebook. Therefore, element ummatched
"[black
irage
light color, passillyy gray beige.
unmatched.
Therefore, the lst of matches would (1. 0g
Does this image match the eloments in
"a logg [C] (withs pirk caler) in a Dight background]"
Exectly follow neturn formet list where motcheed unmatcchdd neturn only. longth coual
LLaVA
(31.0)
Phooe prowid some explonation
LLaVA
Cartanly The image you ve previded features loge than is composed of on shope, which matches the first element in the description The X colored in chade of prik, which matches the second elenent the description However, the bockground of the image not light: it dark, which does not match third element desioppton Thenefore, the matches woud
Loge Matched
3. Bnckground Net Matched
This results in the list (3.
Figure 8: Examples of prompting LLaVA for aspect accuracy measurement in cases of unsuccessful editing.
D Implementation Details
Our proposed ParallelEdits is based the Latent Consistency Model [32]. with the publicly available
LCM which is finetuned from Stable Diffusion v1.5. We then follow 2 to leverage their proposed inversion- free technique in ParallelEdits for image editing. During sampling, we perform LCM sampling (32) with 15 denoising steps, and the classifier-free guidance (CFG) is set to 4.0. Paral- lelEdits can control the editing strength by adjusting the CFG There's trade-off between achieving satisfactory inversion and robust editing ability. A higher CFG tends to produce stronger editing effects but may lower inversion results and identity preservation. We also set the hyper-parameter as 0.9 and / as 0.8 in our experiments, where 0, 8 are used to determine the edit type of given edit action.
17

---

Change
Add Object
Delete
Object
Content
Pose Color Material Background Style
Object
#Edited Aspect
302
98
120
188
99
112
165
178
119
#Edited Token
316
155
227
205
116
175
424
507
381
Table 5:
Summary of Editing Types and Categories in PIE-Bench+++ dataset. There 10 different categories in PIE--encch++ and a total number of 700 images.
In the inversion-free multi-branch editing approach, for < n < N, the noise estimation also con- ditioned on text conditioning Cn in branch This can be expressed as c(n2,, oft = cæ ce(2(n), T, Cx). Here, 01 corresponds to the source prompt. ey corresponds the target prompt, and c, represents the prompt that includes all aspect edits up to branch n2
E
Additional Details of PIE-Bench++
E.1
PIE-Bench++ Details
Unlike existing benchmarks that primarily focus single-aspect edits, PIE--Bnch+++ is tailored to multiple aspect edits, reflecting the complexities inherent in real-world editing tasks. Our enhanced dataset, PIE-Bench++ builds upon the TIEBeench [1 by incorporating 700 images across nine diverse categories, covering both natural and artificial scenes, with a significant focus on multi-aspect editing scenarios. Specifically, the Change Object category involves swapping objects in the scene with different yet reasonable alternatives. Add Object adds new elements to the scene. Delete Object focuses removing objects, testing the model's ability to erase elements seamlessly. Change Object Content alters the content of specific objects, such as changing the design on shirt or the pattern on wall. Change Object Pose includes changes in the shape of objects, humans, or animals. Change Object Color assesses the model's 's ability to apply accurate color changes. Change Object Material evaluates the rendering of different textures and materials. Change Background involves editing scenarios where there is a distinct foreground object and main background. This type of edit focuses on seamlessly integrating new background elements while preserving the integrity of the foreground object. Change Image Style involves the application of style transfer techniques to the entire image while ensuring the original content remains intact. For example, this could involve transforming photograph to adopt a cartoon style. Each category is carefully curated to provide comprehensive evaluation of the dataset's 's multi-aspect editing capabilities, the summary of the dataset is shown in Table
E.2 Dataset Annotation
The annotation process involves primary annotator who labels the source prompt, describing the original image, and the target prompt, which outlines the desired modifications to generate the target image. The target prompt is carefully annotated include all editing pairs expected to be reflected in the target image. Subsequently, second annotator reviews the annotations for accuracy and consistency, ensuring the reliability of the dataset. The majority of target prompts in PIE-Bench++ feature at least two edited aspects. Nevertheless, within the categories that solely changing background and image styles, the number of edits is usually constrained to one or two aspects. This limitation is due to the intrinsic characteristics of these attributes, such as each image having only one background or style.
Annotation format details. Each image in the dataset annotation is associated with key elements as shown in Fig. 9 a source prompt, a target prompt, an edit action, and a mapping of aspects. The edit action specifies the position index in the source prompt where changes are be made, the type of edit to be applied, and the operation required to achieve the desired outcome. The aspect mapping connects objects undergoing editing to their respective modified attributes, enabling the identification of which objects are subject to editing.
18

---

Source Image
Text Text-based based annotation
@oource_prompi1 colorful bird standing on a branch", "target_prompr" "a brown owl standing red flower".
'edit_action":
(""owi-("positio"" "eeeeii "a "rae"ddd"atioo"""""iirr "brown" "brown":("position" 11, "eddi__ttypp t: "action""ccolorf""""" "colorfull'
"loosrr(""osiitionn" "ediit_type":1, "'octtonn"""bannn"""""
("ed"(("psittion"" 6," "edii__yyp""d "action":" "]]] "}}.
sepect__mappine ("owo':""bron") ",tp:n"n",fffooor "'"lower""""ree"""""
"source potumcceprompt: "the galoxy the durdle door".
"aroget_premptt: "the pink sunset and rainbow over the durdle door",
"edit_ actit__ccoon:: ("pink":"" "ppink'("posiitonn==l wedit__ype""a "action":"="),, "actiio"" "galaxxy")
"sunsert:("posittion" =1, "and rainbow' :( position": 22 2'"edit_type 22," :tttoo"""++])
"aspect_mapping"" ("sunset" "'rainboo""""""o"""""""""
"source_ ppomptt : "a slanted mountain bicycle on the road in front of building",
"target_prompt" slanted rusty mountain motorcycle the-road front a fence",
'edit_action":
("rusty": "ruusty"""position"" 2,, 2.,edit _edit_type":7,7 "action":"="),
"motorcycle" "("position" 33, seiit_type"i "actti""io"" :bbicccle""..."""
"fence":" pasition" "edit_ sadit_typee"aa " "on the road" "'ad""""pstttion" :4," "eep"dddttyee""""" 3, "'actionn"" ")),
"aspect_mapping" ("motorydd" ["ree"''rood"""
"source_ pporrct_pompt: "a round cake with orange frosting a wooden plate",
Boarget_promp"" "a square cake with strawberry frosting on plastic plate",
"edit_ _ilit_action": ("square": "ppuuare"""sssition"" "edit,_ee"____ "eaii__ttte""tt________ "biri"""
"strawberry frosting"(oosition"" "ddit sedit_type"tt 16, ""action"""orane frosting"), position" "plostic": ""osttic"("positioon" "'ddt__ "edit_type""7 type": "action":"wodden"" "woodenn"),
posstiion"""
"ospect_mapping"" ("cake" ""cake""""qquee ["square" "strawberry frosting"). J. plaate"("plastc""
Figure 9: Annotation examples from PIE-Bench++. Each annotation containing Source Prompt, Target Prompt, Edit Action, and Aspect Mapping. Edit action contains the specific instructions including the desired modification index in source prompt position, edit type among 9 catergories and the action {8, + } The aspect mapping indicts the pair between object and attribute.
19

---

Additional Qualitative Results
We also provide more qualitative results in Fig. 10 showing the effectiveness of our proposed method in handling multi-aspect editing tasks. These examples showcase the model's proficiency in executing intricate edits. For instance, as depicted in Fig. T0 (b), our method successfully removes cup while accurately reconstructing the obscured parts of the lamp behind it. In Fig. (a), the model demonstrates ability to swap and add aspects, while preserving the composition the scene. The results underscore the model's adeptness in interpreting and executing sophisticated editing instructions, leading to visually consistent and contextually fitting edited images. Additional, we also provide the results for sequential editing methods with different editing order in Fig. 11 and Fig.
clouds
micof
(a)
(b)
Flow
church
sits hill Rower field with clouds the sky
Ikstrattoo lomp
fildd
Nive windew wnt
0400 ouTs
wall
S
fork
S
(d)
wooden
Junury bedrosm ieter or with fieid
Iuoury bedicom interior with wall conon cahade
aboal of stroubarries and Skunberrise etriped
woadte etrow hornee striped hobiecleth
cream
kitchen
! table
(e)
metal chair
chair and pottern and
weodinn boal ron with spoon lotchhon
with flarel pathers
lage X color
nountoliss
(g)
(h)
light
lake with mountsins the bazkground
boct in lake with house and ICE mountare the bockgreund.
a logo of gind Shope it black beokground
logo of withh color light beckgnound
Figure 10: Qualitative results from ParallelEdits. ParallelEdits is able swap, add and delete multiple aspects. The image pair failure case ParallelEdits.
20

---

InfEdit Sequentid
InfEdit Sequentill
Source Prompt: dog is loying down white bookground
witte brow weeden bow
sunglasses
cupcskes
Target Prompt:: white cot wearing sunglosses loying down en white background"
whiee cuptases
Toract Promst
bI PrP Sequential
bDIM + PaP Sequential
Figure 11: Sequential editing using single-aspect text-driven image editing methods. The sequential editing might accumulate errors and undo previous edits. also fails to edit significantly overlapped objects.
Seuartial
Securrial
Goldr Retrisser
cing feace
Target Prempt
vearig kother lalor
from
Figure 12: Sequential editing with different orders. Sequential editing with different orders can yield varying final results. Additionally, it may lead to error accumulation and potentially overwrite previous edits.
21

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification:
The main claims made in the abstract and introduction accurately reflect the paper's contributions and scope.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or
NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
is fine to include aspirational goals as motivation as long as is clear that these goals not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations the work performed by the authors?
Answer: [Yes]
Justification:
We include the limitation and failure cases of the work in Sec.
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create a separate "Limitations" section their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution is low images are taken in low lighting. Or speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.
While the authors might fear that complete honesty about limitations might used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't 't acknowledged judgment and recognize that individual actions in favor of transparency play impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
the paper. The authors should use their best
3. Theory Assumptions and Proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [NA]
22

---

Justification:
This paper does not include theoretical result.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced the statement of any theorems.
The proofs can either appear the main paper or the supplemental material, but they appear in the supplemental material, the authors are encouraged to provide short proof sketch to provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental Result Reproducibility
Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Question:
Answer: [Yes]
Justification: We fully disclose the information needed to reproduce the main experimen- tal results
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g.. in the case of large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily a new algorithm, the paper should make it clear how reproduce that algorithm. (b) If the contribution is primarily
model architecture, the paper should describe the architecture clearly and fully.
(c) If the contribution is a new model (e.g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome deseribe the particular way they provide for reproducibility, In the case of closed-source models, it may be that access to the model is limited in some way (e.g., registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described in supplemental material?
23

---

Answer:
[Yes]
Justification:
The code and data will be open-sourced for academic use.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please the NeurlPS code and data submission guidelines https://nips..e cc/
blic/ wwwgggg//oodduuuuaeeaaee soi.iiiiiiiiiyiiioicooccc for more details.
While we encourage the release of code and data, we understand that this might not be possible, so "No" an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines ( https://nips...
for more details,
The authors should provide instructions on data access and preparation, including how access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information possible in supplemental material (appended to the paper) recommended, but including URLs to data and code permitted.
6. Experimental Setting/Details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification:
We specify all the training and test details
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper to level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, or as supplemental material.
7. Experiment Statistical Significance
Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Answer: [Yes]
Justification: The paper reports error bars
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, statistical significance tests, at least for the experiments that support main claims of the paper.
The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, overall run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call a library function, bootstrap, etc.)
The assumptions made should be given (e.g.. Normally distributed errors).
should be clear whether the error bar is the standard deviation the standard error of the mean.
24

---

It is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of enrors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out range (e.g. negative error rates).
If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.
8. Experiments Compute Resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?
Question:
Answer: [Yes]
Justification: The paper provide sufficient information on the computer resources
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU GPU, internal group, or cloud provider, including relevant memory and storage.
The paper should provide the amount compute required for each of the individual experimental runs well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g.. preliminary or failed experiments that didn't make into the paper).
9. Code Of Ethies
Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics https:
Answer: [Yes]
Justification:
The code follows the NeurIPS Code of Ethics.
Guidelines:
The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special cireumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g., if there is a special consid- eration due to laws or regulations their jurisdiction).
10. Broader Impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
Justification: The paper includes the discussion of potential societal impacts.
Guidelines:
The answer NA means that there is no societal impact of the work performed.
If the authors answer NA or No. they should explain why their work has no societal impact or why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that improvement in the quality of generative models could be used to
25

---

generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
there are negative socictal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML).
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification: The paper poses such risks.
Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards challenging, and many papers do require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Are the creators original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Question:
Answer: [Yes]
Justification: CC-BY 4.0 for PIE-Bench.
Guidelines:
The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package dataset.
The authors should state which version of the asset is used and, if possible, include URL.
The name of the license (e.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (c.g., website), the copyright and terms of service of that source should be provided.
assets are released, the license, copyright information, and terms of in the package should be provided. For popular datasets, paperswit thoode. com/ cwmmcatttttt.e datasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if has changed) should be provided.
this information is not available online, the authors are encouraged to reach out to the asset's is creators.
13. New Assets
Question: Are new assets introduced in the paper well documented and the documentation provided alongside the assets?
26

---

Answer: [Yes]
Justification:
The documentation provided alongside the assets
Guidelines:
The answer NA means that the paper does release new assets.
Researchers should communicate the details the dataset/code/moodel as part their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsourcing and Research with Human Subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well as details about compensation (if any)?
Answer: [NA]
Justification: The paper does not involve crowdsourcing nor research with human subjects
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then much detail as possible should be included the main paper.
According to the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional Review Board (IRB) Approvals or Equivalent for Research with Human Subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [NA]
Justification: The paper does not involve crowdsourcing research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Depending on the country in which research is conducted, IRB approval (or equivalent)
may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and expect authors to adhere the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
27


Reviews:
[{"review_id": "eQNQRP1V78", "reviewer": "Reviewer_6th7", "summary": "In this paper, the authors present a novel multi-aspect image editing method ParallelEdits, by incorporating the attention distribution mechanism and multi-branch editing. Besides, this paper introduces a new dataset PIE-Bench++ for evaluating multi-aspect image editing. Extensive experiments demonstrate the effectiveness of this method.", "questions": "See the weakness part above.", "limitations": "The author has discussed the limitations and potential negative societal impacts.", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. The paper is well-written and well-organized.\n2. The proposed method is novel and effectively addresses the multi-aspect image editing problem.\n3. The results appear to have impressive visual effects. Edits of different aspects are combined seamlessly.\n4. The authors conducted concrete experiments, providing rich quantitative results and qualitative results to demonstrate the effectiveness of the proposed ParallelEdits.", "weaknesses": "1. The pairing process of $E^{i \\rightarrow j}$ imposes an additional burden on users, especially when multiple objects are present in the source image.\n2.  It is not clear how to choose the hyperparameter $\\lambda$ for assigning a type for rigid and non-rigid edit actions, and how it affects the cross-branch interaction and editing results.\n3. This paper lacks a discussion on controlling the editing strength of multiple editing actions, which is crucial for harmonic and flexible editing.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "**Q:** The pairing process imposes an additional burden on users when performing multi-object editing.\n\n**A:** The algorithm receives the pairing process (editing action) to determine which aspect is added, removed, swapped, or left unaltered. Meta-data of this form is not a burden but a necessity because it precisely reflects the user's intent when editing without errors. A user-friendly UI can conveniently capture such meta-data. Moreover, prior works also benefit from meta-data e.g. [1] needs rich text input and [2,3] need editing pairs.\n\n**Q:** How the hyperparameter $\\lambda$ to be chosen and how it affects the editing.\n\n**A:** The hyperparameter $\\lambda$ is the threshold for determining whether an edit is classified as a rigid edit or a non-rigid edit (see equation 3 of the paper). The hyperparameter $\\lambda$ was selected through cross-validation by identifying the optimal $\\lambda$ that gave the highest aspect accuracy. Experiments showed that both quantitative results and qualitative results do not vary for different choices of $\\lambda$. Please see the table below and the Figure 2 of the PDF appended to this rebuttal. We will gladly share additional results if the reviewer desires.\n\n\n|    $\\lambda$    | 0.8   | 0.85  | 0.9 (ours)   | 0.95  |\n|--------|-------|-------|-------|-------|\n| AspAcc-CLIP | 50.09 | 50.68 | 51.05 | 50.53 |\n\n**Q:** Lacking discussions on controlling the editing strength of multiple editing.\n\n**A:** We concur with the reviewer that edit strength control is important. As shown in [4,5], single attribute editing tasks provide clear editing strength and flexibility. As for multi-aspect editing, the reviewer would agree that it is unclear if edit for multiple aspect is itself possible. Moreover, evaluation metrics and benchmark datasets for multi-aspect editing are not yet community-standardized. Thus, instead of handling editing flexibility, contemporary efforts [6,7] focus on feasibility aspects of simultaneous multiple edits. \n\n\n[1] Expressive text-to-image generation with rich text. Ge et al, ICCV23.\n\n[2] Prompt-to-prompt image editing with cross attention control. Hertz, et al, ICLR23.\n\n[3] Null-text inversion for editing real images using guided diffusion models. Mokady, et al, CVPR23.\n\n[4] An edit friendly ddpm noise space: Inversion and manipulations. Huberman-Spiegelglas et al, CVPR24.\n\n[5] Imagen editor and editbench: Advancing and evaluating text-guided image inpainting. Wang, et al, CVPR23.\n\n[6] Ground-A-Score: Scaling Up the Score Distillation for Multi-Attribute Editing. Chang, et al, Arxiv24.\n\n[7] Iterative Multi-granular Image Editing using Diffusion Models. Joseph, et al, WACV24.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "iiX9Ww3lG1", "reviewer": "Reviewer_J7sk", "summary": "The paper introduces ParallelEdits, a method that manages simultaneous edits efficiently without compromising quality. This is achieved through a novel attention distribution mechanism and multi-branch design. Additionally, the authors present the PIE-Bench++ dataset, an expanded benchmark for evaluating multi-object and multi-attribute image-editing tasks.", "questions": "I noticed that the man has changed after editing in Fig. 2. Could you please explain and address it?", "limitations": "See above.", "rating": 5, "confidence": 5, "soundness": 3, "presentation": 2, "contribution": 3, "strengths": "1. The task of multi-aspect editing seems interesting and useful in real life. \n2. The proposed method is efficient for application.\n3. The overall editing results seem promising.", "weaknesses": "1. The authors claim the proposed method is based on the DDCM process [1]. However, when I checked the source paper of [1], I found something strange, especially its Algorithm 1. The dozens of iterations come to a simple conclusion that the output $z$ actually equals $z_0$, which means there is no need for such Virtual Inversion. Also, this process cannot be interpreted by Consistency Models. Hence, I wish the authors could revise their theoretical bases.\n\n2. I think it is better to present the intermediate editing results for sequential editing, which can help readers identify what type of editing the previous methods are not good at.\n\n3. I noticed that there was a benchmark [2] proposed for image editing. I advise the authors to use it for comprehensive evaluation.\n\n[1] Inversion-Free Image Editing with Natural Language (https://arxiv.org/abs/2312.04965)\n\n[2] Diffusion-Model Based Image Editing: A Survey (https://arxiv.org/abs/2402.17525)", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "**Q:** The DDCM process in [1] comes to a simple conclusion that the output z equals $z_0$ after dozens of iterations.\n\n**A:** Indeed, ParallelEdits' source branch uses DDCM, as described in [1], which uses the consistency sampling step popular in consistency model literature [6,7]. DDCM process for source branch was chosen based on [1]'s observations that the method does not suffer cumulative errors over sampling, making the overall method more efficient (fewer samples to generate target image), which is desirable for image-editing applications. Although $z=z_0$ in every iteration of Algorithm 1, the difference between the reconstructed output $z_\\tau$ and $z_0$ as captured by $\\epsilon_\\text{cons}$ continues to evolve (see Figure 3b of [1]). The target branch is calibrated using $\\epsilon_\\text{cons}$ thereby introducing the desired edit. \n\n**Q:** Results about intermediate editing results for sequential editing\n\n**A:** We have included further results in Figures 3 and 4 of the attached document in the general response. As previously stated in line 48 of the main paper, sequential editing is an ineffective approach due to the potential for undoing previous edits and accumulating errors over time.\n\n**Q:** Additional benchmark needs to be included for comprehensive evaluation.\n\n**A:** We thank the reviewer for highlighting this interesting benchmark. Although it predominantly contains single-aspect editing samples, unlike the more complex multi-aspect samples in PIE-Bench++, we evaluated our method on it and reported the results in Table 1 of the global rebuttal. Our method achieves state-of-the-art performance in terms of the overall LMM score proposed in [2]. Inspired by this, we plan to release a benchmark focused on multi-aspect editing, using PIE-Bench++ and the Aspect-accuracy/preservation metrics from our paper. We are confident it will gain strong recognition from the community.\n\n**Q:** Explanation of man has changed after editing in Fig. 2.\n\n**A:** We appreciate the reviewer’s subtle observation. As shown in Figure 2, the “boat” object nearby has been transformed to “white rubber boat” and leaks into the “man” attention map, and the man has been edited to fit the context of changing a traditional boat to a modern rubber boat. Similar with other attention-map-based approaches [1,3,4,5], the attention maps could leak, even though such minor changes do not affect the overall context of this figure. Finally, such leakage can be easily solved by applying a higher threshold to the attention map.\n\n[1] Inversion-Free Image Editing with Natural Language, Xu et al, CVPR24'\n\n[2] Diffusion-Model Based Image Editing: A Survey, Huang et al, ArXiv24'\n\n[3] Prompt-to-Prompt Image Editing with Cross Attention Control, Hertz et al, ICLR23'\n\n[4] DiffEdit: Diffusion-based semantic image editing with mask guidance, Couairon et al, ICLR 23'\n\n[5] MasaCtrl: Tuning-Free Mutual Self-Attention Control for Consistent Image Synthesis and Editing, Cao et al, ICCV23'\n\n[6] Latent consistency models: Synthesizing high-resolution images with few-step inference, Luo et al, ArXiv23'\n\n[7] Consistency Models, Song et al, ICLR23'", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "3hWd8PwGNH", "reviewer": "Reviewer_8ae7", "summary": "This paper aims to make multiple objects or attributes editing, while preserving the quality, named ParallelEdits. The paper also introduces a new dataset PIE-Bench++. Experimental results on both PIE-Bench and PIE-Bench++ demonstrate that this method outperforms many existing editing techniques.", "questions": "Does StyleDiffusion (StyleD) refer to “Controllable disentangled style transfer via diffusion models” or “Prompt-Embedding Inversion for Text-Based Editing” ? If it is the former, please explain why this paper is compared with style transfer methods. If it is the latter, please correct it.\n\nSimultaneously editing multiple objects or attributes is interesting. But some aspects of the method are not adequately explained (e.g., how to get the attention-map M), and some references within the paper are quite casual (e.g., DirectInversion and StyleDiffusion). \n\nThe weaknesses low my initial rate.\n\n---\n**Update after author responses** \nThanks to the authors for their efforts. I'm happy to see the extended discussion and additional comparisons that have resolved my concerns.", "limitations": "Yes", "rating": 6, "confidence": 5, "soundness": 3, "presentation": 2, "contribution": 2, "strengths": "Simultaneously editing multiple objects or attributes is interesting.\n\nThe finding that the order in which aspects are modified when applying single-aspect text-driven image editing methods can affect the quality is interesting.", "weaknesses": "1.\tDirection Inversion is solely a method for inverting real images, not for editing them. Figure 1 is confusing in this context. Direction Inversion can be combined with various editing methods like P2P, PnP, P2P-zero, and MasaCtrl. Which editing method is Direction Inversion combined with in the third row of results? Why is this particular editing method shown?\n2.\tSame problem in the description in the introduction (Line 22, latest methods [3, 1, 4]; Line36, Unlike traditional editing methods (e.g., [1, 2])). [1] (Direction Inversion) is not a editing method.\n3.\tThe paper claims that ParalleEdits can edit multiple objects or attributes simultaneously. Figure 1. illustrates the swapping of multiple objects within the same image, but it only demonstrates the addition of a single object. Can ParalleEdits add multiple objects within a single image, such as both a necktie and sunglasses to a cat?\n4.\tIn aspect grouping, the paper does not explain how to get the attention map M for a real image.\n5.\tIn Figure 2, why does the attention map from the original image combine with the target prompt.\n6.\tThis paper divides aspect grouping based on cross-attention. What if cross-attention is inaccurate? For example, in Figure 2, will each image clearly show a \"ducks\" cross-attention map when \"ducks\" are added to the image?\n7.\tIn Table 1, P2P is an editing method for generated images. When applying it to real images (PIE-Bench++ dataset), it needs to be combined with an inversion technique (i.e., NTI, DI). Need to explain.\n8.\tThe finding that the order in which aspects are modified when applying single-aspect text-driven image editing methods can affect the quality is interesting. However, the paper does not conduct qualitative and quantitative experiments to support this claim.\n\nFurther comments:\n+ Line 119, should z be z_0 ?\n+ Line 135, Xu et al. [3] seems not to use \\cite{} to link the reference?\n+ Line148~Line149, the definition of tokens is not clear for me. Do the tokens represent all nouns and adjectives in a text prompt?", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "**Q: Clarification of the inversion and editing method.**\n\n**A:** We agree with the reviewer. Each baseline in Table 1 and throughout the paper includes a submethod for inversion and another for editing. The table below details these submethods for each baseline. We will update line 36 of the introduction, Table 1/Figure 5 captions, and lines 269-272 to specify the submethods used for inversion and editing for each baseline.\n\n|Method stated in paper|MasaCtrl|P2P|NTI|  StyleD | PnP      |DI|\n|-|-|-|-|-|-|-|\n| Inversion technique | DDIM | DDIM | NTI |StyleD|DDIM|DI|\n|Editing technique |MasaCtrl|P2P|P2P|P2P|PnP|PnP|\n\n**Q:** P2P is an editing method for generated images and needs inversion method to edit real image.\n\n**A:** For P2P, we adopt the default settings in the original P2P paper [1] (introduced in the Real Image Editing section) to conduct DDIM inversion to edit real images.\n\n**Q:** Is ParalleEdits able to add multiple object in the image?\n\n**A:** ParallelEdits can add multiple objects to an image, as shown in Fig 4 of the main paper (adding harness and flowers) and Fig 10 in the Appendix (d,e,f). PIE-Bench++ includes test samples for evaluating multiple object addition. We tested the example provided by the reviewer—a cat wearing a necktie and sunglasses—as detailed in the document attached in the general response (Figure 1), and we welcome any additional challenging examples the reviewer wishes to suggest.\n\n**Q:** Need to explain how to get the attention map for a real image.\n\n**A:** An attention map is generated by running a few diffusion steps (up to 5 steps) as in line 472 of the main paper and it is created by averaging cross-attention matrices of UNet layers, as outlined in P2P (see section 3.2 of [1]). The main paper (line 175) mentions that the aspect grouping process uses a dual-branch diffusion model conditioned on source and target prompt to generate and group attention map. Attention maps from the two branches are associated with source prompt tokens and target prompt tokens, and the attention maps of source branch correspond to the aspects in real/source images. We will include these details in the appendix of the revised manuscript. \n\n**Q:** Figure 2, Why does the attention map from the original image combine with the target prompt.\n\n**A:**  The source and target attention maps are associated with source and target prompt, while both source and target attention maps have been to used to identify the editing type, as shown in Figure 3. We visualize the target attention map overlaid in the original image to illustrate which part of the original image need to be edited based on the attention map. \n\n**Q:** What if cross-attention is inaccurate? For example, is the duck attention map clear to show?\n\n**A:** Yes, adding \"ducks\" to an image creates a cross-attention map as Figure 3 of the main paper shown. Current image editing works [1,2,3,4] have also used attention map to indict object layouts. The iterative diffusion process is able to generate an accurate object layouts in the attention maps ( as described in section 3.2 of P2P paper [1]). Moreover, attention maps have also been utilized for even more intricate tasks such as image segmentation [5]. Although failure situations resulting from faulty attention maps do occur, they are rare.\n\n**Q:** Need qualitative and quantitative experiments to support the claim that order in which aspects are modified when applying single-aspect text-driven image editing methods can affect the quality.\n\n**A:**. Thanks for pointing this out. We have conducted additional qualitative and quantitative experiments to support the claim, please check Table 2 and Figure 4 attached in the general response. We found that the AspAcc-CLIP is significantly different when using varying orders for each single-aspect editing method, and each method achieves the best performance with different orders. Hence, it is hard to choose a proper editing order for sequential editing, not to mention that sequential editing is extremely slow.\n\n**Other minor issues**\n\n**Q:** z should be $z_0$ in Line 119.\n\n**A:** Yes, we will reviese the manuscript.\n\n**Q:** The definition of tokens is not clear.\n\n**A:** The tokens are words obtained from the tokenizer which includes nouns, adjectives and other words splitted by spaces. For example, in the figure 3 the prompt has been broken down into tokens, all the words are token but only the underlining ones are asepcts.\n\n**Q:** Misused of reference.\n\n**A:** Thanks for pointing this out, StyleDiffusion refers \"Prompt-Embedding Inversion for Text-Based Editing\", and Xu et al. refers the [3] Inversion-Free Image Editing with Natural Language. We will update the references in the revision.\n\n[1] Prompt-to-Prompt Image Editing with Cross Attention Control, Hertz et al, ICLR23'\n\n[2] DiffEdit: Diffusion-based semantic image editing with mask guidance, Couairon et al, ICLR 23'\n\n[3] Inversion-Free Image Editing with Natural Language, Xu et al, CVPR24'\n\n[4] MasaCtrl: Tuning-Free Mutual Self-Attention Control for Consistent Image Synthesis and Editing, Cao et al, ICCV23'\n\n[5] Open-Vocabulary Attention Maps with Token Optimization for Semantic Segmentation in Diffusion Models, Marcos et al, CVPR24'", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:
We thank all the reviewers for their time, insightful suggestions, and valuable comments. We are grateful for the positive recognition of the reviewers that our idea and task are interesting (Reviewers 8ae7 and J7sk), the method is efficient for application (Reviewers J7sk), and our editing results are impressive (Reviewers 6th7 and J7sk).

We have responded to each reviewer's comments in detail below. A PDF document has been uploaded to include additional figures and tables. We hope our response will address the reviewers' concerns.

Meta-review: This submission received two weak accept and one borderline accept. This paper aims to make multiple objects or attributes editing, Reviewer 8ae7 and Reviewer J7sk agreed that simultaneously editing multiple objects or attributes is interesting and practical. Reviewer 6th7 recognized the novelty of the method. The editing results are impressive. 
The reviewers raised some concerns like unclear method details and gave some useful suggestions. The authors well addressed reviewers' concerns. Given the strong recommendation from Reviewer 8ae7 & 6th7 and borderline accept recommendation from Reviewer J7sk, the AC recommends this paper for acceptance.
Decision: Accept (poster)