Title: ParallelEdits: Efficient Multi-Aspect Text-Driven Image Editing with Attention Grouping
Venue: NeurIPS 2024
Accepted: yes
Keywords: Diffusion Models

Image Editing; Multi-aspect editing;consistency models
Abstract: Text-driven image synthesis has made significant advancements with the development of diffusion models, transforming how visual content is generated from text prompts. Despite these advances, text-driven image editing, a key area in computer graphics, faces unique challenges. A major challenge is making simultaneous edits across multiple objects or attributes. Applying these methods sequentially for multi-attribute edits increases computational demands and efficiency losses. 
    In this paper, we address these challenges with significant contributions. Our main contribution is the development of ParallelEdits, a method that seamlessly manages simultaneous edits across multiple attributes. In contrast to previous approaches, ParallelEdits not only preserves the quality of single attribute edits but also significantly improves the performance of multitasking edits. This is achieved through innovative attention distribution mechanism and multi-branch design that operates across several processing heads. 
    Additionally, we introduce the PIE-Bench++ dataset, an expansion of the original PIE-Bench dataset, to better support evaluating image-editing tasks involving multiple objects and attributes simultaneously. This dataset is a benchmark for evaluating text-driven image editing methods in multifaceted scenarios.

ParallelEdits: Efficient Multi-Aspect Text-Driven Image Editing with Attention Grouping

Abstract

Text-driven image synthesis has made significant advancements with the development of diffusion models, transforming how visual content is generated from text prompts. Despite these advances, text-driven image editing, a key area in computer graphics, faces unique challenges. A major challenge is making simultaneous edits across multiple objects or attributes. Applying these methods sequentially for multi-aspect edits increases computational demands and efficiency losses. In this paper, we address these challenges with significant contributions. Our main contribution is the development of ParallelEdits, a method that seamlessly manages simultaneous edits across multiple attributes. In contrast to previous approaches, ParallelEdits not only preserves the quality of single attribute edits but also significantly improves the performance of multitasking edits. This is achieved through innovative attention distribution mechanism and multi-branch design that operates across several processing heads. Additionally, we introduce the PIE-Bench++ dataset, an expansion of the original PIE-Bench dataset, to better support evaluating image-editing tasks involving multiple objects and attributes simultaneously. This dataset is a benchmark for evaluating text-driven image editing methods in multifaceted scenarios. Codes are available at: https://mingzhenhuang.github.io/projects/ParallelEdits.html.

1 Introduction

Recently, text-driven image editing has experienced remarkable growth, driven by advances in diffusion-based image generative models. This technique involves modifying existing images based on textual prompts to alter objects, their attributes, and the relationships among various objects. The latest methods [3, 1, 4] can produce edited images that closely match the semantic content described in the prompts while keeping the rest of the image unchanged. Unlike early image editing approaches that required image matting to precisely extract foreground objects using alpha mattes [5], text-driven editing offers a less labor-intensive alternative. User-provided textual prompts guide the edits, with auxiliary inputs like masks facilitating localized modifications [6].

While these methods have showcased promising results, existing methods typically focus on editing a single aspect in the source image. An 'aspect' refers to a specific attribute or entity within the textual prompt that describes the image and can be modified, such as object type, color, material, pose, or relationship. However, the ability to edit multiple aspects through text prompts is rarely explored. We introduce the concept of multi-aspect text-driven image editing to address this gap. Multi-aspect image editing is essential due to the rich and diverse content and structure of digital images, as well as the varied requirements of users. For example, it always occurs that users wish to modify multiple attributes or regions in an image, such as adding a necktie to a cat and changing the background wall to a beach (Fig. 1, Left), or removing a man and replacing a mountain with a castle in the right example. Unlike traditional editing methods (e.g., [1, 2]) that focus on a single aspect, multi-aspect editing allows users to manipulate various aspects simultaneously. Different from full text-to-image synthesis [7, 8], which involves creating content from scratch, multi-aspect editing works with the source image to ensure essential content preservation. It bridges the gap between single-aspect editing and full synthesis, catering to a wide range of editing scenarios.

However, we observe that directly applying the single-aspect text-driven image editing methods in cases where multiple image aspects must be modified often does not yield satisfactory results. A straightforward solution to this problem is to apply the single aspect editing method sequentially -we can order the aspects to be modified and use a single-aspect editing method to change the aspects one by one. Although sequential applications of single-aspect text-driven image editing methods can modify multiple aspects of an image, they may introduce significantly higher computational overhead. More importantly, the order of the aspects modified may affect the quality - changes to later aspects may undo the early ones or accumulate the errors and artifacts, thus reducing the effectiveness of the final editing results, as the last two rows of Fig. 5 and Table 1 show.

In this work, we introduce ParallelEdits as an efficient and effective solution to the problem of multi-aspect text-driven image editing. This method is based on a crucial insight that the editing step can occur in parallel with the image's diffusion steps. Therefore, in ParallelEdits, we build image aspect editing into the diffusion steps to accelerate the editing process. ParrallelEdits is based on an architecture with a fixed number of additional branches dedicated to handling rigid, non-rigid, and style changes. This design ensures scalability independent of the number of prompt aspects altered. In addition, we employ an attention aggregator to accurately assess editing difficulty and route aspects to appropriate branches within the ParallelEdits framework, ensuring precise and efficient editing. To enable subsequent research and evaluation of multi-aspect text-driven image editing methods, we also build the PIE-Bench++ dataset, an extension of the PIE-Bench [1] that has 700 images with detailed text prompts and tailored to facilitate simultaneous edits across multiple image aspects. We propose evaluation metrics and benchmark different text-driven image editing methods on PIE-Bench++. The ParallelEdits outperforms the state-of-the-art image editing methods on PIE-Bench++.

2 Related Works

Diffusion Models for Text-Driven Image Editing . Text-driven image editing aims to manipulate local regions of an image based on textual prompts. The editing has two main goals: ensuring the edits align with provided instructions and preserving essential content. Diffusion models [9] have gained popularity as a preferred image editing model for their capacity for generating highquality samples by incorporating diverse conditions, especially using text [10, 11, 2, 12-14, 1]. This involves transforming the images into the latent space and generating regions using diffusion models conditioned by the text prompt while ensuring accurate reconstruction of unmodified regions

during editing. To avoid the edited image deviating from original image, early text-driven image editing typically requires user-specified masks as additional condition [15-17] or training [18-20] to guided the editing process, which constrain their potential zero-shot application. To address this limitation, recent editing models, such as InfEdit [2], PnP [21], Direct Inversion [1] follow the work Prompt-to-Prompt (P2P) [3], which proposed to obtain an attention map from the cross attention process and either swap or refine the attention map from text prompt for image editing. This design automatically obtains the editing mask and only allows image editing using a text prompt. Another method, MasaCtrl [4], converts existing self-attention in diffusion models into mutual self-attention for non-rigid consistent image synthesis and editing, enabling to query correlated local contents and textures from source images for consistency.

Multi-Aspect Image Editing . While current image editing models have shown promising results in their text-driven image editing benchmarks, we have observed that they work well on single-attribute editing while struggling to edit multiple aspects, especially when editing multiple objects (as shown in Fig. 1). We attribute this limitation to the following reasons. First, existing methods use the attention mask to direct where edits should be made. With multiple attributes, the editing area may expand significantly, incorporating extensive semantic information or scattered regions that are challenging to edit using a single mask. Second, employing a fixed mask from cross-attention maps struggles with edits involving changes in region size (such as pose adjustments), while using an adaptive mask faces challenges in maintaining edit fidelity. Therefore, integrating various attention masks for accurate multi-attribute editing presents a challenging technical problem. Early studies [22, 23] have employed GAN models such as StyleGAN2 [24] to edit multiple attributes in faces. The multiple-attribute editing is realized by training the GAN model with supervised multi-class training and a training dataset of image and attribute vector pairs. This solution heavily relies on the training sets and has limitations in generalizing to new editing types. Few recent works achieve multi-aspect editing with additional inputs: [25] leverages rich text to edit multiple objects and [26] pre-processes the image with grounding to localize multiple edited regions for multi-aspect editing. However, the editing performance highly relies on additional input beyond plain text, either from user input or other off-the-shelf models. A recent work [27] proposes an iterative multi-granular image editor, where a diffusion model can faithfully follow a series of image editing instructions from a user. However, this interactive editing pipeline will result in significant computational overhead.

Image Editing with Multiple Branches. In the literature [4, 3], image editing processes have been conducted by implementing a dual-branch approach. This methodology involves segregating source and target branches throughout the editing process. Specifically, the source branch is reverted to z 0 , while the trajectory of the target branch is iteratively adjusted. By computing the distance from the source branch, the calibration of the target branch occurs at each time-step. Our observation underscores the disparity between the effectiveness of a dual branch in enhancing the editing process and its failure in multi-aspect editing. A singular target branch proves inadequate in calibrating fully from the source branch, leading to imperfect incorporation of all aspects into the image. Hence, our primary proposition advocates for multi-aspect editing by utilizing multiple target branches. Each target branch's trajectory is meticulously calibrated, with simpler concepts addressed in the initial branches and more complex aspects deferred to subsequent ones. In the following section, we will delve deeper into this concept.

3 Diffusion-based Image Generation and Editing

We are provided with an image sample x 0 which transforms the latent space via an encoder/decoder pair E / D , such that z 0 = E ( x 0 ) . Here, z 0 represents the latent representation of the image x 0 . With a slight abuse of notation, we approximate the reconstructed image ¯ x 0 as D (¯ z 0 ) , where ¯ z 0 denotes the reconstructed version of z 0 . These operations are integral to the latent diffusion model [9]. The diffusion process constitutes two steps: the forward step incrementally adds zero-mean white Gaussian noise with time-varying variance to the latent vector z according to discrete-time t * ,

α 1: T represents a variance schedule for t drawn from the interval [1 , T ] . The variance schedule can be different, such as linear or cosine quadratic [28]. The backward step is an iterative process to remove

the noise from the data progressively. Using the same variance schedule α 1: T as in the forward step, a noise schedule σ 1: T and a parameterized noise prediction network ϵ θ with coefficients c pred = √ α t -1 ,

The noise schedule σ 1: T comprises hyperparameters requiring careful selection based on factors like image dimensions or desired performance [29][30]. In the framework of Denoising Diffusion Implicit Models (DDIM) [31], the function f θ is employed for the prediction and reconstruction of ¯ z 0 , based on the input z t . Specifically, we have ¯ z 0 = f θ ( z t , t ) = 1 √ α t z t -√ 1 -α t √ α t ϵ θ ( z t , t ) .

Consistency Models for Inversion-free Image Editing . Consistency models [32, 33] have been introduced to expedite the generation process through a consistent distillation approach. These models exhibit a self-consistency property, ensuring that samples along the same trajectory map to the same initial point. Specifically, the function f θ is rendered self-consistent by satisfying f θ ( z t , t ) = z 0 for a given sample z t at timestep t . As a result, the self-consistency property yields a closed-form solution for the noise predictor ϵ θ . We denote this particular ϵ θ as ϵ cons , which is derived as ϵ cons = z t - √ α t z 0 √ 1 -α t . Since ϵ cons is not parameterized and contains the ground-truth z 0 , Xu et al. [2] propose starting directly with random noise, i.e., z T ∼ N (0 , I ) , at the last time-step T , which is particularly advantageous for image-editing tasks as it eliminates the need for inversion from z 0 to z T . Therefore, starting with z τ = z T ∼ N (0 , I ) , the sampling process proceeds as follows:

After many iterations, the final output is z . Furthermore, [2] demonstrates that the dual-branch paradigm (involving a source and a target branch) used in image editing tasks can be executed in an inversion-free manner. We will delve into this, along with our method description, in Section 4.2.2.

4 Multi-Aspect Image Editing

4.1 Problem Definition

The input to the multi-aspect image editing task includes a source image ( I src ), the source prompt, and a set of edits to be applied to the source image, indicating the changes from the source prompt to target prompt. A text prompt (whether source or target) comprises several independent tokens, of which only a subset is editable. We refer to these editable tokens as Aspects .

Definition 4.1 (Aspect) . We define an i th aspect A i src in the source prompt (or the j th aspect A j edt in the target prompt) as any entity that can be substituted, deleted, or inserted into the text prompt, resulting in a meaningful sentence structure.

Several examples of tokens corresponding to aspects or not are given in Fig. 3. In other words, aspects correspond to single or multiple tokens representing object color, pose, material, content, background, image style, etc. An editing operation E i → j between the editing pair ( A i src , A j edt ) as E i → j ∈ { ⊗ , ⊕ , ⊖ , ⊘ } . Here, ⊗ denotes a swap action, ⊕ denotes an object addition action, ⊖ denotes object deletion, and ⊘ indicates no change in the aspect. Such an editing operation can be inferred directly by appropriately mapping the source and target prompts, or it can be provided as metadata [3, 34]. The editing task is considered successful if the edited source image, I edt , reflects the required edits while preserving the unaffected aspects of the original image.

4.2 Method

Figure 2 outlines the overall pipeline of our method, which has three steps. In the first step (Sec. 4.2.1), we perform aspect grouping using attention maps generated by running a few iterations of the diffusion process. The aspects in the source image are put into up to N groups, each processed by a distinct branch. The second step (Sec. 4.2.2) demonstrates how each branch, which receives a specific group of aspects, performs inversion-free editing. In the last step (Sec. 4.2.3), we perform the necessary adjustments for enabling cross-branch interaction and elucidate the significance of such interaction.

K

Q

V

Q

K

V

Non-Rigid

Editing

sailboat

evening

glow

ducks

Aspect Grouping

ducks

evening glow

Source prompt:

a man sitting in a boat is silhouetted against the sunset with

mountain in the background

Target prompt:

a man sitting in

a

sailboat

is silhouetted against the

evening glow

with

ducks on the water

and

bridge

in the background

Source Branch

Global

Editing

Q K

V

K

Q

V

Q

K

V

K

Q

Q K

V

Q K

V

Edited image

Q K

V

Source image

Attention maps

Branch 3

Self Attention

Q K

V

Q K

V

Cross Attention

bridge

man

man

Branch 4

K

Q

Q

K

V

V

V

Q K

V

Q K

saiboat

K

Q

Rigid

Editing

K

V

bridge

Branch 1

Branch 2

Branch 5

4.2.1 Aspect Grouping

We would like to group aspects in a prompt into N distinct groups using the cross-attention maps of the diffusion UNet [35] to characterize the spatial layouts as in previous studies [36]. Given an editing operation E i → j between the source aspect A i src and the target aspect A j edt , we obtain the corresponding attention maps from both the source and target prompts as ¯ M i src and ¯ M j edt , respectively. The attention map M is defined by the query feature ˆ Q and key feature ˆ K from the cross-attention as M = softmax ( ˆ Q ˆ K T √ d ) . The binarized attention map ¯ M is obtained by normalizing M and thresholding its values. Our aspect grouping proceeds in two steps,

Step 1. Assign a type for every editing operation ( E i → j ) . We consider three possible types of edits, in line with previous works [4], namely a global edit, a local rigid edit or a local non-rigid edit. Rigid local edits, such as changing an object's color or texture, do not alter the layout of objects. Conversely, non-rigid local edits modify the layout of objects, such as adding or deleting objects or changing object poses. Global edits affect background and style changes. The type assignment for the editing operation ( E i → j )

sailboat

evening glow

boat

mountain

man

sunset

Source prompt:

Target prompt:

in the background

Branch 1

Branch 2

Branch 4

Branch 5

Global

a man sitting in a boat is silhouetted against the sunset with mountain

with

ducks on the water

and

bridge

in the background

ducks

< >

Branch 3

< >

bridge

a man sitting in

a

sailboat

is silhouetted against the

evening glow

Rigid

Rigid

Non-Rigid

Non-Rigid

Asrc

A!

src

A6

edt

A

STC

A

STC

is determined by the following rules:

Here, ϕ represent mIoU [37], while γ indicates the alpha mattes of attention maps. λ and β are tunable hyperparameters. For further details, please refer to the supplementary Sec. D.

Step 2. Categorize every editing operation ( E i → j ) into N groups. For each editing operation ( E i → j ) of a specific type, we assess whether ϕ ( ¯ M j edt , ¯ M k edt ) ≥ λ to determine if there exists substantial overlap between any pair of attention maps of that type. If significant overlap is detected, the attention maps are grouped together. On the other hand, if attention maps are isolated like the "boat" and "mountain" in Fig. 3 are categorized into separate groups due to small overall. Therefore, we have a total of N groups. Each group has a dedicated branch, resulting in a total of N > 2 branches.

4.2.2 Inversion-Free Multi-Branch Editing

We use a set of N branches indexed by n . These N branches are in addition to a source branch (also shown in Figure 2) that undergoes a DDCM sampling process [2]. The n th branch is calibrated to its ( n -1) th branch, and the first branch is calibrated to the source branch. The N -way target branch calibration can occur simultaneously, saving significant compute time. For the DDCM sampling process of the n th branch, it has the form of Section 3, Step 1 :

Let us break down Eq. 4 step by step. n = 1 representing the source branch, we have z (1) edt = z src and ϵ (1) edt τ = ϵ src τ . Also, z (1) edt τ = z src τ , which at time step τ = τ 1 , is random noise drawn from N (0 , I ) . Similarly, when n = N , z ( N ) edt represents the final calibrated/edited image containing all the required aspect edits after repeating for τ ∈ { τ 1 , τ 2 , . . . τ T } timesteps. The noise addition on any target branch remains the same as Step 2 , i.e., z ( n ) edt τ = √ α τ z ( n ) edt + √ 1 -α τ ϵ where ϵ ∼ N (0 , I ) . For 1 < n < N , we have ϵ ( n ) edt τ = ϵ θ ( z ( n ) edt τ , τ ) , where ϵ θ represents a parameterized noise predictor network (details in the Appendix Sec. D). A key observation is that the difference in the parameterized noise at the n th branch and ( n -1) th branch is utilized to calculate z ( n ) edt in (4). Finally, ϵ ( n ) cons τ is defined by ϵ ( n ) cons τ = ( z ( n ) edt τ - √ α τ ˆ z ( n -1) edt ) / √ 1 -α τ . Unlike the dual-branch setup in [2], the reference initial input is the estimated latent from the previous branch at a previous diffusion denoising iteration as indicated by ˆ z ( n -1) edt .

4.2.3 Cross-Branch Interactions

For rigid local branches , the cross-attention map M i n from the previous branch is either switched or injected into the current branch, akin to the method used in P2P [3]. This approach facilitates local edits while preserving structural consistency. For non-rigid local branches , we observe that the query features in the shallow layers of UNet [35] can effectively query correlated local contents and textures from the prior branch's latent features, ensuring consistency. Consequently, the key and value features from the prior branch are retained in the current branch to maintain consistent editing. We use a non-rigid editing branch to manage non-rigid local edits. In the current branch n , textures from the previous branch ( n -1) are preserved by replacing the K n -1 and V n -1 features from the last branch with the K n and V n features in the current branch. Only the query features are preserved to maintain layout semantic correspondence. Additionally, the attention mask M n -1 from the previous branch's cross-attention layer is used to guide the editing process by adding it to M n , thereby converting the object layout from M n -1 to M n . This step is crucial for object removal or shape modification edits, where the object mask is derived from the previous branch. For all global branches , there is no replacement of attention features or masks, and the attention mask is not used to guide the editing process, as the entire image is intended to be altered.

5 Experiments

PIE-Bench++ Dataset . We introduce a new dataset, PIE-Bench++, derived from PIE-Bench [1] and dedicated to evaluate the performance of multi-aspect image editing. The PIE-Bench dataset contains 700 images and prompts with single-aspect editing including object-level manipulations (addition,

A Dalmatian dog is lying on the

concrete in front of red

flowers

A Dalmatian dog with a collar is

lying on the snow covered

concrete in front of pink tulips

bread on a wooden table with

tomatoes and a napkin

chocolate bread on a wooden

table with strawberries, kiwi,

and a napkin

A grey horse walks through a

field

A chestnut horse with a

harness walks through a field

of flowers

chocolate

straw-

berry

kiwi

pink tulips

snow-

covered

collar

a painting of a woman in green

blouse and jewelry

a painting of a woman in green

dress and jewelry

blouse

jewelry

harness

flowers

chestnut

a cartoon dog wearing sunglasses

in a white background

scarf

eyeglasses

a cartoon dog wearing

eyeglasses and scarf in a white

background

notebook on white table

money tree

plant and

word

'Hello, World!' is written on the

notebook on black table

black

table

'Hello,

World!'

money

tree

由

Φ

X

区

deletion, or alteration), attribute-level manipulations (changes in content, pose, color, and material), and image-level manipulations that modify background and overall style. Our PIE-Bench++ extends PIE-Bench by enabling multi-aspect edits: 57% of our dataset have two aspect edits per prompt, 19% have more than two edits, and the remaining 24% have a signle aspect edit. For additional details and examples of the PIE-Bench++ dataset, please refer to the supplementary material.

Evaluation Metrics . We introduce two new metrics designed for evaluating multi-aspect text-driven image editing, alongside standard evaluation metrics.

(a) Aspect Accuracy-LLaVA . Drawing inspiration from the remarkable capability of large vision language models in comprehending intricate semantics within images, we propose to innovatively leverage them as an 'omniscient' agent equipped with extensive knowledge to understand various attributes of images. We use the LLaV A [38] model, trained on visual grounding tasks, to evaluate the accuracy of multi-aspect image editing. Given a text prompt with multiple aspects, such as ' A [pink] [taxi] with [colorful] [flowers] on top ', we provide the following prompt with the edited image to the LLaVA model: ' Does the image match the elements in [ ]: A [pink] [taxi] with [colorful] [flowers] on top? Return a list of numbers where 1 is matched and 0 is unmatched. ' We then parse the returned list and compute its average to determine the aspect accuracy. We name this new evaluation metric as AspAcc-LLaVA . Examples and detailed explanations of this evaluation metric are available in the supplementary material.

(b) Aspect Accuracy-CLIP . We also use the similarity of the CLIP [39] to evaluate if an attribute has been successfully edited. Given an edited image I edt and the target prompt P edt with k edited aspects A edt , every time we remove an aspect A j edt from P edt and revert it back to A i src as ˆ P edt . We then extract the CLIP [39] similarity between the edited image I edt and two prompts, i.e., s 1 = CLIP ( I edt , P edt ) and s 2 = CLIP ( I edt , ˆ P edt ) . We expect s 1 > s 2 if the aspect A j edt has been successfully edited. Thus, the aspect accuracy is k s k when a total of k s aspects have been successfully edited among k target edits. Note that in the case of an edited or added object that also involves changes in attributes (such as color or material), we consider it a successful edit only if both the object and its attributes have been successfully modified. We name this metric as AspAcc-CLIP .

(c) Standard Metrics . Several standard metrics widely used for evaluating text-image similarity and image quality are considered, including PSNR, LPIPS [40], MSE, and SSIM [41]. We also use the CLIP [39] score to measure the image-text alignment performance. Additionally, the bi-directional CLIP (D-CLIP) score [42] is reported, which is formulated as follows:

5.1 Quantitative Results

We first conduct experiment on the PIE-Bench++ dataset to compare our method with the state-of-theart text-driven image editing methods combining their corresponding inversion method leads to best

Source Image

Null-text Inversion

DirectInversion

InfEdit*

GPT-4V

PnP*

Source Image

MasaCtrl

Rich-text

✭✭

GPT-4V

InfEdit

"A German Shepherd stands on the grass wearing a collar "

"A German Shepherd stands on the grass with its mouth open

and wears a leather collar among autumn leaves"

Null-text Inversion

"a man walking dog in a city with snow mountains in the background"

Source Image

Null-text Inversion

DirectInversion

InfEdit*

Source Image

MasaCtrl

Rich-text

✭✭

GPT-4V

InfEdit

"a man walking in a town with mountains in the background"

"A bullfinch stands on a mossy branch against a blurred background"

"A yellow bullfinch stands on flowers against a starry night background"

GPT-4V

"white dumplings on brown wooden bowl"

"white cupcakes on black metal bowl"

ParallelEdits (Ours)

PnP*

P2P

P2P

ParallelEdits (Ours)

ParallelEdits (Ours)

ParallelEdits (Ours)

performance, including DDIM+MasaCtrl [4], DDIM+Prompt-to-Prompt (P2P) [3], DDIM+Plug-andPlay (PnP) [21], StyleDiffusion (StyleD) [43]+P2P, Null-text Inversion (NTI) [34]+P2P, DirectInverison (DI)[1]+PnP, and InfEdit [2]. An intuitive way to improve off-the-shelf image editing methods is to apply the single-aspect editing method sequentially. We follow [27] to adapt existing image editing methods into sequential editing processes, where these methods are applied multiple times to achieve multi-aspect editing. Each time, only one aspect is edited. Table 1 presents the metrics in terms of text-image similarity (i.e., CILP and D-CLIP scores), computational efficiency, and aspect accuracy. Our ParallelEdits model outperforms all baselines in editing effectiveness, with a slightly longer runtime than the InfEdit model. Even though sequential editing better aligns the target prompt than their vanilla methods, it significantly increases computational overhead and may propagate editing errors over time. Moreover, although the sequential editing is conducted in the latent space, it would introduce more noise and artifacts to the edited image. Hence, their performance in all editing quality metrics was inferior to our method.

5.2 Qualitative Results

Fig. 4 presents several examples of our method's multi-aspect editing on the PIE-Bench++ dataset. The results demonstrate the effectiveness of our method in handling multiple and varied types of

PIE-Bench++

Ours

DirectInversion

P2P

InfEdit

Ours

DirectInversion

P2P

InfEdit

PIE-Bench++

PIE-Bench

PIE-Bench

66.0

70

68:0

61.0

65.6

65.6

CLIP

40

60

51.3

40.9

51.3

52.150.0

45.046.0

56:0

47.047.0

39.0

45.043.0

LLaVA

65

60

63.9

59.1

61.9

63.4

59.959.460.5

62.0

58.4

63.1

60.9

55.0

59.0

55

28.6

50

1 aspect

1 aspect

2aspects

3+ aspects

1 aspect

1 aspect

2aspects

3+ aspects

edits across diverse image content. Fig. 5 further compares our method with several state-of-the-art models and one popular multi-modal large language model, GPT-4V [44], by providing the source image, source prompt, and target prompt to guide the image editing. The Rich-text [25] model differs from other models, which uses rich-text prompt to edit the image generated from the plain (source) text prompt. The results show that current image editing models even with sequential editing fail to edit multiple aspects, while multi-modal large language models fail to preserve the content of source image. Our method achieves visually convincing results by successfully editing different attributes with good content preservation.

5.3 Ablation Study and Analysis

(a) Impact of Editing Aspect Number. We first examine the performance of our ParallelEdits and baseline methods on various editing aspect numbers by comparing CLIP and LLaVA-based aspect accuracies on the original PIE-Bench [1] and our PIE-Bench++ datasets. The bar charts in Fig. 6 show the outstanding performance of our method across all settings, including single-aspect editing on two datasets and multi-aspect editing. Takeaway: the proposed ParallelEdits demonstrates robustness across varying numbers of editing aspects.

(b) Evaluation on Perservation. We follow [1] to evaluate the background preservation. We first use the PSNR, LPIPS [40], MSE and SSIM [41] to evaluate the background preservation. We measure that metric on a subset of images of our proposed PIE-Bench++ dataset where the background can be well defined in that image, e.g., no image style or background editing, and the background is visible after aspect editing. The results are shown in Table 2, where we compare our method with the top performance methods in Table 1. Moreover, we adopt the similar way as calculating the AspAcc-LLaVA to prompt LLaVA [38] for evaluating how the unchanged aspect preserves in the edited image. We also calculate the CLIP [39] score between the target image and the text prompt after removing all edited aspects. The results are reported in Table 2 noted as CLIP and LLaVA, respectively. Takeaway: preservation is even maintained in ParallelEdits .

(c) Branches numbers and aspect grouping. To demonstrate the effectiveness of our multi-branch design and early aspect grouping, we design additional ablation studies for our method in threefold. (1) We only use one single non-rigid branch to conduct all edits; (2) we remove the aspect categorization process from the pipeline and use the same non-rigid branch for each edit; (3) we adopt one single branch for different type of edits without using any auxillary branches which results a total of three branches (also see Section B for more details). Takeaway: As shown in Table 3, the multi-branch design and aspect grouping play a significant role in enhancing the performance of our proposed ParallelEdits .

Table 4: Comparison on each category in PIE-Bench++. Our ParallelEdits achieves the best performance on most of the categories from the dataset.

(d) Performance comparison on each category. Recall that our dataset includes nine different categories for editing. We compare the performance of baseline models and our approach across the nine categories, as presented in Table 4. Takeaway: Our proposed ParallelEdits achieves state-of-theart performance across most categories.

Limitations and Failure Cases. The proposed ParallelEdits has several limitations. First, it cannot handle the text editing in the image, as shown in the last image pair of Fig. 4. Second, ParallelEdits fails to edit dramatic background changes, as examples shown in the supplementary material.

6 Conclusion

In this work, we propose a new research task, multi-aspect text-driven image editing, to modify multiple object types, attributes, and relationships. We introduce a dedicated method, ParallelEdits, to multi-aspect text-driven image editing as an effective and efficient solution to this problem. Due to the lack of evaluation benchmark, we introduce PIE-Bench++, an improved version of PIE-Bench [1] tailored for simultaneous multiple-aspect edits within images. ParallelEdits achieves better quality and performance than existing methods on proposed PIE-Bench++. Our work introduces ParallelEdits, a novel approach that adeptly handles multiple attribute edits simultaneously, preserving the quality of edits across single and multiple attributes through a unique attention grouping mechanism without adding computational complexity. There are several future works we would like to explore. First, different aspects of an image have a specific semantic order. Editing these aspects according to their intrinsic order will simplify the editing process. Secondly, the current ParallelEdits still has limitations, as shown in Fig. 4. It will be of interest to study approaches to improve these aspects.

Ethics Statement . In anticipation of contributing to the academic community, we plan to make the dataset and associated code publicly available for research. Nonetheless, we acknowledge the potential for misuse, particularly by those aiming to generate misinformation using our methodology. We will release our code under an open-source license with explicit stipulations to mitigate this risk. These conditions will prohibit the distribution of harmful, offensive, or dehumanizing content or negatively representing individuals, their environments, cultures, religions, and so forth through the use of our model weights.

Acknowledgement . This work was supported in part by the National Science Foundation (NSF) Projects under grants SaTC-2153112, No.1822190, and TIP-2137871. Prof. Lokhande thanks support provided by University at Buffalo Startup funds. We thank Sudhir Kumar Yarram for the insightful discussions on the project.

References

Xuan Ju, Ailing Zeng, Yuxuan Bian, Shaoteng Liu, and Qiang Xu. Direct inversion: Boosting diffusion-based editing with 3 lines of code. 2023.

Sihan Xu, Yidong Huang, Jiayi Pan, Ziqiao Ma, and Joyce Chai. Inversion-free image editing with natural language. arXiv preprint arXiv:2312.04965 , 2023.

Amir Hertz, Ron Mokady, Jay Tenenbaum, Kfir Aberman, Yael Pritch, and Daniel Cohen-Or. Prompt-to-prompt image editing with cross attention control. 2022.

Mingdeng Cao, Xintao Wang, Zhongang Qi, Ying Shan, Xiaohu Qie, and Yinqiang Zheng. Masactrl: Tuning-free mutual self-attention control for consistent image synthesis and editing. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV) , pages 22560-22570, October 2023.

Jizhizi Li, Jing Zhang, and Dacheng Tao. Referring image matting. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , pages 22448-22457, 2023.

Yuhao Liu, Jiake Xie, Xiao Shi, Yu Qiao, Yujie Huang, Yong Tang, and Xin Yang. Tripartite information mining and integration for image matting. In Proceedings of the IEEE/CVF International Conference on Computer Vision , pages 7555-7564, 2021.

Shuyang Gu, Dong Chen, Jianmin Bao, Fang Wen, Bo Zhang, Dongdong Chen, Lu Yuan, and Baining Guo. Vector quantized diffusion model for text-to-image synthesis. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 10696-10706, 2022.

Ming Ding, Wendi Zheng, Wenyi Hong, and Jie Tang. Cogview2: Faster and better text-toimage generation via hierarchical transformers. Advances in Neural Information Processing Systems , 35:16890-16902, 2022.

Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Björn Ommer. Highresolution image synthesis with latent diffusion models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 10684-10695, 2022.

Guillaume Couairon, Jakob Verbeek, Holger Schwenk, and Matthieu Cord. Diffedit: Diffusionbased semantic image editing with mask guidance. In The Eleventh International Conference on Learning Representations , 2023.

Gaurav Parmar, Krishna Kumar Singh, Richard Zhang, Yijun Li, Jingwan Lu, and Jun-Yan Zhu. Zero-shot image-to-image translation. In ACM SIGGRAPH 2023 Conference Proceedings , pages 1-11, 2023.

Bahjat Kawar, Shiran Zada, Oran Lang, Omer Tov, Huiwen Chang, Tali Dekel, Inbar Mosseri, and Michal Irani. Imagic: Text-based real image editing with diffusion models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 6007-6017, 2023.

Guillaume Couairon, Jakob Verbeek, Holger Schwenk, and Matthieu Cord. Diffedit: Diffusionbased semantic image editing with mask guidance. arXiv preprint arXiv:2210.11427 , 2022.

Thao Nguyen, Yuheng Li, Utkarsh Ojha, and Yong Jae Lee. Visual instruction inversion: Image editing via image prompting. Advances in Neural Information Processing Systems , 36, 2023.

Andreas Lugmayr, Martin Danelljan, Andres Romero, Fisher Yu, Radu Timofte, and Luc Van Gool. Repaint: Inpainting using denoising diffusion probabilistic models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 11461-11471, 2022.

Omri Avrahami, Dani Lischinski, and Ohad Fried. Blended diffusion for text-driven editing of natural images. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 18208-18218, 2022.

Alex Nichol, Prafulla Dhariwal, Aditya Ramesh, Pranav Shyam, Pamela Mishkin, Bob McGrew, Ilya Sutskever, and Mark Chen. Glide: Towards photorealistic image generation and editing with text-guided diffusion models. arXiv preprint arXiv:2112.10741 , 2021.

Jooyoung Choi, Sungwon Kim, Yonghyun Jeong, Youngjune Gwon, and Sungroh Yoon. Ilvr: Conditioning method for denoising diffusion probabilistic models. arXiv preprint arXiv:2108.02938 , 2021.

Sihan Xu, Ziqiao Ma, Yidong Huang, Honglak Lee, and Joyce Chai. Cyclenet: Rethinking cycle consistency in text-guided diffusion for image manipulation. Advances in Neural Information Processing Systems , 36, 2024.

Min Zhao, Fan Bao, Chongxuan Li, and Jun Zhu. Egsde: Unpaired image-to-image translation via energy-guided stochastic differential equations. Advances in Neural Information Processing Systems , 35:3609-3623, 2022.

Narek Tumanyan, Michal Geyer, Shai Bagon, and Tali Dekel. Plug-and-play diffusion features for text-driven image-to-image translation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 1921-1930, 2023.

Hao Wang, Guosheng Lin, Ana García del Molino, Anran Wang, Zehuan Yuan, Chunyan Miao, and Jiashi Feng. Maniclip: Multi-attribute face manipulation from text. arXiv preprint arXiv:2210.00445 , 2022.

Siavash Khodadadeh, Shabnam Ghadar, Saeid Motiian, Wei-An Lin, Ladislau Bölöni, and Ratheesh Kalarot. Latent to latent: A learned mapper for identity preserving editing of multiple face attributes in stylegan-generated images. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision , pages 3184-3192, 2022.

Tero Karras, Samuli Laine, Miika Aittala, Janne Hellsten, Jaakko Lehtinen, and Timo Aila. Analyzing and improving the image quality of stylegan. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , pages 8110-8119, 2020.

Songwei Ge, Taesung Park, Jun-Yan Zhu, and Jia-Bin Huang. Expressive text-to-image generation with rich text. In Proceedings of the IEEE/CVF International Conference on Computer Vision , pages 7545-7556, 2023.

Hangeol Chang, Jinho Chang, and Jong Chul Ye. Ground-a-score: Scaling up the score distillation for multi-attribute editing. arXiv preprint arXiv:2403.13551 , 2024.

KJ Joseph, Prateksha Udhayanan, Tripti Shukla, Aishwarya Agarwal, Srikrishna Karanam, Koustava Goswami, and Balaji Vasan Srinivasan. Iterative multi-granular image editing using diffusion models. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision , pages 8107-8116, 2024.

Alexander Quinn Nichol and Prafulla Dhariwal. Improved denoising diffusion probabilistic models. In International Conference on Machine Learning , pages 8162-8171. PMLR, 2021.

Ting Chen. On the importance of noise scheduling for diffusion models. arXiv preprint arXiv:2301.10972 , 2023.

Tero Karras, Miika Aittala, Timo Aila, and Samuli Laine. Elucidating the design space of diffusion-based generative models. Advances in Neural Information Processing Systems , 35:26565-26577, 2022.

Jiaming Song, Chenlin Meng, and Stefano Ermon. Denoising diffusion implicit models. arXiv preprint arXiv:2010.02502 , 2020.

Yang Song, Prafulla Dhariwal, Mark Chen, and Ilya Sutskever. Consistency models. arXiv preprint arXiv:2303.01469 , 2023.

Simian Luo, Yiqin Tan, Longbo Huang, Jian Li, and Hang Zhao. Latent consistency models: Synthesizing high-resolution images with few-step inference. arXiv preprint arXiv:2310.04378 , 2023.

Ron Mokady, Amir Hertz, Kfir Aberman, Yael Pritch, and Daniel Cohen-Or. Null-text inversion for editing real images using guided diffusion models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 6038-6047, 2023.

Olaf Ronneberger, Philipp Fischer, and Thomas Brox. U-net: Convolutional networks for biomedical image segmentation. In Medical Image Computing and Computer-Assisted Intervention-MICCAI 2015: 18th International Conference, Munich, Germany, October 5-9, 2015, Proceedings, Part III 18 , pages 234-241. Springer, 2015.

Raphael Tang, Linqing Liu, Akshat Pandey, Zhiying Jiang, Gefei Yang, Karun Kumar, Pontus Stenetorp, Jimmy Lin, and Ferhan Ture. What the daam: Interpreting stable diffusion using cross attention. arXiv preprint arXiv:2210.04885 , 2022.

Tsung-Yi Lin, Michael Maire, Serge Belongie, James Hays, Pietro Perona, Deva Ramanan, Piotr Dollár, and C Lawrence Zitnick. Microsoft coco: Common objects in context. In European conference on computer vision , pages 740-755. Springer, 2014.

Haotian Liu, Chunyuan Li, Qingyang Wu, and Yong Jae Lee. Visual instruction tuning. In NeurIPS , 2023.

Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervision. In International conference on machine learning , pages 8748-8763. PMLR, 2021.

Richard Zhang, Phillip Isola, Alexei A Efros, Eli Shechtman, and Oliver Wang. The unreasonable effectiveness of deep features as a perceptual metric. In Proceedings of the IEEE conference on computer vision and pattern recognition , pages 586-595, 2018.

Zhou Wang, Alan C Bovik, Hamid R Sheikh, and Eero P Simoncelli. Image quality assessment: from error visibility to structural similarity. IEEE transactions on image processing , 13(4):600612, 2004.

Chen Henry Wu and Fernando De la Torre. Unifying diffusion models' latent space, with applications to cyclediffusion and guidance. arXiv preprint arXiv:2210.05559 , 2022.

Senmao Li, Joost van de Weijer, Taihang Hu, Fahad Shahbaz Khan, Qibin Hou, Yaxing Wang, and Jian Yang. Stylediffusion: Prompt-embedding inversion for text-based editing. 2023.

OpenAI. Gpt-4v(ision) system card. 2023.

Ron Mokady, Amir Hertz, Kfir Aberman, Yael Pritch, and Daniel Cohen-Or. Null-text inversion for editing real images using guided diffusion models. arXiv preprint arXiv:2211.09794 , 2022.

Narek Tumanyan, Omer Bar-Tal, Shai Bagon, and Tali Dekel. Splicing vit features for semantic appearance transfer. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 10748-10757, 2022.

Appendix

A ParallelEdits: The Algorithm

In this section we provide Algorithm 1: Early Aspect Grouping and Algorithm 2: ParallelEdits on a particular branch . These algorithms describe the overall idea behind ParallelEdits. They are also pictorially illustrated in Figures 2 and 3 of the main paper. Let us denote an arbitrary branch and the timestep in the diffusion process by n and t respectively. Firstly, in Algorithm 1, we demonstrate how Early Aspect Grouping is conducted over the attention maps. Recall that we refer to this as 'early" aspect grouping because only a few steps (maximum of 5 ) are sufficient to perform the grouping. This phase of ParallelEdits takes as an input, the edit action set { E i → j } and the corresponding cross-attention maps for every token A j src , and outputs the grouped edit actions set ¯ A c edt . Recall from Section 4 of the paper that E i → j ∈ { ⊗ , ⊕ , ⊖ , ⊘ } , with ⊗ denoting a swap action, ⊕ denoting an add action, ⊖ denoting aspect deletion, and ⊘ indicating no change in the aspect. Once grouped edit actions set is computed, it is fed into Algorithm 1 to conduct multi-aspect editing and obtain the edited latent features. In Algorithm 2, we implement several operations on the attention masks, similar to the P2P method [3], and describe them as follows.

Replace : Swapping token attention mask M n -1 in the prompt from previous branch, overriding M n ;

Refine : Injecting only the attention mask that corresponds to the unchanged part of the prompt from M n -1 to M n ;

Retain : Keeping the attention mask M n unchanged.

Algorithm 1 Early Aspect Grouping

B Some More Details on ParallelEdits

In the literature [4, 3], image editing processes have been conducted through the implementation of a dual-branch approach. This method involves utilizing a source and target branches for editing.

Specifically, the source branch is reverted to z 0 , while the trajectory of the target branch is iteratively adjusted. By computing the distance from the source branch and ϵ cons with Latent Consistency Model [32], the target branch is calibrated at each time step.

Our experiments, as seen in Section 5 of the main paper, show the ineffectiveness of a dual-branch procedure for multi-aspect editing tasks. Specifically, a single target branch is inadequate, leading to imperfection in the target image. Thereby we advocate multi-aspect editing through the use of multiple target branches. Each target branch handles a group of aspects, with simpler aspects such as non-rigid local edits directed to initial branches, and more complex aspects such as rigid local edits deferred to subsequent ones. Note that however, all the branches operate simultaneously.

Auxiliary Rigid / Non-Rigid Branches. In the main paper, it was noted that there was one dedicated branch for each type of edit: non-rigid, rigid, and global edit. The Early Aspect Grouping algorithm 1 classifies aspects into these three categories. Our experiments revealed that sometimes, due to low overlap between attention maps, aspects may not always be grouped into dedicated rigid or non-rigid branches. In such cases, it becomes necessary to include an auxiliary branch to handle the ungrouped aspects. Therefore, ParallelEdits may involve a single rigid branch and additional auxiliary branches to manage ungrouped aspects, and similarly, a single non-rigid branch and supplementary auxiliary branches to address ungrouped aspects. An ablation study on auxiliary branches is provided in Table 3.

C More Details on Evaluation Metrics

In this section, we describe more details of our evaluation metrics.

LLaVA aspect accuracy . We show how we leverage LLaVA [38] to evaluate the multi-aspect editing accuracy in Fig. 7 and Fig. 8, we also prompt LLaVA [38] for explanation to human readers. LLaVA [38] could provide detailed summary for the image and also explanations for the mismatched between edited aspects and image.

Other evaluation metrics . Moreover, even though the Structure Distance [46] has been used in PIE-Bench [1] to evaluate the structure between source and target image while ignoring appearance information, it could not serve as a good evaluation metric for multi-aspect editing. This is due to the fact that the structure of multi-aspect edited target image may necessitate substantial modifications, particularly when it involves adding or removing multiple objects.

Does this image match the elements in []:

"a [sailboat] is silhouetted against the [evening glow] with [snow] mountain in the backgrounds, [winter scene]. "

Exactly follow the return format as a list where 1 is matched and 0 is unmatched, return list only.

The list length is equal to 4.

[1,1,1,1]

Please provide some explanation.

The image matches the elements in the description as follows:

1.

A sailboat is present in the image, which is silhouetted against the evening glow.

The evening glow is visible in the sky, creating a beautiful backdrop for the sailboat.

3.

The mountains in the background are covered with snow, which is consistent with the description.

4.

The overall scene is a winter scene, as indicated by the snow-covered mountains and the calm, cold appearance of the water.

Therefore, all elements in the description are matched by the image, resulting in a list of 4 ones.

🌋

LLaVA

🌋

LLaVA

User

Does this image match the elements in []:

"a man [standing] in a boat is silhouetted against the sunset and [ducks] on the water with [Alps mountain] in the background"

Exactly follow the return format as a list where 1 is matched and 0 is unmatched, return list only.

The list length is equal to 4.

[1,1,1,1]

Please provide some explanation.

The image matches all the elements in the given description:

1. A man is standing in a boat.

2. The man is silhouetted against the sunset.

3. There are ducks on the water.

4. The background features a mountain range, which could be described as Alps-like.

Therefore, the elements in the description are all present in the image, and the list would be [1, 1, 1, 1].

🌋

LLaVA

🌋

LLaVA

Does this image match the elements in []:

"a [boat] in a lake with a [house] and [ice mountain] in the background"

Exactly follow the return format as a list where 1 is matched and 0 is unmatched, return list only.

The list length is equal to 3.

[1,1,1]

Please provide some explanation.

The image shows a boat on a lake with a house and a mountain in the background. The mountain appears to be covered with snow, which could be described as

an "ice mountain." The elements in the brackets match the description provided:

A boat is present in the image.

A house is visible in the background.

The mountain in the background is covered with snow, which could be described as an "ice mountain."

Therefore, the elements in the brackets match the description, and the list would be [1, 1, 1].

🌋

LLaVA

🌋

LLaVA

Does this image match the elements in []:

"[money tree plant] and [word 'Hello, World!'] is written on the notebook on [black table]"

Exactly follow the return format as a list where 1 is matched and 0 is unmatched, return list only.

The list length is equal to 3.

[1,0,0]

Please provide some explanation.

Certainly! Here's an explanation for the elements in the image and their match with the elements in the provided format:

1, "[money tree plant]" - The image shows a plant that is commonly known as a money tree. It is a type of bonsai tree with a twisted trunk and glossy green

leaves. So, this element is matched.

2, "[word 'Hello, World!'] is written on the notebook" - The notebook in the image has the word "Hello, World!" written on it. However, the format

specifies that the word should be written on the notebook, which is not the case here. The word is actually written on a whiteboard or a similar surface

that is placed in front of the notebook. Therefore, this element is unmatched.

3, "[black table]" - The table in the image is not black; it is a light color, possibly gray or beige. So, this element is unmatched.

Therefore, the list of matches would be [1, 0, 0].

🌋

LLaVA

🌋

LLaVA

Does this image match the elements in []:

"a logo of [X] [with pink color] in a [light background]"

Exactly follow the return format as a list where 1 is matched and 0 is unmatched, return list only.

The list length is equal to 3.

[1,1,0]

Please provide some explanation.

Certainly! The image you've provided features a logo that is composed of an X shape, which matches the first element in the description. The X is colored in

a shade of pink, which matches the second element in the description. However, the background of the image is not light; it is dark, which does not match

the third element in the description. Therefore, the list of matches would be:

1, Logo is an X - Matched

2, X is pink - Matched

3, Background is light - Not Matched

This results in the list [1, 1, 0].

🌋

LLaVA

🌋

LLaVA

User

D Implementation Details

Our proposed ParallelEdits is based on the Latent Consistency Model [32], with the publicly available LCM † which is finetuned from Stable Diffusion v1.5. We then follow [2] to leverage their proposed inversion-free technique in ParallelEdits for image editing. During sampling, we perform LCM sampling [32] with 15 denoising steps, and the classifier-free guidance (CFG) is set to 4.0. ParallelEdits can control the editing strength by adjusting the CFG . There's a trade-off between achieving satisfactory inversion and robust editing ability. A higher CFG tends to produce stronger editing effects but may lower inversion results and identity preservation. We also set the hyper-parameter θ as 0.9 and β as 0.8 in our experiments, where θ, β are used to determine the edit type of a given edit action.

In the inversion-free multi-branch editing approach, for 1 < n < N , the noise estimation is also conditioned on a text conditioning c n in branch n . This can be expressed as ϵ ( n ) edt τ = ϵ θ ( z ( n ) edt τ , τ, c n ) . Here, c 1 corresponds to the source prompt, c N corresponds to the target prompt, and c n represents the prompt that includes all aspect edits up to branch n .

E Additional Details of PIE-Bench++

E.1 PIE-Bench++ Details

Unlike existing benchmarks that primarily focus on single-aspect edits, PIE-Bench++ is tailored to multiple aspect edits, reflecting the complexities inherent in real-world editing tasks. Our enhanced dataset, PIE-Bench++, builds upon the PIE-Bench [1] by incorporating 700 images across nine diverse categories, covering both natural and artificial scenes, with a significant focus on multi-aspect editing scenarios. Specifically, the Change Object category involves swapping objects in the scene with different yet reasonable alternatives. Add Object adds new elements to the scene. Delete Object focuses on removing objects, testing the model's ability to erase elements seamlessly. Change Object Content alters the content of specific objects, such as changing the design on a shirt or the pattern on a wall. Change Object Pose includes changes in the shape of objects, humans, or animals. Change Object Color assesses the model's ability to apply accurate color changes. Change Object Material evaluates the rendering of different textures and materials. Change Background involves editing scenarios where there is a distinct foreground object and a main background. This type of edit focuses on seamlessly integrating new background elements while preserving the integrity of the foreground object. Change Image Style involves the application of style transfer techniques to the entire image while ensuring the original content remains intact. For example, this could involve transforming a photograph to adopt a cartoon style. Each category is carefully curated to provide a comprehensive evaluation of the dataset's multi-aspect editing capabilities, the summary of the dataset is shown in Table 5.

E.2 Dataset Annotation

The annotation process involves a primary annotator who labels the source prompt, describing the original image, and the target prompt, which outlines the desired modifications to generate the target image. The target prompt is carefully annotated to include all editing pairs expected to be reflected in the target image. Subsequently, a second annotator reviews the annotations for accuracy and consistency, ensuring the reliability of the dataset. The majority of target prompts in PIE-Bench++ feature at least two edited aspects. Nevertheless, within the categories that solely changing background and image styles, the number of edits is usually constrained to one or two aspects. This limitation is due to the intrinsic characteristics of these attributes, such as each image having only one background or style.

Annotation format details . Each image in the dataset annotation is associated with key elements as shown in Fig. 9: a source prompt, a target prompt, an edit action, and a mapping of aspects. The edit action specifies the position index in the source prompt where changes are to be made, the type of edit to be applied, and the operation required to achieve the desired outcome. The aspect mapping connects objects undergoing editing to their respective modified attributes, enabling the identification of which objects are subject to editing.

"source_prompt": "a colorful bird standing on a branch",

"edit_action":   {"owl":{"position":2,"edit_type":1,"action":"bird"}, "brown":{"position":1,"edit_type":6,"action":"colorful"}, "flower":{"position":6,"edit_type":1,"action":"branch"}, "red":{"position":6,"edit_type":6,"action":"+"}},

"aspect_mapping": {"owl":["brown"],"flower":["red"]}

Source Image

"target_prompt": "a brown owl standing on a red flower",

Text-based annotation

"source_prompt": "a round cake with orange frosting on a wooden plate",

"edit_action":   {"square":{"position":1,"edit_type":4,"action":"bird"},

"strawberry frosting:{"position":4,"edit_type":6,"action":"orange frosting"},

"plastic":{"position":8,"edit_type":7,"action":"wooden"}},

"aspect_mapping": {"cake":["square","strawberry frosting"], "plate":["plastic"]}

"target_prompt": "a square cake with strawberry frosting on a plastic plate",

"source_prompt": "a slanted mountain bicycle on the road in front of a building",

"edit_action":   {"rusty":{"position":2,"edit_type":7,"action":"+"}, "motorcycle":{"position":3,"edit_type":1,"action":"bicycle"}, "fence":{"position":11,"edit_type":8,"action":"building"}, "on the road":{"position":4,"edit_type":3,"action":"-"}},

"aspect_mapping": {"motorcycle":["rusty"],"fence":["red"],"road":[]}

"target_prompt": "a slanted rusty mountain motorcycle on the road in front of a fence",

"source_prompt": "the galaxy over the durdle door",

"edit_action":   {"pink":{"position":1,"edit_type":6,"action":"+"}, "sunset":{"position":1,"edit_type":8,"action":"galaxy"}, "and rainbow":{"position":2,"edit_type":2,"action":"+"}},

"aspect_mapping": {"sunset":["pink"],"rainbow":[]}

"target_prompt": "the pink sunset and rainbow over the durdle door",

F Additional Qualitative Results

We also provide more qualitative results in Fig. 10, showing the effectiveness of our proposed method in handling multi-aspect editing tasks. These examples showcase the model's proficiency in executing intricate edits. For instance, as depicted in Fig. 10 (b), our method successfully removes a cup while accurately reconstructing the obscured parts of the lamp behind it. In Fig. 10 (a), the model demonstrates its ability to swap and add aspects, while preserving the composition of the scene. The results underscore the model's adeptness in interpreting and executing sophisticated editing instructions, leading to visually consistent and contextually fitting edited images. Additional, we also provide the results for sequential editing methods with different editing order in Fig. 11 and Fig. 12.

a bowl of strawberries and

blueberries on a striped

tablecloth

a wooden bowl of strawberries,

blueberries, and ice cream ball on

a striped tablecloth with a fork

fork

wooden

a wooden bowl of rice with a

spoon in the kitchen

a bowl of icecream with a spoon

wooden

rice

kitchen

illustration of a cup and a lamp

on a table by a window

illustration of

a cup and

a lamp

on a table by a window with a

moon outside

cup

moon

luxury bedroom interior with

marble wall and field outside

luxury

bedroom interior with

stone wall and ocean outside

a metal chair and a table sits in

front of a wall with a floral

pattern and a mirror

a chair sits in front of a wall

with a floral pattern

stone

wall

ocean

metal

chair

table

mirror

a white church sits on a hill in a

field

a pink house sits on a hill in a

flower field with clouds on the

sky

clouds

pink

house

flowers

ice cream

ball

(a)

(b)

(c)

(d)

(e)

(f)

a lake with mountains in the

background

a boat in a lake with a house and

ice mountains in the background

"rustic"

house,

boat

ice -

mountains

a logo of bird shape in a black

background

a logo of

X with pink color in

a light background

light

pink color

(g)

(h)

cupcakes

black bowl

metal bowl

Source Prompt:

white dumplings on

Target Prompt:

white cupcakes on

black metal bowl

white cat

InfEdit Sequential

InfEdit Sequential

DDIM + PnP Sequential

Source Prompt:

A dog is laying down

on a white background

Target Prompt:

A white cat wearing

sunglasses is laying

down on a white

background'

cat

sunglasses

DI + PnP Sequential

ocean

leather collar

rose

German Shepherd

Source Prompt:

A Golden Retriever

holding a tulip sitting

on the ground in front

of fence

Target Prompt:

A German Shepherd

wearing a leather collar

holding a rose sitting on

the ground in front of

the ocean

InfEdit Sequential

rose

rose

German Shepherd

German Shepherd

ocean

ocean

leather collar

leather collar

ocean

leather collar

rose

German Shepherd

rose

rose

German Shepherd

German Shepherd

ocean

ocean

leather collar

leather collar

DDIM + PnP Sequential

NeurIPS Paper Checklist

1. Claims

Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?

Answer: [Yes]

Justification: The main claims made in the abstract and introduction accurately reflect the paper's contributions and scope.

Guidelines:

The answer NA means that the abstract and introduction do not include the claims made in the paper.

The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or NA answer to this question will not be perceived well by the reviewers.

The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.

It is fine to include aspirational goals as motivation as long as it is clear that these goals are not attained by the paper.

2. Limitations

Question: Does the paper discuss the limitations of the work performed by the authors?

Answer: [Yes]

Justification: We include the limitation and failure cases of the work in Sec. 5.

Guidelines:

The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.

The authors are encouraged to create a separate "Limitations" section in their paper.

The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.

The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on a few datasets or with a few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.

The authors should reflect on the factors that influence the performance of the approach. For example, a facial recognition algorithm may perform poorly when image resolution is low or images are taken in low lighting. Or a speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.

The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.

If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.

While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an important role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.

3. Theory Assumptions and Proofs

Question: For each theoretical result, does the paper provide the full set of assumptions and a complete (and correct) proof?

Answer: [NA]

Justification: This paper does not include theoretical result.

Guidelines:

The answer NA means that the paper does not include theoretical results.

All the theorems, formulas, and proofs in the paper should be numbered and crossreferenced.

All assumptions should be clearly stated or referenced in the statement of any theorems.

The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide a short proof sketch to provide intuition.

Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.

Theorems and Lemmas that the proof relies upon should be properly referenced.

4. Experimental Result Reproducibility

Question: Does the paper fully disclose all the information needed to reproduce the main experimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?

Answer: [Yes]

Justification: We fully disclose all the information needed to reproduce the main experimental results

Guidelines:

The answer NA means that the paper does not include experiments.

If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.

If the contribution is a dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.

Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g., in the case of a large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.

While NeurIPS does not require releasing code, the conference does require all submissions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example

(a) If the contribution is primarily a new algorithm, the paper should make it clear how to reproduce that algorithm.

(b) If the contribution is primarily a new model architecture, the paper should describe the architecture clearly and fully.

(c) If the contribution is a new model (e.g., a large language model), then there should either be a way to access this model for reproducing the results or a way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).

(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome to describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.

5. Open access to data and code

Question: Does the paper provide open access to the data and code, with sufficient instructions to faithfully reproduce the main experimental results, as described in supplemental material?

Answer: [Yes]

Justification: The code and data will be open-sourced for academic use.

Guidelines:

The answer NA means that paper does not include experiments requiring code.

Please see the NeurIPS code and data submission guidelines ( https://nips.cc/pu blic/guides/CodeSubmissionPolicy ) for more details.

While we encourage the release of code and data, we understand that this might not be possible, so 'No' is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for a new open-source benchmark).

The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines ( https://nips.cc/public/guides/CodeSubmissionPolicy ) for more details.

The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.

The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only a subset of experiments are reproducible, they should state which ones are omitted from the script and why.

At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).

Providing as much information as possible in supplemental material (appended to the paper) is recommended, but including URLs to data and code is permitted.

6. Experimental Setting/Details

Question: Does the paper specify all the training and test details (e.g., data splits, hyperparameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?

Answer: [Yes]

Justification: We specify all the training and test details

Guidelines:

The answer NA means that the paper does not include experiments.

The experimental setting should be presented in the core of the paper to a level of detail that is necessary to appreciate the results and make sense of them.

The full details can be provided either with the code, in appendix, or as supplemental material.

7. Experiment Statistical Significance

Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?

Answer: [Yes]

Justification: The paper reports error bars

Guidelines:

The answer NA means that the paper does not include experiments.

The authors should answer "Yes" if the results are accompanied by error bars, confidence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper.

The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).

The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)

The assumptions made should be given (e.g., Normally distributed errors).

It should be clear whether the error bar is the standard deviation or the standard error of the mean.

It is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have a 96% CI, if the hypothesis of Normality of errors is not verified.

For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out of range (e.g. negative error rates).

If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.

8. Experiments Compute Resources

Question: For each experiment, does the paper provide sufficient information on the computer resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?

Answer: [Yes]

Justification: The paper provide sufficient information on the computer resources

Guidelines:

The answer NA means that the paper does not include experiments.

The paper should indicate the type of compute workers CPU or GPU, internal group, or cloud provider, including relevant memory and storage.

The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute.

The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make it into the paper).

9. Code Of Ethics

Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics https://neurips.cc/public/EthicsGuidelines ?

Answer: [Yes]

Justification: The code follows the NeurIPS Code of Ethics.

Guidelines:

The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.

If the authors answer No, they should explain the special circumstances that require a deviation from the Code of Ethics.

The authors should make sure to preserve anonymity (e.g., if there is a special consideration due to laws or regulations in their jurisdiction).

10. Broader Impacts

Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?

Answer: [Yes]

Justification: The paper includes the discussion of potential societal impacts.

Guidelines:

The answer NA means that there is no societal impact of the work performed.

If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact.

Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.

The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to

generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.

The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.

If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML).

11. Safeguards

Question: Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?

Answer: [NA]

Justification: The paper poses no such risks.

Guidelines:

The answer NA means that the paper poses no such risks.

Released models that have a high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.

Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.

We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make a best faith effort.

12. Licenses for existing assets

Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?

Answer: [Yes]

Justification: CC-BY 4.0 for PIE-Bench.

Guidelines:

The answer NA means that the paper does not use existing assets.

The authors should cite the original paper that produced the code package or dataset.

The authors should state which version of the asset is used and, if possible, include a URL.

The name of the license (e.g., CC-BY 4.0) should be included for each asset.

For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.

If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswithcode.com/datasets has curated licenses for some datasets. Their licensing guide can help determine the license of a dataset.

For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.

If this information is not available online, the authors are encouraged to reach out to the asset's creators.

13. New Assets

Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?

Answer: [Yes]

Justification: The documentation provided alongside the assets

Guidelines:

The answer NA means that the paper does not release new assets.

Researchers should communicate the details of the dataset/code/model as part of their submissions via structured templates. This includes details about training, license, limitations, etc.

The paper should discuss whether and how consent was obtained from people whose asset is used.

At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.

14. Crowdsourcing and Research with Human Subjects

Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well as details about compensation (if any)?

Answer: [NA]

Justification: The paper does not involve crowdsourcing nor research with human subjects

Guidelines:

The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.

Including this information in the supplemental material is fine, but if the main contribution of the paper involves human subjects, then as much detail as possible should be included in the main paper.

According to the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.

15. Institutional Review Board (IRB) Approvals or Equivalent for Research with Human Subjects

Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?

Answer: [NA]

Justification: The paper does not involve crowdsourcing nor research with human subjects. Guidelines:

The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.

Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.

We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution.

For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.

Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper aims to make multiple objects or attributes editing, while preserving the quality, named ParallelEdits. The paper also introduces a new dataset PIE-Bench++. Experimental results on both PIE-Bench and PIE-Bench++ demonstrate that this method outperforms many existing editing techniques.\n\nSTRENGTHS:\nSimultaneously editing multiple objects or attributes is interesting.\n\nThe finding that the order in which aspects are modified when applying single-aspect text-driven image editing methods can affect the quality is interesting.\n\nWEAKNESSES:\n1.\tDirection Inversion is solely a method for inverting real images, not for editing them. Figure 1 is confusing in this context. Direction Inversion can be combined with various editing methods like P2P, PnP, P2P-zero, and MasaCtrl. Which editing method is Direction Inversion combined with in the third row of results? Why is this particular editing method shown?\n2.\tSame problem in the description in the introduction (Line 22, latest methods [3, 1, 4]; Line36, Unlike traditional editing methods (e.g., [1, 2])). [1] (Direction Inversion) is not a editing method.\n3.\tThe paper claims that ParalleEdits can edit multiple objects or attributes simultaneously. Figure 1. illustrates the swapping of multiple objects within the same image, but it only demonstrates the addition of a single object. Can ParalleEdits add multiple objects within a single image, such as both a necktie and sunglasses to a cat?\n4.\tIn aspect grouping, the paper does not explain how to get the attention map M for a real image.\n5.\tIn Figure 2, why does the attention map from the original image combine with the target prompt.\n6.\tThis paper divides aspect grouping based on cross-attention. What if cross-attention is inaccurate? For example, in Figure 2, will each image clearly show a \"ducks\" cross-attention map when \"ducks\" are added to the image?\n7.\tIn Table 1, P2P is an editing method for generated images. When applying it to real images (PIE-Bench++ dataset), it needs to be combined with an inversion technique (i.e., NTI, DI). Need to explain.\n8.\tThe finding that the order in which aspects are modified when applying single-aspect text-driven image editing methods can affect the quality is interesting. However, the paper does not conduct qualitative and quantitative experiments to support this claim.\n\nFurther comments:\n+ Line 119, should z be z_0 ?\n+ Line 135, Xu et al. [3] seems not to use \\cite{} to link the reference?\n+ Line148~Line149, the definition of tokens is not clear for me. Do the tokens represent all nouns and adjectives in a text prompt?\n\nQUESTIONS:\nDoes StyleDiffusion (StyleD) refer to “Controllable disentangled style transfer via diffusion models” or “Prompt-Embedding Inversion for Text-Based Editing” ? If it is the former, please explain why this paper is compared with style transfer methods. If it is the latter, please correct it.\n\nSimultaneously editing multiple objects or attributes is interesting. But some aspects of the method are not adequately explained (e.g., how to get the attention-map M), and some references within the paper are quite casual (e.g., DirectInversion and StyleDiffusion). \n\nThe weaknesses low my initial rate.\n\n---\n**Update after author responses** \nThanks to the authors for their efforts. I'm happy to see the extended discussion and additional comparisons that have resolved my concerns.\n\nLIMITATIONS:\nYes", "TITLE": "Official Review", "RECOMMENDATION": 6, "REVIEWER_CONFIDENCE": 5, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2024-06-23"}

comments: SUMMARY:
The paper introduces ParallelEdits, a method that manages simultaneous edits efficiently without compromising quality. This is achieved through a novel attention distribution mechanism and multi-branch design. Additionally, the authors present the PIE-Bench++ dataset, an expanded benchmark for evaluating multi-object and multi-attribute image-editing tasks.

STRENGTHS:
1. The task of multi-aspect editing seems interesting and useful in real life. 
2. The proposed method is efficient for application.
3. The overall editing results seem promising.

WEAKNESSES:
1. The authors claim the proposed method is based on the DDCM process [1]. However, when I checked the source paper of [1], I found something strange, especially its Algorithm 1. The dozens of iterations come to a simple conclusion that the output $z$ actually equals $z_0$, which means there is no need for such Virtual Inversion. Also, this process cannot be interpreted by Consistency Models. Hence, I wish the authors could revise their theoretical bases.

2. I think it is better to present the intermediate editing results for sequential editing, which can help readers identify what type of editing the previous methods are not good at.

3. I noticed that there was a benchmark [2] proposed for image editing. I advise the authors to use it for comprehensive evaluation.

[1] Inversion-Free Image Editing with Natural Language (https://arxiv.org/abs/2312.04965)

[2] Diffusion-Model Based Image Editing: A Survey (https://arxiv.org/abs/2402.17525)

QUESTIONS:
I noticed that the man has changed after editing in Fig. 2. Could you please explain and address it?

LIMITATIONS:
See above.
TITLE: Official Review
SOUNDNESS_CORRECTNESS: 3
DATE: 2024-07-08

comments: SUMMARY:
In this paper, the authors present a novel multi-aspect image editing method ParallelEdits, by incorporating the attention distribution mechanism and multi-branch editing. Besides, this paper introduces a new dataset PIE-Bench++ for evaluating multi-aspect image editing. Extensive experiments demonstrate the effectiveness of this method.

STRENGTHS:
1. The paper is well-written and well-organized.
2. The proposed method is novel and effectively addresses the multi-aspect image editing problem.
3. The results appear to have impressive visual effects. Edits of different aspects are combined seamlessly.
4. The authors conducted concrete experiments, providing rich quantitative results and qualitative results to demonstrate the effectiveness of the proposed ParallelEdits.

WEAKNESSES:
1. The pairing process of $E^{i \rightarrow j}$ imposes an additional burden on users, especially when multiple objects are present in the source image.
2.  It is not clear how to choose the hyperparameter $\lambda$ for assigning a type for rigid and non-rigid edit actions, and how it affects the cross-branch interaction and editing results.
3. This paper lacks a discussion on controlling the editing strength of multiple editing actions, which is crucial for harmonic and flexible editing.

QUESTIONS:
See the weakness part above.

LIMITATIONS:
The author has discussed the limitations and potential negative societal impacts.
TITLE: Official Review
SOUNDNESS_CORRECTNESS: 3
DATE: 2024-07-12